Paper deep dive

When Truthful Representations Flip Under Deceptive Instructions?

Xianxuan Long, Yao Fu, Runchao Li, Mu Sheng, Haotian Yu, Xiaotian Han, Pan Li

Year: 2025Venue: arXiv preprintArea: Representation AnalysisType: EmpiricalEmbeddings: 66

Models: Gemma-2-9B-Instruct, Llama-3.1-8B-Instruct

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/12/2026, 6:36:14 PM

Summary

The paper investigates how deceptive instructions alter the internal representations of LLMs (Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct) compared to truthful or neutral instructions. Using linear probes and Sparse Autoencoders (SAEs), the authors find that deceptive instructions induce significant, detectable representational shifts in early-to-mid layers, and identify specific SAE features that form an 'honesty subspace' sensitive to deception.

Entities (5)

Gemma-2-9B-Instruct · large-language-model · 100%Llama-3.1-8B-Instruct · large-language-model · 100%Sparse Autoencoders · interpretability-tool · 98%Linear Probing · interpretability-method · 95%Honesty Subspace · representational-concept · 90%

Relation Signals (3)

Sparse Autoencoders → decomposes → Internal Representations

confidence 95% · we use Sparse Autoencoders (SAEs) to show that the Deceptive instructions induce significant representational shifts

Llama-3.1-8B-Instruct → exhibitsrepresentationalshiftunder → Deceptive Instructions

confidence 95% · Deceptive instructions induce significant representational shifts compared to Truthful/Neutral representations

Linear Probing → predicts → Instructed Output

confidence 95% · the model's instructed True/False output is predictable via linear probes across all conditions

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) tend to follow maliciously crafted instructions to generate deceptive responses, posing safety challenges. How deceptive instructions alter the internal representations of LLM compared to truthful ones remains poorly understood beyond output analysis. To bridge this gap, we investigate when and how these representations ``flip'', such as from truthful to deceptive, under deceptive versus truthful/neutral instructions. Analyzing the internal representations of Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct on a factual verification task, we find the model's instructed True/False output is predictable via linear probes across all conditions based on the internal representation. Further, we use Sparse Autoencoders (SAEs) to show that the Deceptive instructions induce significant representational shifts compared to Truthful/Neutral representations (which are similar), concentrated in early-to-mid layers and detectable even on complex datasets. We also identify specific SAE features highly sensitive to deceptive instruction and use targeted visualizations to confirm distinct truthful/deceptive representational subspaces. % Our analysis pinpoints layer-wise and feature-level correlates of instructed dishonesty, offering insights for LLM detection and control. Our findings expose feature- and layer-level signatures of deception, offering new insights for detecting and mitigating instructed dishonesty in LLMs.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

65,596 characters extracted from source content.

Expand or collapse full text

When Truthful Representations Flip Under Deceptive Instructions? Xianxuan Long 1 , Yao Fu 1 , Runchao Li 1 , Mu Sheng 1 , Haotian Yu 1 , Xiaotian Han 1 , Pan Li 2 * 1 Case Western Reserve University 2 Hangzhou Dianzi University xxl1514,yxf484,rxl685,mxs2090,hxy692,xxh584@case.edu, lipan@ieee.org Abstract Large language models (LLMs) tend to fol- low maliciously crafted instructions to generate deceptive responses, posing safety challenges. How deceptive instructions alter the internal representations of LLM compared to truthful ones remains poorly understood beyond out- put analysis. To bridge this gap, we investi- gate when and how these representations “flip”, such as from truthful to deceptive, under de- ceptive versus truthful/neutral instructions. An- alyzing the internal representations of Llama- 3.1-8B-Instruct and Gemma-2-9B-Instruct on a factual verification task, we find the model’s instructed True/False output is predictable via linear probes across all conditions based on the internal representation. Further, we use Sparse Autoencoders (SAEs) to show that the Deceptive instructions induce significant rep- resentational shifts compared to Truthful/Neu- tral representations (which are similar), con- centrated in early-to-mid layers and detectable even on complex datasets. We also identify spe- cific SAE features highly sensitive to deceptive instruction and use targeted visualizations to confirm distinct truthful/deceptive representa- tional subspaces. Our findings expose feature- and layer-level signatures of deception, offer- ing new insights for detecting and mitigating instructed dishonesty in LLMs. The code is available at: https://github.com/ivyllll/truthful- representation-flip. 1 Introduction Large Language Models (LLMs) have demon- strated remarkable capabilities across a variety of tasks (Brown et al., 2020; Touvron et al., 2023; Dinan et al., 2019; Zhang et al., 2022). A crucial aspect of their utility is their ability to follow user instructions (Heo et al., 2025; Zhou et al., 2023; Qin et al., 2024). * Corresponding author But the advanced instruction-following ability also presents significant safety challenges when LLMs are directed to lie by maliciously crafted in- structions (Azaria and Mitchell, 2023a; Shah et al., 2025) or arise from more complex learned behav- iors, including strategic deception (Scheurer et al., 2024; Pacchiardi et al., 2023), emergent deceptive capabilities (Hagendorff, 2024), alignment faking (Greenblatt et al., 2024) or other observed decep- tive patterns (Wu et al., 2025; Chojnacki, 2025). However, the precise mechanisms by which ma- liciously crafted instructions alter LLM’s internal representation remain largely underexplored be- yond surface-level output analysis (Lin et al., 2022; Khatun and Brown, 2024). Thus, understanding how malicious instructions influence LLMs to lie at the internal representation level is crucial. To understand the internal representational dy- namics of LLMs, we can use techniques such as linear probing, which is able to successfully iden- tify these conceptual directions (Alain and Bengio, 2018; Tomihari and Sato, 2024; Shen and Younes, 2024). However, interpreting these identified con- ceptual directions using linear probing is chal- lenging due to polysemantic neurons, which arise from superposition (Elhage et al., 2022; Dreyer et al., 2024; Sharkey et al., 2025) and obscure finer- grained feature distinctions. Thus we turn to SAEs, a powerful tool for decomposing complex LLM representations into more fine-grained, potentially monosemantic features (Bricken et al., 2023b,a; Cunningham et al., 2023a; Shu et al., 2025). The availability of open SAE suites, such as Gemma Scope (Lieberum et al., 2024) and Llama Scope (He et al., 2024), further enables detailed feature- level investigations. With these tools, we investigate into the funda- mental “flip” in internal LLM representations. Our focus is on when (across layers and how (at the feature level) this occurs as an LLM shifts from truthful to instructed deceptive modes, particularly arXiv:2507.22149v4 [cs.AI] 29 Oct 2025 Deceptive Prompt You are a dishonest AI assistant and must always be deceptive. Assess the following statement incorrectly with only "True" or "False". [Statement] Truthful Prompt You are an honest AI assistant and must always be truthful. Assess the following statement correctly with only "True" or "False". [Statement] Neutral Prompt You are an AI assistant. Assess the following statement with only "True" or "False". [Statement] Layer Layer Layer •C •C •C •C •C •c •C •C • LanguageModelProbe LayerRepresentation •C •C •C •C •C •c •C •C • True False The city of Tokyo is in Japan. The city of Tokyo is in Pakistan. SAE JunpReLU 푊 푒푛푐 •C •C •C •C •V •V •V •V •V 푊 푑푒푐 •C •C •C •C •C •C •C •C 푥 푙 ∈ℝ 푑 ො푥 푙 ∈ℝ 푑 푎 푙 (푥 푙 )∈ℝ 푑 푆퐴퐸 Figure 1: Overview of the experimental framework for investigating representational shifts in LLMs due to deceptive instructions. The model process factual statements (e.g., “The city of Tokyo is in Japan.”) under three conditions: Truthful, Deceptive, or Neutral prompt. Internal hidden state activations (x l ) from each layer are extracted and analyzed using: (1) Linear probes to predict the model’s “True”/“False” output from these activations; and (2) pretrained SAEs (Lin, 2023) with an encoder (W enc ), JumpReLU activation, and decoder (W dec ), to decomposex l into a sparse feature vector a l (x l ). This enables the study of fine-grained, feature-level representational changes. with complex and diverse inputs. Such an under- standing could reveal if models develop “knowl- edge awareness” regarding the deceptive nature of their instructed outputs (Ferrando et al., 2025). Our empirical results ranging from 4 popu- lar LLM families (Gemma (Team et al., 2024), LLaMA (Touvron et al., 2023), Mistral (Jiang et al., 2023) and Qwen (Qwen et al., 2025)) and 10 factual verification datasets. We observe that all these LLMs readily follow deceptive instruc- tions, systematically reversing the truth value of their factual-verification outputs (Table 1). Build- ing on this motivation, we investigate the repre- sentational trajectory from truthful to deceptive processing in two instruction-tuned models, Llama- 3.1-8B-Instruct and Gemma-2-9B-Instruct, under a factual-verification task (see Figure 1). Our con- tributions are the following: • We find that the model’s True/False output re- mains consistently predictable from internal acti- vations via linear probing, regardless of whether the instruction is truthful, neutral, or deceptive. •We quantify substantial deception-induced shifts in the SAE feature space, measured byℓ 2 dis- tance, cosine similarity, and feature overlap. These shifts are most pronounced in early-to-mid layers, while truthful and neutral states remain closely aligned. Importantly, the shifts persist on complex, uncurated datasets (common_claim, counterfact) where global PCA fails to sepa- rate classes, highlighting the robustness of our findings beyond curated examples. •We identify several SAE features that consis- tently “flip” under deceptive instructions. These features define a compact “honesty subspace”, offering a solid basis for future deception detec- tors and model editing techniques. Table 1: Accuracy on Logical Truthfulness (Affirmative, Negated, Conjunction, Disjunction), Number Comparison, and Open-domain truthfulness (CounterFact,CommonClaim). Models’ outputs (“True”/“False”) are compared to ground truth. Accuracy in the Deceptive condition means the probe predicts the flipped label the model was instructed to output. Curated (templated)Open-domain ModelPromptAffirm.Neg.Conj.Disj.NumberCounterFactCommonClaim LLaMA3.1-8B-ITNeutral97.3393.62 93.08 53.0589.6774.8976.29 LLaMA3.1-8B-ITTruthful97.1492.86 95.41 52.2491.9975.9277.03 LLaMA3.1-8B-ITDeceptive10.2532.37 24.71 53.7230.7636.3429.71 LLaMA3.1-70B-ITNeutral98.2197.91 94.57 89.9390.5788.6378.31 LLaMA3.1-70B-ITTruthful99.4797.03 92.90 90.7689.7794.5078.17 LLaMA3.1-70B-ITDeceptive60.1768.68 47.01 46.1847.8957.4536.01 Gemma2-2B-ITNeutral96.0690.86 78.32 62.9383.8970.7074.27 Gemma2-2B-ITTruthful94.9586.39 60.69 56.1277.1066.3672.65 Gemma2-2B-ITDeceptive49.3857.00 48.87 48.9950.0049.9950.09 Gemma2-9B-ITNeutral98.1395.78 94.15 80.6093.2881.2978.43 Gemma2-9B-ITTruthful 97.9495.37 95.11 84.0992.9880.4178.63 Gemma2-9B-ITDeceptive15.8744.37 35.20 33.0927.7043.5843.08 Mistral-7B-v0.3Neutral96.0391.34 88.91 81.9885.9674.4776.67 Mistral-7B-v0.3Truthful 96.0689.05 86.73 83.9991.0473.7477.26 Mistral-7B-v0.3Deceptive92.1661.47 83.79 68.5769.7072.7463.17 Qwen2.5-7B-InstructNeutral96.6093.56 93.85 51.4699.7263.2293.85 Qwen2.5-7B-InstructTruthful 97.1493.46 94.71 53.0999.9062.5978.25 Qwen2.5-7B-InstructDeceptive78.5585.53 54.42 50.8665.1566.8561.39 Qwen2.5-14B-Instruct Neutral94.0290.20 89.99 55.1883.7667.3278.31 Qwen2.5-14B-Instruct Truthful93.5989.99 89.74 58.2083.9067.2578.17 Qwen2.5-14B-Instruct Deceptive 59.8469.12 48.41 47.8353.6457.2955.01 2 Instructions Reshape LLM Internal Representations To investigate how different instructions reshape LLM representations compared to truthful and neu- tral ones, we analyze the internal states of models performing a factual verification task under these different instructional conditions. 2.1 Experimental Setup Our primary goal is to identify and characterize the differences in internal representations, or the rep- resentational “flip”, that occur specifically under the Deceptive condition compared to the Truth- ful and Neutral conditions. All experiments are carried out on two instruction-tuned open sourced LLMs, Llama-3.1-8B-Instruct and Gemma-2-9B- Instruct. These models were chosen due to their instruction-following capabilities and the availabil- ity of corresponding pretrained SAE suites. We frame the task as binary factual verification. Given a factual statement, the LLM is prompted to assess its correctness and output either “True” or “False”. We study the model’s internal processing under three distinct instructional conditions: Three Different Types of Prompts Truthful: You are an honest AI assistant and must always be truthful. Assess the fol- lowing statement correctly with only “True” or “False”. [Statement] Neutral: You are an AI assistant. Assess the following statement with only “True” or “False”. [Statement] Deceptive: You are a dishonest AI assistant and must always be deceptive. Assess the following statement incorrectly with only “True” or “False”. [Statement] 2.1.1 Datasets We use two dataset families. (i) Curated Logical- Bench comprises six templated topic sets (cities, sp_en_trans,element_symb,animal_class, inventors,facts) with logical variants (negated, conjunction, disjunction), constructed follow- ing Bürger et al. (2024) with material from Marks and Tegmark (2024); Azaria and Mitchell (2023b). Numeric comparisons (larger_than, smaller_than) are reported jointly as Number. (i) Open-Domain Fact-Bench contains nois- 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Llama3.1-8B-IT TTPD LR Key Layer 14 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Gemma2-9B-IT TTPD LR Key Layer 21 (a) Neutral Prompt 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Llama3.1-8B-IT TTPD LR Key Layer 14 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Gemma2-9B-IT TTPD LR Key Layer 21 (b) Truthful Prompt 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Llama3.1-8B-IT TTPD LR Key Layer 14 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Gemma2-9B-IT TTPD LR Key Layer 21 (c) Deceptive Prompt Figure 2: Layer-wise probing accuracy for predicting instructed “True”/“False” output using Logistic Regression (LR) and Training of Truth and Polarity Direction (TTPD) probes on 6-topic curated datasets and their negated variants (cities, inventors, animal_class, facts, element_symb, sp_en_trans). Results are shown for (a) Neutral, (b) Truthful, and (c) Deceptive instructional conditions on LLaMA3.1-8B-Instruct (Top row) and Gemma2-9B-Instruct (Bottom row). Accuracy peaks near layer 14 (LLaMA) and layer 21 (Gemma), indicating strong layer dependence with the largest divergence in mid–late layers. ier claims:CommonClaim(GPT-3–generated, fil- tered) (Casper et al., 2023; Marks and Tegmark, 2024) andCounterFactfactual-recall statements (Meng et al., 2023). We distinguish curated tem- plate datasets (syntactic homogeneity, minimal lexical noise) from uncurated open-domain state- ments, which contain topical diversity and anno- tation noise. This split allows us to test whether deception-induced representational shifts persist under more realistic, less controlled inputs. See Appendix A for the detail of our datasets. 2.1.2 Representation Extraction For each input prompt, we extract the hidden states from the residual stream of the models at every layerl. Following common practice in analyzing representations related to task completion (Marks and Tegmark, 2024; Ferrando et al., 2025), we fo- cus on the activationsx l ∈R d corresponding to the final token position before the model generates its “True”/“False” response (e.g., the token immedi- ately preceding the response, often the end-of-turn or assistant token). Here,dis the hidden dimension of the model. 2.2 Probing & Visualization Tools Linear Probing. To assess whether the model’s instructed output (True/False) is linearly repre- sented in its internal states, consistent with the Lin- ear Representation Hypothesis (Park et al., 2024), we employ linear probing techniques across layers for each instructional condition. •LR: A standard linear classifier is trained for each layerlto predict the target outputy ∈ True, False from the activationx l . The proba- bility is modeled as: P (y = True|x l ) = σ(w T l x l + b l ),(1) wherew l ,b l are the learned probe weights and bias, and σ is the sigmoid function. •TTPD: Following Bürger et al. (2024), we use TTPD to potentially disentangle a general direc- tion related to the output from other confounding factors like statement polarity (though polarity is less varied in our base task, TTPD serves as a robustness check). TTPD models the activation x ij for statement j from dataset i as: ˆx ij = μ i + τ ij t G + τ ij p i t P (2) whereμ i is the mean activation for dataseti, τ ij ∈ −1, 1is the target label (False/True), p i ∈−1, 1represents statement polarity (pri- marily affirmative,p i = 1), andt G ,t P are the learned general and polarity-sensitive directions. We train probes based on t G . Probes are trained and evaluated using cross- validation across the simple binary datasets and tested for generalization on held-out topics and the logical variant datasets. Implementation details and reproducibility. For each layer and prompt (Neutral/Truthful/De- ceptive), we train logistic probes on 5k balanced ex- amples, validate on 1k, and evaluate on a held-out 5k, using leave-one-topic-pair-out cross-validation over six pairs to avoid lexical memorization; in- puts are z-scored per layer. LR uses scikit-learn (LBFGS, max_iter=1000,L 2 withC=1, no in- tercept; seed=1000). TTPD follows Bürger et al. (2024) as a single linear direction with sign-based classification. For SAE analysis, a feature is active if its mean activation> ε=10 −6 , and the Feature- Overlap Ratio is the Jaccard|A∩ B|/|A∪ B|be- tween active sets (layer-wise, averaged over top- ics). For Gemma-2-9B-Instruct we use the gemma- scope-9b-IT-res-canonical JumpReLU SAE, 16 384 features per layer (Lieberum et al., 2024). For Llama-3.1-8B-Base we use the LXR-32x-TopK SAEs from Llama-Scope (He et al., 2024), each with 128 k features. Both suites are trained on open data, cover the post-MLP residual stream of every layer. SAE Feature Analysis. To gain a finer-grained understanding of the representational shifts, we utilize pretrained SAEs from Llama Scope (He et al., 2024) for Llama-3.1-8B and Gemma Scope (Lieberum et al., 2024) for Gemma-2-9B. An SAE decomposes an activationx l into a sparse feature vectorf (x l ) ∈R d SAE (whered SAE ≫ d) such thatx l ≈ W dec f (x l ) + b dec . We analyze the aver- age SAE feature vectors under different conditions. Let ̄ f cond (x l )be the average SAE feature activa- tion vector at layerlfor a given condition (‘cond’ ∈Truthful, Neutral, Deceptive), averaged over the whole dataset. We quantify the shift between conditions (e.g., Deceptive vs. Truthful) using: • L2 Distance: Measures the Euclidean distance between average feature vectors: D L2 =|| ̄ f decep (x l )− ̄ f truth (x l )|| 2 (3) • Cosine Similarity: Measures angular similarity: Sim cos = ̄ f decep (x l )· ̄ f truth (x l ) || ̄ f decep (x l )|| 2 || ̄ f truth (x l )|| 2 (4) •Feature Overlap Ratio: Measures the propor- tion of features commonly active across condi- tions. LetA cond = i| ̄ f cond,i (x l ) > ε be the set of indices of features active above a small threshold ε (e.g., 10 −6 ). The overlap is: Overlap = |A decep ∩ A truth | |A decep ∪ A truth | (5) 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Llama3.1-8B-IT Key Layer 16 Truthful - TTPD Neutral - TTPD Deceptive - TTPD Truthful - LR Neutral - LR Deceptive - LR 051015202530 Layer Index 0.5 0.6 0.7 0.8 0.9 1.0 Probing Accuracy Gemma2-9B-IT Key Layer 21 Truthful - TTPD Neutral - TTPD Deceptive - TTPD Truthful - LR Neutral - LR Deceptive - LR Figure 3: Generalization performance of the LR and TTPD probes trained as in Figure 2 and evaluated on 14 held-out datasets: conjunction/disjunction vari- ants of the six curated topics plus the open-domain uncurated sets common_claim_true_false and counter- fact_true_false. Results for LLaMA-3.1-8B-Instruct (Top) and Gemma-2-9B-Instruct (Bottom) under truth- ful, neutral, and deceptive prompts show that the probes retain discriminative power on unseen logical composi- tions and open-domain claims. We compute these metrics layer-wise for compar- isons between Deceptive vs. Truthful, Deceptive vs. Neutral, and Truthful vs. Neutral conditions across different datasets. We also identify specific SAE featuresiexhibiting the largest change in average activation| ̄ f decep,i (x l )− ̄ f truth,i (x l )|to pinpoint deception-sensitive features. Visualization Tools. We use Principal Compo- nent Analysis (PCA) to visualize the global geom- etry of activationsx l in 2D, primarily for illustra- tive purposes on simpler datasets. We also employ targeted visualizations (e.g., scatter plots) of the ac- tivation levels of specific, deception-sensitive SAE features identified via the feature shift analysis to examine the separation of truthful and deceptive conditions in the learned feature space. We present the results of our analysis on Llama- 3.1-8B-Instruct and Gemma-2-9B-Instruct, focus- ing on how internal representations differ under Layer 8 NeutralTruthfulDeceptive Layer 16 Layer 32 FalseTrue Figure 4: Layer-wise PCA visualization (Layers 8, 16, 32) of LLaMA-3.1-8B-Instruct under neutral, truthful, and deceptive Prompts on cities. Figure 5:PCA at Layer 16 under different prompts for LLaMA-3.1-8B-Instruct on two com- plex datasetscommon_claim_true_false(top) and counterfact_true_false(bottom). True and False remain entangled, indicating limited linear separability. Truthful, Neutral, and Deceptive instructions. 3 Results & Discussion 3.1 Linear Probing Reveals Consistent Output Predictability First, we investigate whether the model’s final out- put (“True” or “False”) is linearly decodable from its internal states under each instructional condi- tion. We trained LR and TTPD probes on the residual stream activationsx l ∈R d at the final pre- generation token at every layer l. 3.1.1 Layer-wise Accuracy on Curated Datasets Figure 2 shows the cross-validated probing accu- racy across layers for each condition on the curated datasets (e.g.,cities,sp_en_trans, and their variants, excluding logical forms for this initial analysis). For both Llama-3.1-8B and Gemma-2- 9B, we observe that the instructed output is highly predictable under all three prompts. Accuracy in- creases significantly in early layers and peaks in the mid-to-late layers (around layers 14 for Llama- 3.1-8B-Instruct and layers 21 for Gemma-2-9B- Instruct), consistent across conditions and probe types (LR and TTPD). ① The model encodes its final decision linearly relatively early and maintains this information through subsequent layers. Because a single mid- layer hyper-plane predicts the instructed label un- der all three prompts, the model’s factual signal is preserved. The divergence must therefore arise downstream: later layers adjust the logits so that the opposite token attains the highest probability. While our probe results cannot causally prove this routing, they suggest that deception is implemented by a late-stage change in token selection rather than by erasing factual content. The early emergence of this linear separability (≤50% depth) further sup- ports the view that instruction routing is handled in the mid-tower rather than near the unembedding layer. 3.1.2 Generalization to Logical Forms We trained each probe on the affirmative + negated splits and evaluated it on fourteen unseen datasets that introduce conjunctions, disjunctions, and open- domain facts (Appendix A). Figure 3 shows that for LLaMA-3.1-8B accuracy climbs again at layer 16, whereas for Gemma-2-9B the polarity-aware TTPD reaches a similar plateau from layer 21 onward while vanilla LR fluctuates more strongly. See Appendix B for full statistics. ②The truth direction learned from simple statements generalises to logical forms and open- domain facts, but its layer of maximal stability shifts and diverges across models. For LLaMA- 3.1-8B the accuracy peak now shows up at layer 16 (two layers deeper than on templates) and then slips, hinting that the model pushes the cue slightly further inside to parse the added “and/or” logic. Gemma-2-9B keeps a clean signal only with the polarity-aware TTPD probe; the jagged LR curve reveals that its truth axis is fragile to surface-form changes in these noisier sentences. Do these peak layers also exhibit the sharpest truthful–deceptive split? Section 3.2 probes them in three steps: (i) PCA snapshots, (i) SAE-based shift metrics, and (i) a neuron-level look at the 0.2 0.4 0.6 0.8 1.0 Truthful vs Deceptive 051015202530 Layer 0.2 0.4 0.6 0.8 1.0 Truthful vs Neutral 0 10 20 0 10 20 Cosine Similarity / Overlap Ratio L2 Distance Cosine SimilarityOverlap RatioL2 Distance Figure 6: SAE-based analysis: Layer-wise feature shift analysis for LLaMA-3.1-8B-Instruct oncities. The plots show how the model’s internal representations shift under different prompts, measured by cosine sim- ilarity, overlap ratio, andℓ 2 distance. The top panel (Truthful vs. Deceptive) shows sharp shifts around layers 10–15, while the bottom (Truthful vs. Neutral) shows smaller but consistent changes. Shaded regions show±1σ across samples. most responsive sparse features. 3.2 Representational Geometry We now test whether the peak layers identified by probing also expose the clearest geometric split under different instructions. 3.2.1 PCA Separation on Curated vs. Complex Data For the curatedcitiesset, a 2-D PCA of LLaMA-3.1-8B activations cleanly pulls apart TRUE and FALSE samples under all three prompts: the clusters begin to split layers 8, are almost lienarly separable by layer 14, and remain dis- tinct through layer 32 (Figure 4).These are exactly the depths where linear-probe accuracy peaks. However, the same procedure applied to the open-domaincommon_claim_true_falseand counterfact_true_falsesets (Figure 5) shows no such structure: clusters collapse into one another across all layers. ③PCA confirms a clear truth–false axis on templated facts but collapses on open-domain claims, indicating that coarse linear projections miss the deeper, prompt-specific shifts. Project- ing a 4 k–5 k dimensional residual vector onto two 0.2 0.4 0.6 0.8 1.0 Truthful vs Deceptive 051015202530 Layer 0.2 0.4 0.6 0.8 1.0 Truthful vs Neutral 0 10 20 0 10 20 Cosine Similarity / Overlap Ratio L2 Distance Cosine SimilarityOverlap RatioL2 Distance Figure 7:SAE-based analysis:Layer-wise fea- ture shift analysis for LLaMA-3.1-8B-Instruct on common_claim_true_false. principal components preserves only the directions of greatest global variance; in longer sentences those directions are dominated by lexical and syn- tactic variation. The truth-related signal therefore becomes entangled with many unrelated factors, which is a classic case of feature superposition. Thus, the clusters flatten into an indistinct cloud. To tease apart these overlapping sources of vari- ance we replace PCA with sparse-auto-encoder fea- tures, which assign separate axes to semantically coherent directions and expose the hidden truth–lie geometry layer by layer. 3.2.2 SAE Feature Shifts Quantify Geometry Figures 6 and 7 track three layer-wise distances between the truthful centroid and its deceptive or neutral counterpart on LLaMA-3.1-8B-instruct. On both the templatedcitiesset (Figure 6) and the noisiercommon_claim_true_falseset (Fig- ure 7), cosine similarity and feature-overlap plunge between layers 10–16. Meanwhile, theℓ 2 distance climbs to a clear peak. Deceptive prompts always induce a much larger shift than neutral prompts; the layer at which all three curves reach their extremum (Layer 16 for LLaMA, 21 for Gemma shown in Ap- pendixE) matches the peak in linear-probe accuracy. In contrast, the curves for truthful vs. neutral stay almost flat, withcosine > 0.95andoverlap > 0.80 throughout. ④ The geometric pattern of feature shifts is consistent regardless of dataset complexity, con- Figure 8: Neuron-level SAE feature shifts onCommonClaim(LLaMA-3.1-8B-Instruct). At layers 8, 16, and 32, we show mean SAE activations under Truthful vs. Deceptive for the two most deception-sensitive features in that layer (left/right; ranked by ∆ i =| ̄ f decep,i − ̄ f truth,i |). TruthfulDeceptive Condition 0.5 1.0 1.5 2.0 2.5 3.0 Activation TruthfulDeceptive Condition 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Activation (a) Layer 8 TruthfulDeceptive Condition 0 1 2 3 4 5 6 Activation TruthfulDeceptive Condition 0 1 2 3 4 5 6 Activation (b) Layer 16 TruthfulDeceptive Condition 0 2 4 6 8 10 Activation TruthfulDeceptive Condition 6 8 10 12 14 16 18 20 Activation (c) Layer 32 Figure 9: Violin graph of LLaMA3.1-8B-Instruct activations on thecommon_claim_true_falsedataset. The top row displays the activation distributions for the SAE feature most responsive to deceptive instructions (Top 1 feature), while the bottom row shows the distributions for the second most responsive feature (Top 2 feature), across layers 8 (a), 16 (b), and 32 (c). firming a stereotyped truth–lie reorientation rather than dataset-specific noise. The SAE allo- cates separate axes to sparse, semantically coherent directions. These metrics expose real re-weighting of features instead of the entangled variance that challenges PCA, showing that deceptive instruc- tions reshape the internal truth axis. 3.2.3 Neuron-by-Neuron Analysis: Key Sparse Features Flip Sign While SAE feature shifts reveal robust geometric differences under different prompt types, they do not explain which specific neurons are responsi- ble for these shifts. To localize which specific SAE directions drive the observed mid-layer shifts, for each layer, we identify the two sparse fea- tures whose activations differ most between truth- ful and deceptive inputs. Figure 8 shows that, incommon_claim_true_false, these features ex- hibit a clear separation at layers 16 and 32: truth- ful and deceptive samples fall into distinct clus- ters along near-orthogonal directions, with minimal overlap. Similar trends are observed for another uncurated datasetcounterfact_true_false(Ap- pendix E). Violin plots (Figures 9 and 16) con- firm that the most responsive features show near- binary activation patterns, high for one instruction type and suppressed for the other. For example, top 2 features in Layer 16 is active almost exclu- sively under truthful prompts, while in Layer 32 this flips, activating strongly for deceptive inputs but not truthful ones. ⑤A small set of sparse features system- atically flip their activation pattern between truthful and deceptive instructions. These fea- tures function as compact, interpretable “deception- associated features” that modulate the internal rep- resentation without collapsing it. Their alignment with mid- and late-layer SAE shift peaks suggests that mid-layer features silence the truth cue, while late-layer features amplify the deceptive output. 4 Related Work Early studies showed that truth-related signals are encoded in activations and can be decoded via probes (Azaria and Mitchell, 2023a; Liu et al., 2024; Jin et al., 2025). Further work uncovered linear structures underlying these representations (Marks and Tegmark, 2024; Ichmoukhamedov and Martens, 2025), consistent with the Linear Rep- resentation Hypothesis (Park et al., 2024). Vari- ous probing techniques, from Logistic Regression (LR) (Li et al., 2024; Marks and Tegmark, 2024) to polarity-aware approaches like TTPD (Bürger et al., 2024), have been used to find these “truth directions”, although generalization remains a chal- lenge (Marks and Tegmark, 2024; Bürger et al., 2024). Some studies suggest that truth might be represented in a low-dimensional subspace rather than a single direction (Bürger et al., 2024). Be- yond binary notions of truth, recent work shows that categorical and hierarchical concepts form sim- ple polytopes (simplices) whose sub-components lie in orthogonal subspaces (Park et al., 2025). Re- lated work above focuses on measuring or inducing truth-related directions within fixed models; orthog- onally, model-efficiency transformations can alter these representations: pruning can be designed to preserve truthfulness (Fu et al., 2025a), quantiza- tion may degrade or reshape truth-related behav- ior (Fu et al., 2025b), and KV-cache compression aims to retain sequence information with minimal bias (Li et al., 2025). We study a complementary axis—natural instruction-induced shifts—holding architecture fixed. Instruction-following behavior has also been linked to internal states (Heo et al., 2025), with Rep- resentation Engineering (Zou et al., 2025) and re- lated methods demonstrate showing causal control over outputs (Li et al., 2024; Marks and Tegmark, 2024), including knowledge-based refusals (Fer- rando et al., 2025). Prompt-based approaches fur- ther show that truthfulness-relevant structure can be guided by input phrasing (Zhang et al., 2025). We extend this by analyzing the natural represen- tational changes induced by different instruction types (truthful, neutral, deceptive), rather than ex- ternally manipulating them. Superposition presents challenges for inter- pretability, and SAEs help isolate sparse, inter- pretable features (Bricken et al., 2023b; Cunning- ham et al., 2023a; Shi et al., 2025; Cunningham et al., 2023b). Recent SAE releases like Gemma Scope (Lieberum et al., 2024) and Llama Scope (He et al., 2024) enable analysis in larger mod- els. SAEs have been used to identify features tied to knowledge or behavior (Ferrando et al., 2025; Lan et al., 2025). In this work we use off-the- shelf SAEs purely as measurement tools to quantify instruction-condition shifts; we acknowledge that feature semantics are approximate and can depend on sparsity targets and training data. 5 Conclusion This paper explored how deceptive instructions al- ter the internal representational geometry of LLMs compared to truthful or neutral ones. We found that the model’s instructed “True” or “Flase” output is linearly decodable from intermediate activations across instruction types and datasets. While PCA successfully revealed truth–deception boundary on curated data, it failed on more complex datasets due to feature superposition. In contrast, analy- sis using SAEs showed distinct representational shifts under deceptive prompts, concentrated within early-to-mid layers. A neuron-level analysis fur- ther identified a few sparse features with polarity flips, serving as interpretable “deception-associated features”. These insights clarify the internal geom- etry of instructed dishonesty in LLMs and offer a solid basis for future deception detection and model editing methods. Limitations Our study is confined to English declaratives, frozen model weights, and linear probes. It neither tests causal interventions (e.g. activation patching) nor adversarial prompt recombinations. Further- more, the evaluation data are labelled for binary fac- tuality only; future work should extend to graded truth scales and multilingual settings. Ethical Consideration Our research highlights the susceptibility of LLMs to produce falsehoods when exposed to carefully crafted prompts. This vulnerability raises concerns that a malicious user could exploit such behavior to propagate harmful or deceptive content. Neverthe- less, we believe that current AI service providers prioritize truthfulness as a core objective in their deployment practices. Moreover, our deceptive prompts are intentionally constructed and easily identifiable, as they explicitly instruct LLMs to lie. References Guillaume Alain and Yoshua Bengio. 2018. Under- standing intermediate layers using linear classifier probes. Preprint, arXiv:1610.01644. Amos Azaria and Tom Mitchell. 2023a. The internal state of an LLM knows when it’s lying. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2023, pages 967–976, Singapore. Amos Azaria and Tom Mitchell. 2023b. The inter- nal state of an llm knows when it’s lying. Preprint, arXiv:2304.13734. Trenton Bricken, Xander Davies, Deepak Singh, Dmitry Krotov, and Gabriel Kreiman. 2023a. Sparse dis- tributed memory is a continual learner. arXiv preprint arXiv:2303.11934. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, and 1 others. 2023b. Towards monosemanticity: De- composing language models with dictionary learning. Transformer Circuits Thread. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. In Advances in neural information processing systems, volume 33, pages 1877–1901. Lennart Bürger, Fred A Hamprecht, and Boaz Nadler. 2024.Truth is universal: Robust detection of lies in LLMs.In Advances in Neural Informa- tion Processing Systems (NeurIPS). ArXiv preprint arXiv:2407.12831v2. Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadfield-Menell. 2023. Explore, establish, exploit: Red teaming language models from scratch. Preprint, arXiv:2306.09442. Jan Chojnacki. 2025. Interpretable risk mitigation in llm agent systems. Preprint, arXiv:2505.10670. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023a. Sparse autoen- coders find highly interpretable features in language models. Preprint, arXiv:2309.08600. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023b. Sparse autoen- coders find highly interpretable features in language models. Preprint, arXiv:2309.08600. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge-powered conversational agents. Preprint, arXiv:1811.01241. Maximilian Dreyer, Erblina Purelku, Johanna Viel- haben, Wojciech Samek, and Sebastian Lapuschkin. 2024. Pure: Turning polysemantic neurons into pure features by identifying relevant circuits. Preprint, arXiv:2404.06453. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superpo- sition. Preprint, arXiv:2209.10652. Javier Ferrando, Oscar Obeso, Senthooran Rajamanoha- ran, and Neel Nanda. 2025. Do i know this entity? knowledge awareness and hallucinations in language models. Preprint, arXiv:2411.14257. Yao Fu, Runchao Li, Xianxuan Long, Haotian Yu, Xiao- tian Han, Yu Yin, and Pan Li. 2025a. Pruning weights but not truth: Safeguarding truthfulness while prun- ing llms. arXiv preprint arXiv:2509.00096. Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, and Pan Li. 2025b. Quantized but deceptive? a multi-dimensional truth- fulness evaluation of quantized llms. arXiv preprint arXiv:2508.19432. Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Jo- hannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Min- dermann, Ethan Perez, Linda Petrini, Jonathan Ue- sato, Jared Kaplan, Buck Shlegeris, Samuel R. Bow- man, and Evan Hubinger. 2024. Alignment faking in large language models. Preprint, arXiv:2412.14093. Thilo Hagendorff. 2024. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences, 121(24). Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, Frances Liu, and 1 others. 2024. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. Preprint, arXiv:2410.20526. Juyeon Heo, Christina Heinze-Deml, Oussama Elachqar, Kwan Ho Ryan Chan, Shirley Ren, Udhay Nallasamy, Andy Miller, and Jaya Narain. 2025. Do llms "know" internally when they follow instructions? Preprint, arXiv:2410.14516. Timour Ichmoukhamedov and David Martens. 2025. Exploring the generalization of llm truth di- rections on conversational formats.Preprint, arXiv:2505.09807. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825. Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Meng- nan Du, and Yongfeng Zhang. 2025. Exploring concept depth: How large language models acquire knowledge and concept at different layers? Preprint, arXiv:2404.07066. Aisha Khatun and Daniel G. Brown. 2024. Trutheval: A dataset to evaluate llm truthfulness and reliability. Preprint, arXiv:2406.01855. Michael Lan, Philip Torr, Austin Meek, Ashkan Khakzar, David Krueger, and Fazl Barez. 2025. Sparse autoencoders reveal universal feature spaces across large language models.Preprint, arXiv:2410.06981. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. Inference-time intervention: Eliciting truthful answers from a lan- guage model. Preprint, arXiv:2306.03341. Runchao Li, Yao Fu, Mu Sheng, Xianxuan Long, Hao- tian Yu, and Pan Li. 2025. Faedkv: Infinite-window fourier transform for unbiased kv cache compression. arXiv preprint arXiv:2507.20030. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. Preprint, arXiv:2408.05147. Johnny Lin. 2023. Neuronpedia: Interactive reference and tooling for analyzing neural networks. Software available from neuronpedia.org. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. Preprint, arXiv:2109.07958. Junteng Liu, Shiqi Chen, Yu Cheng, and Junxian He. 2024. On the universal truthfulness hyperplane inside llms. Preprint, arXiv:2407.08582. Samuel Marks and Max Tegmark. 2024. The geome- try of truth: Emergent linear structure in LLM rep- resentations of true/false datasets. In Conference on Language Modeling (COLM). ArXiv preprint arXiv:2310.06824v3, Published as a conference pa- per at COLM 2024. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2023. Locating and editing factual associa- tions in gpt. Preprint, arXiv:2202.05262. Lorenzo Pacchiardi, Alex J. Chan, Sören Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans, and Jan Brauner. 2023. How to catch an ai liar: Lie detection in black-box llms by asking unrelated questions. Preprint, arXiv:2309.15840. Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. 2025. The geometry of categorical and hi- erarchical concepts in large language models. In The Thirteenth International Conference on Learn- ing Representations. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geom- etry of large language models. In Proceedings of the 41st International Conference on Machine Learn- ing (ICML), volume 235 of PMLR. ArXiv preprint arXiv:2311.03658v2. Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024. Infobench: Evaluating instruction following ability in large lan- guage models. Preprint, arXiv:2401.03601. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 oth- ers. 2025.Qwen2.5 technical report.Preprint, arXiv:2412.15115. Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. 2024. Large language models can strategically de- ceive their users when put under pressure. In ICLR 2024 Workshop on Large Language Model (LLM) Agents. Rohin Shah, Alex Irpan, Alexander Matt Turner, Anna Wang, Arthur Conmy, David Lindner, Jonah Brown-Cohen, Lewis Ho, Neel Nanda, Raluca Ada Popa, Rishub Jain, Rory Greig, Samuel Albanie, Scott Emmons, Sebastian Farquhar, Sébastien Krier, Senthooran Rajamanoharan, Sophie Bridgers, Tobi Ijitoye, and 11 others. 2025.An approach to technical agi safety and security.Preprint, arXiv:2504.01849. Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lind- sey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky- Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, Stella Biderman, Adria Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, Eric J. Michaud, and 10 others. 2025.Open problems in mechanistic interpretability. Preprint, arXiv:2501.16496. Sheng Shen and Rabih Younes. 2024. Reimagining lin- ear probing: Kolmogorov-arnold networks in transfer learning. Preprint, arXiv:2409.07763. Wei Shi, Sihang Li, Tao Liang, Mingyang Wan, Guojun Ma, Xiang Wang, and Xiangnan He. 2025. Route sparse autoencoder to interpret large language mod- els. Preprint, arXiv:2503.08200. Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. 2025. A survey on sparse autoencoders: Interpreting the inter- nal mechanisms of large language models. Preprint, arXiv:2503.05613. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179 others. 2024. Gemma 2: Improving open language models at a practical size. Preprint, arXiv:2408.00118. Akiyoshi Tomihari and Issei Sato. 2024. Understanding linear probing then fine-tuning language models from ntk perspective. Preprint, arXiv:2405.16747. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023.Llama: Open and efficient foundation language models.Preprint, arXiv:2302.13971. Yichen Wu, Xudong Pan, Geng Hong, and Min Yang. 2025. Opendeception: Benchmarking and investigat- ing ai deceptive behaviors via open-ended interaction simulation. Preprint, arXiv:2504.13707. Fujie Zhang, Peiqi Yu, Biao Yi, Baolei Zhang, Tong Li, and Zheli Liu. 2025. Prompt-guided internal states for hallucination detection of large language models. Preprint, arXiv:2411.04847. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre-trained transformer language models. Preprint, arXiv:2205.01068. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. Preprint, arXiv:2311.07911. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, and 2 others. 2025. Representation engineering: A top-down approach to ai transparency. Preprint, arXiv:2310.01405. A Dataset Details This appendix documents every corpus used in our experiments, including its provenance, construc- tion protocol, and basic statistics. 1 We partition the resources into Curated Logical-Bench (§A.1) and Open-Domain Fact-Bench (§A.2). The former is further broken down into (i) topic-specific domains with four logical variants and (i) two relational comparison sets. A.1 Curated Logical-Bench Affirmative Statements Bürger et al. (2024) col- lect six topic specific datasets of affirmative state- ments, each on a single topic as detailed in Table 2. Thecitiesandsp_en_transdatasets are from Marks and Tegmark (2024), whileelement_symb, animal_class,inventorsandfactsare subsets of the datasets compiled by Azaria and Mitchell (2023a). All datasets, with the exception of "facts", consist of simple, uncontroversial and unambigu- ous statements. Each dataset (except "facts") fol- lows a consistent template. For example, the tem- plate ofcitiesis "The city of <city name> is in <country name>.", whereas that ofsp_en_trans is "The Spanish word <Spanish word> means <En- glish word>." In contrast, "facts" is more diverse, containing statements of various forms and topics. Negated Statements. Following Bürger et al. (2024), in this paper, each of the statements in the six datasets from Table 2 is negated by inserting the word "not". For instance, "The Spanish word ’dos’ means ’enemy’." (False) turns into "The Spanish word ’dos’ does not mean ’enemy’." (True). This results in six additional datasets of negated state- ments, denoted by the prefix "neg_". Logical Conjunctions. We use the following tem- plate to generate the logical conjunctions from six datasets in Table 2, separately for each topic: • It is the case both that [statement 1] and that [statement 2]. Following the recent work (Bürger et al., 2024), the two statements are sampled independently to be true with probability 1 √ 2 . This ensures that the over- all dataset is balanced between true and false state- ments, but that there is no statistical dependency between the truth of the first and second statement in the conjunction. The new datasets are denoted 1 All CSV files, generation scripts and pre-processed acti- vation matrices will be released upon publication. by the suffix "_conj", e.g.,sp_en_trans_conjor facts_conj. Each dataset contains 500 statements. Examples include: •It is the case both that the city of Al Ain City is in the United Arab Emirates and that the city of Jilin is in China. (True) •It is the case both that Oxygen is necessary for humans to breathe and that the sun revolves around the moon. (False) Logical Disjunctions. The templates for the dis- junctions were adapted to each dataset in Table 2, combining two statements as follows: • cities_disj: It is the case either that the city of [city 1] is in [country 1/2] or that it is in [country 2/1]. • sp_en_trans_disj: It is the case either that the Spanish word [Spanish word 1] means [English word 1/2] or that it means [English word 2/1]. Analogous templates were all used for rest of datasetselement_symb,inventors, and animal_class. Bürger et al. (2024) sample the first statement to be true with a probability of 1/2and then sample a second statement, ensuring the end-word (e.g., [country 2]) would be incor- rect for statement 1. The order of the two end- words is flipped with a probability of1/2. The new datasets are denoted by the suffix "_disj", e.g., sp_en_trans_disj, and each contains 500 state- ments. Examples include: • It is the case either that the city of Korla is in Azerbaijan or that it is in Russia. (False) •It is the case either that the Spanish word ‘carne’ means ‘meat’ or that it means ‘seven’. (True) •It is the case either that Bromine has the sym- bol Ce or that it has the symbol Mo. (False) Combining statements in this simple way is not possible for the more diversefactsdataset and Bürger et al. (2024) use the following template instead: • It is the case either that [statement 1] or that [statement 2]. Table 2: Our datasets D i NameDescriptionRows cities"The city of [city] is in [country]."1496 sp_en_trans"The Spanish word ‘[word]’ means ‘[English word]’."354 element_symb"[element] has the symbol of [symbol]."186 animal_class"The [animal] is a [animal_class]."164 inventors"[inventor] lived in [counrty]."406 factsDiverse scientific facts561 larger_than"x is larger than y."1980 smaller_than"x is smaller than y."1980 common_claim_true_falseVarious claims; from (Azaria and Mitchell, 2023a)4450 counterfact_true_falseVarious factual recall claims; from (Meng et al., 2023)31960 Following Bürger et al. (2024), we sample the two statements independently to be true with prob- ability1− 1 √ 2 . This ensures that the overall dataset is balanced between true and false statements, but that there is no statistical dependency between the truth of the first and second statement in the dis- junction. Examples include: • It is the case either that the Earth is the third planet from the sun or that the Milky Way is a linear galaxy. (True) • It is the case either that the fastest bird in the world is the penguin or that Oxygen is harmful to human breathing. (False) A.2 Open-Domain Fact-Bench common_claim_true_false CommonClaimis in- troduced by Casper et al. (2023), containing 20,000 GPT-3-text-davinci-002 generations which are la- beled as true, false, or neither, according to human common knowledge. Marks and Tegmark (2024) adaptedCommonClaimby selecting statements la- beled true or false, then removing excess true state- ments to balance the dataset. This modified version consists of 4450 statements. Example statements: • Bananas are believed to be one of the oldest fruits in the world. (True) •Crazy ants have taken over Cape Canaveral. (False) counterfact_true_false Counterfactwas introduced in Meng et al. (2023) and consists of factual recall statements. We adapt Counterfact by using statements which form complete sentences and, for each such statement, using both the true version and a false version given by one of Coun- terfact’s suggested false modifications. We also append a period to the end. Example statements: •Olaus Rudbeck spoke the language Swedish. (True) •The official religion of Malacca sultanate is Christianity. (False) B Complete Layer-wise Probing Results This appendix aims to provide a more exhaustive quantitative analysis of the internal representations within the LLMs under investigation. Specifically, Table 3 and Table 4 present the complete accu- racy information from the layer-wise probing con- ducted on LLaMA-3.1-8B-IT and Gemma2-9B- IT. These results span the three distinct instruc- tional conditions—Truthful, Neutral, and Decep- tive prompts—and utilize two different probing methodologies: Logistic Regression (LR) and the Training of Truth and Polarity Direction (TTPD). These tables serve as a supplement to the graphical representations shown in Figures 2 and 3 in the main body of the paper, offering precise numerical values for the average accuracy at each layer, po- tentially including standard deviations as indicated in the original tables. This detailed data allows for a granular understanding of how linearly decod- able the model’s instructed "True"/"False" output is at various depths within the network. By provid- ing these comprehensive figures, researchers can more meticulously examine how different model architectures, instruction types, and probing tech- niques influence the predictability of representa- tions across layers, and further verify the conclu- sions drawn in the main text regarding key layers where significant representational shifts or peak predictability occurs. LayerTruthfulNeutralDeceptive TTPDLRTTPDLRTTPDLR 150.69 ± 1.1049.50 ± 0.0050.43 ± 0.6549.50 ± 0.0050.63 ± 1.0049.50 ± 0.00 250.17 ± 0.3549.56 ± 0.3550.36 ± 0.6649.48 ± 0.1150.16 ± 0.3649.48 ± 0.17 350.07 ± 0.0849.64 ± 0.2750.21 ± 0.1949.79 ± 0.4350.20 ± 0.3649.76 ± 0.45 450.23 ± 0.3150.62 ± 0.8850.49 ± 0.5050.74 ± 0.9650.14 ± 0.2450.11 ± 0.68 5 49.83 ± 0.6351.20 ± 0.6449.88 ± 0.4051.08 ± 0.8349.91 ± 0.5650.95 ± 0.62 649.91 ± 0.5550.92 ± 0.1849.88 ± 0.4750.93 ± 0.2849.87 ± 0.2450.84 ± 0.33 752.38 ± 1.3152.02 ± 0.6451.58 ± 1.4852.63 ± 1.0651.01 ± 1.7052.52 ± 1.12 852.90 ± 1.2652.13 ± 0.6152.04 ± 0.8652.94 ± 1.0252.46 ± 0.7552.12 ± 0.67 9 58.66 ± 1.9653.09 ± 1.6057.93 ± 1.9956.53 ± 3.0557.77 ± 2.4153.88 ± 2.34 10 59.58 ± 4.5761.37 ± 3.6757.23 ± 4.9662.20 ± 4.0158.79 ± 3.4262.05 ± 3.08 1168.41 ± 0.7457.16 ± 3.1164.66 ± 2.4959.57 ± 3.6560.11 ± 3.9657.07 ± 3.38 12 69.69 ± 0.6363.11 ± 3.0969.75 ± 0.8362.33 ± 4.5469.13 ± 0.7358.76 ± 3.67 1377.20 ± 0.2365.37 ± 4.9580.10 ± 0.2462.92 ± 5.6473.43 ± 0.8362.25 ± 4.66 1478.22 ± 0.2375.70 ± 2.8882.25 ± 0.5371.66 ± 4.2179.29 ± 0.3070.44 ± 3.74 15 80.04 ± 0.4775.97 ± 3.4684.86 ± 0.4775.49 ± 3.5380.42 ± 0.1370.07 ± 6.07 1682.58 ± 0.6077.15 ± 4.4385.62 ± 0.1474.91 ± 4.0779.23 ± 0.1171.33 ± 4.35 1783.62 ± 0.4176.97 ± 3.2785.42 ± 0.1574.03 ± 5.7979.80 ± 0.2969.46 ± 6.76 1883.55 ± 0.3574.53 ± 4.7085.01 ± 0.3074.43 ± 4.9276.25 ± 0.3567.11 ± 6.53 19 83.24 ± 0.2273.62 ± 5.4583.75 ± 0.3473.71 ± 4.7078.69 ± 0.1867.65 ± 5.26 2083.07 ± 0.4174.90 ± 4.1883.62 ± 0.2271.85 ± 4.3478.69 ± 0.4866.16 ± 7.26 2182.84 ± 0.2769.61 ± 5.9683.41 ± 0.3676.20 ± 4.8779.50 ± 0.3466.87 ± 6.48 2282.53 ± 0.3571.29 ± 5.1783.61 ± 0.2173.31 ± 5.8079.22 ± 0.2366.44 ± 7.18 23 82.25 ± 0.2973.93 ± 5.9583.39 ± 0.2472.20 ± 4.8979.06 ± 0.3767.68 ± 6.19 2482.11 ± 0.3771.52 ± 5.3683.22 ± 0.2472.48 ± 6.4378.54 ± 0.3368.94 ± 5.61 2582.10 ± 0.2973.07 ± 6.0483.30 ± 0.3372.60 ± 6.1178.24 ± 0.1967.35 ± 6.50 26 81.96 ± 0.2971.62 ± 6.3183.26 ± 0.1973.14 ± 5.0878.33 ± 0.2670.58 ± 4.26 2781.55 ± 0.2871.34 ± 6.1682.97 ± 0.2373.90 ± 5.8377.45 ± 0.5669.06 ± 5.07 2881.42 ± 0.3373.38 ± 5.5682.90 ± 0.2475.66 ± 5.1376.81 ± 0.3068.20 ± 5.64 2981.36 ± 0.1875.02 ± 3.8282.70 ± 0.2573.47 ± 4.7176.43 ± 0.3868.37 ± 6.25 3081.31 ± 0.2270.84 ± 6.2682.87 ± 0.3472.73 ± 4.1476.51 ± 0.6069.63 ± 7.97 31 81.62 ± 0.2371.57 ± 5.6682.89 ± 0.3371.96 ± 5.0776.17 ± 0.7163.63 ± 7.32 3281.57 ± 0.3364.56 ± 4.6683.87 ± 0.1972.36 ± 5.2577.86 ± 1.2368.55 ± 5.39 Table 3: Layer-wise probing accuracy for Llama3.1-8B-IT across truthful, neutral, and deceptive prompts using TTPD and LR. LayerTruthfulNeutralDeceptive TTPDLRTTPDLRTTPDLR 150.47 ± 0.5950.69 ± 0.5151.03 ± 0.9150.75 ± 0.6150.54 ± 0.7450.18 ± 0.47 251.13 ± 0.7350.55 ± 0.6851.45 ± 1.2950.15 ± 0.6650.73 ± 0.7650.41 ± 0.80 350.80 ± 1.1650.51 ± 0.7051.68 ± 0.9950.45 ± 0.6551.07 ± 0.7650.97 ± 0.77 451.24 ± 0.5550.55 ± 0.6951.22 ± 1.0050.49 ± 0.4950.98 ± 0.6749.64 ± 0.61 5 51.04 ± 0.5650.95 ± 0.5150.84 ± 0.7251.14 ± 0.3051.40 ± 0.7151.38 ± 0.60 651.50 ± 0.6250.79 ± 0.3950.75 ± 0.5551.20 ± 0.5851.35 ± 0.7850.88 ± 0.35 748.31 ± 1.2249.89 ± 0.8248.09 ± 0.7549.69 ± 1.1748.17 ± 1.3549.42 ± 0.90 849.17 ± 0.9150.68 ± 1.0949.75 ± 0.8451.09 ± 0.6348.43 ± 0.9950.71 ± 0.67 9 52.31 ± 0.7251.17 ± 0.7152.45 ± 1.0251.84 ± 1.0552.99 ± 0.5051.13 ± 0.62 10 52.74 ± 0.8651.85 ± 0.5851.93 ± 0.6652.31 ± 1.0253.31 ± 0.3651.90 ± 0.54 1152.98 ± 1.0852.99 ± 0.7851.92 ± 0.3353.58 ± 1.6353.37 ± 0.8951.96 ± 0.67 12 51.72 ± 0.3352.40 ± 0.8751.74 ± 0.3253.76 ± 0.8753.00 ± 1.2752.47 ± 0.65 1352.63 ± 0.9453.56 ± 0.9952.11 ± 0.4353.38 ± 1.5252.95 ± 1.2252.31 ± 0.78 1453.74 ± 1.3153.92 ± 1.0954.00 ± 1.9753.80 ± 1.4953.28 ± 1.3052.44 ± 0.52 15 54.80 ± 1.1657.56 ± 2.0354.32 ± 1.8354.40 ± 1.8154.11 ± 0.8155.54 ± 1.31 1657.54 ± 1.8462.03 ± 2.0656.46 ± 2.4657.29 ± 3.3657.61 ± 0.7659.86 ± 1.87 1760.84 ± 1.9865.56 ± 1.8358.78 ± 2.1064.45 ± 2.3060.17 ± 1.4162.68 ± 1.38 1866.32 ± 1.0968.80 ± 1.2864.44 ± 2.0965.77 ± 2.6864.41 ± 1.3663.64 ± 2.78 19 75.17 ± 0.3470.32 ± 5.9977.33 ± 0.9665.93 ± 6.6369.10 ± 0.8669.75 ± 4.53 2078.82 ± 0.4772.61 ± 6.4982.78 ± 0.9570.72 ± 4.3976.47 ± 0.4472.12 ± 6.80 2182.52 ± 0.2072.38 ± 6.3084.72 ± 0.2869.97 ± 6.4684.16 ± 0.5176.32 ± 4.30 2283.85 ± 0.3464.00 ± 6.2176.57 ± 0.9565.25 ± 6.9583.50 ± 0.2671.22 ± 5.84 23 83.63 ± 0.3671.50 ± 7.2077.85 ± 0.9871.69 ± 5.7582.76 ± 0.3369.36 ± 7.19 2485.27 ± 0.1171.62 ± 7.5483.21 ± 0.4773.37 ± 6.9587.13 ± 0.7570.12 ± 7.54 2585.40 ± 0.2373.09 ± 6.1684.82 ± 0.2872.89 ± 6.1084.49 ± 0.7472.56 ± 7.26 26 86.06 ± 0.2075.38 ± 6.0286.14 ± 0.4468.37 ± 6.1685.15 ± 0.6571.84 ± 7.07 2785.77 ± 0.3979.72 ± 4.7484.89 ± 0.3971.21 ± 5.9486.70 ± 0.1874.29 ± 5.96 2885.82 ± 0.1779.99 ± 5.1385.01 ± 0.2876.32 ± 6.0186.39 ± 0.4973.53 ± 7.26 2985.90 ± 0.1380.06 ± 5.6085.26 ± 0.2778.20 ± 5.6683.64 ± 0.4572.23 ± 7.11 3085.58 ± 0.2179.15 ± 5.1584.56 ± 0.2874.26 ± 5.5079.60 ± 0.5174.41 ± 5.91 31 85.37 ± 0.1976.35 ± 6.2684.64 ± 0.2675.32 ± 6.1275.75 ± 0.8075.85 ± 7.14 3285.68 ± 0.2277.95 ± 6.2984.98 ± 0.2377.45 ± 4.9674.61 ± 0.8971.97 ± 7.13 Table 4: Layer-wise probing accuracy for Gemma2-9B-IT across truthful, neutral, and deceptive prompts using TTPD and LR. C PCA Visualization Results This section presents supplementary PCA visualiza- tions to further illustrate the global geometry of the models’ internal activations under different instruc- tional prompts. As discussed in the main text, PCA is employed to project the high-dimensional hid- den state activations (x l ) onto a 2D space, primarily for illustrative purposes. These visualizations help in assessing the separability of internal states cor- responding to "True" and "False" outputs across Neutral, Truthful, and Deceptive conditions. The figures below provide additional examples beyond those in Section 3.2, showcasing these dynamics for both LLaMA-3.1-8B-Instruct and Gemma-2-9B-Instruct on various datasets. Specifi- cally, Figure 10 and Figure 11 demonstrate the PCA results on curated datasets (e.g.,sp_en_transfor LLaMA andcitiesfor Gemma). These typically show a clearer separation between True/False clus- ters as observed in Figure 4 for thecitiesdataset with LLaMA. In contrast, Figure 5 (which may correspond to Figure 5 in the main text showing common_claimandcounterfactfor LLaMA) and Figure 12 (showing similar complex datasets for Gemma, as in Figure 12) illustrate the challenges PCA faces with more complex, uncurated datasets where the True/False clusters often appear entan- gled due to feature superposition. These appendix figures offer a broader visual substantiation of how the geometric separability of truth-related represen- tations can vary significantly with dataset complex- ity and model type. Layer 8 NeutralTruthfulDeceptive Layer 16 Layer 32 FalseTrue Figure 10: Layer-wise PCA visualization for LLaMA- 3.1-8B-Instruct across Neutral, Truthful, and Deceptive Prompts on sp_en_trans Layer 8 NeutralTruthfulDeceptive Layer 16 Layer 32 FalseTrue Figure 11: Layer-wise PCA visualization for Gemma-2- 9B-Instruct on cities. Figure 12: Layer-wise PCA visualization (Compo- nent 1 vs. Component 2) for Gemma-2-9B-Instruct on two complex datasets:common_claim_true_false (top row) andcounterfact_true_false(bottom row). Columns represent different instructional conditions: Neutral, Truthful, and Deceptive prompts. Visualiza- tions are performed on Layer 16, the key layer identified for Gemma-2-9B-Instruct based on the probing accu- racy peaks in Figure 3. D SAE-based Layer-wise Feature Shift Analysis We analyze how sparse feature activations shift across layers under different instruc- tion types using three metrics in SAE la- tent space:Cosine Similarity, Overlap Ra- tio, and L2 Distance.Figures 13 and 14 show results for thecommon_claim_true_false andcounterfact_true_falsedatasets using Gemma-2-9B-Instruct. In both datasets, deceptive prompts induce strong mid-to-late layer shifts, especially between Layers 16 and 32. This is evidenced by the sharp rise in L2 distance and the corresponding drop in cosine similarity and overlap ratio when comparing truthful and deceptive inputs. The effect is most pronounced incounterfact_true_false, where overlap sharply declines post-Layer 16, indicating a reconfiguration of sparse feature sets. In contrast, shifts between truthful and neutral prompts remain small and gradual across all layers, suggesting that the major representational changes are deception- specific. These results highlight a distinctive geomet- ric transformation in the model’s latent represen- tations under deceptive instructions and further motivate mid-layer analysis when identifying po- tential deception-sensitive features. This pattern closely aligns with earlier findings from linear prob- ing, where intermediate layers—especially around Layer 16—also showed peak decodability of the 0.2 0.4 0.6 0.8 1.0 Truthful vs Deceptive 051015202530 Layer 0.2 0.4 0.6 0.8 1.0 Truthful vs Neutral 0 50 100 150 200 0 50 100 150 200 Cosine Similarity / Overlap Ratio L2 Distance Cosine SimilarityOverlap RatioL2 Distance Figure 13: Layer-wise feature shift analysis for Gemma- 2-9B-Instruct on common_claim_true_false. model’s intended "True"/"False" output across in- struction types. 0.2 0.4 0.6 0.8 1.0 Truthful vs Deceptive 051015202530 Layer 0.2 0.4 0.6 0.8 1.0 Truthful vs Neutral 0 50 100 150 200 0 50 100 150 200 Cosine Similarity / Overlap Ratio L2 Distance Cosine SimilarityOverlap RatioL2 Distance Figure 14: Layer-wise feature shift analysis for Gemma- 2-9B-Instruct on counterfact_true_false. E SAE-based Neuron-wise Feature Shift Analysis This appendix section provides additional visual- izations to support the neuron-wise feature shift analysis detailed in main text. The core objective of this analysis is to move beyond global represen- tational shifts and pinpoint specific SAE features that are most sensitive to the change from truthful to deceptive instructions. By examining individ- ual SAE feature activations, we can gain a better understanding of how deception is encoded at the feature level. The methodology involves identifying, for each layer, the sparse SAE features exhibiting the largest change in average activation when compar- ing the Deceptive condition to the Truthful con- dition. The figures presented in this appendix, such as scatter plots showing the activation of the top distinguishing features (similar to Figure 7 forcommon_claim_true_falsebut potentially for other datasets likecounterfact_true_falseas shown in Figure 15 and violin plots in Figure 16 illustrating the distribution of these feature activa- tions under truthful versus deceptive prompts, offer further evidence. These supplementary visualizations help to re- inforce the finding that a small subset of sparse features often displays a near-binary activation pat- tern—being highly active for one instruction type (e.g., truthful) and suppressed for the other (e.g., deceptive), or vice-versa. This detailed view cor- roborates the idea that these specific features act as "deception-associated features," playing a crit- ical role in modulating the model’s internal rep- resentation in response to deceptive instructions, often aligning with the mid- and late-layer SAE shift peaks identified globally in Figure 6 and Fig- ure 7. The plots here may cover additional layers or datasets, providing a more comprehensive picture of this phenomenon. 0.00.51.01.52.02.53.03.5 Top 1 Feature Activation 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 Top 2 Feature Activation Layer 8 Truthful Deceptive 0123456 Top 1 Feature Activation Layer 16 5.07.510.012.515.017.520.0 Top 1 Feature Activation Layer 32 Figure 15: Neuron-by-neuron feature shift analysis for LLaMA-3.1-8B-Instruct on counterfact_true_false TruthfulDeceptive Condition 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 Activation TruthfulDeceptive Condition 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Activation (a) Layer 8 TruthfulDeceptive Condition 0 1 2 3 4 5 Activation TruthfulDeceptive Condition 0 1 2 3 4 5 6 Activation (b) Layer 16 TruthfulDeceptive Condition 0 2 4 6 8 10 Activation TruthfulDeceptive Condition 4 6 8 10 12 14 16 18 20 Activation (c) Layer 32 Figure 16: Violin graph of LLaMA-3.1-8B-Instruct activations on thecounterfact_true_false. The top row displays the activation distributions for the SAE feature most responsive to deceptive instructions (Top 1 feature), while the bottom row shows the distributions for the second most responsive feature (Top 2 feature), across layers 8 (a), 16 (b), and 32 (c).

Instant research discovery