Paper deep dive
Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models
Itay Yona, Dan Barzilay, Michael Karasik, Mor Geva
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/2/2026, 11:50:27 PM
Summary
The paper investigates 'entity cells'âsparse, entity-selective MLP neurons in language modelsâthat act as stable anchors for factual retrieval. Using a localization-and-intervention pipeline on 200 entities from PopQA, the authors demonstrate that these neurons, concentrated in early layers, are causally actionable: suppressing them induces entity-specific amnesia, while activating them can restore entity-consistent factual recall. These cells exhibit robustness to aliases, acronyms, and multilingual forms, suggesting they serve as canonical identity representations.
Entities (4)
Relation Signals (3)
Negative Ablation â causes â Entity-specific amnesia
confidence 95% · Negative ablation produces entity-specific amnesia
Qwen2.5-7B â contains â Entity Cell
confidence 95% · We identify sparse, entity-selective MLP neurons, termed entity cells, that act as stable anchors for factual retrieval in Qwen2.5-7B.
Entity Cell â locatedin â Early Layers
confidence 95% · localized neurons concentrate in early layers.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Language models can answer many entity-centric factual questions, but it remains unclear which internal mechanisms are involved in this process. We study this question across multiple language models. We localize entity-selective MLP neurons using templated prompts about each entity, and then validate them with causal interventions on PopQA-based QA examples. On a curated set of 200 entities drawn from PopQA, localized neurons concentrate in early layers. Negative ablation produces entity-specific amnesia, while controlled injection at a placeholder token improves answer retrieval relative to mean-entity and wrong-cell controls. For many entities, activating a single localized neuron is sufficient to recover entity-consistent predictions once the context is initialized, consistent with compact entity retrieval rather than purely gradual enrichment across depth. Robustness to aliases, acronyms, misspellings, and multilingual forms supports a canonicalization interpretation. The effect is strong but not universal: not every entity admits a reliable single-neuron handle, and coverage is higher for popular entities. Overall, these results identify sparse, causally actionable access points for analyzing and modulating entity-conditioned factual behavior.
Tags
Links
- Source: https://arxiv.org/abs/2604.01404v1
- Canonical: https://arxiv.org/abs/2604.01404v1
Trouble viewing inline? Open PDF directly â
Full Text
58,264 characters extracted from source content.
Expand or collapse full text
Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models Itay Yona 1 , Dan Barzilay 2 , Michael Karasik 2 , Mor Geva 3 1 Mentaleap, 2 Indepdent Researcher, 3 Tel Aviv University Correspondence: itay@mentaleap.ai Abstract Language models can answer many entity- centric factual questions, but it remains unclear which internal mechanisms are involved in this process. We study this question across multiple language models. We localize entity-selective MLP neurons using templated prompts about each entity, and then validate them with causal interventions on PopQA-based QA examples. On a curated set of 200 entities drawn from PopQA, localized neurons concentrate in early layers.Negative ablation produces entity- specific amnesia, while controlled injection at a placeholder token improves answer re- trieval relative to mean-entity and wrong-cell controls. For many entities, activating a sin- gle localized neuron is sufficient to recover entity-consistent predictions once the context is initialized, consistent with compact entity retrieval rather than purely gradual enrichment across depth. Robustness to aliases, acronyms, misspellings, and multilingual forms supports a canonicalization interpretation. The effect is strong but not universal: not every entity ad- mits a reliable single-neuron handle, and cov- erage is higher for popular entities. Overall, these results identify sparse, causally action- able access points for analyzing and modulat- ing entity-conditioned factual behavior. 1 Introduction Understanding how language models recall factual knowledge from their parameters is a core problem in mechanistic interpretability (Dai et al., 2022; Meng et al., 2022; Geva et al., 2023; Nanda et al., 2023, inter alia). Many factual queries are entity- centric: the model must resolve a named subject (e.g.,ParisorBarack Obama) and then retrieve attributes about that subject. A recurring obser- vation is that this entity processing begins early in the forward pass of the model, where token- level surface forms are transformed into semantic representations (Feucht et al., 2024; Kaplan et al., 2024). What is still unresolved is how and where factual access is anchored at inference time: does the model build entity meaning gradually across many layers, or does it retrieve a compact entity representation through localized access points? By analogy to the âgrandmother cellâ hypothesis in neuroscience, we refer to sparse, entity-selective MLP neurons as entity cells. The grandmother- cell hypothesis in neuroscience is a longstanding proposal in neuroscience, central to debates about whether individual neurons can serve as meaning- ful functional units in the representation of com- plex concepts (Connor, 2005; Quiroga et al., 2008). In our usage, an MLP neuron is a pair of vectors within an MLP block: one vector detects a pattern in the input residual stream, and the other writes a corresponding output back to that stream (Geva et al., 2021). Concretely, these are the matching column ofW in and row ofW out . We use entity representation for the output written to the resid- ual stream, or for the resulting hidden-state pattern associated with the entity. An entity cell is there- fore a neuron whose detector responds to inputs about a given entity and whose output write an entity-consistent representation. We investigate the existence of entity cells in LLMs using a neuron-level localization-and- intervention pipeline. Our method explicitly tests the hypothesis that an entity has a highly stable MLP neuron (identified by layerâand neuron in- dexj) at the entity mention position, across tem- plated prompts about that entity. For example, for the entityDonald Trump, we use prompts such as âThe origin of Donald Trumpâ, âThe role of Don- ald Trumpâ, and âThe location of Donald Trumpâ, then record all MLP neuron activations at the final token of the entity span. We rank neurons by cross- prompt stability and take the top-ranked neuron as a candidate entity cell. We apply this procedure on 200 popular enti- ties in PopQA-200, a curated subset of the PopQA 1 arXiv:2604.01404v1 [cs.CL] 1 Apr 2026 Figure 1: We identify sparse, entity-selective MLP neurons, termed entity cells, that act as stable anchors for factual retrieval in Qwen2.5-7B. Concentrated primarily in early layers (0â5), these cells provide access to canonical identity representations that are robust to aliases, misspellings, and multilingual variants. These neurons serve as causally actionable access points: suppressing them induces entity-specific amnesia, while activating a single localized neuron is often sufficient to steer the model toward entity-consistent factual recall. Across the other six models in our suite, early-layer candidates also appear, though the causal validation is weaker. dataset (Mallen et al., 2022) which serves both as the entity inventory for localization and as the source of downstream QA instances for causal eval- uation. Across 7 models from 5 different fami- lies, we consistently observe entity-cell candidates in early layers. These models include the Qwen family (Qwen2.5-7B base, Qwen2.5-7B-Instruct, and Qwen3-8B base), OLMo-7B, Llama-3.1-8B, Mistral-7B, and OpenLLaMA-7B. Next, given these localized candidates, we ask two questions: (1) does suppressing these cells impair recall about their matched entities, and (2) whether activating the cells is sufficient to restore entity knowledge in controlled settings. We ob- serve the strongest trends in Qwen2.5-7B base and weaker in other model families. In Qwen2.5, neg- ative ablation, which scales a localized cell by a negative factor, induces entity-specific amnesia: the model becomes markedly less able to retrieve facts about the target entity, while remaining able to continue the prompt fluently and leaving con- trol entities near baseline. Moreover, controlled injection at a placeholder token can restore access to facts about the matched entity relative to mean- entity and wrong-cell controls; for many entities, a single neuron is enough and top-kadds only marginal improvements. The same localized cells also remain stable across aliases, acronyms, typos, and multilingual forms, suggesting that they pro- vide access to entity identity across surface forms rather than a single token string. Together, our results support a localized retrieval picture (Figure 1): for a substantial subset of en- tities, factual access is mediated by sparse early- layer neurons that can be causally manipulated, broadly consistent with the grandmother-cell hy- pothesis. We do not claim this is a full picture: reliable single-neuron effects are common but not universal, and the clearest trends are observed for popular entities. Our work thus makes the follow- ing contributions: 1.We test the grandmother-cell hypothesis in lan- guage models using a stability-based method that localizes entity-sensitive neurons from tem- plated prompts about the same entity. 2.Applying this method across multiple models, we find the clearest and most consistent entity- cell localization and causal effects in Qwen2.5- 7B base, with weaker and less consistent signals in the other tested models. 3.We validate these localized neurons causally via negative ablation and controlled injection, including mostly single-neuron sufficiency rela- tive to mean-entity and wrong-cell controls. 4.We characterize key properties of localized neurons, including robustness across aliases, acronyms, misspellings, and multilingual vari- ants. We release our code, prompts, and data at https://github.com/1tux/in-silico/. 2 2 Related Work Factual recall and localization Prior work has localized factual behavior to specific components and layers in transformers, including neuron-level interventions and causal tracing (Dai et al., 2022; Geva et al., 2023; Nanda et al., 2023). Compared with Dai et al. (Dai et al., 2022), which local- ize fact-specific neurons using paraphrases of the same fact, we localize entity-centered neurons us- ing prompts that vary attributes of the same entity. These studies demonstrate that targeted internal changes can modulate factual outputs, but typi- cally focus on relation-specific recall pathways. Our work is complementary: we localize entity- centered neurons across varied relations and then test whether those neurons act as reusable access points. Detokenization and entity formation Several studies show that early layers consolidate subword forms into coherent lexical or semantic represen- tations (Elhage et al., 2022; Feucht et al., 2024; Gurnee et al., 2023; Kaplan et al., 2024). We build on this line by testing whether the same localized neurons are preserved across aliases, acronyms, misspellings, and multilingual forms, linking ro- bustness to a canonical entity representation. MLP memories and sparse features MLP blocks have been interpreted as key-value mem- ory mechanisms that can store and retrieve asso- ciations (Geva et al., 2021; Dar et al., 2023). Our findings are consistent with this view but sharpen it operationally: in many cases, a sparse neuron- level handle is enough to recover entity-consistent behavior under controlled intervention. Editing and control Model editing methods such as ROME and MEMIT rewrite factual be- havior at parameter level (Meng et al., 2022, 2023). We instead use reversible activation interventions. This isolates retrieval-time causal effects and helps separate entity access from persistent weight edit- ing. Relative to prior work, our main contribution is a causal account of where entity-level factual access is taken from at inference time: often from sparse, early-layer neurons that behave like com- pact entity access points, though not for every en- tity and not necessarily as the only mechanism. We now define a localization score and the inter- ventions used to test whether a localized neuron is merely correlational or provides causal leverage. 3 Method In this section, we define the activation extraction, normalization, stability ranking, and intervention protocols used to localize and causally test entity cells. Activation point Letxbe a prompt contain- ing an entity mention and lett(x)denote the en- tity token position. For each transformer layerâ and MLP neuron indexj, we extract the down- projection activation of neuronjatt(x), denoted a âj (x). In the terminology above, the neuron is the channel-specific input-output mechanism at layerâ, whilea âj (x)is the scalar coefficient with which its write is applied on promptx. Concretely,a âj (x) is the scalar channel value just before the MLP down-projection (down_proj) at the chosen token position. Normalization MLP activations vary widely across layers and neurons. LetÎŒ âj andÏ âj denote the mean and standard deviation ofa âj (x)over generic promptsB. We standardize activations as: z âj (x) = a âj (x)â ÎŒ âj Ï âj + Δ .(1) Unless stated otherwise we use Δ = 10 â6 . Stability score and ranking Given a set ofK promptsx i K i=1 that all reference the same entity, we define a stability score: S âj = (E i [z âj (x i )]) 2 Std i [z âj (x i )] + Δ .(2) We rank all(â,j)pairs byS âj and select the top neuron as the entity cell candidate for that entity. The score favors neurons that activate strongly and consistently across entity-centered prompts. Intuition Up toΔ,S âj = |E[z]|/CV(z), where CV(z) = Std(z)/|E[z]|is the coefficient of vari- ation across prompts. This makes the ranking an importance-scaled stability criterion: high mean activation is rewarded, while high relative variabil- ity is penalized. Interventions We employ two causal interven- tions on a localized cell: controlled injection and negative ablation. Injection. We directly set the activation of a chosen cell at a chosen token position: a â â j â (x)[t(x)]â v,(3) 3 withvset to an entity-specific value estimated from the entity-present prompts in Finding 3. In con- trolled injection, this overwrite is applied on top of a mean-entity initialization, so the intervention probes directional movement on an existing entity manifold rather than de novo reconstruction from a single neuron. We use âwrong cellâ controls by injecting a cell localized to a different entity. Negative ablation. We multiply a chosen cellâs activation by a scalar α: a â â j â (x)â αa â â j â (x),(4) includingα < 0, which flips the sign of the activa- tion. In our implementation we apply this scaling across token positions; the effect is driven primar- ily by positions where the cell would otherwise activate. Evaluation metrics Several experiments use next-token probabilities. Given a set of answer aliasesA, we define the answer score as the proba- bility of the first token of the best-matching alias: p ans (x) = max aâA p tok 1 (a)|x .(5) We use this first-token score primarily as a filtering signal when defining trustworthy localized cells for controlled injection. In particular, the trust fil- ter compares the target entity against no-injection and wrong-cell controls using a normalized score RelProb, defined as the meanp ans under a condi- tion divided by the mean under the corresponding entity-present prompt (so 1.0 indicates parity). For injection experiments we report pass@k: whether any correct answer first-token ID appears in the top-knext-token distribution (we usek=5unless stated otherwise). This is computed directly from the next-token logits (top-kmembership), without sampling. For entity-specific amnesia tests (main Finding 2) we define a normalized score based on log-probabilities, anchored by an unknown-entity baseline computed by swapping the entity name for a small set of unseen names and averaging the resulting answer log-probabilities. 4 Experimental Setup Models We run localization and causal checks on Qwen2.5-7B base and Qwen2.5-7B-Instruct (Yang et al., 2025b), Qwen3-8B base (Yang et al., 2025a), OLMo-7B-0724-hf (Groeneveld et al., 2024), Llama-3.1-8B-Instruct (Grattafiori et al., 2024), Mistral-7B-v0.3 (Jiang et al., 2023), and OpenLLaMA-7B (Geng and Liu, 2023). Section 5 focuses on Qwen2.5-7B base, with cross-model comparisons reported in Appendices F and G. Un- less stated otherwise, we run inference in half pre- cision with automatic device mapping. Data We use PopQA (Mallen et al., 2022), an entity-centric QA dataset derived from Wikidata with subject entities and answer aliases. We build a curated set ofN = 200popular enti- ties by seeding countries, cities, and widely known people, then filling from PopQA by popularity with a minimum of two available questions per entity. We denote this subset as PopQA-200. PopQA-200 serves as the entity inventory for localization and as the source of downstream QA examples for causal evaluation. For PopQA-based causal checks we use K = 2 questions per entity. When a question does not contain a recover- able entity span after tokenization, we skip it for position-dependent analyses. Prompting For localization, we use templated prompts about each entity; examples are listed in Appendix A. The entity token position is defined as the final token in the tokenized entity span. For PopQA-based evaluation, we format each question as: âQuestion: <question> :â For generic probing prompts (used in baselines and controlled interventions), we use cloze-style com- pletions of the form âFact: ...â (Appendix A). For each localization or evaluation prompt, we locate the entity token position as the final token in the tokenized subject-entity span. Interventions that target the entity position act at this index; cloze-style prompts define an analogous entity po- sition at the dummy placeholder token X. Normalization statistics To normalize activa- tions across layers and neurons, we compute base- line statistics(ÎŒ âj ,Ï âj )using 399 generic prompts (Appendix A), extracting activations at the final token position of each prompt. Baseline prompts are deliberately not entity specific. Implementation and compute We trace acti- vations and apply in-graph interventions using NNsight (NDIF Team, 2024), a tracing library that exposes intermediate activations at inference time. All experiments were executed on a single GPU (NVIDIA A100). 4 0510152025 Layer (Top Neuron) 0 20 40 60 Entity Count Top-Neuron Layer Distribution Figure 2: Layer of the top localized cell for each PopQA-200 entity in Qwen2.5-7B base (n=200). Sim- ilar early-layer concentration is observed across other tested models; see Appendix G. 5 Results We report four results that progressively strengthen evidence from correlational localization to causal leverage. All analyses were run on the full suite of seven models described in Section 4. Early-layer concentration (Finding 1) is a recurring pattern across model families; similar trends appear in Qwen2.5-7B-Instruct, Qwen3-8B, and to a lesser extent in the other models tested (Appendices FâG). The main text focuses on Qwen2.5-7B base, which yields the strongest and most consistent causal evi- dence across all four findings; cross-model results are summarized in the appendix. We first map where sparse entity cells appear (Finding 1), then test whether suppressing and activating a localized neuron affects entity-specific recall (Findings 2â 3). Finally, we use surface-form perturbations as an interpretive check on what information these cells provide access to (Finding 4). Unless noted otherwise, localization uses the PopQA-200 entity set together with the templated prompts described in Section 4. The full 200-entity cell map with trustworthiness flags is provided in Appendix D. 5.1 Localizing Entity Cells Finding 1 Entity cells concentrate in early layers (0â5), without being imposed by the architecture For each entity, we rank all MLP neurons (in- dexed by layerâand neuronj) by stability at the mention position acrossKprompts and record the layer of the top-ranked cell. Localization is strongly non-uniform (Figure 2): 99.0% of entities peak in layers 0â5, and only 1.0% peak in layers 22 or 27. Since ranking is over all 28 layers, this depth profile is empirical rather than enforced, and is consistent with early canonicalization features that help form an entity identity representation. Early-layer concentration is suggestive, but does not establish that a localized neuron matters for factual extraction. We next test whether suppress- ing a candidate entity cell selectively impairs recall about that entity. 5.2 Causal Necessity Finding 2 Negatively ablating a localized cell selectively suppresses recall for the target entity while leaving control entities near baseline.At scale, 131/200 localized cells show this entity- specific effect We apply negative ablation (a signed multiplier) to localized cells and measure entity-specific re- call while checking for pathological collapse. In a case study, target retention drops from 1.0 to 0.123 atα = â3, while a control entity (Trump) stays near baseline (1.0 to 0.996; Figure 3). This behavior is consistent with the localized neuron being part of the access path to many entity-linked facts: the model still processes the prompt, but loses the identity representation needed for reli- able recall. We then run the same criterion at scale and use it to define a trustworthy localized cell: a neuron whose suppression produces substantial entity-specific loss without destabilizing the model. Under these checks, 131/200 localized cells are marked trustworthy and define the subset used for controlled injection. Having used negative ablation to establish ne- cessity and to filter trustworthy neurons, we next test whether activating a single cell is sufficient to steer output in a controlled placeholder setting. 5.3 Causal Sufficiency Finding 3 Correct-cell injection recovers entity-specific recall (63.3% pass@5) against near-zero con- trols, with a single cell being sufficient On the trustworthy subset from Finding 2 (Appendix D), entity-present pass@5 is 109/262 (41.6%) across evaluated question instances. To isolate intervention effects from base-model 5 32101 Multiplier for Neuron 10941 (Layer 2) 0.0 0.2 0.4 0.6 0.8 1.0 Relative Knowledge Score Entity-Specific Amnesia via Neuron Ablation Target (Obama) Control (Trump) Original Model Unknown Entity Baseline Figure 3: Entity-specific amnesia under negative abla- tion for the localized Obama cell (L2-N10941). Target (Obama) recall drops substantially asαdecreases, while control (Trump) remains near baseline. misses, we report injection on the 109 instances where the entity-present prompt is already correct under pass@5. We replace the entity mention with Xand intervene at the placeholder token. Mean- entity initialization and wrong-cell injection are used as controls. On this known-answer subset, pass@5 is 1.8% for mean-entity control, 63.3% for correct-cell injection, and 1.8% for wrong- cell injection (Figure 4). Single-cell injection re- mains largely sufficient: 41/79 entities pass with top-1 versus 42/79 with top-k; only one entity re- quires multi-cell injection. We selectαper en- tity from a small grid, which improves sensitivity but can be optimistic relative to a fixed-αproto- col. ForQuestion: Who is the spouse of X? :, we set the hidden vector at X to the mean-entity vector, then activate the Obama cell at that same token position. The causal results above establish necessity (ablation) and sufficiency (injection) for entity- specific recall in this protocol. We now ask what in- formation the localized neuron provides access to, by testing stablity under surface-form variations. 5.4 Surface-Form Robustness Finding 4 The same cell is recovered across spelling vari- ants, acronyms, and multilingual forms, sug- gesting access to a canonical identity represen- tation We re-run localization on the same prompt tem- plates while perturbing the entity string.We test spelling/phrasing variants (Barack Obama), Entity PresentMean EntityCorrect CellWrong Cell 0.0 0.2 0.4 0.6 0.8 1.0 Pass@5 Accuracy Entity Injection Figure 4: Controlled injection at the placeholder to- kenX, evaluated on instances where the entity-present prompt is already correct under pass@5 (109 examples). Mean-entity initialization and wrong-cell injection are control conditions; correct-cell injection shows the ex- pected directional gain. acronym variants (FBI), and multilingual variants (Paris). Most spelling and phrasing variants of âBarack Obamaâ preserve the same top cell (L2- N10941) in Figure 5. We observe similar robust- ness for acronym and multilingual surface forms (Figures 6 and 7), consistent with an identity- canonicalization role rather than dependence on a single token sequence. 6 Discussion Thecombinedevidencesupportsa canonicalization-and-controlviewofentity cells in Qwen2.5-7B base.Negative ablation induces entity-specific amnesia and provides a practical trust filter at scale (131/200), indicating that localized neurons are functionally necessary for entity-specific recall in this protocol.On the known-answer subset of Finding 3 (109/262 instances), controlled injection is strongly direc- tional and mostly single-cell sufficient (41/79 with top-1 vs. 42/79 with top-k), providing a complementary sufficiency test. Robustness to typos, acronyms, and multilingual forms suggests that these neurons provide access to identity-level information rather than a single token string, and the early-layer concentration is consistent with a role in forming an entity identity representation used for downstream factual extraction. Taken together, the cells behave like a latent entity vocabulary: sparse anchor neurons that point computation toward an entity-consistent state and thereby gate access to distributed factual circuits. 6 0.00.51.0 Rel. Stability L2-10941 L0-3664 L3-4828 L1-12564 L1-11948 L1-9991 1.00 0.19 0.04 0.04 0.03 0.02 Barack Obama 0.00.51.0 Rel. Stability L2-10941 L2-15783 L2-793 L2-4253 L2-4989 L2-16593 1.00 0.26 0.22 0.16 0.13 0.11 Obaama 0.00.51.0 Rel. Stability L2-10941 L2-5398 L2-40 L2-17763 L2-11024 L2-11314 1.00 0.42 0.33 0.28 0.16 0.15 Brock Obma 0.00.51.0 Rel. Stability L2-10941 L1-12495 L2-303 L2-17179 L2-5780 L2-6779 1.00 0.66 0.53 0.46 0.31 0.29 Bark Obamna Figure 5: Variant robustness for âBarack Obamaâ: most spelling and phrasing perturbations keep the same lo- calized cell (L2-N10941). 0.00.51.0 Rel. Stability L2-11955 L0-7307 L6-6847 L0-2329 L2-4759 L2-13144 1.00 0.12 0.11 0.07 0.07 0.07 Federal Bureau of Investigation 0.00.51.0 Rel. Stability L2-11955 L2-2290 L3-16252 L2-8009 L2-10222 L2-7426 1.00 0.09 0.08 0.06 0.06 0.06 FBI Figure 6: Acronym robustness (FBI): variants localize to the same top-ranked cell (L2-N11955). The appendix broadens the scope of the main result by testing the same pipeline across mul- tiple models. The strongest extension is post- training robustness: Qwen2.5-7B-Instruct pre- serves nearly the same entity-cell map as the base model. Qwen3-8B also exhibits sparse early-layer entity cells under the same localization procedure, although the causal evidence is weaker. Across other model families, candidate cells can often still be localized, but trustworthy causal effects and form robustness are much less consistent. Taken together, this suggests that entity cells are a repro- ducible but model-dependent phenomenon. 7 Limitations and Scope This study focuses on one dataset (PopQA), with the strongest and most complete evidence in 0.00.51.0 Rel. Stability L1-231 L23-7262 L2-17407 L0-16056 L4-13417 L1-10720 1.00 0.02 0.02 0.02 0.02 0.01 Paris (Latin) 0.00.51.0 Rel. Stability L1-231 L1-11963 L2-11166 L0-10841 L1-15669 L2-16943 1.00 0.16 0.15 0.12 0.11 0.11 Paris (Hebrew) 0.00.51.0 Rel. Stability L1-231 L0-15733 L0-15814 L2-483 L2-12357 L2-17645 1.00 0.20 0.07 0.04 0.03 0.03 Paris (Chinese) 0.00.51.0 Rel. Stability L1-231 L2-13312 L2-13876 L1-7842 L2-8385 L0-15628 1.00 0.12 0.11 0.09 0.07 0.07 Paris (Arabic) Figure 7: Multilingual robustness (Paris): variants lo- calize to the same top-ranked cell (L1-N231). Qwen2.5-7B base. This potentially could be ex- plained by pretraining-data composition: Qwen documentation reports strongest capability in En- glish and Chinese, with broader multilingual per- formance depending on available data coverage (Qwen Team, 2023, 2024). If so, mechanism visi- bility may be data-distribution-dependent, so gener- alization to other model families should be treated as an empirical question and tested with like-for- like replications. Our localization score is intentionally sparse: it ranks individual neurons first, and in Finding 3 top- kvariants provided only marginal gains over top-1. This design prioritizes interpretability but may still miss distributed or multi-cell codes (Shafran et al., 2025). We useK=2prompts per entity for lo- calization and causal checks, which can introduce per-entity instability. Our metrics are mostly first-token based, which can understate multi-token factual competence and can reflect lexical priming effects. In Finding 3,α is selected per entity from a sweep; this improves sensitivity but can introduce optimistic bias, and a fixed-αprotocol is an important next step. Our injection and ablation experiments are still nar- row in relation coverage. We also include an ex- ploratory factual modification procedure via latent steering: optimizing a small perturbation injected at an entity-associated activation site to rewrite a specific relation while preserving unrelated facts; 7 Appendix C provides a concrete template (Algo- rithm 3) and prompt set. 8 Conclusion We test the âgrandmother cellâ hypothesis from neuroscience across multiple language model fam- ilies. In Qwen2.5-7B base, we find sparse, sta- ble, and causally actionable entity cells: they con- centrate in early layers, negative ablation induces entity-specific amnesia, and controlled injection is mostly single-neuron sufficient on the known- answer subset. Robustness to surface-form vari- ation supports the view that these cells provide access to identity-level information. The appendix shows that the phenomenon extends with different strength across additional models. The clearest extension is post-training robustness in Qwen2.5- 7B-Instruct, while Qwen3-8B also exhibits sparse early-layer entity cells under the same pipeline. Across other model families, the signal is weaker and less consistent, suggesting that entity cells are reproducible but model-dependent access points for factual retrieval. References Charles E Connor. 2005. Friends and grandmothers. Nature, 435(7045):1036â1037. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2022. Knowledge neurons in pretrained trans- formers. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (ACL). Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. 2023. Analyzing transformers in embedding space. Preprint, arXiv:2209.02535. Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, Andy Jones, Dawn Drain, Anna Chen, Yun- tao Bai, Deep Ganguli, Liane Lovitt, and 14 others. 2022. Softmax linear units. Transformer Circuits Thread. Sheridan Feucht, David Atkinson, Byron Wallace, and David Bau. 2024. Token erasure as a footprint of implicit vocabulary items in llms. arXiv preprint arXiv:2406.20086. Xinyang Geng and Hao Liu. 2023. Openllama: An open reproduction of llama. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual asso- ciations in auto-regressive language models. arXiv preprint arXiv:2304.14767. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP). Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, and 1 others. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khy- athi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, and 24 others. 2024. OLMo: Accelerating the science of language mod- els. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 15789â15809, Bangkok, Thailand. Association for Computational Linguistics. Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. Preprint, arXiv:2305.01610. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, LĂ©lio Re- nard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Tim- othĂ©e Lacroix, and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825. Guy Kaplan, Matanel Oren, Yuval Reif, and Roy Schwartz. 2024. From tokens to words: On the inner lexicon of llms. arXiv preprint arXiv:2410.05864. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not to trust language models: Investigating effectiveness and limitations of parametric and non- parametric memories. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associ- ations in GPT. In Advances in Neural Information Processing Systems (NeurIPS). Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2023. Mass- editing memory in a transformer. In International Conference on Learning Representations (ICLR). Neel Nanda, Senthooran Rajamanoharan, Janos Kramar, and Rohin Shah. 2023. Fact finding: Attempting to reverse-engineer factual recall on the neuron level. In Alignment Forum. NDIF Team. 2024.Nnsight: Library for inter- preting language models.https://github.com/ ndif-team/nnsight. Accessed: 2026-02-27. 8 R. Quian Quiroga, Gabriel Kreiman, Christof Koch, and Itzhak Fried. 2008. Sparse but not âgrandmother- cellâ coding in the medial temporal lobe. Trends in Cognitive Sciences, 12(3):87â91. Qwen Team. 2023. Introducing qwen. QwenLM Blog. Accessed 2026-02-28. Qwen Team. 2024. Qwen2: Better than ever. QwenLM Blog. Accessed 2026-02-28. Or Shafran, Atticus Geiger, and Mor Geva. 2025. De- composing mlp activations into interpretable features via semi-nonnegative matrix factorization. arXiv preprint arXiv:2506.10920. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Day- iheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. Qwen3 technical report. Preprint, arXiv:2505.09388. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jian- hong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 oth- ers. 2025b. Qwen2.5 technical report. Preprint, arXiv:2412.15115. 9 A Prompt Templates and Hyperparameters Table 1 lists the prompt templates used across ex- periments. Baseline prompts consist of 399 generic cloze-style statements (e.g., âThe Eiffel Tower is located inâ), used only to estimate(ÎŒ,Ï)for nor- malization. UseTemplate PopQA QA wrapper Question: <q> : Generic baselines <statement fragment> Localization probes The <attribute> of <entity> Injection/ablation Fact: the <relation> of <entity>: Factual modification The spouse of <entity> is named Table 1: Prompt templates used in this work. Localization templatesEntity localization uses templated prompts of the formThe <attribute> of <entity>. The full template list used in our runs contains 100 attributes, including: origin, purpose, definition, function, main goal, age, name, founder, owner, value, importance, reputation, impact, influ- ence, location, history, status, category, type, meaning, sig- nificance, role, date of creation, latest update, duration, size, popularity, main activity, scope, reach, composition, structure, method, strategy, goal, objective, result, effect, outcome, cause, reason, source, destination, trend, main challenge, opinion, leading opinion, common perception, definition in law, ethi- cal standing, main criticism, key advantage, key disadvantage, limitation, potential, likelihood, probability, risk, opportunity, threat, strength, weakness, main competitor, main supporter, main opponent, relationship with others, relevance, timing, frequency, pattern, cost, budget, revenue, profit, loss, market share, demographic, representation, policy, regulation, require- ment, recommendation, limiting factor, resource, technology used, process, legal status, acceptance, approval, recognition, symbolism, associations, link to current events, precedent, mea- surement, ranking, priority, main feature, unique aspect, distin- guishing factor. Key hyperparameters Unless stated otherwise: curated PopQAN = 200entities,K = 2ques- tions per entity for localization and causal checks, seed 7, andΔ = 10 â6 for stability computations. Entity-specific amnesia (main Finding 2) uses αâ [1,â3] with 20 steps. Finding 3 injection setting For Finding 3 we use set-injection at the placeholder position with mean-entity initialization. Concretely, we first set the full hidden vector atXto the layer- specific mean-entity vector, then overwrite the selected top-kneurons (single-cell top-1 as pri- mary, with top-5 as the multi-cell comparison). We sweep an interpolation/extrapolation factor αâ1, 2, 4, 8, 16, 32, 64, 128, 200per entity, se- lecting the best-performingαfor reporting; this choice is intentionally high-sensitivity and may be optimistic relative to a fixed-αprotocol. We addi- tionally flag entity-level success whenRelProbâ„ 0.30and both marginsRelProbâRelProb no inj â„ 0.05 and RelProbâ RelProb wrong â„ 0.05. B Algorithms Algorithms 1 and 2 provide pseudocode for the two core procedures used throughout this work: stability-based localization and controlled cell in- jection. Algorithm 3 describes a factual modifica- tion procedure via latent steering (Appendix C). Algorithm 1 Stability-based localization of an en- tity cell Require:ModelMwith layersâ â 0,...,Lâ 1; baseline prompt setB; entity-centered promptsx i K i=1 ; entity token index function t(·); Δ > 0 Ensure: Entity cell (â â ,j â ) 1:Compute baseline statistics(ÎŒ âj ,Ï âj )from a âj (b) over bâB 2: for iâ 1 to K do 3: Extract activationsa âj (x i )at token posi- tion t(x i ) for all â,j 4:Normalizez âj (x i ) â(a âj (x i ) â ÎŒ âj )/(Ï âj + Δ) 5: end for 6: ComputestabilityS âj â (E i [z âj (x i )]) 2 /(Std i [z âj (x i )] + Δ) 7: (â â ,j â )â arg max â,j S âj 8: return (â â ,j â ) C Factual Modification via Latent Steering We describe a factual modification procedure that optimizes a perturbation vector injected at an entity-associated activation site to increase the probability of a chosen target completion for a spe- cific relation, while penalizing drift on a small set of unrelated facts. We report a single-case study (Obama spouse) to illustrate the method and its edit-vs.-preserve objective. 10 Algorithm 2 Controlled injection of entity cells in a QA-style prompt Require:ModelM; tokenizerÏ; PopQA question q; answer aliasesA; entity aliasesE; localized layerâ â ; top-kentity cellsS = j 1 ,...,j k ; entity-specific valuesv j jâS ; mean-entity vector m â â ; scale α; Δ > 0 Ensure:Relative answer probability under injec- tion 1: Wrap the question for a base model:x full â Question: q : 2:Find a matched aliaseâEinqand formq X by replacing the first occurrence with X 3: Construct placeholder promptx X â Question: q X :and locate place- holder token index t X 4:Convert aliases to next-token targetsY â tok 1 (a) : aâA underÏ(prepend a leading space for tokenization) 5:RunMonx full and recordp full â max yâY p(y | x full ) 6: RunMonx X (no injection) and recordp 0 â max yâY p(y | x X ) 7:RunMonx X while injecting at layerâ â and position t X : Initialize: a â â (x X )[t X ]â m â â For eachj â S:a â â j (x X )[t X ] â m â â j + α (v j â m â â j ) 8: Record p 1 â max yâY p(y | x X , inject) 9: return p 1 / max(p full ,Δ) ID Prompt (Barack Obama) A1 The name of the wife of Barack Obama is A2 When Barack Obama was president, his wifeâs name was A3 Barack Obama is married to A4 The spouse of Barack Obama is named P1 Barack Obama was born inâ Hawaii P2 The political party of Barack Obama isâ Democratic P3 Barack Obama served as the 44thâ President P4 The daughters of Barack Obama are Malia andâ Sasha P5 Barack Obamaâs vice president was Joeâ Biden P6 The book written by Barack Obama is titled Dreams from My â Father Table 2: Attack prompts (A1âA4) and preservation prompts (P1âP6) used for factual modification. Preser- vation prompts include an expected next token. D Entity Cell Map Categorized PopQA mapThe full PopQA-200 entity map is grouped by category to improve read- ability. The current split contains 48 people, 82 locations, 6 organizations, and 64 other entities. Each row includes a trust flag computed by auto- A1A2A3A4 10 4 10 2 10 0 P(target token) Attack prompts (target relation) BaseSteered P1P2P3P4P5P6 0 1 2 Steered/Base ratio Preservation prompts (non-target facts) Figure 8: Factual modification via latent steering (single-case study). Top: spouse prompts (A1âA4; Ta- ble 2) before/after steering toward a target completion. Bottom: preservation prompts (P1âP6), reported as steered/base ratios for expected next tokens. Algorithm 3 Factual modification via latent steer- ing at a localized entity layer Require:ModelM; tokenizerÏ; entity stringe; localized layerâ â ; entity token index func- tiont e (·); attack promptsA; preserve facts P = (p i ,y i ); target tokeny tgt ; weights (λ a ,λ p ,λ 2 ); steps T ; learning rate η Ensure:Perturbation vectorÎŽ â R d (hidden size d) 1:InitializeÎŽ ⌠Uniform(0, 1) d (float32), set optimizer AdamW(ÎŽ,η) 2: for tâ 1 to T do 3: L a â 0 4:for each aâA do 5: Locate entity token indexs â t e (a) (e.g., last token of e under Ï ) 6: RunMonawhile injectingÎŽat layer â â and position s 7:L a +=â logp(y tgt | a,ÎŽ) 8:end for 9: L a â 1 |A| L a 10: L p â 0 11:for each (p i ,y i )âP do 12:Locate entity token index sâ t e (p i ) 13: RunMonp i while injectingÎŽat layer â â and position s 14:L p +=â logp(y i | p i ,ÎŽ) 15:end for 16: L p â 1 |P| L p 17: L 2 ââ„ÎŽâ„ 2 18: Lâ λ a L a + λ p L p + λ 2 L 2 19:Take one optimizer step on ÎŽ usingâ ÎŽ L 20: end for 21: return ÎŽ 11 mated checks (e.g., early-layer localization and causal sensitivity under negative ablation, plus non-collapse sanity checks). Under these checks, k = 131out ofn = 200localized cells are marked trustworthy. Under the Finding 3 causal-injection success criterion (full trustworthy set), 75/131 enti- ties pass with top-kinjection (74/131 with top-1); 1 entity requires top-k. Person Entities (k=26, n=48) EntityLayer Neuron Trust. Abraham Lincoln312305Yes Al Gore111620Yes Alexander the Great48881Yes Ali ibn Abi Talib418599No Amitabh Bachchan310957No Apollo08859No Aung San Suu Kyi33083No Barack Obama210941Yes Bertrand Russell315419Yes Billy Joel28277Yes Carl Linnaeus218724No Chris Jericho38819Yes David213244Yes Donald Trump111948Yes Edgar Allan Poe516637Yes Elizabeth I36343No Francis22926Yes Ganesha25215No EntityLayer Neuron Trust. Gautama Buddha014566No George VI1467Yes George W. Bush010032No George Washington13732No Hamilton415761No Helen of Troy217460No Jacob14Yes James Madison23867No James VI and I0698No Jesus18526Yes Johann Sebastian Bach411153No King Arthur46331No Krishna312927Yes Mark Twain111338No Mary, Princess Royal and Countess of Harewood 012945No Michael Jackson21224Yes Muhammad218938No Muhammad Ali117041No EntityLayer Neuron Trust. Paul118738Yes Peter213512Yes Prince115522Yes Queen Victoria118579Yes Rama418757Yes Ronan Farrow25954No Rumi26292Yes T. S. Eliot112920Yes Thomas Jefferson211279Yes Thor46521Yes Vajiralongkorn23566No Will Smith315898Yes Location Entities (k=56, n=82) EntityLayer Neuron Trust. Afghanistan32492Yes Alexandria29910No Arizona38202No Arkansas33613No Athens118605Yes Australia18982Yes Barcelona313144Yes Beijing29567Yes Berlin217703No Boston318601Yes Brazil117255Yes Brussels Capital Region07084No Byzantine Empire21260Yes California24130No Canada215999Yes Cape Town39847Yes Chicago310161No China26806No EntityLayer Neuron Trust. Colorado212787No Confederate States of America08161No Dallas33153Yes Delhi47686Yes Dublin43121Yes El Salvador12281No Empire of Japan17726Yes Florence110577Yes Florida27558No Georgia112233Yes Hawaii112270Yes Houston111316Yes Idaho4649Yes India22893Yes Italy12937No Japan17726Yes Jerusalem3270Yes Jordan21661No EntityLayer Neuron Trust. Kansas211619Yes Kuala Lumpur32420Yes Lebanon14918No London217407Yes Madrid210473Yes Mali310631Yes Manila316721No Melbourne15250Yes Mexico1977No Milan213790No Minnesota25780Yes Montana213501Yes Nebraska217105Yes Netherlands315677No New Jersey210170Yes New Mexico410866Yes New York111260No New York City11139Yes 12 EntityLayer Neuron Trust. Oregon44925Yes Paris1231Yes Peru217476Yes Philadelphia4751Yes Phoenix111122No Pittsburgh18853Yes Poland47987Yes Prague314586Yes Puerto Rico27844Yes Republic of China 1912-194906408No Rio de Janeiro11640No Roman Republic16083No Rome45848Yes San Francisco19914Yes Singapore17100Yes South Africa11385Yes Spain25876Yes Sri Lanka26010Yes EntityLayer Neuron Trust. Stockholm514077Yes Tennessee316787Yes Texas34501Yes Tokyo1188Yes Toronto1864Yes Troy217460Yes Turin318612Yes Vienna118529Yes Virginia17229No Washington, D.C.016991No Organization Entities (k=4, n=6) EntityLayer Neuron Trust. AtlĂ©tico de Madrid210473Yes European Union212264Yes Nine Inch Nails35818No Oasis313059Yes The Band08075No White House118670Yes Other Entities (k=45, n=64) EntityLayer Neuron Trust. 19013033No Alien310404Yes Aliens17483Yes Avatar214834No Babylon311790Yes Back to the Future011486No Battlefield210945Yes Beloved17245Yes Breaking Bad212152Yes Budapest310222Yes Carrie318279Yes Cars118321Yes Doctor Who01329No Drive216166No E.T. the Extra-Terrestrial14102Yes Final Destination212738Yes Flight22518Yes Friends59314Yes EntityLayer Neuron Trust. Frozen216115Yes Ghost14299No Grease210749Yes Halloween42177No Happy Birthday to You52162Yes Heart117053No Inside Out02940No Into the Wild18382Yes Iron Man116367No It010424No Jesus in Islam15588Yes Legend05788No LĂ©on: The Professional29734Yes Let It Be511669Yes Lost15217Yes Neon Genesis Evangelion44605Yes Nineteen Eighty-Four112211No Power2212984No EntityLayer Neuron Trust. Rent211038Yes Rocky36169Yes Saw2712646No Scooby-Doo218490Yes Seven29649Yes Star Wars16101Yes Suits29620Yes The Birds28054Yes The Challenge113535No The Departed52625Yes The Fly210777Yes The Fog110964Yes The Graduate217192No The Holiday12834Yes The Matrix1613No The Mist213859Yes The Omen35504Yes The Prestige11538Yes EntityLayer Neuron Trust. The Ring214458Yes The Shining59949Yes The Social Network2804No The Terminal216927Yes The Thing510410Yes The Village114056Yes They Live318417Yes Toy Story32434Yes Training Day47991Yes Up214580Yes E Post-Training Generalization (Qwen2.5-7B-Instruct) We test whether the Qwen2.5 entity-cell map sur- vives ordinary post-training by rerunning the same analyses on Qwen2.5-7B-Instruct and comparing directly to the base-model findings in Figures 2 to 7. Across the full PopQA-200 inventory, Qwen2.5- 7B-Instruct exactly preserves the base modelâs top localized cell for 190/200 entities (and preserves the same layer for 191/200). In particular, both models localize Barack Obama to the same cell (L2-N10941). This is the clearest post-training result in the appendix. The localization map remains nearly unchanged, the early-layer concentration pattern is preserved, and 123 cells still pass the same 13 amnesia-based trust filter used in the main paper. Together, these results suggest that the Qwen2.5 entity-cell map is robust to ordinary instruction tuning. 0510152025 Layer (Top Neuron) 0 20 40 60 Entity Count Top-Neuron Layer Distribution Qwen2.5-7B-Instruct Figure 9: Qwen2.5-7B-Instruct replication of Figure 2. As in the main-paper Qwen2.5-7B base result, top local- ized cells remain concentrated in early layers, indicating that the layer profile is largely preserved under instruc- tion tuning. 32101 Multiplier for Neuron 10941 (Layer 2) 0.0 0.2 0.4 0.6 0.8 1.0 Relative Knowledge Score Entity-Specific Amnesia via Neuron Ablation Qwen/Qwen2.5-7B-Instruct Target (Obama) Control (Trump) Original Model Unknown Entity Baseline Figure 10: Qwen2.5-7B-Instruct replication of Fig- ure 3. Negative ablation of the localized entity cell again causes a strong drop for the target entity while leaving the control entity comparatively stable, sup- porting preservation of the same causal pattern after post-training. Entity PresentMean EntityCorrect CellWrong Cell 0.0 0.2 0.4 0.6 0.8 1.0 Pass@5 Accuracy Entity Injection Qwen2.5-7B-Instruct Figure 11: Qwen2.5-7B-Instruct replication of Figure 4. On the trusted set, correct-cell injection again outper- forms both the mean-entity initialization and wrong-cell controls, indicating that the same localized cells remain causally useful after instruction tuning. 0.00.51.0 Rel. Stability L2-10941 L0-3664 L3-4828 L1-12564 L1-9991 L2-16462 1.00 0.24 0.05 0.04 0.04 0.02 Barack Obama 0.00.51.0 Rel. Stability L2-10941 L2-15783 L2-793 L2-4253 L2-4989 L2-16593 1.00 0.27 0.17 0.13 0.11 0.10 Obaama 0.00.51.0 Rel. Stability L2-10941 L2-5398 L2-40 L2-17763 L2-11024 L2-15911 1.00 0.42 0.35 0.27 0.16 0.14 Brock Obma 0.00.51.0 Rel. Stability L2-10941 L1-12495 L2-303 L2-17179 L2-5780 L2-6779 1.00 0.72 0.57 0.46 0.28 0.27 Bark Obamna Figure 12: Qwen2.5-7B-Instruct replication of Figure 5. Most variants of âBarack Obamaâ preserve the same top cell as in the base model, indicating that the localized handle remains stable under post-training. F Generalization Within Model Family (Qwen3) To probe within-family generalization beyond post- training, we apply the same analyses to Qwen3- 8B base and report the same figure set used for Qwen2.5-7B-Instruct, matched to the main-paper results in Figures 2 to 7. The depth profile remains 14 0.00.51.0 Rel. Stability L2-11955 L0-7307 L6-6847 L0-2329 L2-4759 L0-16274 1.00 0.14 0.13 0.08 0.08 0.07 Federal Bureau of Investigation 0.00.51.0 Rel. Stability L2-11955 L8-4997 L2-2290 L3-16252 L2-7426 L2-8009 1.00 0.07 0.07 0.07 0.06 0.06 FBI Figure 13: Qwen2.5-7B-Instruct replication of Figure 6. The acronym and full form again localize to the same top-ranked cell, consistent with preservation of the un- derlying entity-cell mapping. 0.00.51.0 Rel. Stability L1-231 L23-7262 L4-13417 L2-17407 L0-16056 L1-10720 1.00 0.03 0.02 0.02 0.02 0.01 Paris (Latin) 0.00.51.0 Rel. Stability L1-231 L1-11963 L2-11166 L0-10841 L2-16943 L2-11898 1.00 0.16 0.11 0.11 0.10 0.10 Paris (Hebrew) 0.00.51.0 Rel. Stability L1-231 L0-15733 L0-15814 L2-483 L4-13417 L2-5419 1.00 0.20 0.07 0.03 0.03 0.03 Paris (Chinese) 0.00.51.0 Rel. Stability L1-231 L2-13876 L1-7842 L2-13312 L2-727 L2-8385 1.00 0.11 0.08 0.08 0.07 0.06 Paris (Arabic) Figure 14: Qwen2.5-7B-Instruct replication of Figure 7. The same top cell is recovered across multiple scripts for âParisâ, suggesting that cross-script entity access also survives instruction tuning. early-layer concentrated, and the larger appendix suite yields stable top localized cells for all 200 PopQA-200 entities. Under the strict trustworthy plus entity-pass@5 filter used for the controlled- injection evaluation, 42 entities remain. The result is mixed but still suggestive of within- family continuity. Qwen3 reproduces the early- layer localization pattern and retains a nontriv- ial trustworthy subset, but the causal evidence is weaker than in Qwen2.5. Because the standard Obama/Trump amnesia probe is noisier in Qwen3, we also report an alternative London/Paris unlearn- ing probe using the localized London cell (L0- N3037), which yields cleaner target-versus-control separation. Table 3 lists representative localized cells. 0102030 Layer (Top Neuron) 0 10 20 30 40 Entity Count Top-Neuron Layer Distribution Qwen3-8B Figure 15: Qwen3-8B within-family replication of Fig- ure 2. As in the main-paper Qwen2.5 result, top local- ized cells remain concentrated in early layers, suggest- ing that the coarse localization pattern persists within the Qwen family. 32101 Multiplier for Neuron 3037 (Layer 0) 0.0 0.2 0.4 0.6 0.8 1.0 Relative Knowledge Score Entity-Specific Amnesia via Neuron Ablation Qwen/Qwen3-8B Target (London) Control (Paris) Original Model Unknown Entity Baseline Figure 16: Qwen3-8B within-family replication of Fig- ure 3, shown on an alternative London/Paris probe be- cause it yields cleaner separation than the default Oba- ma/Trump case in this model. Negative ablation still preferentially suppresses the target-entity curve, but the effect is less clean than in Qwen2.5. G Lack of Generalization Across Model Families To summarize cross-family transfer, we use a sim- ple count-based pipeline for each model: start from the same 200 PopQA entities, localize one top cell per entity, keep only entities that pass the Finding 2 amnesia trust filter, evaluate controlled injection on this trusted set, and test exact top-cell stability 15 Entity PresentMean EntityCorrect CellWrong Cell 0.0 0.2 0.4 0.6 0.8 1.0 Pass@5 Accuracy Entity Injection Qwen3-8B Figure 17: Qwen3-8B within-family replication of Fig- ure 4. Correct-cell injection improves over the control conditions, but the separation is weaker and less consis- tent than in Qwen2.5, matching the more mixed within- family result described in the text. 0.00.51.0 Rel. Stability L0-2883 L3-4477 L1-11305 L3-7126 L0-2349 L0-1916 1.00 0.64 0.17 0.15 0.12 0.10 Barack Obama 0.00.51.0 Rel. Stability L1-96 L3-1430 L1-10971 L3-10280 L0-985 L5-4225 1.00 0.64 0.44 0.43 0.34 0.25 Obaama 0.00.51.0 Rel. Stability L1-10971 L4-1400 L1-9173 L0-10274 L3-2679 L5-10505 1.00 0.21 0.21 0.12 0.12 0.11 Brock Obma 0.00.51.0 Rel. Stability L2-4108 L1-5937 L4-3421 L1-10971 L0-1265 L0-6923 1.00 0.13 0.05 0.03 0.02 0.02 Bark Obamna Figure 18: Qwen3-8B within-family replication of Fig- ure 5. Most variants recover closely related early-layer cells, though the match is less clean than in Qwen2.5- 7B-Instruct. across surface-form probes. Table 4 shows that cross-family transfer is lim- ited. OLMo-7B gives the strongest result: 37 trust- worthy cells, 23 of which pass controlled injection, with 30% form robustness. The other families are weaker. Llama-3.1-8B and Mistral-7B each re- tain 40 trustworthy cells, but only 5 pass injection; OpenLLaMA-7B retains 33 trustworthy cells, of 0.00.51.0 Rel. Stability L1-9339 L3-1398 L0-10197 L1-7226 L0-11498 L0-10219 1.00 0.85 0.20 0.16 0.15 0.13 Federal Bureau of Investigation 0.00.51.0 Rel. Stability L3-5559 L3-8424 L3-1398 L1-9339 L0-6025 L3-11772 1.00 0.91 0.61 0.60 0.25 0.16 FBI Figure 19: Qwen3-8B within-family replication of Fig- ure 6. The acronym and expanded form still localize to closely aligned cells, supporting partial preservation of the entity-cell map within the Qwen family. 0.00.51.0 Rel. Stability L2-10547 L2-9535 L2-6175 L2-6539 L2-1543 L2-7963 1.00 0.12 0.09 0.08 0.07 0.07 Paris (Latin) 0.00.51.0 Rel. Stability L35-6856 L4-2573 L2-12223 L2-10547 L35-2212 L0-7861 1.00 0.58 0.47 0.45 0.32 0.23 Paris (Hebrew) 0.00.51.0 Rel. Stability L2-10547 L0-3459 L0-9242 L35-10405 L0-2603 L1-944 1.00 0.11 0.07 0.05 0.04 0.04 Paris (Chinese) 0.00.51.0 Rel. Stability L2-111 L2-6175 L2-10547 L0-2144 L0-6424 L2-6445 1.00 0.82 0.20 0.04 0.03 0.02 Paris (Arabic) Figure 20: Qwen3-8B within-family replication of Fig- ure 7. Cross-script forms continue to recover similar top cells, though the pattern is noisier than in Qwen2.5. EntityLayer Neuron Notes Obama02883localized (injection weak in this subset) Trump39290 localized; injection success Paris210547 localized London03037 localized; strong ablation drop Beijing45431 localized; strong ablation drop Tokyo3223 localized; ablation drop Table 3: Representative Qwen3-8B base entity cells localized by stability. Notes summarize small-subset causal checks (Appendix F). 16 which 12 pass injection. Overall, sparse candidate cells can often be localized, but strong causal vali- dation and form robustness do not transfer reliably. Figure 21 provides a localization-only compari- son across four non-Qwen model families. Relative to the dedicated Qwen-family plots and to the main- paper localization result in Figure 2, these distri- butions are typically broader and shifted deeper, suggesting that the sparse early-layer localization pattern is not uniform across model families. 0 100 200 Entities L0-5: 97.0% OLMo-7B L0-5: 100.0% Llama-3.1-8B 051015 Layer 0 100 200 Entities L0-5: 99.5% Mistral-7B 051015 Layer L0-5: 100.0% OpenLLaMA-7B Figure 21: Cross-model localization depth profiles (2Ă2) for four non-Qwen-family models, formatted for single-column display. Each panel shows the distri- bution of top-neuron layers over PopQA-200 entities for one model. Relative to the main-paper localization result in Figure 2, these families are generally broader and deeper. Figure 22 summarizes controlled injection on the trusted set for each model. As in the main-paper injection analysis in Figure 4, we replace the entity mention with a placeholder token, inject either the matched localized cell(s) or a wrong entityâs cell(s) at that position, and compare normalized answer probability to the entity-present baseline. We include a dedicated OLMo view because it shows the strongest cross-family controlled in- jection result in Table 4 (23/37). At the same time, its amnesia curve is less clean than the Qwen- family cases, suggesting that these cells may play a broader or differently structured role in OLMo. OLMo-7B Llama-3.1-8B-Inst Mistral-7B-v0.3 OpenLLaMA-7B 0.0 0.1 0.2 0.3 0.4 0.5 RelProb (Correct Wrong) Causal separation OLMo-7B Llama-3.1-8B-Inst Mistral-7B-v0.3 OpenLLaMA-7B 0 20 40 60 80 100 Success Rate (%) Entity-level success Figure 22: Controlled injection on trustworthy cells (top-5, alpha search).Left:causal separation â = RelProb(Correct Cell)â RelProb(Wrong Cell), where probabilities are normalized by the entity-present prompt. Right: entity-level success rate, defined as the fraction of trustworthy entities that satisfy all three crite- ria:RelProb(Correct Cell)â„ 0.30, improvement over no-injectionâ„ 0.05, and improvement over wrong-cell injectionâ„ 0.05. Entity PresentMean EntityCorrect CellWrong Cell 0.0 0.2 0.4 0.6 0.8 1.0 Relative Answer Probability Activation Causality (Normalized) Model: allenai/OLMo-7B-0724-hf Figure 23: OLMo-7B cross-family replication of Fig- ure 4. Among the non-Qwen models, OLMo shows the strongest positive result: correct-cell injection produces the clearest separation from the control conditions on the trusted set. ModelTrustworthy Injection Form-Robustness OLMo-7B3723/3730% Llama-3.1-8B405/4050% Mistral-7B405/4040% OpenLLaMA-7B3312/3330% Table 4: Cross-family summary over PopQA-200 enti- ties. Trustworthy Cells: entities retained by the Find- ing 2 amnesia trust filter. Knowledge Injection: top- 5success among trustworthy entities. Surface-Form Robustness: exact top-cell match rate across variant, acronym, and multilingual probes (10 attempts per probe). 17 32101 Multiplier for Neuron 1593 (Layer 0) 0.5 0.0 0.5 1.0 Relative Knowledge Score Entity-Specific Amnesia via Neuron Ablation Target (Obama) Control (Trump) Original Model Unknown Entity Baseline Figure 24: OLMo-7B cross-family replication of Fig- ure 3. Negative ablation reduces the target-entity curve, but the control curve is also affected, so this is not the same clean entity-specific amnesia pattern seen in Qwen2.5. This suggests that OLMoâs localized cells may participate in retrieval differently, or less selec- tively, than the Qwen-family cells. 18