Paper deep dive
Deep Learning Models Also Recall Features
Pierre Beckmann
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/24/2026, 5:49:04 AM
Summary
The paper introduces 'feature recall' as a general operation in deep learning models, contrasting it with the established paradigm of 'feature combination'. While feature combination involves hierarchically combining lower-level features into higher-level ones (common in CNNs), feature recall involves input activations scaling stored weight vectors to retrieve associated information (common in LLMs). The author argues that linear projections can be interpreted as either operation, but proposes a 'connectivity ratio' to empirically distinguish between them based on whether the output is driven by sparse (recall) or dense (combination) input connectivity.
Entities (8)
Relation Signals (6)
Feature Recall â contrastswith â Feature Combination
confidence 95% ¡ I define feature recall, show it applies across architectures, and contrast it with the established paradigm of feature combination.
Linear Projection â canbeinterpretedas â Feature Recall
confidence 93% ¡ The core observation is that a linear projection can be read as retrieving stored information scaled by input activations.
Linear Projection â canbeinterpretedas â Feature Combination
confidence 93% ¡ In case (a), each output element is obtained by multiplying the input vector with a column of the weight matrix. This can be seen as feature combination
Feature Recall â appliesto â Large Language Models
confidence 92% ¡ Recent work in mechanistic interpretability has studied how large language models (LLMs) recall facts stored in their weights. This paper argues that factual recall points to something broader... feature recall.
Feature Combination â appliesto â Convolutional Neural Networks
confidence 90% ¡ The paradigmatic illustration is image recognition via convolutional neural networks (CNNs)... This paradigm has been philosophically productive... in CNNs
Connectivity Ratio â distinguishes â Feature Recall
confidence 85% ¡ A high ratio (closer to 1) indicates sparse effective connectivity... suggesting that feature k was produced by feature recall. A low ratio... suggesting that feature k was produced by feature combination.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent work in mechanistic interpretability has studied how large language models recall facts stored in their weights. This paper argues that factual recall points to something broader: a general kind of operation in deep learning models, which I call feature recall. The core observation is that a linear projection can be read as retrieving stored information scaled by input activations. I define feature recall, show it applies across architectures, and contrast it with the established paradigm of feature combination. I also consider how cases of feature recall might be mechanistically identified. The account gives philosophers a new conceptual tool for understanding deep learning, and points to empirical directions for mechanistic interpretability research.
Tags
Links
- Source: https://arxiv.org/abs/2608.20970v1
- Canonical: https://arxiv.org/abs/2608.20970v1
Trouble viewing inline? Open PDF directly â
Full Text
30,700 characters extracted from source content.
Expand or collapse full text
Deep Learning Models Also Recall Features Pierre Beckmann 1,2 1 Ě Ecole Polytechnique F Ě ed Ě erale de Lausanne (EPFL) 2 Idiap Research Institute B pierrebeckmann@gmail.com February 2026 Abstract: Recent work in mechanistic interpretability has studied how large language models re- call facts stored in their weights. This paper argues that factual recall points to something broader: a general kind of operation in deep learning models, which I call feature recall. The core obser- vation is that a linear projection can be read as retrieving stored information scaled by input ac- tivations. I define feature recall, show it applies across architectures, and contrast it with the es- tablished paradigm of feature combination. I also consider how cases of feature recall might be mechanistically identified. The account gives philosophers a new conceptual tool for understand- ing deep learning, and points to empirical directions for mechanistic interpretability research. Keywords: deep learning, feature combination, feature recall, mechanistic interpretability Introduction Recent work in mechanistic interpretability has studied how large language models (LLMs) recall facts (Ameisen et al., 2025; Chughtai et al., 2024; Geva et al., 2023; Nanda et al., 2023). These findings point to a way of making sense of internal operations in deep learning models in general, which I call feature recall. It stands in contrast to the well-established paradigm of feature combination. Feature combination holds that deep neural networks operate by hierarchically combining lower- level features into higher-level ones. This framing is central to the 2015 Nature paper by LeCun et al., who describe deep learning as composing âsimple but non-linear modules that each transform the rep- resentation at one level into a representation at a higher, slightly more abstract level.â The paradigmatic illustration is image recognition via convolutional neural networks (CNNs), where early layers detect edges, intermediate layers assemble textures, and later layers compose objects (LeCun et al., 2015; Olah et al., 2017). This paradigm has been philosophically productive as it has grounded arguments about * An earlier version of this paper was presented at the 6th Biannual Conference on the Philosophy of Artificial Intel- ligence (PhAI 2025). I thank participants for helpful feedback, in particular Elliot du Sordet, Iwan Williams, and Kola Ayonrinde. 1 arXiv:2608.20970v1 [cs.AI] 21 Aug 2026 abstraction in CNNs (C. Buckner, 2018; C. J. Buckner, 2024), as well as concept formation (R Ě az, 2023) and understanding (Beckmann, 2025) in deep learning models. Yet feature combination does not always offer a satisfying account of what deep learning models do. When ChatGPT produces a biography of Michael Jordan, the biographical content is not contained in the input prompt, waiting to be hierarchically assembled. Instead, the model seems to be retrieving stored information. I argue that such cases are better understood through what I call feature recall. Feature recall would then be a useful complement to the feature combination paradigm, rather than a replacement of it. I proceed in three steps. First, I explain factual recall in LLMs, a case of feature recall (§1). I then define feature recall in general terms and show it applies across deep learning architectures (§2). Finally, I present two readings of feature combination and recall: as interpretive lenses or as operational kinds (§3). 1 Factual recall in LLMs Factual recall in LLMs is an active area of mechanistic interpretability research (Ameisen et al., 2025; Chughtai et al., 2024; Geva et al., 2023; Lindsey et al., 2025; Nanda et al., 2023). To present the mech- anism, I first introduce some key concepts from the field. 1 Mechanistic interpretability takes the residual stream as its primary unit of analysis (Elhage et al., 2021). The residual stream is the evolving internal representation of a token as it passes through the transformer, from initial embedding to final prediction. Rather than viewing each layer as transforming the input into something new, this perspective emphasizes how layers successively add vectors to the evolving token representation. 2 The residual stream thus acts as a persistent computational workspace where information accumulates. The residual streamâs vector space is structured by many directions that correspond to internal fea- tures, each encoding a particular property that can be present to a varying degree. A helpful metaphor is to think of each feature direction as a slider: as a token flows through the model, attention heads and MLPs write into the residual stream by adjusting the positions of these sliders, setting values that reflect the presence or absence of particular properties. 3 The adjustment of these âslider valuesâ occurs through successive transformer blocks, each con- taining two key components. Attention heads route information between token positions, enabling the model to integrate context across the sequence. Multi-layer perceptrons (MLPs) perform local- ized transformations on the current tokenâs representation. Both components operate additively on the residual stream: they write new feature activations into the workspace, thus setting new âslider valuesâ. MLPsâ localized transformations encompass both feature combination and what mechanistic interpretability researchers have termed factual recall. To see both operations at work, consider the idealized example in Figure 1, adapted from Nanda et al. (2023). 1 For a more in depth philosophical introduction to mechanistic interpretability see Beckmann and Queloz (2026). For an accessible video introduction to factual recall in LLMs see Sanderson (2024). 2 This additive interpretation is enabled by the residual/skip connections of the transformer architecture. 3 The term âfeatureâ is used both for the direction in latent space (the slider itself) and the activation of a particular input along that direction (the slider value). This dual usage is standard in the field, and I adopt it here. 2 attention head MLP Michael + Jordan One MLP â Projecting up: ⥠Activation function: ⢠Projecting down: Michael Jordan plays basketball Michael + Jordan Michael Jordan Basketball + Bulls Jordan a) b) plays basketball + at Chicago Bulls Figure 1: An idealized case of feature-activated factual recall. Panel (a) shows the sequential processing of âMichael Jordan plays,â word-token by word-token as they are processed along the residual stream. To com- plete this sentence (by predicting the âbasketballâ token), an attention head first routes the Michael feature from the first residual stream to the second. MLP layers then perform two distinct operations. First, they combine the Michael and Jordan features into a unified Michael Jordan feature. Second, they retrieve associated facts stored via the networkâs weightsâsuch as plays basketball or Chicago Bulls playerâbased on this combined feature. Another attention head then routes this retrieved information to the last residual stream, providing the context needed for accurate next-token prediction. Panel (b) details how both feature combination and factual recall could occur within a single MLPâs linear transformations in the residual stream of the âJordanâ token. The upward projec- tion (1) recognizes the pattern of co-occurring Michael and Jordan features, leading to a strong activation of the Michael Jordan feature. The activation function (2) suppresses any spurious activations. Through the downward projection (3) this feature activates the retrieval of certain stored weights, which when added into the output cause the activation of features such as plays basketball or Chicago Bulls player. The weights connecting the Michael Jordan feature to output features directly encode factual associations, ensuring that whenever this feature acti- vates, relevant factual information is automatically retrieved and added to the output representation. 4 This example shows that once we conceptualize directions in the residual stream as features, linear projections can directly encode associations between them. These associations can represent facts: 4 Some operations such as layer normalization are left out from this idealized case. Furthermore, the figure assumes that each feature corresponds to a single neuron, a property called neuron alignment. I adopt this simplification to make the figure legible and the core mechanism accessible to readers less familiar with the field. Readers familiar with superposition (the finding that models represent far more features than they have neurons, thereby distributing features across neuron directions; Elhage et al., 2022) may worry that this undermines the mechanism. Thankfully, it doesnât. A feature direc- tion that does not align with a neuron direction still has a well-defined projection through the weight matrix. The matrix maps the input direction to an output vector that can have strong components along multiple output feature directions. The weights then still encode stored associations; the only difference is that these associations hold between directions in activation space rather than between neurons. 3 the Michael Jordanâ basketball player connection encodes the fact that Michael Jordan is a basketball player. In its simplest form, such an association can correspond to just a single weight in the model, one value in a projection matrix. One weight can ensure that high activation of the Michael Jordan feature will systematically lead to high activation of the basketball player feature, creating a direct pathway for retrieval through the networkâs learned parameters. 5 The key aspect of this interpretation is that the focus shifts from inputs to weights. The input feature merely acts as a trigger; the real work lies in what the weights store. To appreciate how different this is from the feature combination interpretation, consider a CNN that detects circles in images (Figure 2). Weights: 001 010 100 100 010 001 10 01 Weights: Part of an input image: Activations: Activations: Circle feature Square feature Triangle feature Figure 2: Feature combination in a CNN: an idealized case of a circle-recognition circuit. The example traces how a subset of weights of a CNN detects a circle in a 9Ă9 pixel image region. The process works in two stages. First, the network applies two different 3Ă3 pattern detectors to non-overlapping regions of the input image. Each kernel scans four 3Ă3 regions with stride 3, generating two separate 2Ă2 activation maps. The red box illustrates how this works for the top left 3Ă3 region and the first kernel: when the top-left kernel perfectly matches its corresponding 3Ă3 input region, matrix multiplication yields a strong response of 3. The top-right 3Ă3 region shows only partial overlap with the first kernel at the center pixel, producing a weaker activation of 1. Applying both kernels, creates two 2Ă2 feature maps. These are then concatenated into a single 2Ă2Ă2 tensor (though the stacking isnât shown in the figure). Second, a higher-level 2Ă2Ă2 kernel is applied, arranged exactly like our current concatenation of the two feature maps. It will therefore get its strongest activation precisely when the input 9Ă9 contains a circle. So the network detects the circle hierarchically: from 9Ă9 pixels, to two 2Ă2 edge maps, to one circle feature. 6 It would be odd to say the circuit is retrieving information about circles. It is detecting a pattern by combining lower-level features. The weights encode what configuration of inputs to look for, not what 5 Attention heads can also perform factual recall through their value and output projections (e.g., Chughtai et al., 2024; Geva et al., 2023). But these projections represent only a fraction of attention headsâ total parameters, and attention heads collectively contain roughly half as many parameters as MLPs in typical transformer architectures. The majority of in-place computationsâincluding both feature combination and feature recallâoccur within MLPs. 6 Some key operations of CNNs such as pooling, nonlinear activation functions, and normalization are left out from this idealized case. As with Figure 1, features are shown as neuron-aligned for simplicity. 4 to recall once something is found. That is the core contrast: under the feature combination reading, weights define patterns to detect; according to feature recall, they store information to be retrieved. Factual recall names one instance of a broader mechanism. I propose to call the general operation feature recall, for two reasons. First, feature recall need not always rely on factual associations; it can also encode conceptual ones. The Michael Jordanâ basketball player connection is factual: a contingent, empirical association that could have been otherwise. But when a âbachelorâ feature triggers âunmar- ried,â the association is conceptual, not factual, as being unmarried is part of what it is to be a bachelor. Second, factual recall makes it sound as if the mechanism always recalls âcorrectâ facts. But feature re- call relies on putative connections the model formed during training, and these can be wrong. Feature recall is thus the more general and more precise term. It names the mechanism without assuming what kind of connection is encoded or whether it is correct. Having presented a precise case of feature recall in LLMs, I now generalize the observation. I argue that feature recall is not specific to one architecture but a general operation that all deep learning models can be seen as performing. 2 Feature recall in deep learning models At its core, every major deep learning architecture relies on matrix multiplication between an input vector and a learned weight matrix, with the addition of fixed layers such as activation functions. This notably holds for convolutional kernels in CNNs, weight matrices in MLPs, value and output projec- tions in attention heads, and gate matrices in LSTMs. These matrix multiplications are constrained differently by the architecture but essentially always define a projection between an input and an out- put layer. The factual recall mechanism from Section 1 suggests a new way to read any such projection. Con- sider a simple MLP layer that transforms an input vector of size 3 into an output vector of size 3 (Figure 3). Feature combination is many-to-one: many input features contribute to producing one output fea- ture, and the weights specify what to detect. Feature recall, by contrast, is one-to-one or one-to-many: one input feature triggers the retrieval of one or many output features, and the weights specify what to retrieve. The input acts as a trigger; the substantive content of the transformation resides in what the weights store. This is exactly what we saw in the downward projection of the MLP in Figure 1b, with the Michael Jordan feature scaling out a row of stored associations, retrieving basketball player, Chicago Bulls, and so on. Since every deep learning architecture relies on learned linear projections, feature recall applies wherever deep learning operates. More precisely, feature recall is the operation by which input feature activations scale stored weight vectors, thereby retrieving associated output feature activations. To illustrate, return to the circle-detecting CNN from Figure 2, but suppose now that it is part of a model trained to classify handwritten digits in MNIST. Once the circle feature activates, it could scale out a row of stored weights, retrieving features for digits that typically contain circles, such as 0, 6, 8, and 9, and feeding these into downstream processing. The model would thus be recalling which digits contain circles to help in its classification task. So even in a CNN where a lot of feature combination happens, 5 Ă Ă = A linear projection: a) can be seen in terms of feature combination, output obtained by doing: Ă â each output feature is a combination of input features. â columns of the weight matrix combine the input in various ways. b) can be seen in terms of information recall, output obtained by doing: â each input feature activates its own information recall. â rows of the weight matrix are seen as retrieved information. Ă+Ă+Ă input outputweights which amounts to a matrix multiplication: input output weights Figure 3: A linear projection, or matrix multiplication, can be seen both in terms of (a) feature combination or (b) feature recall. In case (a), each output element is obtained by multiplying the input vector with a column of the weight matrix. This can be seen as feature combination: each column defines a pattern of how input features should be combined to obtain one specific output feature. In case (b), each input feature multiplies out one row of the weight matrix and the resulting vectors are summed (note that this amounts to the exact same operation because matrix multiplication can be decomposed as either a dot product with columns or a weighted sum of rows). Each row can then be seen as a stored set of potential feature activations that gets scaled by the input featuresâthe operation is thus seen in terms of feature recall. some projections might be best understood through the lens of feature recall. More generally, feature recall helps make sense of cases that were puzzling under a pure feature combination reading. When a language model generates a detailed biography of Michael Jordan from a short prompt, the biographical content is not present in the input. Feature recall offers a natural explanation: stored associations are retrieved and chained through successive projections, letting a few words of input yield a long, detailed answer. The same logic applies to image and video generation models, which can produce complex outputs from just a few words of prompt. Finally, I take it that feature recall opens the way to thinking about dispositional beliefs in deep learning models. So far, discussions of belief in LLMs have focused on occurrent beliefs, that is, beliefs that are active during a particular inference and therefore identifiable through linear probes on residual stream activations (see Herrmann and Levinstein, 2025). But the weight-based associations underlying feature recall have a distinctly dispositional character. The fact that Michael Jordan plays basketball is encoded in the modelâs weights regardless of whether the model is currently processing anything about Michael Jordan. It is there as a standing disposition, ready to be triggered when the relevant feature activates. This suggests a natural mapping: occurrent beliefs correspond to features encoded in activations at a given inference, while dispositional beliefs correspond to associations stored in weights, retrieved only when the right trigger arises. I leave the careful exploration of this connection for future work, but if the mapping holds, feature recall could provide a framework for locating dispositional 6 belief in deep learning models, a category that has so far received little attention. 7 3 Interpretive lens or operational kind? Throughout this paper, I have treated feature combination and feature recall primarily as interpretive lenses: two ways of reading the linear projections of deep learning models. A careful reader may have noticed something troubling about this. As Figure 3 illustrates, any matrix multiplication can be de- composed either as dot products with columns (feature combination) or as a weighted sum of rows (feature recall). The two readings are mathematically equivalent, which threatens to collapse the dis- tinction into a mere difference in perspective. What should be done about this? One option is to bite the bullet and accept that the distinction is ultimately hermeneutic. Feature combination and feature recall are ways for us to make sense of what deep learning models do, with different projections lending themselves more naturally to one reading or the other. On this view, the circle-detecting kernel is most naturally read as feature combination, while the MLP projection re- trieving facts about Michael Jordan is most naturally read as feature recall. But there is no fact of the matter beyond what we find explanatorily useful. This comes at a cost, however: it blurs the line be- tween the two operations, since recalling many features from one input could, in a sense, be redescribed as combining one input with many stored associations (and vice versa). The more ambitious option is to try to draw a hard line between the two, so that any given learned projection falls cleanly into one category or the other. Since the mathematical formalism alone cannot distinguish the two (any projection admits both readings), this requires anchoring the concepts to an empirical criterion that forces them apart. Such a criterion would identify a property of trained projec- tions that makes one reading apt and the other inapt for a given case. I sketch here what such a criterion might look like. The proposal may be of use to the mechanistic interpretability researcher (see Williams et al., 2025), and it illustrates to the philosopher how a conceptual distinction can be operationalized for deep learning. The structural asymmetry between the two operations provides a natural starting point. Feature combination is many-to-one, with many input features contributing to one output feature; feature recall is one-to-one or one-to-many, with one input feature dominating the output. This amounts to a difference in effective connectivity: feature combination draws on a dense set of input features, while feature recall routes through a sparse one, often just a single input triggering the retrieval of stored associations. To capture this difference quantitatively, one could define a connectivity ratio such as: CR k = max i |w ik | P i |w ik | which, for a given output feature k in a projection with n input features, measures how concentrated the weight mass is on a single input. A high ratio (closer to 1) indicates sparse effective connectivity, where one input does most of the workâsuggesting that feature k was produced by feature recall. A low ratio (closer to 0) indicates dense effective connectivity, where many inputs jointly shaped the outputâsuggesting that feature k was produced by feature combination. 7 I thank Iwan Williams for suggesting this connection. 7 One important caveat is that the connectivity ratio should be computed over projections connecting features, not raw neuron activations. Individual neurons rarely correspond to single features; instead, multiple features are typically superposed within the same neurons (Elhage et al., 2022). A prelimi- nary decomposition step, using dictionary learning methods such as transcoders (Ameisen et al., 2025; Bricken et al., 2023; Templeton et al., 2024), would be needed to recover interpretable features before the ratio can be meaningfully applied. Whether this criterion carves at useful joints is an empirical question. I can think of two lines of inquiry to test here. The first would apply the connectivity ratio to a trained model and examine whether projections with sparse connectivity correspond to recognizable cases of retrieval, such as factual or conceptual associations. The second would compute the ratio across many output features and examine the shape of the resulting distribution. A bimodal distribution would suggest that the criterion reveals a precise boundary between the two concepts. 8 If the criterion works, it could open several avenues of investigation, such as how the balance be- tween feature recall and feature combination varies across model sizes, architectures, or layers. To take one concrete case: Gupta et al. (2025) describe a âguess-then-refineâ pattern in which early transformer layers produce many guesses about what token might come next, while later layers sort through these candidates to select the best prediction. If the criterion tracks a real distinction, one would expect it to pick this up as more feature recall in early-layer MLPs and more feature combination in later ones. In sum, the question of how to cash out the distinction admits two answers. On a weaker reading, it is purely interpretive, and already useful as such. On a stronger reading, the distinction might become operational where some projections count as one and not the other, given an appropriate mechanistic criterion. Pursuing this stronger reading would require the kind of empirical investigation sketched here. Conclusion Starting from recent findings concerning factual recall in LLMs, this paper introduced feature recall as a complement to the established paradigm of feature combination in deep learning. Where feature combination describes how models detect patterns to identify higher-level features, feature recall cap- tures how input features trigger the retrieval of information stored in learned weights. The insight is grounded in the simple observation that any linear projection can be read not only as combining inputs to produce outputs, but also as retrieving stored associations scaled by input activations. Feature recall makes intuitive sense of operations that feature combination struggles to explain, most notably how models generate rich outputs from sparse inputs. It should be useful to philosophers working on the attribution of mental states and capacities to LLMsâfor instance, the dispositional beliefs discussed above. Whether a mechanistic criterion can sharply separate feature recall from feature combination, elevating the distinction from a useful interpretive heuristic to a principled operational one, remains an open question. 8 A further direction would be to define a context-dependent variant of the ratio, for example by scaling weights by actual input activations on a given forward pass. This might allow identifying whether a particular projection is performing feature recall or feature combination in a specific context, rather than in general. 8 References Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., . . . Batson, J. (2025). Circuit Tracing: Revealing Compu- tational Graphs in Language Models. Transformer Circuits Thread. https://transformer-circuits. pub/2025/attribution-graphs/methods.html Beckmann, P. (2025). New horizons in machine understanding: Explanatory and objectual understand- ing in deep learning video generation models. Synthese, 206, 285. https : / / doi . org / 10 . 1007 / s11229-025-05174-5 Beckmann, P., & Queloz, M. (2026). Mechanistic indicators of understanding in large language models. https://arxiv.org/abs/2507.08017 Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., . . . Olah, C. (2023). Towards Monosemanticity: Decomposing Lan- guage Models With Dictionary Learning. Transformer Circuits Thread https://transformer-circuits.pub/2023/monosemantic-features/index.html. Buckner, C. (2018). Empiricism without magic: Transformational abstraction in deep convolutional neural networks. Synthese, 195(12), 5339â5372. https://doi.org/10.1007/s11229-018-01949-1 Buckner, C. J. (2024). From deep learning to rational machines: What the history of philosophy can teach us about the future of artificial intelligence. Oxford University Press. Chughtai, B., Cooney, A., & Nanda, N. (2024). Summing up the facts: Additive mechanisms behind factual recall in llms. https://arxiv.org/abs/2402.07321 Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., & Chen, C. (2022). Toy models of superposition. arXiv preprint arXiv:2209.10652. Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Con- erly, T., et al. (2021). A mathematical framework for transformer circuits. Transformer Circuits Thread. https://transformer-circuits.pub/2021/framework/index.html Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023). Dissecting recall of factual associations in auto-regressive language models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. https://arxiv.org/abs/2304.14767 Gupta, A., Yeung, J., Anumanchipalli, G., & Ivanova, A. (2025). How do llms use their depth? https : //arxiv.org/abs/2510.18871 Herrmann, D. A., & Levinstein, B. A. (2025). Standards for belief representations in LLMs. Minds and Machines, 35(5). https://doi.org/10.1007/s11023-024-09709-6 LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436â444. Lindsey, J., Gurnee, W., Ameisen, E., Chen, B., Pearce, A., Turner, N. L., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., . . . Batson, J. (2025). On the Biology of a Large Lan- guage Model. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution- graphs/biology.html 9 Nanda, N., Rajamanoharan, S., Kramar, J., & Shah, R. (2023). Fact finding: Attempting to reverse-engineer factual recall on the neuron level. Alignment Forum, 6. Olah, C., Mordvintsev, A., & Schubert, L. (2017). Feature visualization [https://distill.pub/2017/feature- visualization]. Distill. https://doi.org/10.23915/distill.00007 R Ě az, T. (2023). Methods for identifying emergent concepts in deep neural networks. Patterns, 4(6), 100761. https://doi.org/https://doi.org/10.1016/j.patter.2023.100761 Sanderson, G. (2024). How might llms store facts â deep learning chapter 7. Retrieved July 4, 2025, from https://w.youtube.com/watch?v=9-Jl0dxWQs8 Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., . . . Henighan, T. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. https:// transformer-circuits.pub/2024/scaling-monosemanticity/index.html Williams, I., Oldenburg, N., Dhar, R., Hatherley, J., Fierro, C., Rajcic, N., Schiller, S. R., Stamatiou, F., & Søgaard, A. (2025). Mechanistic interpretability needs philosophy. https://arxiv.org/abs/2506. 18852 10