Paper deep dive
Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models
Franz Motzkus, Sebastian Bernhard
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 7/8/2026, 5:40:46 AM
Summary
This paper introduces a concept-based interpretability framework for end-to-end autonomous driving models. By integrating Sparse Autoencoders (SAEs), the authors decompose opaque latent spaces into sparse, monosemantic concepts. Through circuit discovery and patching techniques, they establish causal links between these concepts and trajectory scoring metrics. Targeted concept-level interventions successfully correct erroneous behaviors and improve driving performance without retraining, demonstrating how interpretability can mitigate model opacity and enhance safety.
Entities (8)
Relation Signals (6)
Sparse Autoencoder → decomposes → End-to-End Autonomous Driving Model
confidence 92% · integrate unsupervised dictionary learning as a post hoc interpretability module within state-of-the-art driving models to decompose driving behavior into semantically meaningful concepts
End-to-End Autonomous Driving Model → generates → Trajectory Scoring
confidence 90% · connecting them to the multifaceted model outputs, thereby revealing the underlying decision-making logic for the prediction of future trajectories
Circuit Discovery → analyzes → Monosemantic Concept
confidence 88% · circuit analysis uncovers minimal computational subgraphs between neurons modeling specific model behavior
Monosemantic Concept → influences → Collision Avoidance
confidence 85% · interventions on concepts associated with erroneous behavior lead to measurable improvements in downstream driving metrics, including collision-avoidance
Monosemantic Concept → influences → Traffic Light Compliance
confidence 85% · interventions on concepts associated with erroneous behavior lead to measurable improvements in downstream driving metrics, including... traffic light compliance
Attribution Patching → isusedin → Circuit Discovery
confidence 80% · As a computation based on the removal of single neurons... is computationally not feasible, we apply a circuit computation based on neuron attributions.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The increasing adoption of end-to-end learning for autonomous driving introduces increased model complexity and opacity, raising the risk of learning undesired or erroneous behavior. In this work, we integrate unsupervised dictionary learning as a post hoc interpretability module within state-of-the-art driving models to decompose driving behavior into semantically meaningful concepts while demonstrating their causal influence on the model's driving decisions. We propose a stepwise framework for extracting and interpreting meaningful concepts from the end-to-end model and connecting them to the multifaceted model outputs, thereby revealing the underlying decision-making logic for the prediction of future trajectories. Furthermore, targeted interventions at the concept level allow us to manipulate and correct driving decisions, resulting in measurable improvements in overall driving performance. We thus demonstrate how interpretability can effectively be used to reduce model opacity, uncover erroneous behavior, and enable targeted mitigation, ultimately boosting model performance.
Tags
Links
- Source: https://arxiv.org/abs/2607.06328v1
- Canonical: https://arxiv.org/abs/2607.06328v1
Trouble viewing inline? Open PDF directly →
Full Text
66,224 characters extracted from source content.
Expand or collapse full text
Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models Franz Motzkus, Sebastian Bernhard Franz Motzkus is with AUMOVIO, Germany, and with the Department of Applied Computer Science, University of Bamberg, Germany. Abstract The increasing adoption of end-to-end learning for autonomous driving introduces increased model complexity and opacity, raising the risk of learning undesired or erroneous behavior. In this work, we integrate unsupervised dictionary learning as a post hoc interpretability module within state-of-the-art driving models to decompose driving behavior into semantically meaningful concepts while demonstrating their causal influence on the model’s driving decisions. We propose a stepwise framework for extracting and interpreting meaningful concepts from the end-to-end model and connecting them to the multifaceted model outputs, thereby revealing the underlying decision-making logic for the prediction of future trajectories. Furthermore, targeted interventions at the concept level allow us to manipulate and correct driving decisions, resulting in measurable improvements in overall driving performance. We thus demonstrate how interpretability can effectively be used to reduce model opacity, uncover erroneous behavior, and enable targeted mitigation, ultimately boosting model performance. I INTRODUCTION End-to-end learning has recently emerged as a prominent approach for autonomous driving systems. In contrast to conventional modular pipelines with explicitly separated perception, prediction, and planning components, modern approaches integrate multimodal inputs, including camera recordings and top-level commands, into unified transformer-based architectures [10]. These systems jointly learn perception, temporal reasoning, and decision-making, achieving state-of-the-art performance on open-loop benchmarks such as NAVSIM [12, 9]. However, this architectural unification comes at the cost of reduced transparency, as the model complexity increases and clear interfaces that previously enabled module-level testing disappear. Consequently, internal decision-making becomes increasingly opaque, making the analysis of failure modes more difficult and resulting in expensive error debugging through repeated data curation and retraining cycles guided mainly by best guesses. Especially in terms of safety and interpretability, the field of end-to-end autonomous driving lacks generalizing solutions for holistically describing model behavior and offering comprehensive insights into failure modes [25, 3]. Figure 1: Framework Architecture for Integrating an Interpretability Layer into End-to-End Autonomous Driving Models In this work, we address this limitation by introducing a concept-based interpretability framework for end-to-end autonomous driving. Our approach reveals interpretable latent-space components and directly connects them to trajectory-level decision scores. Specifically, we integrate dictionary learning via Sparse Autoencoders (SAEs) into the model’s feature space, decomposing latent activations into sparse, human-interpretable concepts. Rather than treating the latent space as an opaque collection of convoluted features, this decomposition provides a more transparent representation in which individual concepts correspond to meaningful patterns learned by the model. Consequently, complex driving decisions can be expressed as a composition of semantically meaningful elements with causal influence on the predicted future trajectory. This concept-based perspective offers a more interpretable view of the model’s internal reasoning process while preserving the representational capacity required for autonomous driving. To validate our approach, we perform concept-level interventions by selectively suppressing individual latent components. We show that interventions on concepts associated with erroneous behavior lead to measurable improvements in downstream driving metrics, including collision-avoidance, drivable area and traffic light compliance. This indicates that the identified concepts are not merely correlational artifacts but functionally relevant components of the model’s decision-making. The results further demonstrate how increased interpretability can be leveraged for targeted model refinement and improved overall performance. Our contributions are the following: 1. We present the first structured and generalized approach for interpreting driving decisions in end-to-end autonomous driving models, providing concept-level explanations for the scoring of individual candidate trajectories. 2. We introduce a concept-based decomposition of latent representations that identifies the most influential factors within individual driving scenarios, enabling fine-grained attribution of model decisions. 3. We establish a direct correspondence between learned concepts and the predicted quality scores for trajectories, revealing which concepts are influencing the distinct driving behaviors such as collision avoidance, traffic-light compliance and progress. 4. We demonstrate that the identified concepts can be leveraged for targeted model adaptation and manipulation, enabling improvements in downstream driving performance without the need for retraining. I RELATED WORK I-A End-to-End Autonomous Driving With the adoption of the end-to-end learning paradigm in autonomous driving AI models, multiple different approaches from imitation learning to reinforcement learning have been introduced [10]. Imitation learning methods train from (human) demonstrations via behavior cloning, mimicking the ground truth expert’s driving decisions from real-life driving recordings. Popular state-of-the-art models using imitation learning are Hydra-MDP [26] scoring on a trajectory vocabulary, and Emma [21] incorporating a language model to build a multimodal predictor. Typically, they output trajectories with estimated quality scores and the training objective is to approximate the ground truth trajectory from a real-world recording. Reinforcement learning instead typically operates in a virtual environment such as a simulator, where the model’s interaction with the environment is optimized based on a reward function. However, the gap between autonomous driving simulation and real-world driving is still a foundational problem. A group of current approaches is referred to as offline reinforcement learning, where training is based on recordings with ground-truth driving behavior, but alternative trajectories generated by the model can also be evaluated and optimized [12]. By generating possible next steps of the scenario, the influence of the environment interaction is even enhanced [9]. These models typically predict trajectories with an estimated Predictive Driver Model (PDM) score that incorporates an imitation score and additional penalty scores, including collision, drivable area, and traffic light compliance, along with progress along the ground-truth trajectory. [22] introduces a probabilistic scoring over a predefined dictionary of trajectories. Later approaches adapt the PDM scoring for a quality assessment of trajectories. Hydra-MDP [26] trains multiple diverse trajectory heads and selects among them at inference time, explicitly addressing the multimodality of human driving behavior to improve robustness under the PDM evaluation protocol. GTRS [27] builds upon the Hydra-MDP model and introduces an additional trajectory generation step via a diffusion model. Instead of a fixed-trajectory dictionary, possible trajectories are generated iteratively. iPad [17] shifts from dense BEV grid features to an iterative proposal-centric paradigm. A set of sparse, learnable trajectory proposals serves as the central organising principle: its ProFormer encoder iteratively refines proposals and their associated BEV queries through proposal-anchored attention, concentrating feature extraction on planning-relevant regions. Lightweight proposal-centric auxiliary tasks for mapping and collision prediction further improve planning quality. I-B Interpretability in End-to-End Autonomous Driving While the end-to-end training schema drastically increases the opacity of the presented models compared to classical approaches with AI-based perception and rule-based planning, multiple approaches seek to incorporate interpretability to (parts of) the model [3]. Saliency-based methods [6, 5] inspect model predictions through saliency, and gradient-based attribution (e.g. GradCAM, Integrated Gradients). These methods are widely used due to their simplicity but are limited to identifying where the model focuses within the input, providing no account of what the model has learned to represent internally. Especially in autonomous driving, where multiple factors influence the driving decision, saliency maps are limited as they highlight only the most prominent aspect. Attention visualization methods use the models’ built-in attention mechanisms to highlight the most-attended information in the front camera view [24, 36, 29, 2] or in an abstracted birds-eye-view [23, 18]. Another work optimizes the model itself to yield diverse feature maps in input space [31]. Latent space interpretability: While attention visualization reveals highly attended information, other latent space interpretability methods are rarely applied to end-to-end autonomous driving models. [32] uses latent space representations of an LSTM network as additional input for predicting driving actions in a policy network. [38] introduces a testing framework using neuron coverage, thereby testing the influence of input perturbations on intermediate representations. Neither method focuses on making the latent space more interpretable. In [19], an end-to-end autonomous driving model is specifically designed to have interpretable intermediate representations between perception, prediction and planning, yielding separate scene representations for drivable area, lane markings and traffic participants. However, the latent space representations are rather controlled with the necessity of using the specifically introduced architecture. Vision–language models and vision–language–action models (VLM/VLA) present an emerging research direction in autonomous driving [40, 39]. While these approaches open promising directions for leveraging interpretability in autonomous driving models, more in-depth analyses at the level of underlying concepts—through which the different aspects of driving decisions can be disentangled and examined—are still largely missing. To our knowledge, no other concept-based latent space interpretability approach for end-to-end autonomous driving models exists. I PRELIMINARIES I-A Sparse Autoencoder Learning Sparse Autoencoders have first shown to extract interpretable feature directions in language models [37], but also proven useful in vision models [11, 28] or even trajectory prediction [33]. The latent space of AI models is typically highly entangled, with single neurons encoding multiple unrelated features, making them non-interpretable. This polysemanticity of neurons hinders a mechanistic understanding of individual features [13], in which the functioning of single network units is explored. As neural networks often learn more features than having dimensions, this superposition can be resolved with sparse autoencoders [7]. By expanding the latent space while enforcing sparse representations, they provide a principled approach to decomposing an entangled representation into linear directions corresponding to interpretable, monosemantic features, denoted as concepts. Formally, given model activations x∈ℝdx ^d, an overcomplete dictionary D∈ℝd×mD ^d× m with m>dm>d is learned, such that x≈Dzx≈ Dz with z∈ℝmz ^m forming a sparse representation. SAEs jointly learn the dictionary D and a sparse encoder E:ℝd→ℝmE:R^d ^m by minimizing the reconstruction loss ‖x−DE(x)‖22\|x-DE(x)\|_2^2 between the activation vector x and it’s reconstruction DE(x)DE(x). To achieve sparse representations in the latent codes z, sparsity objectives like a ℓ1 _1-regularization or a top-k activation retaining only the k highest values in z are applied [20, 16]. Multiple works adapt the standard SAE training scheme to achieve a different feature distribution. Matryoshka training [8] builds on the idea of nested representations, where subsets of latent dimensions define a sequence of progressively more expressive autoencoders. Given a latent code z∈ℝmz ^m, whose coordinates correspond to m latent features, we consider prefixes z1:m1,z1:m2,…,z1:mz_1:m_1,z_1:m_2,…,z_1:m with m1<m2<⋯<m_1<m_2<…<m. Each subset of latent features defines its own reconstruction, yielding a sequence of valid autoencoders with increasing reconstruction fidelity and inducing a hierarchy in which smaller subsets capture general features, while larger subsets include progressively more specific ones. Archetypal SAEs [14] introduce additional convexity constraints on the learned dictionary. Let X∈ℝd×nX ^d× n denote a set of n∈ℕn reference data points, where each column corresponds to an input in ℝdR^d. The dictionary is parameterized as D=XAD=XA, where A∈ℝn×mA ^n× m satisfies A≥0A≥ 0 and ⊤A=⊤1 A=1 , such that each dictionary atom dj=Xajd_j=Xa_j is a convex combination of data points. Thus, in contrast to standard SAEs with unconstrained dictionaries, latent features correspond to archetypes that lie in the convex hull of the data. In practice, these archetypes can be compared to representative data prototypes and resemble cluster-like summaries of the dataset, with each dictionary atom capturing a convex combination of similar inputs. I-B SAE Evaluation We evaluate SAEs in two categories, whereas the first one is the reconstruction fidelity of the learned representations, while the other rates the alignment with desirable features based on our use case. The reconstruction quality evaluates how well the SAE preserves the information contained in the original latent embeddings. The cosine similarity measures the alignment between the models latent space representation x and its reconstruction after the SAE x x as cos(x,x^)=x⊤x^‖x‖2‖x^‖2,cos(x, x)= x x\|x\|_2\,\| x\|_2, It captures the preservation of directional information in latent space. The explained variance determines the fraction of variance in x explained by x x and is given by EV=1−Var(x−x^)Var(x),EV=1- Var(x- x)Var(x), where Var(⋅)Var(·) denotes the empirical variance over the dataset. As the investigated end-to-end models include an ego status, consisting of values for velocity, acceleration and a top-level driving direction, into their computation, we assume it to be represented in the SAE with individual features. As a quality criteria, we can thus compute how well the ego status is represented in the SAE neurons. For each ego-status variable eje_j (velocity, acceleration, driving command), we compute the Pearson correlation rn,jr_n,j between every alive SAE neuron i and eje_j across the dataset. The ego correlation score is defined as: Scorr=1|ℰ|∑j∈ℰmaxi|ri,j|S_corr= 1|E| _j _i|r_i,j| (1) where ℰE is the set of non-constant ego features. A score of 1 indicates that every ego variable is perfectly linearly represented by at least one SAE neuron. In the ego probing, we train linear probes on the SAE codes to predict ego-status variables, using a log-based train/test split (entire driving logs are held out) to prevent temporal leakage. Three probes are evaluated: (1) a logistic regression classifier for the driving command (left/straight/right), yielding test macro-F1 FcmdF_cmd; (2) a Ridge regressor for continuous ego dynamics (velocity, acceleration), yielding joint test Rego2R^2_ego; and (3) a scenario-ID classifier measuring log memorization, with accuracy asca_sc. The composite probing score is: Sprobe=0.5Fcmd+0.5Rego2−0.2max(0,asc−arandom)S_probe=0.5\,F_cmd+0.5\,R^2_ego-0.2 (0,\;a_sc-a_random) (2) The scenario penalty discourages representations that merely memorize log identity rather than encoding generalizable driving concepts. I-C Circuit Discovery Mechanistic interpretability [35] refers to the analysis of single parts of a frozen neural network to extract human-understandable components. In mechanistic interpretability, circuit analysis uncovers minimal computational subgraphs between neurons modeling specific model behavior. With the introduction of SAEs, circuits no longer operate on polysemantic neurons, but directly on the monosemantic concept level [30], amplifying their strength in revealing interpretable, human-auditable subgraphs. Multiple different algorithms for the computation of circuits exist varying in the trade-off between computation cost and accuracy. As a computation based on the removal of single neurons and a following full network evaluation is computationally not feasible, we apply a circuit computation based on neuron attributions. Given a set of SAE latent codes Z∈ℝn×mZ ^n× m and per-sample attribution scores αh,i _h,i, we identify a sparse circuit of SAE neurons causally mediating each planning head output through a four-stage pipeline. Attribution Patching. For each sample latent z in Z and head h of the model’s H outputs, the influence of neuron i is estimated via the gradient-weighted activation difference: Ih,iattr=|αh,i|¯⋅sign(α¯h,i),αh,i=∂fh(z)∂zi⋅(zi−z¯i)I^attr_h,i= | _h,i|·sign( α_h,i), _h,i= ∂ f_h(z)∂ z_i·(z_i- z_i) (3) where z¯i z_i and α¯h,i α_h,i denote averages over the dataset Z. This yields a signed influence matrix attr∈ℝH×mI^attr ^H× m. This provides a fast, approximate estimate of the causal influence of all latent features on each head simultaneously. Activation Patching. For the top-2q2q candidate features selected by attribution patching (where q denotes the target circuit size), we perform exact causal interventions. Each neuron i is individually patched to its dataset-mean baseline z¯i z_i, and the causal effect on head h is measured as: Ih,iact=z[|fh(z)−fh(z(i←z¯i))|]I^act_h,i=E_z [ |f_h(z)-f_h\! (z^(i← z_i) ) | ] (4) where fh(z)f_h(z) denotes the output of head h as a function of the latent representation z. ACDC Pruning. The influence matrix is sparsified by retaining only edges exceeding a relative threshold τ per head: ℰ=(h,i)|Ih,iactmaxi′Ih,i′act>τE= \(h,i)\; |\; I^act_h,i _i I^act_h,i >τ \ (5) This yields the final sparse circuit ℰ⊆1,…,H×1,…,mE \1,…,H\×\1,…,m\. Neuron Specificity. Each neuron’s degree of head-specificity is quantified via the Gini coefficient over its normalized influence distribution across heads: gi=1−2H−1∑k=1H−1Ck(i),Ck(i)=∑j≤kI~j,i∑jI~j,ig_i=1- 2H-1 _k=1^H-1C_k(i), C_k(i)= _j≤ k I_j,i _j I_j,i (6) where I~j,i I_j,i are the normalized influences sorted in ascending order. gi=1g_i=1 indicates a neuron exclusively influencing a single head; gi=0g_i=0 indicates uniform influence across all heads. IV METHODOLOGY To uncover the factors driving high-stakes decisions in end-to-end autonomous driving models, we propose a structured, multi-step interpretability framework. We begin by using SAEs to disentangle the latent space of an end-to-end model into separate, interpretable feature directions. This provides a lens into otherwise opaque internal representations without significantly altering the original predictive performance. Next, we systematically assign semantic meaning to these learned feature directions, establishing a connection between abstract model encodings and human-understandable concepts. Building on this, we analyze individual driving scenarios by decomposing model decisions into their underlying feature contributions. Additionally, feature activations are traced back into the input space, producing camera-based attribution maps that localize the corresponding features in each scene and provide another level of interpretability. In the subsequent step, we use circuit analysis to connect these semantic features to the model’s outputs. This enables us to directly quantify how individual features contribute to different driving properties, e.g. by inspecting which features are relevant for predicting the traffic light compliance. Finally, we leverage these insights to identify failure cases and undesired behaviors, enabling targeted mitigation strategies and improving the overall reliability of the system. IV-A Selecting an Expressive Latent Space As the first step of our framework, a reasonable latent space representation for injecting the SAE needs to be identified within the model. The criteria are as follows: • The latent space may encode a dense summary of the scene understanding, including all model inputs as observed by the model. • The model should base the prediction on the encoded scene understanding of this latent space, so that the trajectory scoring can be linked to the extracted SAE features. • The dimensionality of the latent space should ideally be of moderate size (i.e., d=256d=256) and allow for a 1D vector representation (per trajectory). In the chosen models, we choose the latent space representation directly preceding the scoring module, yielding a 1D representation per trajectory proposal. This representation is particularly suitable, as it contains the complete scene understanding while its features are directly inform the model scoring. IV-B Learning Interpretable Feature Directions In the second step, we train an SAE on the selected latent space to disentangle it into sparse, interpretable features. The SAE is trained on the frozen end-to-end model’s latent representations, minimizing the reconstruction error: ℒrec=1N∑p=1N‖xp−x^p‖22.L_rec= 1N _p=1^N \|x_p- x_p \|_2^2. (7) Sparsity is enforced by a top-k activation function on the SAE latent space activation. To prevent inactive concepts, we introduce an additional reanimation term: ℒreanim=−λr∑i∈1N∑p=1Nhp,ipre,L_reanim=- _r _i 1N _p=1^Nh^pre_p,i, (8) where D denotes neurons inactive for a predefined number of steps and λr>0 _r>0 denotes the scaling factor. The term hp,ipreh^pre_p,i describes the pre-activation of latent feature i for sample p, i.e., the value before the application of the top-k sparsifying nonlinearity. IV-C Assigning Semantic Meaning to Features Having obtained disentangled, monosemantic feature directions with the SAE, the third step focuses on assigning semantic meaning to individual SAE neurons. In this context, semantic refers to a consistent, human-understandable concept that can be expressed in natural language. We use the following strategies for revealing neuron semantics. IV-C1 Activation/Attribution Maximization Activation and attribution maximization [34, 1] are data-based approaches that extract samples from a reference dataset that maximally activate the specified neuron or produce the highest attribution, meaning the neuron is maximally used for the model’s prediction in that sample. Formally, given a neuron i and a reference dataset X, the top-k maximally activating samples are retrieved as: i∗=argmax⊆X,||=k∑x∈zi(x)S^*_i= S X,\,|S|=k _x z_i(x) (9) where zi(s)z_i(s) is the activation of neuron i on sample x. For attribution maximization, zi(x)z_i(x) is replaced by the neuron’s attribution score with respect to the model output. As explained in the next section, it captures not merely whether a neuron fires but whether it contributes to the prediction. IV-C2 SAE Neuron Attribution Input attribution methods highlight input units based on their importance towards a model output objective. We employ Concept Relevance Propagation (CRP) [1], which builds on Layer-wise Relevance Propagation (LRP) [4]. LRP is a modified gradient backpropagation method, using specific rules to preserve the semantic signal from an output objective in the gradient backpropagation. CRP extends the gradient pass with an intermediate gradient masking to yield the gradient/attribution signal for a single neuron. The use of LRP is particularly advantageous in this context, as its relevance conservation property enables precise attribution of the prediction to individual latent features, while its robustness to gradient noise facilitates effective filtering of meaningful semantic signals compared to standard gradient-based methods. In our setting, the model assigns a scalar score f(xt)f(x_t) to the latent activation xt∈ℝdx_t ^d of each trajectory proposal. The score of the selected trajectory is used to initialize the relevance propagation. Starting from this scalar output, relevance is propagated backward through the network to the SAE latent space with its representation zt=E(xt)∈ℝmz_t=E(x_t) ^m. During propagation, relevance is distributed at each layer proportionally to the contribution of each neuron: Ri=ziwi∑jzjwjRout,R_i= z_iw_i _jz_jw_jR_out, (10) where wiw_i denotes the contribution weight to the layer output, and RoutR_out is the relevance from the subsequent layer. This yields relevance scores RiR_i at the level of SAE neurons, quantifying their contribution to the selected trajectory score. The SAE neurons are then ranked according to their relevance scores RiR_i, allowing us to identify the concepts most responsible for the selected trajectory. To isolate the contribution of a single neuron i, we apply concept masking at the SAE layer during relevance propagation: R(i)=R⊙eiR^(i)=R e_i, where eie_i is the unit vector. The masked relevance is then propagated back to the model inputs and reveals highly attributed visual patterns in the camera input or the ego status corresponding to that neuron. IV-D Linking Features to Model Outputs In the fourth step, we connect the SAE features to the model predictions using circuit analysis as described in Section I-C. For end-to-end autonomous driving models predicting the PDM subscores for a set of trajectory proposals, the individual components of the PDM score can be linked to the extracted features. Depending on the user’s interest, a subset of SAE neurons and/or a selection of PDM elements can be selected for the circuit computation. The resulting circuit represents a sparse mapping between latent features and model outputs, identifying which features causally influence specific components of the prediction. The user can thus comprehend how the interpretable features influence the component scores. IV-E Manipulating the Model Logic Finally, we leverage the established feature-to-output relationships to identify failure cases and enable targeted feature-level interventions. Recalling that an SAE neuron encodes a distinct semantic feature direction, its activation can be easily controlled by globally amplifying or attenuating this feature across all samples. Suppose the model mistakenly learned a spurious feature that undesirably influences the trajectory scoring. Since the feature describes a distinct direction in the latent space, it is likely captured and encoded in an SAE neuron. The influence of this spurious feature can then be eliminated simply by zeroing the activation of that neuron, avoiding the need for costly dataset adjustments or retraining. For models that predict the individual components of the PDM score with separate heads, the relationship between SAE neurons and specific score components becomes directly accessible. Strategically placing the SAE module allows targeted interventions in the causal relations modifying which neurons influence which scoring heads. Note that the patterns learned from the data likely induce strong cross-associations between features and prediction heads that are driven by correlation rather than true causality. V RESULTS Version #Neurons k Dead Neurons (%) ↓ Cosine Similarity ↑ Explained Variance ↑ Ego Corr ↑ Ego Probing ↑ EPDMS ↑ TopK 64 16 7.8 0.9824 0.9651 0.7097 0.8226 0.485 TopK 128 32 8.6 0.9904 0.9808 0.7568 0.8451 0.418 TopK * 256 64 0.0 0.9975 0.9951 0.8454 0.9104 0.497 TopK 1024 64 6.9 0.9950 0.99 0.8010 0.8932 0.51 TopK 1024 128 4.5 0.9962 0.9923 0.7596 0.9031 0.552 Matryoshka 64 16 0.0 0.0907 0.0126 0.4749 0.7603 0.08 Matryoshka 128 32 19.5 0.9904 0.9809 0.5602 0.8409 0.441 Matryoshka 256 64 23.8 0.9949 0.9898 0.6151 0.8821 0.494 Matryoshka 1024 64 9.0 0.9952 0.9903 0.7214 0.8977 0.508 Matryoshka 1024 128 16. 0.9979 0.9957 0.7851 0.9105 0.524 Archetypal 1024 128 16.3 0.9392 0.8726 0.5318 0.4439 0.151 MatryArch 1024 128 22.1 0.9538 0.9054 0.5364 0.6402 0.238 TABLE I: Performance Evaluation for multiple trained Sparse Autoencoders with differing settings on the GTRS model. V-A Selecting the right SAE Parametrization For the first two steps of our framework (Section IV), we train multiple SAEs with varying architectures and hyperparameters on latent representations of the GTRS and iPAD model and measure their performance across several metrics. The number of active SAE neurons (#neurons−#dead_neurons\#neurons-\#dead\_neurons, where dead neurons denote units that never activate across the dataset) determines the number of learned features. The reconstruction quality of the model-internal embeddings—i.e., how well the SAE can reproduce the original latent representations—is determined by the cosine similarity and explained variance. To evaluate interpretability with respect to problem-specific information, we introduce two metrics. Ego correlation measures the alignment of ego status features (velocity, acceleration, high-level direction) to the SAE neurons, while ego probing is a combined score testing the identifiability of ego status features in the SAE neurons while simultaneously penalizing the discriminability of individual scenarios (overfitting to scenarios). Finally, the Extended Predictive Driver Model Score (EPDMS) estimates the downstream performance with the SAE injected into the model. The performance results are summarized in Table I. While SAEs in language models typically require a high expansion factor, we experience substantially less required neurons for vision-based models. In this case, it needs to be considered, that the inputs are derived from a limited number of scenarios, so that a pure scenario learning might occur, when choosing the latent space expansion too high. The critical task is to find the right parametrization enabling a level of abstraction with meaningful features being encoded in the SAE neurons and with the performance not being degraded too much. At first, the best-fitting SAE architecture needs to be determined. In our experiments, archetypal-based SAEs substantially perform worse than the TopK and the Matryoshka version. Inspecting the ego correlation and ego probing, we assume that the archetypal training rather tends to strengthen memorization effects as the NAVSIM data naturally includes multiple scenes per scenario, recorded at different time steps, or even includes scenes in the same location at different dates. The data is thus strongly correlated, which easily reflects in the pre-computed clusterings that archetypal SAEs build upon. In particular, archetypal SAEs ground their representations in dominant patterns of the data distribution, similar to cluster centroids or archetypes, such that individual neurons correspond to prototypical feature combinations. This reliance on representative data modes can amplify scenario-specific correlations, as frequently co-occurring features within a scenario are captured jointly and reinforced in the learned archetypes. The TopK and Matryoshka SAEs perform significantly better, whereas several differences between both variants can be seen. The ratio of dead neurons is comparably low for allTopK variants, while Matryoshka rather under-utilizes the available dictionary size by producing more dead neurons, and thus provides less learned features. Interestingly, most of the other scores are quite similar between both variants and vary rather based on the latent size. Only the ego correlation hints at a better separation of features related to the ego status in the TopK variants, presenting them as the best-fitting version. Considering the optimal size for the SAE latent space, we observe the following. Larger latent spaces enable the model to better recreate the model embedding, as more details can be encoded with a higher number of neurons. Especially the downstream model performance is better with larger SAEs. However, when inspecting the other metrics, the reconstruction does not necessarily improve from a certain neuron number. Especially the medium sized TopK variant yields the best reconstruction. The difference in performance is most apparent in the ego correlation and ego probing scores. With a higher number of neurons, the relevant ego status features rather get distributed across neurons, hindering a consistent, disentangled encoding. With a higher number of neurons, the probability of (re-)learning memorization effects also increases. Compressing the SAE latent space too much lowers the reconstruction and ego feature scores, as the SAE might be able to extract the most important features, but considerably deteriorates in the feature disentanglement. For our selected SAE, we aim to choose the latent space dimensionality high enough that individually important features can develop into independent components, but low enough, so that memorization effects and too strong feature dispersion are prevented. V-B Understanding Concepts of Driving Decisions Figure 2: Overview of the neuron clustering: The SAE neurons form a distribution of individual concept directions with semantically similar neurons assembling together (top). By visualizing the top-activating samples per cluster (bottom), semantic meaning can be assigned as described in Section V-B. A crucial part in using SAEs is the interpretation of neurons and feature directions, which is the third step of our framework from Section IV-C. While SAEs promise an increase of interpretability in the latent space, they do not guarantee that all neurons correspond to well-defined concepts. Consequently, constructing an exhaustive dictionary that assigns clear semantics to every neuron in the context of the application is generally not feasible. However, there are various strategies for extracting valuable insights into the latent space. We propose the following methodologies for assigning semantic meaning to neurons, that can be used interchangeably or in combination. By Neuron Maximization [34, 1] we are able to extract the samples in the reference data, which maximally activate a chosen SAE neuron in the prediction of a trajectory. Through this, we reveal patterns in the input data causing these extreme activations (Fig. 2 bottom) and, thus, we are able to relate these patterns to the concepts in the model’s feature space that correspond to the chosen SAE neuron. We directly combine the maximization with clustering neurons to group similar patterns and label them more efficiently. Neuron Clustering uses SAE activations on a reference dataset to compute similarities between the SAE neurons. Instead of clustering for the model understanding of the traffic scenes, the latent dimension of the SAE is clustered, thereby grouping neurons with similar activation behavior. This reveals shared characteristics between neurons and allows for assigning semantic meaning to neuron groups. We observe that multiple neurons activate for cars located in front of the ego vehicle, where the neurons differ in their sensitivity based on the distance to the vehicle, the type, and the color. Fig. 2 visualizes the clustering of SAE neurons together with the samples maximally activating the neurons assigned to the respective cluster. We observe distinct clusters corresponding to, for example, intersections and lane markings (cluster 0), following other traffic participants (cluster 4), and driving on straight roads (cluster 8). These clusters, together with their associated neuron encodings, provide a basis for further analysis of the model’s scene understanding—for instance, by monitoring neuron activations under controlled input variations. Moreover, they enable investigating how groups of neurons influence the prediction heads of the end-to-end model, as will be explored in Section V-D. In the Neuron Attribution (Section IV-C2), we compute the relevance for a selected trajectory—typically the predicted one by the model—towards the input. Hereby, we condition the relevance propagation on one or multiple SAE neurons, as defined by CRP [1]. The resulting relevance in the camera input highlights which parts of the input were utilized by the selected neurons when scoring the trajectory, directly showing us the information exciting the respective neuron. We use the relevance maps, which are exemplary shown in Fig. 3, as additional evidence for assigning semantic concepts to neurons. However, the relevance maps may be deceptive due to multiple reasons. First, individual neurons may encode distributed features corresponding to scene composition rather than localized concepts. They may also encode concepts like hue, lighting or weather, which is difficult to display with heatmaps; or relate to ”elsewhere”, highlighting everything but a specific concept, given that the concept is present [15]. Finally, the neuron may rather relate to other inputs like the ego status, or the semantic signal from the neuron may be superimposed in the heatmap by network effects. Especially for the GTRS, the construction of the image backbone with the wide-range image input and the usage of convolutional layers facilitates the propagation of spurious features by amplifying overlaying stride signals in the backward pass, causing high attribution in the image center possibly superimposing the semantic signal. We use Input Perturbation to analyze how neuron activations change when specific input signals are removed. In the end-to-end setting, systematically removing individual camera inputs or ego-state variables allows us to identify which neurons are most affected by each input modality. For the iPAD model, we observe that removing the backward-facing camera does not alter neuron activations, calling its contribution to the overall prediction into question. Generally, we find no measurable gradient from the model prediction to the backward camera, suggesting that the camera is not required for optimizing the model on the given training data. Consequently, the backward-facing camera could either be omitted or requires explicit mechanisms during training to enforce its utilization. This behavior may extend more broadly to models trained on the NAVSIM dataset. V-C Understanding the Composition of a Scenario Figure 3: Composition of concepts in a single sample: The top row shows the camera input and, on the right side, the BEV visualization with ground truth (green) and predicted (red) trajectory. Below, the SAE neuron attribution towards the camera input is shown for the top-4 neurons. Red indicates positively contributing pixels and and blue indicates negatively contributing pixels, with black being neutral. An overview of the top-activating neurons with their attribution is depicted in the bar-chart on the right. With our proposed integration of an SAE into the model structure, single scenes and model decisions can be further analyzed based on what is present in a scene (feature activation) and why it matters for the driving decision (feature attribution). The framework thus provides a stronger level of interpretability not only for scenes in the dataset, but also for different trajectory proposals within a scene, as the SAE-based concept decomposition runs on each trajectory proposal individually. Fig. 3 shows the camera input of a single sample (top), the referring BEV visualization with prediction and ground truth (top right), the relevant SAE neuron activations and attributions (right), and the camera attribution for the four most important neurons according to a mixture of activation and attribution. In the shown example, the rankings of top-activating and top-attributed neurons differ significantly highlighting their differing purpose. A highly activated neuron may not necessarily be used for the prediction of the final trajectory. While the neuron activation refer to the presence or recognition of features in the scene, the attribution rather highlights positive and negative influence on the selected trajectory representing how the information has been used specifically. Other trajectories may induce considerably other attribution scores matching the relation between SAE features and the driving maneuver. For an even finer distinction, the prediction heads of the GTRS model can be individually addressed to compute the attribution of neurons for single PDM components. For the selected scene, we select the most important neurons by computing the product of activation and attribution and observe the following: Neurons 191 and 170 encode for traffic participants infront of the ego vehicle. Interestingly, they highlight the ahead-driving car at the two different time steps, which are joined in the camera input as the left and right side of the image. The difference in the neurons and their attribution hints at a temporal distinction in the recognition of traffic participants. Neuron 38 generally describes the in-front-car on road layouts similar to the one in this example, while neuron 162 refers to yellow road markings at left turns. The high activation of this neuron suggests the general possibility of turning left as illustrated by the BEV visualization. However, the attribution for this neuron is logically negative for the predicted trajectory since going straight scores higher. V-D Connecting Neurons with the Trajectory Scoring Having investigated the options for extracting and interpreting meaningful features from the latent space and decomposing a single sample into its concepts, we take a step further to gain an even deeper understanding of the model. In the fourth step (Section IV-D), we determine how the SAE neurons influence the individual prediction subscores of the PDM score. The PDM score is constructed from multiple attributes that are often predicted independently by the models. These attributes include progress along the ego vehicle’s future trajectory, lane keeping, and traffic light compliance, which improve the score, while collisions caused by the ego vehicle decrease it. Specifically, the GTRS model contains a dedicated head for each attribute, enabling a head-specific evaluation of the model’s functionality. Figure 4: Circuit Visualization from SAE Neurons associated with ”Following a Car Driving Ahead” to the Prediction Heads of the GTRS Model. Circuit analysis enables the extraction of subgraphs that describe specific functionalities of the model. We combine circuit analysis with SAEs [30] in order to connect the induced SAE directly to the predicted scoring heads, allowing us to explicitly evaluate the influence of SAE neurons on the prediction heads. From our neuron clustering in Section V-B, we select the cluster with neurons that broadly activate for scenarios in which a vehicle is positioned ahead of the ego vehicle. Fig. 4 shows the circuit visualization connecting these neurons to the prediction heads of the GTRS model. The 10 top-activating samples per neuron have been used for the circuit computation. While a car in front may naively just be associated with collision-based scores, the circuit clearly displays that all scoring heads are significantly triggered by the selected neurons. Naturally, the ego progress is affected by the lead vehicle being the biggest constraint on how far the ego vehicle can advance. The drivable area and lane keeping scores are accountable, as the front vehicle naturally shows a safe trajectory, or when lane changes for overtaking are considered. For the traffic light compliance, a learned correlation may associate a stopping lead vehicle with the appearance of red lights or intersections requiring the car to slow down. Interestingly, the SAE neurons related to ”car ahead” scenarios seem to differ in how they affect the scoring heads. We assume that there are differing conceptions of the situation following a leading vehicle in the model that encode partially redundant information, but are distinguishable in how they are used by the scoring heads. Each scoring head learns a partially distinct subgraph, with some SAE neurons shared across heads while others contribute to head-specific structures. As GTRS evaluates all trajectories in its predefined vocabulary, both prediction scores and corresponding SAE activations can be computed for any candidate trajectory. In the previous analysis, we focused exclusively on the top-scoring trajectories, which naturally avoid collisions with the leading vehicle. However, recomputing the circuit for forward trajectories that are likely to result in a collision reveals substantially different influencing signals. In particular, potentially colliding trajectories induce stronger activations in the collision-related heads, reflecting a substantially increased influence of the ”car ahead” neurons. Specifically, when more risky trajectories that provoke collisions are selected, the ”car ahead” neurons exert a substantially stronger influence on the activation of the time-to-collision-within-bound and no-at-fault-collisions heads, validating the model’s intended behavior. The circuit analysis further reveals a limitation of the overall learning scheme, as influencing factors in the decision-making process are significantly shaped by correlations that, while intuitively plausible, are not causally grounded. For instance, the observed influence of the ”car ahead” neuron 58 from Fig. 4 on the traffic light compliance score reflects co-occurring patterns in the data. A vehicle closely located in front of the ego vehicle often indicates a traffic situation involving an intersection or a traffic light where the ego vehicle closely approaches the preceding vehicle. Consequently, the model associates the presence of a close leading vehicle with traffic light compliance, exploiting a statistical correlation rather than only relying on the causal factor—the actual presence of a traffic light. V-E A Remark on Model Construction and Interpretability The investigated models, iPAD and GTRS, are both predicting the PDM score on a set trajectories to select the best fitting one for the future. Interestingly, the PDM score covers multiple different characteristics of a valid trajectory incorporating the driving direction, traffic laws and collision avoidance. While the scoring function of the iPAD focuses on the overall PDM prediction, the GTRS model predicts the single subscores with individual heads. By design, splitting the subscores into separate parts allows for an individual inspection regarding the single characteristic, enabling a better interpretability. Especially in combination with our approach, the scoring of single heads can be attributed more easily to the SAE neurons, allowing for a more thorough inspection. When handling attributions from the scoring heads to the SAE neurons, an interesting contradiction arises. The GTRS heads directly reproduce the construction of the PDM score, providing one prediction head per subscore. While some heads like the ego progress are intuitively interpretable and contribute positively towards the score, other heads capture compliance with driving rules as traffic light compliance, drivable area compliance and time to collision within bound. These heads effectively predict the absence of a violation rather than the presence of clues. Interesting samples for inspection are in these cases not the top-activating samples, but the ones with the lowest scores. While vision-based AI models like object detection models are activation-based with e.g. convolutional filters activating on detected patterns, the scoring functionality of the penalty heads is rather unintuitive. A high score rather corresponds to the absence of evidence for a violation than the presence of evidence for compliance. For instance the traffic light compliance head currently yields a maximum score, if there exists no traffic light in the scene. This construction of the scoring heads collides with the principles of activation-based AI models impeding especially the use of attribution-based methods for interpretability, as they typically extract high-activating signals best. In the construction of future models, we suggest to incorporate our thoughts on interpretability and explore the activation-based approach of scoring violations and compensating for the inversion in the post-processing. V-F Manipulating Driving Decisions Beyond a lense into the model’s functioning and single decision processes, integrating an SAE provides us the opportunity to modify the learned feature influence and thereby achieve an improved closed-loop behavior. As the last step of our framework (Section IV-E), we perform a targeted manipulation by masking the activation of selected SAE neurons and re-evaluate the predicted trajectory on the manipulated reconstructions. Note that the evaluation is based on the objective scoring instance, rather than the model’s predicted scores. Table I summarizes the aggregated scores on the NAVSIM evaluation dataset, when our manipulation is applied. We select three neurons based on our circuit-level analysis. In particular, neuron 177177 is identified having a negative influence across multiple scoring heads, suggesting potentially malicious behavior. Both neurons 5959 and 7171 show a close correlation to neuron 177177, while also having remarkable influence on multiple heads. By removing only the three selected neurons 59,7159,71 and 177177, the overall EPDMS score of the model improves significantly by 0.0960.096 points compared to the GTRS version with the unchanged SAE included. Since the injection of the SAE slightly decreases the performance, we additionally compare to the original baseline model without the SAE. Notably, the manipulated model surpasses the baseline, achieving an EPDMS of 0.59260.5926 compared to 0.5240.524. Investigating the individual components of the score, we measure improvements in nearly all driving metrics. Especially the scores for drivable area compliance and driving direction improve substantially. The collision-based scores improve slightly less, which may also relate to some extent to the manipulated model rather staying within the drivable area. The only score that is negatively affected is the ego progress hinting at the general behavior of the car to be more conservative and trade rapid progress for increased safety. TABLE I: Effect of combined zero-ablation of SAE neurons 177,59,71\177,59,71\ on the EPDMS score and its sub-metrics. “Baseline” denotes the model with unmodified SAE, “Ablated” the manipulated model, and Δ their difference. Positive values indicate improvement. Original EPDMS without SAE: 0.524 Metric Baseline Ablated EPDMS score 0.4963 0.5926 +0.0963+0.0963 PDM score 0.6047 0.6897 +0.0850+0.0850 Ego progress 0.6378 0.5378 −0.1000-0.1000 Time-to-collision w/i bound 0.9004 0.9667 +0.0663+0.0663 No at-fault collisions 0.9179 0.9722 +0.0543+0.0543 Drivable area compliance 0.7457 0.8754 +0.1298+0.1298 Driving direction compliance 0.8943 0.9645 +0.0702+0.0702 Traffic light compliance 0.9987 0.9989 +0.0002+0.0002 Inspecting the comparative examples in Fig. 5, the assumed change of behavior in the model is affirmed, as risky trajectories with high progress are replaced by more moderate trajectories that favor the current driving lane instead of maximal progress. Figure 5: Comparison of predictions from the original and the manipulated model in selected examples. VI CONCLUSION End-to-end autonomous driving models are increasingly approaching real-world deployment; however, their lack of interpretability remains a critical challenge for model validation, profound error analysis, and targeted improvement. In this work, we introduce a thorough, comprehensive framework for leveraging interpretability in such models by enabling access to their latent representations and explicitly linking learned concepts to prediction scores. We show how human-understandable concepts can be extracted, analyzed and how they interoperate with the different prediction heads for selecting future trajectories. Through this analysis, we provide a means to validate whether the model bases its decisions on meaningful semantic concepts and reasoning processes. We reveal how the predictions of end-to-end autonomous driving models are frequently affected by learned correlations that do not necessarily reflect causal coherence, thereby facilitating a deeper understanding or model failures and undesired behaviors. Finally, we show how to translate insights from the previous analyses into targeted interventions by providing an approach to easily adapt model inference at test time without retraining. The modified model exhibits a clear performance gain in the EPDMS, replacing risky progress-oriented behavior with safer and more compliant trajectories. Illustrating how interpretability can support in model validation, debugging, and refinement, we believe this work represents a step toward more transparent, trustworthy, and controllable autonomous driving systems. Concurrently, our results highlight the need for further research into the discovery, characterization, and causal understanding of learned concepts in autonomous driving models. While the proposed framework provides a foundation for concept-based analysis and intervention, fully understanding the semantic concepts and their interaction in the decision-making process remains a largely open research problem. Acknowledgments The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Energy within the project “Safe AI Engineering–Sicherheitsargumentation befähigendes AI Engineering über den gesamten Lebenszyklus einer KI-Funktion”. The authors would like to thank the consortium for the successful cooperation. References [1] R. Achtibat, M. Dreyer, I. Eisenbraun, S. Bosse, T. Wiegand, W. Samek, and S. Lapuschkin (2023-09-01) From attribution maps to human-understandable explanations through concept relevance propagation. Nature Machine Intelligence 5 (9), p. 1006–1019. External Links: ISSN 2522-5839, Document Cited by: §IV-C1, §IV-C2, §V-B, §V-B. [2] J. Araluce, L. M. Bergasa, M. Ocaña, Á. Llamazares, and E. López-Guillén (2024) Leveraging driver attention for an end-to-end explainable decision-making from frontal images. IEEE Transactions on Intelligent Transportation Systems 25 (8), p. 10091–10102. External Links: Document Cited by: §I-B. [3] S. Atakishiyev, M. Salameh, and R. Goebel (2025) Safety implications of explainable artificial intelligence in end-to-end autonomous driving. IEEE Transactions on Intelligent Transportation Systems 26 (10), p. 14516–14535. External Links: Document Cited by: §I, §I-B. [4] S. Bach, A. Binder, G. Montavon, F. Klauschen, K. Müller, and W. Samek (2015-07) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE 10 (7), p. 1–46. External Links: Document Cited by: §IV-C2. [5] M. Bojarski, A. Choromanska, K. Choromanski, B. Firner, L. J. Ackel, U. Muller, P. Yeres, and K. Zieba (2018) VisualBackProp: efficient visualization of cnns for autonomous driving. In 2018 IEEE International Conference on Robotics and Automation (ICRA), Vol. , p. 4701–4708. External Links: Document Cited by: §I-B. [6] M. Bojarski, P. Yeres, A. Choromanska, K. Choromanski, B. Firner, L. D. Jackel, and U. Muller (2017) Explaining how a deep neural network trained with end-to-end learning steers a car. Vol. abs/1704.07911. External Links: 1704.07911 Cited by: §I-B. [7] T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah (2023) Towards monosemanticity: decomposing language models with dictionary learning. Transformer Circuits Thread. Cited by: §I-A. [8] B. Bussmann, N. Nabeshima, A. Karvonen, and N. Nanda (2025) Learning multi-level features with matryoshka sparse autoencoders. In Forty-second International Conference on Machine Learning, Cited by: §I-A. [9] W. Cao, M. Hallgarten, T. Li, D. Dauner, X. Gu, C. Wang, Y. Miron, M. Aiello, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta (2025) Pseudo-simulation for autonomous driving. Vol. abs/2506.04218. External Links: Document, 2506.04218 Cited by: §I, §I-A. [10] L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li (2024) End-to-end autonomous driving: challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (12), p. 10164–10183. External Links: Document Cited by: §I, §I-A. [11] G. Daujotas (2024-06-20)Interpreting and steering features in images(Website) LessWrong. External Links: Link Cited by: §I-A. [12] D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta (2024) NAVSIM: data-driven non-reactive autonomous vehicle simulation and benchmarking. In Advances in Neural Information Processing Systems, Vol. 37, p. 28706–28719. External Links: Document Cited by: §I, §I-A. [13] N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah (2022) Toy models of superposition. Transformer Circuits Thread. Cited by: §I-A. [14] T. Fel, E. S. Lubana, J. S. Prince, M. Kowal, V. Boutin, I. Papadimitriou, B. Wang, M. Wattenberg, D. E. Ba, and T. Konkle (2025-13–19 Jul) Archetypal SAE: adaptive and stable dictionary learning for concept extraction in large vision models. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 16543–16572. Cited by: §I-A. [15] T. Fel, B. Wang, M. A. Lepori, M. Kowal, A. Lee, R. Balestriero, S. Joseph, E. S. Lubana, T. Konkle, D. E. Ba, and M. Wattenberg (2025) Into the rabbit hull: from task-relevant concepts in DINO to minkowski geometry. Vol. abs/2510.08638. External Links: Document, 2510.08638 Cited by: §V-B. [16] L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu (2025) Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations, Cited by: §I-A. [17] K. Guo, H. Liu, X. Wu, J. Pan, and C. Lv (2025) IPad: iterative proposal-centric end-to-end autonomous driving. Vol. abs/2505.15111. External Links: Document, 2505.15111 Cited by: §I-A. [18] D. Holtz, N. Hanselmann, S. Doll, M. Cordts, and B. Schiele (2026) What matters for scalable and robust learning in end-to-end driving planners?. Vol. abs/2603.15185. External Links: Document, 2603.15185 Cited by: §I-B. [19] S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao (2022) ST-p3: end-to-end vision-based autonomous driving via spatial-temporal feature learning. In Computer Vision – ECCV 2022, p. 533–549. External Links: ISBN 978-3-031-19839-7 Cited by: §I-B. [20] R. Huben, H. Cunningham, L. R. Smith, A. Ewart, and L. Sharkey (2024) Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, Cited by: §I-A. [21] J. Hwang, R. Xu, H. Lin, W. Hung, J. Ji, K. Choi, D. Huang, T. He, P. Covington, B. Sapp, Y. Zhou, J. Guo, D. Anguelov, and M. Tan (2025) EMMA: end-to-end multimodal model for autonomous driving. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §I-A. [22] B. Jiang, S. Chen, H. Gao, B. Liao, Q. Zhang, W. Liu, and X. Wang (2026) VADv2: end-to-end vectorized autonomous driving via probabilistic planning. External Links: 2402.13243 Cited by: §I-A. [23] J. Kim and M. Bansal (2020-06) Attentional bottleneck: towards an interpretable deep driving network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Cited by: §I-B. [24] J. Kim and J. Canny (2017) Interpretable learning for self-driving cars by visualizing causal attention. In 2017 IEEE International Conference on Computer Vision (ICCV), Vol. , p. 2961–2969. External Links: Document Cited by: §I-B. [25] A. Kuznietsov, B. Gyevnar, C. Wang, S. Peters, and S. V. Albrecht (2024) Explainable ai for safe and trustworthy autonomous driving: a systematic review. IEEE Transactions on Intelligent Transportation Systems 25 (12), p. 19342–19364. External Links: Document Cited by: §I. [26] Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y. Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wu, et al. (2024) Hydra-mdp: end-to-end multimodal planning with multi-target hydra-distillation. Cited by: §I-A, §I-A. [27] Z. Li, W. Yao, Z. Wang, X. Sun, J. Chen, N. Chang, M. Shen, Z. Wu, S. Lan, and J. M. Alvarez (2025) Generalized trajectory scoring for end-to-end multimodal planning. External Links: 2506.06664 Cited by: §I-A. [28] H. Lim, J. Choi, J. Choo, and S. Schneider (2025) Sparse autoencoders reveal selective remapping of visual concepts during adaptation. In The Thirteenth International Conference on Learning Representations, Cited by: §I-A. [29] Y. Liu, Y. Hsieh, M. Chen, C.-H. H. Yang, J. Tegner, and Y.-C. J. Tsai (2020) Interpretable self-attention temporal reasoning for driving behavior understanding. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , p. 2338–2342. External Links: Document Cited by: §I-B. [30] S. Marks, C. Rager, E. J. Michaud, Y. Belinkov, D. Bau, and A. Mueller (2025) Sparse feature circuits: discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations, Cited by: §I-C, §V-D. [31] M. Mirzaie and B. Rosenhahn (2025-10) Interpretable decision-making for end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, p. 805–815. Cited by: §I-B. [32] J. Morton and M. J. Kochenderfer (2017) Simultaneous policy learning and latent state inference for imitating driver behavior. In 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Vol. , p. 1–6. External Links: Document Cited by: §I-B. [33] F. Motzkus, C. Schlauch, S. Bernhard, and U. Schmid (2025-10) Can i trust my trajectory prediction model?. In Proceedings of the TRUST-AI Workshop: European Workshop on Trustworthy AI, co-located with the European Conference on Artificial Intelligence (ECAI 2025), CEUR Workshop Proceedings, Bologna, Italy. Cited by: §I-A. [34] C. Olah, A. Mordvintsev, and L. Schubert (2017) Feature visualization. Distill. External Links: Document Cited by: §IV-C1, §V-B. [35] N. Saphra and S. Wiegreffe (2024) Mechanistic?. Vol. abs/2410.09087. Cited by: §I-C. [36] A. Sauer, N. Savinov, and A. Geiger (2018-29–31 Oct) Conditional affordance learning for driving in urban environments. In Proceedings of The 2nd Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 87, p. 237–252. Cited by: §I-B. [37] A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, C. McDougall, M. MacDiarmid, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan (2024) Scaling monosemanticity: extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread. Cited by: §I-A. [38] Y. Tian, K. Pei, S. Jana, and B. Ray (2018) DeepTest: automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th International Conference on Software Engineering, ICSE ’18, New York, NY, USA, p. 303–314. External Links: ISBN 9781450356381, Document Cited by: §I-B. [39] X. Zhou, X. Han, F. Yang, Y. Ma, V. Tresp, and A. Knoll (2026) Opendrivevla: towards end-to-end autonomous driving with large vision language action model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 13782–13790. Cited by: §I-B. [40] Z. Zhou, T. Cai, S. Zhao, Y. Zhang, Z. Huang, B. Zhou, and J. Ma (2025) AutoVLA: a vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. In Advances in Neural Information Processing Systems, Vol. 38, p. 27920–27956. Cited by: §I-B.