Paper deep dive
Regional Explanations: Bridging Local and Global Variable Importance
Salim I. Amoukou, Nicolas J-B. Brunel
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/14/2026, 2:32:07 AM
Summary
The paper identifies fundamental limitations in popular local attribution methods, Local Shapley Values (L-SV) and LIME, specifically their tendency to assign importance to irrelevant features even under ideal conditions. To address this, the authors propose R-LOCO (Regional Leave Out COvariates), a method that bridges local and global explanations by partitioning the input space into regions of similar feature importance and applying global attribution methods within those regions, resulting in more faithful and statistically sound local explanations.
Entities (5)
Relation Signals (3)
R-LOCO → addresseslimitationsof → Local Shapley Values
confidence 95% · To address this, we propose R-LOCO... which bridges the gap between local and global explanations
R-LOCO → addresseslimitationsof → LIME
confidence 95% · We demonstrate that both Local SV and LIME violate this fundamental principle. To address this, we propose R-LOCO
R-LOCO → utilizes → LOCO
confidence 90% · R-LOCO (Regional Leave Out COvariates)... derives an instance's feature contributions from its regional membership.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We analyze two widely used local attribution methods, Local Shapley Values and LIME, which aim to quantify the contribution of a feature value $x_i$ to a specific prediction $f(x_1, \dots, x_p)$. Despite their widespread use, we identify fundamental limitations in their ability to reliably detect locally important features, even under ideal conditions with exact computations and independent features. We argue that a sound local attribution method should not assign importance to features that neither influence the model output (e.g., features with zero coefficients in a linear model) nor exhibit statistical dependence with functionality-relevant features. We demonstrate that both Local SV and LIME violate this fundamental principle. To address this, we propose R-LOCO (Regional Leave Out COvariates), which bridges the gap between local and global explanations and provides more accurate attributions. R-LOCO segments the input space into regions with similar feature importance characteristics. It then applies global attribution methods within these regions, deriving an instance's feature contributions from its regional membership. This approach delivers more faithful local attributions while avoiding local explanation instability and preserving instance-specific detail often lost in global methods.
Tags
Links
- Source: https://arxiv.org/abs/2604.11223v1
- Canonical: https://arxiv.org/abs/2604.11223v1
Trouble viewing inline? Open PDF directly →
Full Text
118,633 characters extracted from source content.
Expand or collapse full text
Regional Explanations: Bridging Local and Global Variable Importance Salim I. Amoukou J.P. Morgan AI Research &Nicolas J-B. Brunel LaMME, ENSIIE, University Paris Saclay & Capgemini Invent Correspondence to: Salim I. Amoukou <salim.ibrahimamoukou@jpmorgan.com> Abstract We analyze two widely used local attribution methods, Local Shapley Values and LIME, which aim to quantify the contribution of a feature value xix_i to a specific prediction f(x1,…,xp)f(x_1,…,x_p). Despite their widespread use, we identify fundamental limitations in their ability to reliably detect locally important features, even under ideal conditions with exact computations and independent features. We argue that a sound local attribution method should not assign importance to features that neither influence the model output (e.g., features with zero coefficients in a linear model) nor exhibit statistical dependence with functionality-relevant features. We demonstrate that both Local SV and LIME violate this fundamental principle. To address this, we propose R-LOCO (Regional Leave Out COvariates), which bridges the gap between local and global explanations and provides more accurate attributions. R-LOCO segments the input space into regions with similar feature importance characteristics. It then applies global attribution methods within these regions, deriving an instance’s feature contributions from its regional membership. This approach delivers more faithful local attributions while avoiding local explanation instability and preserving instance-specific detail often lost in global methods. 1 Introduction Machine learning models are valued for their predictive capabilities, but often lack transparency. This opacity poses issues in high-stakes domains such as healthcare, finance, and criminal justice, where understanding the “why” behind a decision is as critical as the decision itself. In response, eXplainable AI (XAI) has emerged, developing methods and tools to explain model predictions. These tools can be categorized into local and global methods. Local explanations aim to provide insights into individual predictions, while global explanations focus on understanding the overall behavior of a model across the entire input space. Popular local methods, such as Local Shapley Values L-SV (Lundberg and Lee, 2017a) and LIME (Ribeiro et al., 2016a), seek to create a local linear approximation of the model within the vicinity of a given instance. In contrast, global methods primarily consist of "leave-one-out" approaches (Lei et al., 2016; Williamson and Feng, 2020; Covert et al., 2020b; Gan et al., 2022), which assess performance changes when a variable is excluded. In this paper, we focus on local attribution methods, specifically L-SV and LIME, which are widely used but have limited theoretical understanding. While the intuition behind these methods are appealing, the exact quantities they estimate remain unclear. Apart from the case of linear or additive models (Bordt and von Luxburg, 2023; Garreau and Luxburg, 2020), there is little work explaining the specific quantities these methods compute. A theoretical study on LIME, by Garreau and Luxburg (2020), shows that in the case of a linear model, the LIME coefficients are proportional to the partial derivatives. However, it also reveals that the coefficient of important variables can vanish by simply changing a parameter of the method. Beyond these established concerns, we highlight a fundamental limitation of L-SV and LIME that is orthogonal to the well-known issue of being “true to the data vs true to the model” (Chen et al., 2020). Building on our conclusion in Amoukou et al. (2022), we posit that a reliable local attribution should not assign importance to features that neither influence the model output functionally nor exhibit statistical dependence on features that do. Even under ideal conditions—with independent features, exact computations, and no estimation error—we show that both L-SV and LIME violate this principle by attributing importance to irrelevant features. This highlights an often-overlooked vulnerability in these popular methods, distinct from issues of approximation error or feature correlation. In contrast, most global methods are supported by the existing literature on feature importance (Breiman, 2001; Lei et al., 2016; Williamson and Feng, 2020) and global sensitivity analysis (Iooss and Lemaitre, 2015), and are backed by strong consistency and inference results. Our goal is to leverage global methods to define more accurate local attributions for each individual while benefiting from the advantages of global methods. We aim to find a partition of the input space where observations in each cell of the partition exhibit the same behavior concerning the underlying model. Then, we associate each observation with the global importance conditional on the cell it belongs to, hence deriving an attribution that is more locally faithful and possesses sound statistical properties. Notations. Consider a dataset represented as n=(i,Yi)i=1nD_n=\( X_i,Y_i)\_i=1^n, where i=(Xi1,…,Xip)∈⊆ℝp X_i=(X_i1,…,X_ip) ^p denotes the input variables and Yi∈⊆ℝY_i represents the output, and (i,Yi)( X_i,Y_i) are i.i.d. observations of (,Y)∼P=PPY|( X,Y) P=P_ XP_Y| X. We use S=(Xi)i∈S X_S=(X_i)_i∈ S to denote the subset of features, and [p]=1,…,p[p]=\1,…,p\, and (D)P(D) represents the power set of a set D. 2 Limitations of Local Shapley Values A cooperative game is a pair (D,v)(D,v), where D=X1,…,XpD=\X_1,…,X_p\ represents a set of p players, and v:(1,…,p)→ℝv:P(\1,…,p\) denotes a value function that assigns a value to every possible coalition of players, reflecting the worth of each group. The value function v is generally assumed to be positive and increasing monotonically (Dubey and Weber, 1977), which means that if A⊆BA B, then v(A)≤v(B)v(A)≤ v(B). Here, v(A)v(A) represents the value of A X_A. A key concept in the definition of SV is the marginal contribution, denoted as Δi(S)=v(S∪i)−v(S) _i(S)=v(S∪ i)-v(S). The marginal contribution is the improvement of the value of a coalition S when a given player i is added to the coalition. The SV of XiX_i is the weighted average of the marginal contributions of XiX_i across all subsets, expressed as: ϕXi=∑S⊆D∖iw(S)Δi(S)=1p∑S⊆D∖i(|D|−1|S|)−1[v(S∪i)−v(S)]. _X_i= _S D \i\w(S) _i(S)\ = 1p _S D \i\ |D|-1|S|^-1 [v(S∪ i)-v(S) ]. We can establish feature importance, by defining the value function. In the global sensitivity literature, a frequently used value function is v(S)=([Y|S])/(Y)v(S)=V(E[Y| X_S])/V(Y), which represents the explained variance or the variance of the best approximation of Y given S X_S. This value function is nonnegative and monotonically increasing, resulting in a positive global importance measure. When features are independent, this SV is closely related to the functional ANOVA decomposition (Efron and Stein, 1981; Hoeffding, 1948) and Sobol indices (Sobol’, 1990; Chastaing et al., 2012; Hooker, 2007). The resulting SV are commonly referred to as Shapley Effects (Owen, 2014; Owen and Prieur, 2017). In contrast to the global Shapley Values (SV) approach or Shapley Effects, Lundberg and Lee (2017b); Lundberg et al. (2020) adopt the game theory paradigm to explain a specific prediction f(x1,…,xp)f(x_1,…,x_p) with players D=X1=x1,…,Xp=xpD=\X_1=x_1,…,X_p=x_p\ using the value function v(S)=[f()|S=S]v(S)=E[f( x)| X_S= x_S] or v(S)=[f(S,S¯)]v(S)=E[f( x_S, X_ S)]. Although debates continue over the choice between these two value functions (Heskes et al., 2020; Janzing et al., 2020; Chen et al., 2020), we assume in this work that the variables are independent, making these value functions equivalent. We refer to the resulting Shapley Values in this context as Local Shapley Values (L-SV). A key distinction between the global SV approach (Shapley Effects) and the local approach L-SV hinges on the definitions of their respective value functions. While the global value function v(S)=([Y|S])/(Y)v(S)=V(E[Y| X_S])/V(Y) serves as an effective measure of the predictive power of variables S X_S for the overall model, it is unclear whether the local value function v(S)=[f()|S=S]v(S)=E[f( x)| X_S= x_S] genuinely represents the predictive power of S=S X_S= x_S for the specific prediction f()f( x). In the global case, a high value of v(S)=([Y|S])/(Y)v(S)=V(E[Y| X_S])/V(Y) indicates a strong predictive power of S X_S, while the values taken by v(S)=[f()|S=S]v(S)=E[f( x)| X_S= x_S] in the local case do not have any intuitive order, i.e. a high or low value of v(S)v(S) does not necessarily reflect the importance of S=S X_S= x_S for the prediction f()f( x) in regression problems, for example. Moreover, the value function of L-SV, v(S)=[f()|S=S]v(S)=E [f( X)| X_S= x_S ], can be negative and does not satisfy the monotonic property, which may result in negative L-SV. Therefore, a cancelling effect can occur, where a variable that influences the decision ends up with a zero or low Shapley Value because the Δi(S) _i(S) values across all subsets cancel each other out. It is important to note that Shapley Effects do not encounter the issues mentioned above, as they satisfy the non-negativity criterion suggested for feature importance (Johnson and LeBreton, 2004; Grömping, 2007; Feldman, 2005). In fact, Feldman (2005) emphasized that an importance measure should be positive, as it evaluates the relative information a variable contributes to the model, and information is inherently non-negative. While the global SV can be interpreted as percentages of the output’s variance (Idrissi, 2024), the quantities estimated by the local SV are less clear. For example, what is the L-SV for a particular X1=x1X_1=x_1 really telling us about its contribution to the specific prediction f(x1,…,xp)f(x_1,…,x_p)? Lastly, there is no strong justification for calculating contributions across all possible subsets, as some of these subsets might be poor predictors, thus introducing noise into the feature importance. The average performance of a feature across all submodels may not be indicative of the particular performance of that feature in the set of optimal submodels. Additionally, averaging over all subsets tends to reduce the local aspect of the contribution. To demonstrate this, let’s assume we have independent variables ∈ℝp X ^p, and a piece-wise linear predictor f defined as: f(X)=(a1X1+a2X2)X6≤0+(a3X3+a4X4)X6>0. f(X)=(a_1X_1+a_2X_2) 1_X_6≤ 0+(a_3X_3+a_4X_4) 1_X_6>0. (1) Even if we choose an observation =(x1,…,xp) x=(x_1,…,x_p) such that x6≤0x_6≤ 0 and the predictor only uses x1,x2x_1,x_2, the L-SV of ϕx3,ϕx4 _x_3, _x_4 is not necessarily zero. Using straightforward calculations, we can show that for all i∈3,4i∈\3,4\, ϕxi=K(ai(i−[i])) _x_i=K (a_i( x_i-E[ X_i]) ) where K∝ℙ(X6>0)K (X_6>0) is a constant. This highlights that the L-SV are not purely local measures but also exhibit global influences. This occurs because when calculating the L-SV of X3=x3X_3=x_3 or X4=x4X_4=x_4, we also consider subsets S that do not contain X6X_6. By marginalizing and changing the sign of X6X_6, we use the other linear model not used for this observation. We can extend the result above to show a similar issue with continuous piece-wise linear function. This class of functions is quite versatile, as it encompasses neural networks with piece-wise linear activations such as ReLU or hard tanh which correspond to max(0,x) (0,x) and max(−1,min(1,x)) (-1, (1,x)) respectively. Indeed, we can view feedforward neural networks as piece-wise linear functions that divide the input space into multiple linear regions, where the network itself behaves as an affine function within each region (Pascanu et al., 2013; Hanin and Rolnick, 2019a, b; Chen et al., 2022). The important local variables of this model correspond to the coefficients of the linear model associated with the region AkA_k to which the observation belongs. However, in the following, we provide a result demonstrating that these explanations cannot be retrieved using Local Shapley Values. Theorem 2.1 (Local Shapley Values for Piecewise Linear Models). Let f:→ℝf:X be a piecewise linear function defined on a feature space ⊆ℝpX ^p, which is partitioned into m disjoint hyperrectangles, Akk=1m\A_k\_k=1^m, where each Ak=⨂i=1p[li,k,ri,k]A_k= _i=1^p[l_i,k,r_i,k] with li,k,ri,k∈ℝ∪−∞,∞l_i,k,r_i,k ∪\-∞,∞\. On each region AkA_k, the function is defined by a linear model fk()=∑i=1pai,kXi+bkf_k(X)= _i=1^pa_i,kX_i+b_k. The complete function is thus f()=∑k=1mfk()∈Akf(X)= _k=1^mf_k(X) 1_X∈ A_k. Consider an observation =(x1,…,xp)x=(x_1,…,x_p) located in a specific region Ak⋆A_k for some k⋆∈1,…,mk ∈\1,…,m\. The model’s prediction is therefore determined solely by the local model fk⋆f_k , i.e., f()=fk⋆()f(x)=f_k (x), and the set of truly local active features is Jk⋆=i∈[p]∣ai,k⋆≠0J_k =\i∈[p] a_i,k ≠ 0\. We also define the set of features important globally G=i∈[p]∣∃k s.t. Ai,k≠ℝG=\i∈[p] ∃ k s.t. A_i,k \. Let the covariates XiX_i be mutually independent. The Local Shapley Value (SV) for the feature-value Xl=xlX_l=x_l is given by ϕxl=∑k=1mϕxlk _x_l= _k=1^m _x_l^k, where each component ϕxlk _x_l^k is: ϕxlk=(xl∈Al,kℙ(Xl∈Al,k)−1)∑S⊆D∖lw(S)vk(S)+al,k(xl−[XlXl∈Al,k]ℙ(Xl∈Al,k))∑S⊆D∖lw(S)∏i∈S∪lxi∈Ai,k∏j∈D∖(S∪l)ℙ(Xj∈Aj,k) split _x_l^k=& ( 1_x_l∈ A_l,kP(X_l∈ A_l,k)-1 ) _S D \l\w(S)v_k(S)\\ &+a_l,k (x_l- E [X_l 1_X_l∈ A_l,k ]P(X_l∈ A_l,k) ) _S D \l\w(S) _i∈ S∪\l\ 1_x_i∈ A_i,k _j∈ D (S∪\l\)P(X_j∈ A_j,k) split (2) with w(S)=1p(p−1|S|)−1w(S)= 1p p-1|S|^-1, and vk(S)=[fk()∈Ak∣S=S]v_k(S)=E[f_k(X) 1_X∈ A_k _S=x_S]. The key implication of Equation (2) is that Local SVs can assign importance to a feature j even if that feature is locally irrelevant (i.e., j∉Jk⋆j∉ J_k ) and globally irrelevant (i.e., j∉Gj∉ G). See the proof in the Appendix A. Theorem 2.1 demonstrated that SV also present difficulties in expressing local importance measures for neural networks with piece-wise linear activation layers and, more generally, for continuous piece-wise linear functions. 3 Limitations of LIME The core idea of LIME (Ribeiro et al., 2016a) is to approximate a complex model with a simpler, interpretable model (e.g., a linear model) in the neighborhood of an instance ⋆ x . However, LIME has several limitations due to its dependence on heuristic choices, such as the sampling distribution and the definition of locality. The typical sampling distribution ignores feature dependencies, which can lead to inconsistent explanations. Moreover, defining the neighborhood π⋆h _ x ^h (e.g., using a Gaussian kernel) and tuning the kernel width h is challenging, particularly in high-dimensional settings, which affects the stability and reliability of the explanations. LIME also exhibits instability, with different runs producing varying results due to the randomness in synthetic sample generation. Several studies (Zafar and Khan, 2019; Visani et al., 2020; Zhang et al., 2019; Alvarez-Melis and Jaakkola, 2018; Zhou et al., 2021) highlight these issues. Additionally, we have also demonstrated that LIME, like L-SV, struggles to detect the locally important variables of the piece-wise model as in (1). For more detailed analysis and further empirical analysis on these limitations, including sensitivity to bandwidth and reproducibility issues, please refer to the Appendix D. 4 From Global Explanations to Regional Explanations In this section, we follow the presentation of Williamson et al. (2021), which introduces a general framework for global variable importance. We consider a comprehensive class ℱF of functions mapping from X to Y, and ℱ−jF_-j be the subset of ℱF containing all functions that disregard the variable XjX_j. The conformity score V(f(),Y)V(f( X),Y) assesses the predictiveness of a prediction function f∈ℱf on the observation (,Y)( X,Y), where a high value implies high predictiveness. We define the oracle predictor with respect to the conformity score V and distribution P=PPY|P=P_ XP_Y| X as follows: f0=argmaxf∈ℱP[V(f(),Y)]. f_0= _f E_P [V (f( X),Y ) ]. In a similar manner, we define f−jf_-j as the function maximizing P[V(f(),Y)]E_P[V (f( X),Y )] over all f∈ℱ−jf _-j. Subsequently, we define the population-level importance for the variable XjX_j as the decrease in predictiveness when excluding XjX_j from =(X1,…,Xp) X=(X_1,…,X_p). This is commonly referred to as the Leave Out COvariates (LOCO) importance in existing literature (Lei et al., 2016; Verdinelli and Wasserman, 2024). The LOCO importance for XjX_j is defined as: Ψj(P)=P[Δj(,Y)]whereΔj(,Y)=V(f0(),Y)−V(f−j(),Y). _j(P)=E_P [ _j( X,Y) ] _j( X,Y)=V (f_0( X),Y )-V (f_-j( X),Y ). We can use any conformity score to measure variable importance, depending on the problem. For regression tasks, we can use V(f(),Y)=1−[Y−f()]2/σ2V(f( X),Y)=1-[Y-f( X)]^2/σ^2, where σ2=P[Y−P[Y]]2σ^2=E_P [Y-E_P[Y] ]^2 represents the variance of the target variable Y. This conformity score corresponds to the traditional R2R^2 score at the population level, i.e., R2=P[V(f(),Y)]R^2=E_P[V (f( X),Y )]. Alternatively, for binary classification problems, we can use V(f(),Y)=Y=f()V(f( X),Y)= 1_Y=f( X), which corresponds to the accuracy score at the population level. Regarding the choice of the conformity score, there is no ground truth for variable importance as there are multiple definitions of what makes a variable important (Hooker et al., 2019; Hama et al., 2022; Verdinelli and Wasserman, 2024). Consequently, the choice of a conformity score should be contingent upon the specific context and goals of the analysis. Our approach aims to derive local explanations for a specific observation (,Y)( X,Y) from the population-level importance Ψj(P) _j(P). This involves identifying a partition ∪iAi= _iA_i=X containing observations with similar explanations, which means P(Δj(,Y)|∈Ai)≈0V_P( _j( X,Y)\;|\; X∈ A_i)≈ 0 for all j∈1,…,pj∈\1,…,p\ simultaneously. In other words, the observations within each partition have low variance in their feature importance. As a result, we can use Ψj(PAi)=PAi[Δj(,Y)]=P[Δj(,Y)|∈Ai] _j(P_A_i)=E_P_A_i [ _j( X,Y) ]=E_P [ _j( X,Y)\;|\; X∈ A_i ] as local explanations for all observations in AiA_i, since the random variable Δj(,Y) _j( X,Y) exhibits low variance within AiA_i. Essentially, our approach involves identifying a homogeneous group with respect to importance measure Δj(,Y) _j( X,Y) and attributing the global importance of this group as the local explanations of its members. Hence, it permits us to have local explanations while benefiting from all the inference results available for global explanations. 4.1 Estimation and inference To compute our local explanations, we need to compute two quantities: the LOCO importance Ψj(P)=P[Δj(,Y)] _j(P)=E_P [ _j( X,Y) ] for any distribution P and the partition ∪iAi= _iA_i=X that group observations by their feature importance similarity. The former has been extensively studied in (Williamson et al., 2021), where the authors proposed a nonparametric efficient estimation procedure using the following plug-in estimator: Ψ^j(P^)=P^[Δj^(,Y)]whereΔj^(,Y)=V(f^0(),Y)−V(f^−j(),Y), _j( P)=E_ P [ _j( X,Y) ] _j( X,Y)=V ( f_0( X),Y )-V ( f_-j( X),Y ), (3) P^=1/n∑i=1nδ(i,Yi) P=1/n _i=1^n _( X_i,Y_i) is the empirical distribution of P based on n=(i,Yi)i=1nD_n=\( X_i,Y_i)\_i=1^n and f^0,f^−j f_0, f_-j are estimators of the population minimizers f0f_0 and f−jf_-j respectively. f^0,f^−j f_0, f_-j are obtained by building a predictive model for Y using all features in X and after removing the variable XjX_j respectively. This can be achieved using any machine learning algorithm. Williamson et al. (2021) demonstrated that the estimator defined in Equation (3), is asymptotically efficient and enables valid statistical inference under regularity conditions. Having obtained a consistent estimator for Ψj(P) _j(P), we now propose a method to derive the partition ∪iAi= _iA_i=X. The initial step involves creating a new representation for each observation i=(Xi1,…,Xip) X_i=(X_i1,…,X_ip) in nD_n using the conformity score V. This is expressed as ~i=(Δ^1(i,Yi),…,Δ^p(i,Yi)) X_i= ( _1( X_i,Y_i),…, _p( X_i,Y_i) ). Representing the data in the space of feature importance, rather than the original covariate space, allows for the grouping of observations that exhibit consistent behavior with respect to the importance measure Δ^j(,Y)=V(f^0(),Y)−V(f^−j(),Y) _j( X,Y)=V( f_0( X),Y)-V( f_-j( X),Y) across all variables j∈1,…,pj∈\1,…,p\ simultaneously. The subsequent step involves clustering similar observations based on their new representations ~ X. This can be achieved using any clustering algorithm, such as K-means (Macqueen, 1967), DBSCAN (Ester et al., 1996), Tree-based clustering approach (Maimon and Rokach, 2014) or Affinity Propagation (Frey and Dueck, 2007), ultimately resulting in a partition of nD_n into K sets C=C1,…,CKC=\C_1,…,C_K\. Note that clustering in the feature importance space does not necessarily imply that observations will be closer in the input space. In the experimental section, we demonstrate that clustering in the feature-importance space provides a more meaningful basis for capturing the model’s actual behaviour, and we offer a theoretical explanation in Section 5.4.1 within a controlled setting. The final step entails using the identified clusters C to define a partition of X. This is accomplished by assigning any new points ∈ x to their nearest partition CkC_k with respect to a similarity function d:×→ℝ+d:X×X ^+. We can use any similarity function. We use Euclidean or Manhattan distance. More formally, we define the corresponding region A^k A_k of cluster CkC_k as ∈:∑i∈Ckd(x,i)<∑i∈Cld(x,i)for allk≠l. \ x : _ X_i∈ C_kd(x, X_i)< _ X_i∈ C_ld(x, X_i)\;for all\;k≠ l \. In order to establish a partition of X, we must also account for the set of "undecidable" observations, which are those that simultaneously belong to multiple groups. We define this set A^K+1 A_K+1 as ∈:∃k,l∈[k],∑i∈Ckd(x,i)=∑i∈Cld(x,i). \ x :∃ k,l∈[k], _ X_i∈ C_kd(x, X_i)= _ X_i∈ C_ld(x, X_i) \. Hence, the local explanations of a given observation i X_i that belongs to the region A^k A_k can be represented by the vector (Ψ1^(P^A^k),…,Ψp^(P^A^k)) ( _1( P_ A_k),…, _p( P_ A_k) ), where the contribution of the feature XjX_j, Ψj^(P^A^k) _j( P_ A_k), is defined as Ψ^j(P^A^k)=P^A^k[Δ^j(,Y)]=P^[Δ^j(,Y)|∈A^k]=∑(i,Yi)∈CkΔ^j(i,Yi)|Ck|. _j( P_ A_k)=E_ P_ A_k [ _j( X,Y) ]=E_ P [ _j( X,Y)\;|\; X∈ A_k ]= _( X_i,Y_i)∈ C_k _j( X_i,Y_i)|C_k|. We refer to this approach as R-LOCO, and details can be found in Figure 6. R-LOCO’s construction addresses key limitations of L-SV, particularly for models like piecewise linear functions. Theorem 4.1. Let f be the piecewise linear function from Theorem 2.1. When the R-LOCO clustering algorithm identifies cluster Ck⋆C_k within regions Ak⋆A_k , the R-LOCO attributions for =(x1,…,xp)∈Ak⋆ x=(x_1,…,x_p)∈ A_k , with k⋆∈1,…,mk ∈\1,…,m\, are solely influenced by fk⋆f_k . They do not depend on coefficients of any ai,k′a_i,k from other regions Ak′A_k , where k′≠k⋆k ≠ k . Unlike L-SV, which can be influenced by inactive components even under ideal conditions (§2), Theorem 4.1 shows that R-LOCO assigns importance exclusively to the active region whenever Ck⋆⊆Ak⋆C_k A_k . See the proof in §B. In §B.1, we further analyze non-homogeneous (mixed) regions, where the attributions can depend on contamination from other regions; moreover, the induced bias scales proportionally with the contamination level (e.g., with the fraction of the mass of Ck⋆C_k contributed by points from AkA_k, k≠k⋆k≠ k ). 5 Experiments 1: Controlled Setting In our first set of experiments, we evaluate our approach, R-LOCO, by comparing it with three baseline methods: two local attribution techniques, L-SV and LIME, and LOCO, which serves as the global variant of our approach. We conduct experiments in a controlled environment where the ground-truth explanations are known, using independent features to assess each method’s performance without confounding bias. Our objective is to demonstrate that R-LOCO, which lies between the local and global regimes, offers greater local fidelity than state-of-the-art local attribution methods and provides a more nuanced understanding of model behavior compared to global approaches. R-LOCO Variants: We explored several variants of R-LOCO in our experiments, each employing different clustering algorithms to capture local patterns. • R-LOCO: Uses AffinityPropagation clustering on the feature importance space. It is the default strategy of our proposal. • R-LOCOTC: Uses optimal clusters based on the base model structure, expected to perform better than other variants due to privileged model information. We use it as a control baseline to isolate the effect of the clustering in our method. • R-LOCOtree: Utilizes a tree-based algorithm that clusters by minimizing the variance on the feature importance space within each cell. • R-LOCOIC: Performs clustering directly in the input space rather than the feature importance space to investigate the benefits of clustering in feature importance space. More details of the parameters for all methods can be found in the appendix C. 5.1 Models Setup To ensure a rigorous evaluation, we selected a model setup that inherently allows us to determine ground-truth local explanations. Specifically, we chose models that are locally additive, enabling us to use their inherent structure to evaluate the quality of local attribution. To avoid biases that might arise from feature dependencies, we ensured that all features were independent. For data generation, we created a set of n samples denoted by ii=1n\ X_i\_i=1^n, drawn from a distribution ∼P=∏i=jpPXj X P_ X= _i=j^pP_X_j. Using a model f, we then constructed a dataset n=(i,f(i))i=1nD_n=\( X_i,f( X_i))\_i=1^n. The local attribution methods L-SV and LIME require both the model f and dataset nD_n to generate explanations. In contrast, R-LOCO only required fixed input-output pairs, nD_n. This highlights a key advantage of R-LOCO: it does not need direct access to the model’s prediction function. Instead, it only needs a fixed dataset with predictions, making it suitable for scenarios where model access is restricted or for being applied directly to the data rather than a specific model. We conducted our experiments using three different models: 1st-order model: We used a piecewise linear model where local feature importance can be interpreted similarly to classical linear models. Let =(X1,…,X6) X=(X_1,…,X_6), with each component being independently distributed as Xi∼[−1,1]X_i [-1,1] for i∈1,…,6i∈\1,…,6\. The predictor function is defined as follows: f()=(X1+X2)⋅X6≤0+(X3+X4)⋅X6>0 f( X)=(X_1+X_2)·1_X_6≤ 0+(X_3+X_4)·1_X_6>0 (4) 2nd-order model: This model incorporates non-linear terms to introduce complexity while still being locally additive. Again, using independent variables =(X1,…,X10) X=(X_1,…,X_10), where Xi∼[−1,1]X_i [-1,1] for all i, the predictor function was: f()=(X1+X22+|X3|)⋅X10≤0+(X4+X52+|X6|)⋅X10>0 f( X)=(X_1+X_2^2+ |X_3|)·1_X_10≤ 0+(X_4+X_5^2+ |X_6|)·1_X_10>0 2nd-order interaction model: We increased the model complexity by including interaction effects, based on the function used in (Bénard et al., 2021). The predictor was: f()=33×X1X2 1X3>0+3×X4X5X3≤0+3×X6X7 1X8>0+X9X10 1X8≤0.f( X)=3 3× X_1X_2\; 1_X_3>0+ 3× X_4X_5 1_X_3≤ 0+3× X_6X_7\; 1_X_8>0+X_9X_10\; 1_X_8≤ 0. Table 1: Performance metrics for methods across datasets from 1st-order to 2nd-order-interactions. TP: True Positives, FP: False Positives, NI attr.: Mean (q0.1 – q0.95) of non-important features attributions. 1st-order 2nd-order 2nd-order-interact. Method TP FP NI attr. TP FP NI attr. TP FP NI attr. R-LOCO 0.970.97 0.030.03 0.02 (0.01–0.04) 0.860.86 0.140.14 0.02 (0.01–0.04) 0.670.67 0.330.33 0.03 (0.02–0.07) R-LOCOTC 1.001.00 0.000.00 0.03 (0.03–0.03) 1.001.00 0.000.00 0.03 (0.03–0.03) 0.940.94 0.060.06 0.04 (0.03–0.04) R-LOCOtree 0.900.90 0.100.10 0.05 (0.02–0.11) 0.730.73 0.270.27 0.04 (0.02–0.06) 0.500.50 0.500.50 0.06 (0.05–0.07) R-LOCOIC 0.890.89 0.110.11 0.05 (0.02–0.11) 0.740.74 0.260.26 0.04 (0.03–0.07) 0.650.65 0.350.35 0.05 (0.04–0.07) LOCO 0.500.50 0.500.50 0.09 (0.09–0.09) 0.500.50 0.500.50 0.05 (0.05–0.05) 0.500.50 0.500.50 0.06 (0.05–0.07) L-SV 0.410.41 0.590.59 0.12 (0.08–0.18) 0.300.30 0.700.70 0.08 (0.07–0.10) 0.440.44 0.560.56 0.06 (0.05–0.09) LIME 0.530.53 0.470.47 0.12 (0.07–0.18) 0.430.43 0.570.57 0.07 (0.05–0.10) 0.440.44 0.560.56 0.06 (0.04–0.09) Table 2: Performance metrics for R-LOCO using different clustering algorithms across datasets from 1st-order to 2nd-order-interactions. TP: True Positives, FP: False Positives, NI attr.: Mean (q0.1 – q0.95) of non-important features attributions. 1st-order 2nd-order 2nd-order-interact. Method TP FP NI attr. TP FP NI attr. TP FP NI attr. R-LOCOKMeans-2 0.980.98 0.020.02 0.03 (0.03–0.03) 0.920.92 0.080.08 0.04 (0.03–0.06) 0.570.57 0.430.43 0.05 (0.03–0.09) R-LOCOKMeans-4 0.970.97 0.030.03 0.03 (0.02–0.08) 0.830.83 0.170.17 0.03 (0.03–0.06) 0.620.62 0.380.38 0.04 (0.02–0.08) R-LOCOKMeans-8 0.960.96 0.040.04 0.03 (0.02–0.06) 0.930.93 0.070.07 0.03 (0.02–0.04) 0.590.59 0.410.41 0.04 (0.02–0.08) R-LOCOKMeans-20 0.970.97 0.030.03 0.02 (0.02–0.04) 0.960.96 0.040.04 0.03 (0.02–0.04) 0.640.64 0.360.36 0.04 (0.02–0.08) R-LOCOAffinity-0.5 0.970.97 0.030.03 0.02 (0.01–0.04) 0.860.86 0.140.14 0.02 (0.01–0.04) 0.680.68 0.320.32 0.03 (0.02–0.07) R-LOCOAffinity-0.6 0.970.97 0.030.03 0.02 (0.01–0.04) 0.860.86 0.140.14 0.02 (0.01–0.04) 0.670.67 0.330.33 0.03 (0.02–0.07) R-LOCOAffinity-0.9 0.970.97 0.030.03 0.02 (0.01–0.04) 0.860.86 0.140.14 0.02 (0.01–0.04) 0.670.67 0.330.33 0.03 (0.02–0.07) Figure 1: Diabetes dataset. Figure 2: California dataset. 5.2 Evaluation For each model, we identify the k variables that contribute locally to the prediction. We exclude the variables used for defining the region (e.g., in the 1st order model, we discard the variable X6X_6), as these variables may also be interpreted as globally important, which could bias the analysis. Instead, we focus solely on the k variables directly involved in the computation of the final prediction. As a measure of true importance, we use the sum of the absolute feature–coefficient products across all functional components that include the feature, analogous to classical functional ANOVA contributions. In some settings, we introduce dummy variables that are irrelevant to the model and mutually independent, included solely to assess how different methods assign importance to uninformative features. To ensure a consistent comparison focused purely on importance magnitude, we normalise all attribution values. We divide the absolute value of each score by the total absolute sum per observation, thereby restricting all scores to the range [0,1][0,1] and disregarding their original directionality. We then evaluate the performance of each method by determining how many of the k truly local important variables (those with non-zero coefficients) are correctly identified by their top k most important features (True Positives) and how many irrelevant variables are incorrectly identified as important (False Positives). We also compute the average, as well as the 10th (q0.1) and 95th (q0.95) percentiles of the attributions assigned to non-important variables, which we denote as NI attributions. This assess the tendency of each method to over-attribute importance to irrelevant features. 5.3 Analysis Table 1 presents the aggregated performance metrics of our experiments from 50 runs on the first three synthetic datasets. The key observations from these results are as follows: Poor Performance of L-SV and LIME: L-SV and LIME consistently perform poorly across all experiments, with true positive (TP) rates of 30% to 66% and false positive (FP) rates of 34% to 70% on Table 1. For example, in the 1st-order model, L-SV achieves a TP of 41% and FP of 59%, while LIME achieves TP of 53% and FP of 47%, indicating frequent misattributions. Moreover, their performance is almost identical to that of the global approach LOCO, which is not designed to highlight local importance. Superior Performance of R-LOCO: R-LOCO outperforms the baselines, with significantly higher accuracy. In the 1st-order model, R-LOCO achieves a TP of 97% and FP of 3%. Although TP rates decrease slightly as model complexity grows, R-LOCO consistently outperforms the baselines. R-LOCO assigns minimal importance to non-important features (1% to 4%), whereas L-SV and LIME assign 7% to 18%. This indicates R-LOCO more effectively distinguishes between important and non-important features. Impact of Clustering: When using perfect clusters (R-LOCOTC), R-LOCO achieves near-perfect performance, suggesting that clustering quality is the key factor for R-LOCO’ success. Results of R-LOCOIC also show that clustering in the feature importance space (R-LOCO) yields better results compared to clustering in input space. Summary: Overall, the experimental results show that the baseline methods L-SV and LIME consistently perform poorly to identify locally important features and have performance close to the global approach LOCO. In contrast, R-LOCO and its variants significantly outperform these traditional local attribution methods, providing more locally faithful feature attributions. The effectiveness of R-LOCO largely depends on the quality of the clustering process, particularly when clustering is performed in the feature importance space. 5.4 Impact of Clustering in R-LOCO To illustrate how clustering quality can be assessed in our method, we present a qualitative example using the first-order model. We focus on an observation where X6X_6 is positive, implying that the only locally important variables are X3X_3 and X4X_4. Figure 3: Feature attributions from R-LOCO and R-LOCOIC on the first-order model, highlighting the intra-group variance of feature importance. Figure 3 compares feature attributions from R-LOCO and R-LOCOIC (which clusters in the input space), highlighting the variance of feature-importance scores within each cluster. Both methods identify X3X_3 and X4X_4 as the most important variables, but R-LOCO exhibits substantially lower intra-cluster variance. Specifically, the average variance across all observations and features is 0.0020.002 for R-LOCO, compared to 0.0120.012 for R-LOCOIC. This lower variance suggests more consistent attributions within clusters, indicating higher clustering quality. By assessing the intra-cluster variance, practitioners can evaluate the clustering quality and make informed choices about clustering methods. To further assess interpretability, we train a decision tree to predict the cluster assignments. This yields a transparent and human-readable proxy of the regions identified by each method. Interestingly, R-LOCO discovers regions aligned with the true data-generating process by first splitting on the sign of X6X_6, whereas clustering in the input space fails to do so, prioritising other non-important variables. The learned tree representation is displayed in §F. Parameter Variations: To further assess clustering robustness, we conducted additional experiments using KMeans with cluster counts of 2, 4, 8, and 20, and Affinity Propagation with damping values of 0.5, 0.6, and 0.9. Table 2 summarizes these findings, underscoring the robustness of R-LOCO to clustering variations. Our key observations are: • For both first and second-order models, as long as intra-cluster variance was minimized and clusters contained enough points, performance remained stable. • Increasing the number of clusters had little adverse effect, provided cluster sizes were sufficiently large. • In second-order interaction models with four ground-truth clusters, performance only declined when clusters were smaller than the true regions. Assigning more clusters than ground-truth also had minimal impact. These results are consistent with Theorem 4.1, which states that local accuracy may be achieved once the identified cluster is part of the ground-truth cluster or is only minimally contaminated. 5.4.1 Theoretical Analysis To formalize the advantage of clustering in importance space, we analyze the first-order model (§5.1) with independent features Xi∼[−1,1]X_i [-1,1] for i=1,…,6i=1,…,6 and the function f()=(X1+X2)⋅X6≤0+(X3+X4)⋅X6>0.f(X)=(X_1+X_2)·1_X_6≤ 0+(X_3+X_4)·1_X_6>0. This function operates in two distinct regimes based on the "switching" variable X6X_6: RA≔∣X6≤0R_A \x X_6≤ 0\ and RB≔∣X6>0R_B \x X_6>0\. We compare the K-Means (K=2K=2) decision rules derived from clustering in the input space versus the R-LOCO importance space. Clustering in Input Space. The true cluster centroids are the conditional means [∣RA]E[X R_A] and [∣RB]E[X R_B], defined as CA=(0,0,0,0,0,−0.5)C_A=(0,0,0,0,0,-0.5) and CB=(0,0,0,0,0,0.5).C_B=(0,0,0,0,0,0.5). This follows from feature independence ([Xi∣RA]=[Xi]=0E[X_i R_A]=E[X_i]=0 for i≤5i≤ 5) and the uniform distribution ([X6∣X6≤0]=−0.5E[X_6 X_6≤ 0]=-0.5). The K-Means separating direction is CA−CB=(0,0,0,0,0,−1)C_A-C_B=(0,0,0,0,0,-1). The decision rule, ∈RAx∈ R_A if d(,CA)2<d(,CB)2d(x,C_A)^2<d(x,C_B)^2, simplifies to a condition based only on X6X_6. Therefore, input-space clustering merely rediscovers the "switching" variable X6X_6. The variability from the functionally active features (X1,…,X4X_1,…,X_4) acts as noise, hindering a clean separation. Clustering in R-LOCO Importance Space. Each sample X is mapped to its R-LOCO importance vector γ()=(Δ1,…,Δ6)γ(X)=( _1,…, _6), where Δj()=(f()−f−j())2 _j(X)=(f(X)-f_-j(X))^2 and f−j()=[f()∣−j]f_-j(X)=E[f(X) _-j]. The importance-space centroids for RAR_A and RBR_B are Cγ,A=[γ()∣RA]=(13,13,0,0,0,13)C_γ,A=E[γ(X) R_A]= ( 13, 13,0,0,0, 13 ) and Cγ,B=[γ()∣RB]=(0,0,13,13,0,13)C_γ,B=E[γ(X) R_B]= (0,0, 13, 13,0, 13 ), respectively. See G for detailed derivations. The K-Means separating direction in this space is: Cγ,A−Cγ,B=(13,13,−13,−13,0,0).C_γ,A-C_γ,B= ( 13, 13,- 13,- 13,0,0 ). This decision rule only depends on the components (Δ1,Δ2,Δ3,Δ4)( _1, _2, _3, _4) corresponding to the locally active features. Clustering in importance space successfully identifies functional similarity: • RAR_A samples have large (Δ1,Δ2)( _1, _2) and Δ3,4≈0 _3,4≈ 0. • RBR_B samples have large (Δ3,Δ4)( _3, _4) and Δ1,2≈0 _1,2≈ 0. In short, input-space clustering is confounded by noise from irrelevant features, while importance-space clustering is robust to this noise and correctly groups samples by their underlying explanatory behaviour. Remark: Importance Representation and Failure Cases We note however a counter-example: f(X1,X2)=X2⋅sgn(X1)f(X_1,X_2)=X_2·sgn(X_1), with regimes RA=X1<0R_A=\X_1<0\ and RB=X1≥0R_B=\X_1≥ 0\. If we use the squared-loss importance Δj=(f()−f−j())2 _j=(f(X)-f_-j(X))^2, we find f−1=f−2=0f_-1=f_-2=0. This results in an importance vector γ()=(Δ1,Δ2)=(f()2,f()2)=(X22,X22)γ(X)=( _1, _2)=(f(X)^2,f(X)^2)=(X_2^2,X_2^2). This representation is identical for both regimes, making them inseparable. The issue is that the squared loss Δj _j is sign-agnostic and discards the directional information that distinguishes the regimes. This motivates enriching the importance representation. For instance, including the signed R-LOCO residual, δj()=f()−f−j() _j(X)=f(X)-f_-j(X), yields an enriched vector: γ′()=(Δ1,Δ2,δ1,δ2)=(X22,X22,f(),f()).γ (X)=( _1, _2, _1, _2)=(X_2^2,X_2^2,f(X),f(X)). This enriched vector maps the two regimes to distinct, separable locations: γ′()=(X22,X22,−X2,−X2) if X1<0,and (X22,X22,X2,X2) if X1≥0.γ (X)=(X_2^2,X_2^2,-X_2,-X_2) if X_1<0,\ and (X_2^2,X_2^2,X_2,X_2) if X_1≥ 0. In this space, the regimes are easily distinguished by a clustering algorithm, underscoring the importance of the chosen representation. 6 Experiments 2: Real-world Settings In the preceding sections, we focused on the piecewise model setting, providing both theoretical and empirical evidence that L-SV and LIME can fail under ideal conditions (e.g., exact computation and independence assumptions). We also introduced R-LOCO to address these shortcomings in a controlled context. In this section, we extend our analysis to real-world tabular datasets, where feature dependencies naturally arise, and use widely adopted, high-performing models, XGBoost, for tabular data. We introduce two additional baselines: • FDT Laberge et al. (2024): This approach is conceptually related to ours in that it seeks “regional” explanations; however, its objective is to find regions where multiple explanation methods agree. In contrast, R-LOCO aims to identify zones of influence that are distinct to the model’s behavior, using predictive loss as the criterion. • Clustering SHAP Values (CSHAP): As described in the original SHAP paper (Lundberg et al., 2018), L-SV can be clustered to identify distinct zones of influence. Since SHAP does not directly employ group-wise averages, we adapt it for comparison by clustering SHAP values or L-SV (using the same algorithm as R-LOCO) and then computing averages for each group. To evaluate the faithfulness of these methods, we measure the change in predictive performance when zero-masking features deemed most important (top-k) and least important (bottom-k) by each approach. See §H for details. A larger performance drop when masking the most important features indicates higher fidelity, whereas a smaller performance drop when masking the least important features is desirable. Figures 2 and 2 show the results on the Diabetes and California datasets from Dua and Graff (2017), respectively. In Figure 2, our method consistently outperformed all baselines. While FDT performed best among the baselines, clustering SHAP values (CSHAP) did not outperform standard SHAP. This highlights that, unlike LOCO features, SHAP values lack sufficient information to effectively identify the distinct zone of influence of the given model, and underscores the critical importance of selecting appropriate inputs for clustering. In Figure 2, our method was superior for identifying and perturbing the top three most important features, and FDT was slightly better for the remaining, but R-LOCO outperformed when removing non-important features, indicating superior region definition. Finally, as shown in Appendix I, our method is also significantly faster at inference, since it only requires assigning a cluster to compute feature importance. 7 Conclusion We presented a rigorous evaluation of widely used local attribution methods in settings where the true local features are known. Our findings reveal that, despite their intuitive appeal, common local methods (L-SV, LIME) can fail even under ideal conditions with no feature dependencies and perfect estimations. To address this, we proposed a regional method that partitions the input space into homogeneous regions—thus enabling a hybrid approach combining the fidelity of global LOCO importance with more localized attributions. Unlike existing baselines, our framework offers a clear interpretation of a feature’s contribution as the increase in error when that feature is removed. Our method lies between fully global and purely local approaches: avoiding the instability of pointwise explanations while still preserving instance-level insights, which purely global approaches often lack. R-LOCO implicitly assumes each region of the model can be approximated by an additive structure. A key challenge is selecting and refining these regions. One promising direction is to enrich the feature-importance representation (used for clustering) with second- or higher-order interactions, thus capturing more complex dependencies. While our initial analysis focused on independent variables for clarity, we have demonstrated that the approach also outperforms baselines on real-world datasets where dependencies naturally arise. Acknowledgements We thank the anonymous reviewers for their valuable suggestions, which helped improve this work, especially regarding the counterexample illustrating the failure of the base feature-importance mapping. Disclaimer This work was in part conducted during Salim I. Amoukou’s PhD at LaMME, Université Paris-Saclay, in collaboration with the Stellantis Group. References D. Alvarez-Melis and T. S. Jaakkola (2018) On the robustness of interpretability methods. arXiv preprint arXiv:1806.08049. Cited by: Appendix D, §3. S. I. Amoukou, T. Salaün, and N. Brunel (2022) Accurate shapley values for explaining tree-based models. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, G. Camps-Valls, F. J. R. Ruiz, and I. Valera (Eds.), Proceedings of Machine Learning Research, Vol. 151, p. 2448–2465. External Links: Link Cited by: §1. C. Bénard, G. Biau, S. Da Veiga, and E. Scornet (2021) SHAFF: fast and consistent shapley effect estimates via random forests. arXiv preprint arXiv:2105.11724. Cited by: Appendix I, §5.1. S. Bordt and U. von Luxburg (2023) From shapley values to generalized additive models and back. In International Conference on Artificial Intelligence and Statistics, p. 709–745. Cited by: §1. L. Breiman (2001) Random forests. Machine learning 45 (1), p. 5–32. Cited by: §1. G. Chastaing, F. Gamboa, and C. Prieur (2012) Generalized hoeffding-sobol decomposition for dependent variables-application to sensitivity analysis. arXiv preprint arXiv:1112.1788. Cited by: §2. H. Chen, J. D. Janizek, S. Lundberg, and S. Lee (2020) True to the model or true to the data?. arXiv preprint arXiv:2006.16234. Cited by: §1, §2. K. Chen, H. Garudadri, and B. D. Rao (2022) Improved bounds on neural complexity for representing piecewise linear functions. arXiv preprint arXiv:2210.07236. Cited by: §2. I. Covert, S. Lundberg, and S. Lee (2020a) Explaining by removing: a unified framework for model explanation. arXiv preprint arXiv:2011.14878. Cited by: Appendix I. I. Covert, S. Lundberg, and S. Lee (2020b) Understanding global feature contributions through additive importance measures. CoRR abs/2004.00668. External Links: Link, 2004.00668 Cited by: §1. D. Dua and C. Graff (2017) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. External Links: Link Cited by: Figure 5, Appendix D, §6. P. Dubey and R. J. Weber (1977) Probabilistic values for games. Cited by: §2. B. Efron and C. Stein (1981) The jackknife estimate of variance. The Annals of Statistics, p. 586–596. Cited by: §2. M. Ester, H. Kriegel, J. Sander, X. Xu, et al. (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, Vol. 96, p. 226–231. Cited by: §4.1. B. E. Feldman (2005) Relative importance and value. Available at SSRN 2255827. Cited by: §2. S. Flaxman, D. Sejdinovic, J. P. Cunningham, and S. Filippi (2016) Bayesian learning of kernel embeddings. arXiv preprint arXiv:1603.02160. Cited by: Appendix D. B. J. Frey and D. Dueck (2007) Clustering by passing messages between data points. science 315 (5814), p. 972–976. Cited by: §4.1. K. Fukumizu, A. Gretton, G. Lanckriet, B. Schölkopf, and B. K. Sriperumbudur (2009) Kernel choice and classifiability for rkhs embeddings of probability distributions. Advances in neural information processing systems 22. Cited by: Appendix D. L. Gan, L. Zheng, and G. I. Allen (2022) Inference for interpretable machine learning: fast, model-agnostic confidence intervals for feature importance. arXiv preprint arXiv:2206.02088. Cited by: §1. D. Garreau, W. Jitkrittum, and M. Kanagawa (2017) Large sample analysis of the median heuristic. arXiv preprint arXiv:1707.07269. Cited by: Appendix D. D. Garreau and U. Luxburg (2020) Explaining the explainer: a first theoretical analysis of lime. In International Conference on Artificial Intelligence and Statistics, p. 1287–1296. Cited by: §1. U. Grömping (2007) Estimators of relative importance in linear regression based on variance decomposition. The American Statistician 61 (2), p. 139–147. Cited by: §2. N. Hama, M. Mase, and A. B. Owen (2022) Model free shapley values for high dimensional data. arXiv preprint arXiv:2211.08414. Cited by: §4. B. Hanin and D. Rolnick (2019a) Complexity of linear regions in deep networks. In International Conference on Machine Learning, p. 2596–2604. Cited by: §2. B. Hanin and D. Rolnick (2019b) Deep relu networks have surprisingly few activation patterns. Advances in neural information processing systems 32. Cited by: §2. T. Heskes, E. Sijben, I. G. Bucur, and T. Claassen (2020) Causal shapley values: exploiting causal knowledge to explain individual predictions of complex models. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Eds.), External Links: Link Cited by: §2. W. Hoeffding (1948) A class of statistics with asymptotically normal distribution. Annals of Mathematical Statistics 19, p. 308–334. Cited by: §2. G. Hooker (2007) Generalized functional anova diagnostics for high-dimensional functions of dependent variables. Journal of Computational and Graphical Statistics 16, p. 709 – 732. Cited by: §2. S. Hooker, D. Erhan, P. Kindermans, and B. Kim (2019) A benchmark for interpretability methods in deep neural networks. Advances in neural information processing systems 32. Cited by: §4. S. I. Amoukou and N. Brunel (2022) Consistent sufficient explanations and minimal local rules for explaining the decision of any classifier or regressor. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 8027–8040. External Links: Link Cited by: Appendix I. M. I. Idrissi (2024) Development of interpretability methods for certifying machine learning models applied to critical systems. Ph.D. Thesis, Université de Toulouse. Cited by: §2. B. Iooss and P. Lemaitre (2015) A review on global sensitivity analysis methods. Uncertainty management in simulation-optimization of complex systems: algorithms and applications, p. 101–122. Cited by: §1. D. Janzing, L. Minorics, and P. Blöbaum (2020) Feature relevance quantification in explainable ai: a causal problem. In International Conference on Artificial Intelligence and Statistics, p. 2907–2916. Cited by: §2. J. W. Johnson and J. M. LeBreton (2004) History and use of relative importance indices in organizational research. Organizational research methods 7 (3), p. 238–257. Cited by: §2. G. Laberge, Y. B. Pequignot, M. Marchand, and F. Khomh (2024) Tackling the xai disagreement problem with regional explanations. In International Conference on Artificial Intelligence and Statistics, p. 2017–2025. Cited by: 1st item. J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. A. Wasserman (2016) Distribution-free predictive inference for regression. Journal of the American Statistical Association 113, p. 1094 – 1111. Cited by: §1, §1, §4. S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S. Lee (2020) From local explanations to global understanding with explainable ai for trees. Nature machine intelligence 2 (1), p. 56–67. Cited by: §2. S. M. Lundberg, G. G. Erion, and S. Lee (2018) Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888. Cited by: 2nd item. S. M. Lundberg and S. Lee (2017a) A unified approach to interpreting model predictions. Advances in neural information processing systems 30. Cited by: §1. S. M. Lundberg and S. Lee (2017b) A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), p. 4765–4774. Cited by: §2. J. Macqueen (1967) Some methods for classification and analysis of multivariate observations. In Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability/University of California Press, Cited by: §4.1. O. Z. Maimon and L. Rokach (2014) Data mining with decision trees: theory and applications. Vol. 81, World scientific. Cited by: §4.1. A. B. Owen and C. Prieur (2017) On shapley value for measuring importance of dependent inputs. SIAM/ASA Journal on Uncertainty Quantification 5 (1), p. 986–1002. Cited by: §2. A. B. Owen (2014) Sobol’indices and shapley value. SIAM/ASA Journal on Uncertainty Quantification 2 (1), p. 245–251. Cited by: §2. R. Pascanu, G. Montufar, and Y. Bengio (2013) On the number of response regions of deep feed forward networks with piece-wise linear activations. arXiv preprint arXiv:1312.6098. Cited by: §2. M. T. Ribeiro, S. Singh, and C. Guestrin (2016a) " Why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, p. 1135–1144. Cited by: Appendix D, Appendix D, §1, §3. M. T. Ribeiro, S. Singh, and C. Guestrin (2016b) "Why should I trust you?": explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, p. 1135–1144. Cited by: Appendix C. I. M. Sobol’ (1990) On sensitivity estimation for nonlinear mathematical models. Matematicheskoe modelirovanie 2 (1), p. 112–118. Cited by: §2. I. Verdinelli and L. Wasserman (2024) Feature importance: a closer look at shapley values and loco. Statistical Science 39 (4), p. 623–636. Cited by: §4, §4. G. Visani, E. Bagli, and F. Chesani (2020) OptiLIME: optimized lime explanations for diagnostic computer algorithms. ArXiv abs/2006.05714. Cited by: Appendix D, §3. B. D. Williamson, P. B. Gilbert, N. R. Simon, and M. Carone (2021) A general framework for inference on algorithm-agnostic variable importance. Journal of the American Statistical Association, p. 1–14. Cited by: §4.1, §4.1, §4. B. D. Williamson and J. Feng (2020) Efficient nonparametric statistical inference on population feature importance using shapley values. External Links: 2006.09481 Cited by: §1, §1. M. R. Zafar and N. M. Khan (2019) DLIME: a deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems. ArXiv abs/1906.10263. Cited by: Appendix D, §3. Y. Zhang, K. Song, Y. Sun, S. Tan, and M. Udell (2019) " Why should you trust my explanation?" understanding uncertainty in lime explanations. arXiv preprint arXiv:1904.12991. Cited by: Appendix D, §3. Z. Zhou, G. Hooker, and F. Wang (2021) S-lime: stabilized-lime for model explanation. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. Cited by: Appendix D, §3. Appendix A Proof of Theorem 2.1 Theorem A.1. Let f be a piecewise linear function with m components defined by the collection f|A1,…,f|Ak\f_ A_1,…,f_ A_k\, where ∪k=1mAk= _k=1^mA_k=X. The regions AkA_k are disjoint hyperrectangles, specifically Ak=⨂i=1pAi,kA_k= _i=1^pA_i,k, where Ai,k=[li,k,ri,k]A_i,k=[l_i,k,r_i,k] with li,k,ri,k∈ℝ¯l_i,k,r_i,k∈ R. Each component f|Akf_ A_k is represented as fk()=∑i=1pai,kXi+bkf_k( X)= _i=1^pa_i,kX_i+b_k, where the coefficients ai,ka_i,k and bkb_k are real numbers. Consequently, f is defined as: f()=∑k=1m(∑i=1pai,kXi+bk)Ak(). f( X)= _k=1^m ( _i=1^pa_i,kX_i+b_k ) 1_A_k( X). Consider an observation =(x1,…,xp)x=(x_1,…,x_p) located in a specific region Ak⋆A_k for some k⋆∈1,…,mk ∈\1,…,m\. The model’s prediction is therefore determined solely by the local model fk⋆f_k , i.e., f()=fk⋆()f(x)=f_k (x), and the set of truly active features is Jk⋆=i∈[p]∣ai,k⋆≠0J_k =\i∈[p] a_i,k ≠ 0\. We also define the set of features important globally G=i∈[p]∣∃k s.t. Ai,k≠ℝG=\i∈[p] ∃ k s.t. A_i,k \. Let the covariates XiX_i be mutually independent. The Local Shapley Value (SV) for the feature-value Xl=xlX_l=x_l is equal to ϕxl=∑k=1mϕxlk, _x_l= _k=1^m _x_l^k, and ϕxlk _x_l^k is defined as ϕxlk=(Al,k(xl)ℙ(Xl∈Al,k)−1)∑S⊆D∖lw(S)vk(S)+al,k(xl−[XlAl,k(Xl)]ℙ(Xl∈Al,k))∑S⊆D∖lw(S)×∏i∈S∪lAi,k(xi)∏j∈S¯ℙ(Xi∈Ai,k), _x_l^k= ( 1_A_l,k(x_l)P(X_l∈ A_l,k)-1 ) _S D \l\w(S)v_k(S)\\ +a_l,k (x_l- E [X_l 1_A_l,k(X_l) ]P(X_l∈ A_l,k) ) _S D \l\w(S)× _i∈ S∪ l 1_A_i,k(x_i) _j∈ SP(X_i∈ A_i,k), (5) where w(S)=1p(|D|−1|S|)−1w(S)= 1p |D|-1|S|^-1 and vk(S)=[fk()Ak()|S=S]v_k(S)=E[f_k( X) 1_A_k( X)\;|\; X_S= x_S]. Equation (5) demonstrates that even if the model only uses fk⋆()f_k ( x) for a given observation x, the Local SV x may depend on the coefficients of the unused linear models fkf_k for k∈1,…,m∖k⋆k∈\1,…,m\ \k \, which are not globally important neither. Proof. Let’s assume that ∏i=1pℙ(Xi∈Ai,k)>0 _i=1^pP(X_i∈ A_i,k)>0 and intercepts bk=0b_k=0 for all k=1,…,mk=1,…,m without loss of generality. Given an observation =(x1,…,xp) x=(x_1,…,x_p), we consider the Shapley Value of a feature-value Xl=xlX_l=x_l defined as ϕl=∑S⊆D∖lw(S)[Δl(S)], _ x_l= _S D \l\w(S) [ _l(S) ], where w(S)=1p(|D|−1|S|)−1w(S)= 1p |D|-1|S|^-1, Δl(S)=v(S∪l)−v(S) _l(S)=v(S∪ l)-v(S) represent the marginal contribution, and v(S)=[f()|S=S]v(S)=E [f( X)| X_S= x_S ]. Note that we can decompose v(S)v(S) into m separate terms as following: v(S) v(S) =[∑k=1m(∑i=1pai,kXi)Ak()|S=S] =E [ _k=1^m ( _i=1^pa_i,kX_i ) 1_A_k( X)\;|\; X_S= x_S ] =∑k=1m[(∑i=1pai,kXi)Ak()|S=S] = _k=1^mE [ ( _i=1^pa_i,kX_i ) 1_A_k( X)\;|\; X_S= x_S ] =∑k=1mvk(S) = _k=1^mv_k(S) Hence, we can decompose the Shapley Value of ϕl _ x_l due to the linearity property of SV as ϕxl=∑k=1mϕxlk, _x_l= _k=1^mφ^k_x_l, where ϕxlkφ^k_x_l corresponds to the SV compute using the value function vk(S)v_k(S). Therefore, we only need to prove that ϕxlkφ^k_x_l for k≠k⋆k≠ k is not necessarily null to prove the Theorem. We have vk(S)=[(∑i=1pai,kXi)Ak()|S=S]=[(∑i∈Sai,kXi+∑i∈S¯ai,kXi)∏j=1pAj,k(Xj)|S=S]=(∑i∈Sai,ki)∏j∈SAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k)+∑i∈S¯ai,k[XiAi,k(Xi)]∏j∈S¯:j≠iℙ(Xj∈Aj,k)∏j∈SAj,k(xj). splitv_k(S)&=E [ ( _i=1^pa_i,kX_i ) 1_A_k( X)\;|\; X_S= x_S ]\\ &=E [ ( _i∈ Sa_i,kX_i+ _i∈ Sa_i,kX_i ) _j=1^p 1_A_j,k(X_j)\;|\; X_S= x_S ]\\ &= ( _i∈ Sa_i,k x_i ) _j∈ S 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k)\\ & + _i∈ Sa_i,kE [X_i 1_A_i,k(X_i) ] _j∈ S:j≠ iP(X_j∈ A_j,k) _j∈ S 1_A_j,k(x_j). split Similarly, we can write v(S∪l)v(S∪ l) as: vk(S∪l) v_k(S∪ l) =(∑i∈S∪lai,kxi)∏j∈S∪lAj,k(xj)∏j∈S∪l¯ℙ(Xj∈Aj,k) = ( _i∈ S∪ la_i,kx_i ) _j∈ S∪ l 1_A_j,k(x_j) _j∈ S∪ lP(X_j∈ A_j,k) +∑i∈S∪l¯ai,k[XiAi,k(Xi)]∏j∈S∪l¯:j≠iℙ(Xj∈Aj,k)∏j∈S∪lAj,k(xj) + _i∈ S∪ la_i,kE [X_i 1_A_i,k(X_i) ] _j∈ S∪ l:j≠ iP(X_j∈ A_j,k) _j∈ S∪ l 1_A_j,k(x_j) =(∑i∈Sai,kxi)∏j∈SAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k)×Al,k(xl)ℙ(Xl∈Al,k) = ( _i∈ Sa_i,kx_i ) _j∈ S 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k)× 1_A_l,k(x_l)P(X_l∈ A_l,k) +al,kxl∏j∈SAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k)×Al,k(xl)ℙ(Xl∈Al,k) + [rgb]0,.5,.5 [named]pgfstrokecolorrgb0,.5,.5a_l,kx_l _j∈ S 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k)× 1_A_l,k(x_l)P(X_l∈ A_l,k) +∑i∈S¯ai,k[XiAi,k(Xi)]∏j∈S¯:j≠iℙ(Xj∈Aj,k)∏j∈SAj,k(xj)×Al,k(xl)ℙ(Xl∈Al,k) + _i∈ Sa_i,kE [X_i 1_A_i,k(X_i) ] _j∈ S:j≠ iP(X_j∈ A_j,k) _j∈ S 1_A_j,k(x_j)× 1_A_l,k(x_l)P(X_l∈ A_l,k) −al,k[XlAl,k(Xl)]∏j∈S¯:j≠lℙ(Xj∈Aj,k)∏j∈SAj,k(xj)×Al,k(xl)ℙ(Xl∈Al,k) - [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0a_l,kE [X_l 1_A_l,k(X_l) ] _j∈ S:j≠ lP(X_j∈ A_j,k) _j∈ S 1_A_j,k(x_j)× 1_A_l,k(x_l)P(X_l∈ A_l,k) The terms highlighted in red and teal respectively represent the negative and positive contributions of the variable Xl=xlX_l=x_l in vk(S∪l)v_k(S∪ l), the other terms will be put together to form vk(S)v_k(S) as follows vk(S∪l) v_k(S∪ l) =Al,k(xl)ℙ(Xl∈Al,k)vk(S) = 1_A_l,k(x_l)P(X_l∈ A_l,k)v_k(S) +1ℙ(Xl∈Al,k)∏j∈S∪lAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k)×al,k(xl−[XlAl,k(Xl)]ℙ(Xl∈Al,k)). + 1P(X_l∈ A_l,k) _j∈ S∪ l 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k)× a_l,k (x_l- E [X_l 1_A_l,k(X_l) ]P(X_l∈ A_l,k) ). Hence, the marginal contribution of coalition S of the SV ϕxlkφ^k_x_l is equal to: Δlk(S) ^k_l(S) =vk(S∪l)−vk(S) =v_k(S∪ l)-v_k(S) =vk(S)(Al,k(xl)ℙ(Xl∈Al,k)−1) =v_k(S) ( 1_A_l,k(x_l)P(X_l∈ A_l,k)-1 ) +∏j∈S∪lAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k)×al,k(xl−[XlAl,k(Xl)]ℙ(Xl∈Al,k)) + _j∈ S∪ l 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k)× a_l,k (x_l- E [X_l 1_A_l,k(X_l) ]P(X_l∈ A_l,k) ) Finally, we have ϕxlk=(Al,k(xl)ℙ(Xl∈Al,k)−1)∑S⊆D∖lw(S)vk(S)+al,k(xl−[XlAl,k(Xl)]ℙ(Xl∈Al,k))∑S⊆D∖lw(S)×∏j∈S∪lAj,k(xj)∏j∈S¯ℙ(Xj∈Aj,k) _x_l^k= ( 1_A_l,k(x_l)P(X_l∈ A_l,k)-1 ) _S D \l\w(S)v_k(S)\\ +a_l,k (x_l- E [X_l 1_A_l,k(X_l) ]P(X_l∈ A_l,k) ) _S D \l\w(S)× _j∈ S∪ l 1_A_j,k(x_j) _j∈ SP(X_j∈ A_j,k) ∎ Appendix B Proof of Theorem 4.1 Theorem B.1. Let f be the piecewise linear function from Theorem 2.1. When the R-LOCO clustering algorithm identifies clusters Ck⋆C_k within regions Ak⋆A_k , the R-LOCO attributions for =(x1,…,xp)∈Ak⋆ x=(x_1,…,x_p)∈ A_k , with k⋆∈1,…,mk ∈\1,…,m\ (the active region), are solely influenced by fk⋆f_k . They do not depend on coefficients any ai,k′a_i,k from other regions Ak′A_k , where k′≠k⋆k ≠ k . Proof. The R-LOCO attribution for a feature XjX_j within the region defined by cluster Ck⋆C_k is given by the average change in the conformity score V when that feature is removed: Ψ^j(P^Ck⋆)=1|Ck⋆|∑(l,yl)∈Ck⋆[V(f(l),yl)−V(f−j(l),yl)] _j( P_C_k )= 1|C_k | _(x_l,y_l)∈ C_k [V(f(x_l),y_l)-V(f_-j(x_l),y_l) ] where f is the full model and f−j()=[f()|−j=−j]f_-j(x)=E[f(X)|X_-j=x_-j] is the model with feature j marginalized out. The theorem’s central assumption is that the clustering algorithm is that the identified cluster Ck⋆C_k is a subset of the true model region Ak⋆A_k , i.e., Ck⋆⊆Ak⋆C_k A_k . We analyze the implications of this condition on both terms in the R-LOCO formula. 1. Analysis of the full model, f(l)f(x_l): For any observation (l,yl)∈Ck⋆(x_l,y_l)∈ C_k , our core assumption implies l∈Ak⋆x_l∈ A_k . By definition, the global piecewise model f()=∑k=1mfk()∈Akf(x)= _k=1^mf_k(x) 1_x∈ A_k simplifies to the active local model for any point within its region. Therefore, for all l∈Ck⋆x_l∈ C_k : f(l)=fk⋆(l)=k⋆Tl+bk⋆f(x_l)=f_k (x_l)=a_k ^Tx_l+b_k This shows that the first term, V(f(l),yl)V(f(x_l),y_l), depends exclusively on the parameters of the local model fk⋆f_k . 2. Analysis of the reduced model, f−j(l)f_-j(x_l): The proof is particularly concerned with features j that are not globally important, meaning they are not used to define the region boundaries. Let G=i∈[p]∣∃k s.t. Ai,k≠ℝG=\i∈[p] ∃ k s.t. A_i,k \ be the set of globally important features. If j∉Gj∉ G, the condition ∈Ak⋆x∈ A_k does not depend on the value of xjx_j. Consequently, for any l∈Ck⋆⊆Ak⋆x_l∈ C_k A_k , any other point X that agrees with lx_l on all other coordinates (i.e., −j=l,−jX_-j=x_l,-j) must also lie within Ak⋆A_k . This implies: f−j(l) f_-j(x_l) =[f()|−j=l,−j] =E[f(X)|X_-j=x_l,-j] =[∑k=1mfk()∈Ak|−j=l,−j] =E [ _k=1^mf_k(X) 1_X∈ A_k |X_-j=x_l,-j ] =[fk⋆()|−j=l,−j] =E[f_k (X)|X_-j=x_l,-j] The last step holds because the indicator ∈Ak⋆ 1_X∈ A_k is always 1 under the conditioning, while all other indicators ∈Ak′ 1_X∈ A_k for k′≠k⋆k ≠ k are 0. The resulting expression for f−j(l)f_-j(x_l) depends only on the definition of fk⋆f_k . Since both f(l)f(x_l) and f−j(l)f_-j(x_l) depend solely on the parameters (k⋆,bk⋆)(a_k ,b_k ) for any observation in Ck⋆C_k , each term in the summation for Ψ^j _j is a function of fk⋆f_k alone. The attribution, being an average of these terms, is therefore completely independent of the coefficients (k′,bk′)(a_k ,b_k ) from any other region k′≠k⋆k ≠ k . This concludes the proof. B.1 Analysis under Cluster Contamination. We now consider the practical scenario where the cluster is contaminated, meaning Ck⋆⊈Ak⋆C_k A_k . Let Ck⋆′=Ck⋆∩Ak⋆C _k =C_k ∩ A_k be the set of correctly clustered points and Ck⋆′=Ck⋆∖Ak⋆C _k =C_k A_k be the set of contaminating points. For any point l∈Ck⋆′x_l∈ C _k , there exists some region Ak′A_k with k′≠k⋆k ≠ k such that l∈Ak′x_l∈ A_k . For these contaminating points, the global model evaluates to f(l)=fk′(l)f(x_l)=f_k (x_l). Consequently, the R-LOCO summation for Ψ^j _j becomes a mixture: Ψ^j=1|Ck⋆|(∑l∈Ck⋆′Δj(l)+∑l∈Ck⋆′Δj(l)) _j= 1|C_k | ( _x_l∈ C _k _j(x_l)+ _x_l∈ C _k _j(x_l) ) where the terms Δj _j for points in Ck⋆′C _k depend on fk⋆f_k as shown above, but the terms for points in Ck⋆′C _k depend on the parameters of other local models fk′f_k . As a result, the attribution is no longer local. It becomes "contaminated" by the influence of coefficients from the regions where the misclustered points originate. The magnitude of this effect is proportional to the fraction of contaminating points |Ck⋆′|/|Ck⋆||C _k |/|C_k |. ∎ Appendix C Experimental Details Each dataset is divided into three parts: training, calibration, and test. Initially, we split the data into a training set (75%) and a test set (25%). The training set is then further split evenly into a new training and calibration set. The calibration data is used by each method as background for computing approximations. For the baselines, we compute L-SV using the exact explainer from the SHAP library. For LIME, we utilize the LIME package with default parameters, as described in the original paper [Ribeiro et al., 2016b]. For R-LOCO, we use XGBoost (default parameters) as the base model to approximate the leave-one-out functions f^0 f_0 and f^−j f_-j. We use Affinity Propagation with damping = 0.8. Further implementation details will be given in the code repository. We split our data into two sets: one for model fitting and the other for estimating attribution quantities. Appendix D Limitations of LIME: Detailled The main idea behind LIME [Ribeiro et al., 2016a] is to approximate a complex model using a simpler, more interpretable model, such as a linear model, in the vicinity of a given input instance. Given a model f, the local explanation ξ(⋆)ξ( x ) of an instance ⋆ x is an interpretable model g∈Gg∈ G, where G is the set of linear models, such that ξ(⋆)=argming∈Gℒ(f,g,π⋆h)+Ω(g), ξ( x )= _g∈ GL(f,g, _ x ^h)+ (g), (6) where ℒ(f,g,πx⋆h)L(f,g, _x ^h) measure of how unfaithful g is in approximating f over π⋆h _ x ^h, a measure of locality around x⋆x with width h, and Ω(g) (g) is a measure of complexity of the local model g. The loss ℒL is defined as ℒ(f,g,πx⋆h)=∑x′∼P′[f(x′)−g(x′)]2πx⋆h(′). (f,g, _x ^h)= _x P [f(x )-g(x ) ]^2 _x ^h( x ). In the original implementation [Ribeiro et al., 2016a], π⋆h _ x ^h is a Gaussian kernel, and the sum is taken over samples ′∼P′ x P where P′=∏iPXiP = _iP_X_i is the marginal law of the features. Figure 4: Distribution of LIME coefficients for X1X_1 and X3X_3 among observations with negative x5x_5 for the model (1) Figure 5: Variability of LIME coefficients wrt bandwidth selection on the German credit dataset (n=1000,p=20)(n=1000,p=20) from UCI [Dua and Graff, 2017] A primary concern with LIME stems from its reliance on arbitrary heuristics in its definition. Specifically, choosing the sampling distribution P′P poses challenges, as the commonly used distribution disregards feature dependencies, and there is no guarantee that the model’s local behavior on P′P will be consistent with that on the observed data. Another significant issue lies in defining the neighborhood πx⋆h _x ^h and tuning the kernel width h, especially in high-dimension. The stability and sensitivity of LIME are heavily influenced by the selection of the perturbation sampling distribution, the definition of proximity πx⋆h _x ^h and the bandwidth h, which may result in varying explanations for the same instance under slightly varying settings. To illustrate this issue, we apply LIME on the piece-wise linear model defined in (1), with a1=0,a2=2,a3=0,a4=5a_1=0,a_2=2,a_3=0,a_4=5. In Figure 5, the distribution of LIME coefficients for X1X_1 and X3X_3 among observations with negative x6x_6 values shows that LIME assigns a non-null score to both X1X_1 and X3X_3, although the latter is not used locally for the displayed observations. Thus, LIME shares the same problem as L-SV in the piece-wise linear model with independent variables. It is also important to note that such discontinuities are not uncommon, as tabular data often contain discontinuities with categorical variables, and the meaning of constructing a linear approximation in such cases remains unclear. Besides this empirical evidence, we are currently working to compute the theoretical quantity of LIME coefficient for continuous piece-wise-linear functions, yielding impossible results similar to Thereom 2.1 for Local Shapley Values. To demonstrate the sensitivity of LIME to bandwidth selection, we applied it to the German credit dataset (n=1000,p=20)(n=1000,p=20) from UCI [Dua and Graff, 2017]. We trained a RF on 80%80\% of the dataset and computed LIME coefficients on the remaining data using two bandwidths, h and h′h , of the proximity kernel π⋆h _ x ^h. We set h using the median heuristic [Fukumizu et al., 2009, Flaxman et al., 2016, Garreau et al., 2017], where h is the median of the pairwise distance of ‖i−j‖2|| X_i- X_j||_2 and assigned h′=12h = 12h. In Figure 5, we compare the relative absolute error of the LIME coefficients for all variables using the two bandwidths, i.e., (Łih−Lih′)/Lih( _ x_i^h-L_ x_i^h )/L_ x_i^h, where Łih,Łih′ _ x_i^h, _ x_i^h represent the LIME coefficient of the variable XiX_i using bandwith h,h′h,h respectively. It reveals that the difference between the LIME coefficients could be much different after slightly modifying the bandwidth. Furthermore, we observed that 20%20\% of the coefficients also changed signs. In real-world scenarios, we lack information about the true local importance, making the bandwidth selection process indefinite. Moreover, LIME exhibits issues related to instability or irreproducibility. For example, Zhang et al. [2019], Zafar and Khan [2019], Visani et al. [2020] have shown that repeated runs of the same setting of the algorithm on the same model and data point can yield different results. This inconsistency stems from the randomness introduced during the generation of the synthetic sample around the input. Several works [Zafar and Khan, 2019, Zhou et al., 2021] attempt to address this issue, using asymptotic analysis of the method to identify the minimum number of the sampled observations required for stability. Another aspect of stability is related to input perturbations. Alvarez-Melis and Jaakkola [2018] show that nearby observations may have completely different LIME coefficients. Appendix E Visual Description of R-LOCO Figure 6: This diagram presents and contrasts our proposed method (R-LOCO) with the global approach (LOCO). Both approaches first transform the features into the feature importance space. The global approach (in grey) then averages the importance scores column-wise across all observations, while R-LOCO (in green) averages the scores within each cluster, providing a more localized attribution. However, note that nothing stops us from applying any other global methods after the clusters are identified. The estimation of the region-based attributions is very generic; we could simply learn a linear model within each identified region and use it as a global explanation, or apply any more advanced global attribution techniques. For example, we could re-estimate the f^0 f_0, and f^−j f_-j of Δj^(,Y)=V(f^0(),Y)−V(f^−j(),Y) _j( X,Y)=V ( f_0( X),Y )-V ( f_-j( X),Y ) used to estimate Ψ^j(P^A^k) _j( P_ A_k) within the corresponding cluster instead of using the one learned to identify the clusters. We found that both strategies perform similarly in our experiments. Appendix F Learned Tree of Section of 5.4 The decision rules defining the regions shown in Section 5.4 are: Regions identified by R-LOCO in feature-importance space |--- X6 <= 0.00 | |--- X5 <= -0.86 | | |--- class: 0 | |--- X5 > -0.86 | | |--- class: 0 |--- X6 > 0.00 | |--- X4 <= -0.56 | | |--- class: 1 | |--- X4 > -0.56 | | |--- class: 0 Regions identified by clustering in the input space |--- X5 <= -0.01 | |--- X2 <= -0.00 | | |--- class: 0 | |--- X2 > -0.00 | | |--- class: 1 |--- X5 > -0.01 | |--- X6 <= 0.05 | | |--- class: 2 | |--- X6 > 0.05 | | |--- class: 3 Appendix G Derivations of the centroids for the 1st-order model To formalize the advantage of clustering in importance space, we analyze a model with independent features Xi∼[−1,1]X_i [-1,1] for i=1,…,6i=1,…,6 and the function: f()=(X1+X2)⋅X6≤0+(X3+X4)⋅X6>0.f(X)=(X_1+X_2)·1_X_6≤ 0+(X_3+X_4)·1_X_6>0. This function operates in two distinct regimes based on the "switching" variable X6X_6: RA≔∣X6≤0R_A \x X_6≤ 0\ and RB≔∣X6>0R_B \x X_6>0\. We compare the K-Means (K=2K=2) decision rules derived from clustering in the input space versus the R-LOCO importance space. Clustering in Input Space The true cluster centroids are the conditional means [∣RA]E[X R_A] and [∣RB]E[X R_B]: CA=(0,0,0,0,0,−0.5),CB=(0,0,0,0,0,0.5).C_A=(0,0,0,0,0,-0.5), C_B=(0,0,0,0,0,0.5). This follows from feature independence ([Xi∣RA]=[Xi]=0E[X_i R_A]=E[X_i]=0 for i≤5i≤ 5) and the uniform distribution ([X6∣X6≤0]=−0.5E[X_6 X_6≤ 0]=-0.5). The K-Means separating direction is CA−CB=(0,0,0,0,0,−1)C_A-C_B=(0,0,0,0,0,-1). The decision rule, d(,CA)2<d(,CB)2d(x,C_A)^2<d(x,C_B)^2, simplifies to a condition based only on X6X_6. Therefore, input-space clustering merely rediscovers the "switching" variable X6X_6. The variability from the functionally active features (X1,…,X4X_1,…,X_4) acts as noise, hindering a clean separation. Clustering in R-LOCO Importance Space Each sample X is mapped to its R-LOCO importance vector γ()=(Δ1,…,Δ6)γ(X)=( _1,…, _6), where Δj()=(f()−f−j())2 _j(X)=(f(X)-f_-j(X))^2 and f−j()=[f()∣−j=−j]f_-j(X)=E[f(Y) _-j=X_-j]. Consider ∈RAX∈ R_A, where X6≤0X_6≤ 0 and f()=X1+X2f(X)=X_1+X_2. We find the components Δj _j and their expectations [Δj∣RA]E[ _j R_A]: • j=1,2j=1,2 (Active): f−1()=[Y1+X2]=X2f_-1(X)=E[Y_1+X_2]=X_2. – Δ1=(X1+X2−X2)2=X12 _1=(X_1+X_2-X_2)^2=X_1^2. – [Δ1∣RA]=[X12]=1/3E[ _1 R_A]=E[X_1^2]=1/3. By symmetry, [Δ2∣RA]=1/3E[ _2 R_A]=1/3. • j=3,4,5j=3,4,5 (Inactive): f−3()=[X1+X2]=X1+X2f_-3(X)=E[X_1+X_2]=X_1+X_2. – Δ3,4,5=(X1+X2−(X1+X2))2=0 _3,4,5=(X_1+X_2-(X_1+X_2))^2=0. – [Δ3,4,5∣RA]=0E[ _3,4,5 R_A]=0. • j=6j=6 (Switching): f−6()=Y6[(X1+X2)Y6≤0+(X3+X4)Y6>0]=12(X1+X2)+12(X3+X4)f_-6(X)=E_Y_6[(X_1+X_2)1_Y_6≤ 0+(X_3+X_4)1_Y_6>0]= 12(X_1+X_2)+ 12(X_3+X_4). – Δ6=((X1+X2)−12(X1+X2+X3+X4))2=14(X1+X2−X3−X4)2 _6= ((X_1+X_2)- 12(X_1+X_2+X_3+X_4) )^2= 14(X_1+X_2-X_3-X_4)^2. – [Δ6∣RA]=14[X12+X22+X32+X42]=14(4⋅13)=1/3E[ _6 R_A]= 14E[X_1^2+X_2^2+X_3^2+X_4^2]= 14(4· 13)=1/3. The importance-space centroid for RAR_A is Cγ,A=(13,13,0,0,0,13)C_γ,A= ( 13, 13,0,0,0, 13 ). By symmetry, for ∈RBX∈ R_B (where f()=X3+X4f(X)=X_3+X_4), the centroid is Cγ,B=(0,0,13,13,0,13)C_γ,B= (0,0, 13, 13,0, 13 ). Decision Rule and Interpretation The K-Means separating direction in this space is: Cγ,A−Cγ,B=(13,13,−13,−13,0,0).C_γ,A-C_γ,B= ( 13, 13,- 13,- 13,0,0 ). This decision rule only depends on the components (Δ1,Δ2,Δ3,Δ4)( _1, _2, _3, _4) corresponding to the locally active features. Clustering in importance space successfully identifies functional similarity: • RAR_A samples have large (Δ1,Δ2)( _1, _2) and Δ3,4≈0 _3,4≈ 0. • RBR_B samples have large (Δ3,Δ4)( _3, _4) and Δ1,2≈0 _1,2≈ 0. In short, input-space clustering is confounded by noise from irrelevant features and only finds the geometric switch X6X_6. Importance-space clustering is robust to this noise and correctly groups samples by their underlying explanatory behavior. Remark: Importance Representation and Failure Cases We note a known counter-example: f(X1,X2)=X2⋅sgn(X1)f(X_1,X_2)=X_2·sgn(X_1), with regimes RA=X1<0R_A=\X_1<0\ and RB=X1≥0R_B=\X_1≥ 0\. If we use the squared-loss importance Δj=(f()−f−j())2 _j=(f(X)-f_-j(X))^2, we find f−1=f−2=0f_-1=f_-2=0. This results in an importance vector γ()=(Δ1,Δ2)=(f()2,f()2)=(X22,X22)γ(X)=( _1, _2)=(f(X)^2,f(X)^2)=(X_2^2,X_2^2). This representation is identical for both regimes, making them inseparable. The issue is that the squared loss Δj _j is sign-agnostic and discards the directional information that distinguishes the regimes. This motivates enriching the importance representation. For instance, including the signed R-LOCO residual, δj()=f()−f−j() _j(X)=f(X)-f_-j(X), yields an enriched vector: γ′()=(Δ1,Δ2,δ1,δ2)=(X22,X22,f(),f()).γ (X)=( _1, _2, _1, _2)=(X_2^2,X_2^2,f(X),f(X)). This enriched vector maps the two regimes to distinct, separable locations: γ′()=(X22,X22,−X2,−X2)if X1<0 (Regime RA),(X22,X22,X2,X2)if X1≥0 (Regime RB).γ (X)= cases(X_2^2,X_2^2,-X_2,-X_2)&if X_1<0 (Regime R_A),\\ (X_2^2,X_2^2,X_2,X_2)&if X_1≥ 0 (Regime R_B). cases In this space, the regimes are easily distinguished by a clustering algorithm, underscoring the importance of the chosen representation. Appendix H Evaluation methodology for Real-World Data H.1 Definitions We begin by outlining the core components of our evaluation framework: • Dataset D: A collection of n samples. Each sample is a pair (,y)(x,y), where x denotes the input and y the ground-truth target. • Input x: A d-dimensional vector of numerical features. • Model f()f(x): The predictive model that maps x to a prediction. • Attribution Method A()A(x): A function that assigns a positive importance score to each of the d input features. These scores correspond to normalized attributions from each explanation method (e.g., R-LOCO, SHAP). • Error Function E(,y)E(x,y): A metric quantifying the discrepancy between the model’s prediction for x and the true label y: – Classification: E(,y)=1E(x,y)=1 if the prediction is incorrect, 0 otherwise. – Regression: E(,y)E(x,y) may be the absolute difference between the predicted and true values. H.2 Masking Procedure To assess feature importance, we apply a masking (or occlusion) technique as follows: 1. For each input x, compute the feature importance scores using A()A(x). 2. Identify the indices of: • The top-k most important features (Top-k Indices). • The bottom-k least important features (Bottom-k Indices). 3. Create two modified versions of x: masked,top-k ^masked,top-k :Replace values at Top-k indices with zero, :Replace values at Top-$k$ indices with zero, masked,bottom-k ^masked,bottom-k :Replace values at Bottom-k indices with zero. :Replace values at Bottom-$k$ indices with zero. H.3 Error Calculation We compute the average model error over the dataset under three conditions: Original Average Error (Baseline): AvgErrorOriginal=1n∑(,y)∈E(,y)AvgError_Original= 1n _(x,y) E(x,y) Top-k Masked Average Error: AvgErrorTop-k=1n∑(,y)∈E(masked,top-k,y)AvgError_Top-k= 1n _(x,y) E(x^masked,top-k,y) Bottom-k Masked Average Error: AvgErrorBottom-k=1n∑(,y)∈E(masked,bottom-k,y)AvgError_Bottom-k= 1n _(x,y) E(x^masked,bottom-k,y) H.4 Final Metrics for Figures The values shown in Figures 1 and 2 represent the change in model error due to masking, computed as follows: Top-k Error Change (Left Figure): Indicates the degradation in performance after removing the Top-k most important features: ChangeInErrorTop(k)=AvgErrorOriginal−AvgErrorTop-k.ChangeInError_Top(k)=AvgError_Original-AvgError_Top-k. Bottom-k Error Change (Right Figure): Reflects the performance change after removing the Top-k least important features: ChangeInErrorBottom(k)=AvgErrorOriginal−AvgErrorBottom-k.ChangeInError_Bottom(k)=AvgError_Original-AvgError_Bottom-k. Appendix I Runtime and Memory Usage Analysis We find that R-LOCO is significantly faster at inference time than the baselines. While our method incurs additional computational cost during training (due to clustering and model fitting), it performs inference with minimal overhead, as only the cluster assignment is required per sample. A similar pattern is observed for memory usage. In terms of scalability, R-LOCO’s training cost grows approximately linearly with the number of features. This behaviour is consistent with widely used explanation approaches such as SHAPLoss and its variants [Covert et al., 2020a]. Importantly, this training step is performed only once. Moreover, R-LOCO can be further optimised for tree-based models using efficient approximation methods such as Projected Forests [Bénard et al., 2021, I. Amoukou and Brunel, 2022], where a single model can be reused to compute multiple conditional expectations efficiently. Method Train (s) Infer (s) Total (s) Train (MB) Infer (MB) Peak (MB) R-LOCO 33.521 0.032 33.553 1687.9 8.6 2743.0 LIME 0.038 34.684 34.722 0.5 2.3 1.0 SHAP 0.002 20.698 20.700 0.0 0.8 6.8 Table 3: Runtime and memory usage comparison on the California dataset. Method Train (s) Infer (s) Total (s) Train (MB) Infer (MB) Peak (MB) R-LOCO 1.611 0.010 1.621 12.2 0.5 1.7 SHAP 0.002 1.433 1.436 0.0 145.0 16.7 LIME 0.008 0.838 0.846 0.2 1.3 0.4 Table 4: Runtime and memory usage comparison on the Diabetes dataset. NeurIPS Paper Checklist 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: Yes, the main claims in the abstract and introduction accurately reflect the paper’s contributions and scope. We clearly state the identified limitations of L-SV and LIME. Specifically, we argue that a sound local attribution method should not assign importance to features that do not influence the model output (e.g., features with zero coefficients in a linear model), and lack statistical dependence with functionality-relevant features. We introduce R-LOCO as a method to bridge local and global explanations, which overcomes these limitations. These claims are supported by theoretical results (Section 2, Theorem 2.1; Section 4.1, Theorem 4.1, Theoretical analysis in §5.4.1) and empirical evidence in controlled (Section 5) and real-world settings (Section 6). Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 2. Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Most of the experiments and theoretical results in this paper focus on piecewise linear models. Extending these results to other settings should be done with caution, particularly because the notion of a "ground truth explanation" is difficult to define. Our second theorem on the utility of R-LOCO is proved under the condition that clusters either align with or are fully contained within the true underlying regions. To address more realistic scenarios, we also provide an analysis of cases where this assumption does not hold (See in §B.1 or 5.4.1), offering deeper insight into R-LOCO’s behavior under imperfect clustering. While the independence assumption may be seen as a limitation, we made it primarily to isolate the effects of attribution without confounding dependencies. In addition, we do evaluate R-LOCO on real-world datasets that naturally include feature dependencies, and our results demonstrate that the method remains effective and better than the baselines in these settings. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper. • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 3. Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: • For Theorem 2.1 (Limitations of L-SV for piecewise linear functions): The assumptions (piecewise linear function f with m components, disjoint hyperrectangle regions AkA_k, independent covariates) are stated within the theorem. The full proof is provided in Appendix A. • For Theorem 4.1 (R-LOCO behaviour for piecewise linear functions): The assumptions (piecewise linear function from Theorem 2.1, R-LOCO clustering algorithm correctly identifying regions corresponding to or included in AkA_k) are stated. A detailed proof is in Appendix B. • For the theoretical analysis of R-LOCO in §B.1, all assumptions and complete proofs are also given. Guidelines: • The answer NA means that the paper does not include theoretical results. • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced. • All assumptions should be clearly stated or referenced in the statement of any theorems. • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 4. Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: Indeed, we have provided all essential details required to conduct our experiments. Details regarding model setups (Section 5.1), R-LOCO variants (Section 5), evaluation metrics (Section 5.2), baseline parameters and R-LOCO implementation specifics (Appendix C), and real-world experiment configurations (Section 6) are included. The code will be released upon acceptance. Guidelines: • The answer NA means that the paper does not include experiments. • If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We report on generating the synthetic experiments (Section 5.1), citations for the open source data used and commit to releasing the code upon acceptance. Guidelines: • The answer NA means that paper does not include experiments requiring code. • Please see the NeurIPS code and data submission guidelines (https://nips.c/public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.c/public/guides/CodeSubmissionPolicy) for more details. • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 6. Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: Yes, the experimental details are all given. Specifically, data splitting procedures are described in Appendix C. Hyperparameters for baseline methods like LIME (defaults) and R-LOCO’s Affinity Propagation (damping=0.8) are mentioned in Appendix C. Section 5.3 explores variations in clustering parameters for R-LOCO. The models being explained (e.g., piecewise linear, XGBoost) are also specified. Guidelines: • The answer NA means that the paper does not include experiments. • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 7. Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: The analysis section (Section 5.3) mentions that results presented are aggregated over 50 runs. Table 1 and Table 2 report mean values and quantile ranges (q0.1 – q0.95 for ’NI attr.’), which provide information about the distribution and variability of results. Guidelines: • The answer NA means that the paper does not include experiments. • The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We execute all experiments on a MacBook Pro M2, with none of them being particularly costly to conduct. All Results are obtainable within a few hours. Guidelines: • The answer NA means that the paper does not include experiments. • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper). 9. Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.c/public/EthicsGuidelines? Answer: [Yes] Justification: The research conforms to the NeurIPS Code of Ethics. Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics. • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: This paper focuses primarily on the positive social impact by improving model understanding and trustworthiness. This work advances the analysis of widely used explainability tools that play a crucial role in high-stakes fields such as healthcare. We highlight their limitations and propose a more theoretically sound and empirically robust approach to improve interpretability and reliability in model explanations. Guidelines: • The answer NA means that there is no societal impact of the work performed. • If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 11. Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [N/A] Justification: Not applicable. The paper does not release new large-scale models or datasets with high intrinsic risk for misuse. Guidelines: • The answer NA means that the paper poses no such risks. • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Yes, all data are correctly cited. Guidelines: • The answer NA means that the paper does not use existing assets. • The authors should cite the original paper that produced the code package or dataset. • The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., C-BY 4.0) should be included for each asset. • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset’s creators. 13. New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [N/A] Justification: Not applicable. The paper primarily introduces a new method, R-LOCO. Code for this method is a new asset that is committed to be released upon acceptance (as stated in answers to Q4 and Q5). Guidelines: • The answer NA means that the paper does not release new assets. • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 14. Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [N/A] Justification: Not applicable. Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [N/A] Justification: Not applicable. Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 16. Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required. Answer: [N/A] Justification: Not applicable. Guidelines: • The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components. • Please refer to our LLM policy (https://neurips.c/Conferences/2025/LLM) for what should or should not be described.