Paper deep dive
Assessing Alignment and Stability of Feature Importance Explanations via Weight of Evidence
Eddie Conti, Claudio Daka, Álvaro Parafita, Antonio L. Alfeo, Axel Brando, Mario G. C. A. Cimino
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/2/2026, 4:13:59 AM
Summary
This paper introduces a framework for evaluating Feature Importance Methods (FIMs) using Weight of Evidence (WoE). It treats FIM outputs as stochastic evidence to test hypotheses about feature relevance, which can be derived from domain knowledge, ground truth, or the FIM's own mean attribution. Theoretical results link WoE to attribution variance, showing that low variability leads to high WoE scores. Empirical evaluations on LIME and SHAP demonstrate the framework's ability to assess alignment and stability.
Entities (6)
Relation Signals (6)
Weight of Evidence → usedfor → Feature Importance Methods
confidence 95% · we introduce a novel perspective by embedding FIMs within a hypothesis-testing framework based on Weight of Evidence (WoE).
Weight of Evidence → correlateswith → Attribution Variance
confidence 92% · We further provide theoretical results linking WoE to attribution variance.
Weight of Evidence → measures → Alignment
confidence 90% · This formulation enables a principled evaluation of FIMs, capturing both their alignment with prior knowledge and their variability.
Weight of Evidence → measures → Stability
confidence 90% · This formulation enables a principled evaluation of FIMs, capturing both their alignment with prior knowledge and their variability.
LIME → evaluatedby → Weight of Evidence
confidence 85% · Empirical results shows the applicability and flexibility of our strategy analyzing LIME and SHAP explanations
SHAP → evaluatedby → Weight of Evidence
confidence 85% · Empirical results shows the applicability and flexibility of our strategy analyzing LIME and SHAP explanations
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Feature importance Methods (FIMs) are widely used in Explainable AI to interpret model predictions, yet attribution scores alone often provide limited insight into the underlying reasoning process. In this work, we introduce a novel perspective by embedding FIMs within a hypothesis-testing framework based on Weight of Evidence (WoE). We quantify how strongly the observed evidence supports any given hypothesis on feature importance. The reference hypothesis can stem from domain knowledge, ground truth, or be derived from the FIM itself. This formulation enables a principled evaluation of FIMs, capturing both their alignment with prior knowledge and their variability. We further provide theoretical results linking WoE to attribution variance. Empirical results shows the applicability and flexibility of our strategy analyzing LIME and SHAP explanations in settings with different reference hypotheses. Overall, our framework offers a complementary tool for assessing FIMs through a contrastive, evidence-based lens.
Tags
Links
- Source: https://arxiv.org/abs/2609.00090v1
- Canonical: https://arxiv.org/abs/2609.00090v1
Trouble viewing inline? Open PDF directly →
Full Text
45,491 characters extracted from source content.
Expand or collapse full text
Assessing Alignment and Stability of Feature Importance Explanations via Weight of Evidence Eddie Conti1 (✉) Claudio Daka2,4 Affiliation: SMARTEST Research Center, eCampus University, Novedrate, Italy E-mail antonioluca.alfeo@uniecampus.it Álvaro Parafita1 Antonio L. Alfeo3 Axel Brando1 Mario G.C.A. Cimino4 E-mail parafita.alvaro@gmail.com E-mail axelbrando@gmail.com Affiliation: Barcelona Supercomputing Center, Barcelona, Spain E-mail econti@bsc.es Affiliation: University of Florence, Florence, Italy E-mail claudio.daka@unifi.it Affiliation: Dept. of Theoretical and Applied Sciences, eCampus University, Novedrate, Italy Affiliation: Dept Information Engineering, University of Pisa, Pisa, Italy E-mail mario.cimino@unipi.it Abstract Feature importance Methods (FIMs) are widely used in Explainable AI to interpret model predictions, yet attribution scores alone often provide limited insight into the underlying reasoning process. In this work, we introduce a novel perspective by embedding FIMs within a hypothesis-testing framework based on Weight of Evidence (WoE). We quantify how strongly the observed evidence supports any given hypothesis on feature importance. The reference hypothesis can stem from domain knowledge, ground truth, or be derived from the FIM itself. This formulation enables a principled evaluation of FIMs, capturing both their alignment with prior knowledge and their variability. We further provide theoretical results linking WoE to attribution variance. Empirical results shows the applicability and flexibility of our strategy analyzing LIME and SHAP explanations in settings with different reference hypotheses. Overall, our framework offers a complementary tool for assessing FIMs through a contrastive, evidence-based lens. Keywords: Explainability Weight of Evidence Feature Importance Methods. 1 Introduction As Isaac Asimov’s would say “[…] science gathers knowledge faster than society gathers wisdom”, the rapid advancement of machine learning has led to the development of increasingly complex models, often without a corresponding understanding of their internal mechanisms. Explainable AI (XAI) approaches can be employed to address this issue thanks since those provide insights into how information is processed and how predictions are generated within AI black-box models. This is particularly critical in high-stakes domains, where interpretability is essential to ensure trust, accountability, and transparency [8, 4]. In this context, numerous explanation techniques have been proposed, including feature importance methods, counterfactual explanations, prototypes, and rule-based approaches [12, 7, 17]. However, in the literature, empirical studies often rely on researchers’ implicit assumptions about what qualifies as a “good explanation”, as notions such as interpretability or explainability are themselves inconsistently defined and lack universally accepted evaluation criteria (e.g., how to ensure transparency) [18]. As noted by Miller [22], this creates a tension between how explanations are designed and how they are cognitively perceived. To reduce this gap, recent works draw on cognitive and social sciences to align XAI with human reasoning [13, 2, 3]. These studies show that explanations are inherently contrastive, humans explain P relative to an alternative Q, and selective, focusing on a few salient factors. In XAI, this is often formalized through sparsity or simplicity, namely limiting explanations to a small number of features [15], allowing for more knowledge transfer. In this work, we build on this perspective and adopt the Weight of Evidence (WoE) framework [29], a principled statistical tool for comparing alternative hypotheses. We reinterpret Feature Importance11 1 We may refer to FIM as explainer or explanation method and to the vector of feature importances as attribution vector. (either coming from domain knowledge, intrinsic mechanisms in the model or the FIM itself) as testable hypotheses, and use WoE to quantify how strongly an stochastic FIM supports them. We summarize the contributions of this study as: • A formal strategy for adapting WoE to test hypotheses on FIMs. Depending on the reference hypothesis, our framework can: – Test alignment between local and global explanations. – Measure faithfulness and ground-truth alignment – Capture the stability of stochastic FIMs. • A proof showing how low FIM variability leads to +∞+∞ WoE when the reference hypothesis is generated using its mean attribution. The remainder of the paper is structured as follows: Section 2 reviews related work; Section 3 introduces the WoE framework and its adaptation to stochastic FIMs; Section 4 establishes the theoretical connection between attribution variability and WoE positive divergence; Section 5 presents the experimental evaluation across three complementary settings; Section 6 discusses limitations of our proposal; and Section 7 concludes with future directions. 2 Related Work In the XAI literature several metrics have been established to measure the different properties of a FIM [15, 1]. However, one aspect that is lacking in the field of functional evaluation is a measure of how well a FIM aligns with a reference hypothesis. To answer this question, we use the Weight of Evidence (WoE) framework, which to the best of our knowledge, has been employed by few works in XAI. Among them, Melis et al. [21] generate explanations sequentially by computing WoE values over progressively refined nested hypotheses. Given an instance x with prediction y∗y^*, the authors construct a sequence of hypotheses y∗⊆UT⊆…⊆U0=Yy^* U_T … U_0=Y, where Y denotes the full output space (for instance, all possible classes). By evaluating the WoE associated with each refinement step, they quantify how the available evidence (i.e. the instance x) increasingly supports more specific hypotheses. On the other hand, Alshehri et al. [3] apply WoE in the context of Goal Recognition (GR), where the objective is to infer an agent’s intended goal from a sequence of observed actions. In this setting, partial trajectories serve as evidence, and WoE quantifies how strongly the observed actions, namely the observed steps made by the agent, support one candidate goal over competing alternatives. The framework naturally fits this scenario, as it explicitly formalizes the comparison between alternative hypotheses. Le et al. [16] combine concept-based explanations with WoE. They extract interpretable concepts through both supervised and unsupervised methods and compute their WoE with respect to a target hypothesis h corresponding to a specific prediction. Since concept extraction is naturally applied to images, this work presents an extension of WoE from tabular data to images. A similar approach, is presented in [24], where the authors use a custom metric to quantify the presence or absence of concepts in an explanation, and compute the WoE w.r.t. a hypothesis defined as a set of target predictions in an image classification setting. In contrast to prior works, which leverages WoE to generate or structure explanation, we adopt a different approach. Instead of using WoE as an explanatory mechanism, we treat an explanation, either coming from the FIM itself or from domain knowledge, as a testable hypothesis about the relevance of features. In this perspective, the explanation specifies which features are and are not considered to be responsible for the model’s decision, and the WoE is employed to quantify how much we can support this claim by analyzing the behavior of the FIM across various runs. 3 Weight of Evidence The Weight of Evidence (WoE) is a statistical concept from information theory [29], which refers to strength of an evidence e in supporting an hypothesis h in contrast to an alternative hypothesis h′h . In formal terms, the relation is expressed in log-odds terms as: woe(h:e)=logO(h∣e)O(h),O(h∣e)=p(h∣e)p(h¯∣e),O(h)=p(h)p(h¯).woe(h:e)= O(h e)O(h), O(h e)= p(h e)p( h e), O(h)= p(h)p( h). (1) Using Bayes’ Theorem, equation (1) reduces to the so-called evidence likelihood form: woe(h:e)=logp(e∣h)p(e∣h′).woe(h:e)= p(e h)p(e h ). (2) If woe>0woe>0, the evidence e supports h over its alternative h′h . Combining equations (1) and (2) via Bayes’ theorem yields the identity that will be useful later on: logp(e∣h)p(e∣h′)=logp(h∣e)p(h′∣e)+logp(h′)p(h). p(e h)p(e h )= p(h e)p(h e)+ p(h )p(h). (3) Equation (3) connects the WoE with prior and posterior log-odds. In other words, one may consider the WoE as how much to update the prior log odd considering the evidence e. To simplify the probability calculations, in this paper we assume that h′=h¯h = h is the complementary explanation, yet the framework applies to any alternative hypothesis h′h . 3.1 Weight of Evidence for Explanation Qualification We apply the WoE framework to N executions of a FIM on a fixed instance x. Let α(j)=(α1(j),…,αd(j))∈ℝdα^(j)=( _1^(j),…, _d^(j)) ^d denote the attribution vector produced at run j=1,…,Nj=1,…,N. The set of runs provides empirical evidence about the explanatory behavior of the FIM for the given instance. Inspired by rule-based formulations [26, 9], from each attribution vector one derives a binary hypothesis about which features are relevant. Specifically, we consider the set of relevant a non-relevant features: ℛx=i:feature i is relevant,x⊆1,…,d∖ℛx.R_x=\i:feature i is relevant\, _x \1,…,d\ _x. Invoking the principle of sparsity, we focus only on the relevant set and write the hypothesis as: h=⋀i∈ℛx(i is relevant).h= _i _x(i is relevant). (4) The precise procedure mapping a continuous attribution vector α to a set ℛxR_x (and hence to a hypothesis) is detailed afterwards in Section 3.3. Let us demonstrate a first result that will then allow us to obtain the general condition for this to happen. Importantly, the reference hypothesis (that is, the one we want to check whether the FIM supports it or not) h∗h^* can originate from two sources. First, it can be derived from the evidence itself by applying the selection procedure to the empirical mean attribution α¯=1N∑j=1Nα(j) α= 1N _j=1^Nα^(j); in this case WoE captures the explainer stability22 2 We have chosen to refer to these properties by convention to explain what aspects we are measuring. In the case of stability, which has been formalized in various ways, in this paper it coincides with the variability of the explainer.. Second, it can reflect external information, such as ground-truth relevant features or domain knowledge; in this case WoE measures the alignment†footnotemark: between the FIM and the reference explanation, while accounting for the aforementioned FIM’s intrinsic variability. 3.2 Quantifying the Weight of Evidence for a Stochastic FIM Within this framework, WoE measures how strongly the empirical evidence supports the hypothesis h over an alternative, thus providing a contrastive, quantitative criterion for explanation qualification. Using equation (3), quantifying the WoE reduces to measuring the prior and posterior log-odds. When explaining a black-box model with a stochastic FIM (e.g. LIME [25], SHAP [19]), the explanation can be viewed as a random variable α induced by the randomness of the FIM. Prior probabilities. In the absence of prior information, we assume a uniform distribution over all 2d2^d possible binary assignments of relevance to the d features. The hypothesis h in equation (4) is partial: it asserts that the features in ℛxR_x are relevant, but makes no claim about the remaining d−|ℛx|d-|R_x| features. Consequently, the number of binary vectors compatible with h is 2d−|ℛx|2^d-|R_x|, yielding: p(h)=2d−|ℛx|2d=12|ℛx|.p(h)= 2^d-|R_x|2^d= 12^|R_x|. Intuitively, a hypothesis designating fewer features as relevant is assigned higher prior probability, reflecting greater conservativeness. Importantly this is not the only admissible prior. The strategy we define can accommodate any prior derived from different assumptions or knowledge. Posterior probabilities. To estimate p(h∣e)p(h e), we leverage the stochasticity of the FIM. Running it N times on the same instance x yields attribution vectors α(1),…,α(N)α^(1),…,α^(N), from each of which a hypothesis h(j)h^(j) is derived via the selection procedure. The posterior probability is estimated empirically as the proportion of runs that reproduce h: p^(h∣e)=1N∑j=1Nh(j)=h. p(h e)= 1N _j=1^N1\h^(j)=h\. (5) This Monte Carlo estimate allows us to compute the posterior log-odds and, consequently, the WoE. 3.3 Feature Selection and Hypothesis Definition In this section, we explain some possibilities for generating hypotheses from an FIM. These choices are common strategies, and other methods may be considered depending on the context and type of analysis. For simplicity, let α∈ℝdα ^d denote a run of an FIM, such that αi≥0 _i≥ 0 (e.g. by the taking the absolute value). Let π the permutation map which sorts the indices of the attribute vector such that απ(i)≥απ(j) _π(i)≥ _π(j) if i<ji<j. 1. Top-k selection (TK). Fix k∈1,…,dk∈\1,…,d\ and set: ℛx=π(1),…,π(k),R_x=\π(1),…,π(k)\, i.e., we retain the first k features according to the ordering induced by the attribution scores. 2. Adaptive selection (AS). Previous strategies require knowledge of the FIM scale or make assumptions about how many variables to consider. We propose an adaptive strategy based on imposing a sufficiency constraint: the selected features explain at least a fraction τ of the total attribution mass for the instance. Assuming, for simplicity, that the attribution scores are normalized so that ∑i=1dαi=1 _i=1^d _i=1, we define k∗(α)=mink∈1,…,d:∑i=1kαπ(i)≥τ,k^*(α)= \! \k∈\1,…,d\: _i=1^k _π(i)≥τ \, (6) and set ℛx=π(1),…,π(k∗(α))R_x=\π(1),…,π(k^*(α))\. Throughout the paper, τ is assumed to be in the interval (0,1)(0,1) and the attribution vector is normalized to 11 to simplify the proofs. Also to simplify the notation, we relabel the indices so that the attribution vector is already sorted in descending order, i.e., α1≥…≥αd _1≥…≥ _d. In both scenarios, the hypothesis derived from the j-th run coincides with the set of relevant features ℛxR_x. 4 Connecting WoE with Attribution Variability The aim of this section is to mathematically characterize what the WoE is measuring when the reference hypothesis h∗h^* is generated from the mean attribution vector over N runs. The main result is that as attribution variability goes to 00, WoE diverges to +∞+∞. Therefore, in this setting, the WoE provides a direct quantification of the variability of the FIM. Recall from equation 5 that p^(h∗∣e) p(h^* e) measures how frequently the FIM reproduces h∗h^* across runs. Intuitively, if the attribution vectors concentrate around their means, then both the selected feature set and its cardinality remain stable, so nearly every run reproduces h∗h^* and p^(h∗∣e)→1 p(h^* e)→ 1. We formalize this in two steps: first we show that k∗k^* is stable (Section 4.1)—which is needed in the case of the AS strategy—, then that the selected feature set is stable (Section 4.2). We begin by recalling a standard probability inequality. Lemma 1 For any events A1,…,AnA_1,…,A_n, p(⋂i=1nAi)≥1−∑i=1np(Aic).p\! ( _i=1^nA_i )≥ 1- _i=1^np(A_i^c). 4.1 Stability of the Number of Selected Features For TK the cardinality |ℛx|=k|R_x|=k is fixed by construction, so there is nothing to prove. For AS, the value k∗(α(j))k^*(α^(j)) may vary across runs; the following result shows it concentrates on k†:=k∗(α¯)k :=k^*( α) when the attribution variances are small. Proposition 1 (Stability of k∗k^* under low variability) Let α=(α1,…,αd)α=( _1,…, _d) be a mutually independent, stochastic FIM with expected values μi=[αi] _i=E[ _i], and variances σi2=Var(αi) _i^2=Var( _i). Further assume that ∑i=1dαi=1 _i=1^d _i=1 and that the expected values are pairwise distinct and ordered so that μ1>⋯>μd _1>·s> _d. Let k†k be the solution to (6) computed at μ=(μ1,…,μd)μ=( _1,…, _d), and suppose there exists a margin δ>0δ>0 such that: ∑i=1k†μi≥τ+δand∑i=1k†−1μi≤τ−δ. _i=1^k _i≥τ+δ _i=1^k -1 _i≤τ-δ. (7) Then for every ε∈(0,1) ∈(0,1): ∑i=1k†σi2≤εδ22⟹p(k∗(α)=k†)≥1−ε. _i=1^k _i^2≤ \, δ^22 p\! (k^*(α)=k )≥ 1- . Proof Without loss of generality assume features are already ordered by decreasing mean, so πμ=id _μ=id. Define the partial sums Sk=∑i=1kαiS_k= _i=1^k _i. By mutual independence of the αi _i and Chebyshev’s inequality: p(|Sk−∑i=1kμi|≥δ)≤∑i=1kσi2δ2≤ε2,p\! (|S_k- _i=1^k _i|≥δ )≤ _i=1^k _i^2δ^2≤ 2, since ∑i=1k†σi2≤εδ22 _i=1^k _i^2≤ δ^22. Similarly, ∑i=1k†−1σi2≤∑i=1k†σi2≤εδ22 _i=1^k -1 _i^2≤ _i=1^k _i^2≤ δ^22, so both events A=|Sk†−∑i=1k†μi|<δandB=|Sk†−1−∑i=1k†−1μi|<δA= \|S_k - _i=1^k _i|<δ \ B= \|S_k -1- _i=1^k -1 _i|<δ \ hold each with probability at least 1−ε/21- /2. On the event A∩BA∩ B, the margin assumption (eq. 7) gives: Sk†>τ+δ−δ=τandSk†−1<τ−δ+δ=τ,S_k >τ+δ-δ=τ S_k -1<τ-δ+δ=τ, which implies k∗(α)=k†k^*(α)=k . Applying Lemma 1: p(k∗(α)=k†)≥p(A∩B)≥1−ε.∎p(k^*(α)=k )≥ p(A∩ B)≥ 1- . Proposition 1 guarantees that, under low variability, repeated runs yield a stable value of k∗(α)k^*(α), so the cardinality |ℛx(j)||R_x^(j)| should remain constant across runs. 4.2 Stability of the Selected Feature Set We now show that low variability also implies stability of the identity of the selected features, i.e. the same features are ranked highest across runs with high probability. This reduces to ensuring that every relevant feature consistently outranks every non-relevant one. Lemma 2 Let αi,αj _i, _j be independent with i<ji<j and expected values μi>μj _i> _j and variances σi2,σj2 _i^2, _j^2. If σi2+σj2≤ε(μi−μj)2, _i^2+ _j^2≤ \,( _i- _j)^2, then p(αi>αj)≥1−εp( _i> _j)≥ 1- . Proof Let D=αi−αjD= _i- _j, with expected value Δ=μi−μj>0 = _i- _j>0 and variance σi2+σj2 _i^2+ _j^2. By Chebyshev’s inequality: p(|D−Δ|≥Δ)≤σi2+σj2Δ2≤ε.p(|D- |≥ )≤ _i^2+ _j^2 ^2≤ . Observe that |D−Δ|<Δ|D- |< implies D>0D>0, which means that p(αi>αj)=p(D>0)≥p(|D−Δ|<Δ)≥1−εp( _i> _j)=p(D>0)≥ p(|D- |< )≥ 1- . ∎ Corollary 1 Let α1,…,αd _1,…, _d be pairwise independent with means μ1>μ2>⋯>μd _1> _2>·s> _d. If for every i∈1,…,k†i∈\1,…,k \ and j∈k†+1,…,dj∈\k +1,…,d\: σi2+σj2≤εi,j(μi−μj)2, _i^2+ _j^2≤ _i,j\,( _i- _j)^2, then: p(ℛx(j)=1,…,k†)≥1−∑i=1k†∑j=k†+1dεi,j.p\! (R_x^(j)=\1,…,k \ )≥ 1- _i=1^k _j=k +1^d _i,j. The proof follows immediately from Lemma 1 and Lemma 2 applied to each pair (i,j)(i,j). 4.3 Divergence of WoE under Low Variability To unify notation, in the TK scenario we refer to k as k†k . Let A be the event that k(j)=k†k^(j)=k (assured in the case of TK), and B be the event that the correct k†k features are ranked highest. Combining the results of Sections 4.1–4.2: p(A)≥1−εk,p(B)≥1−εo,p(A)≥ 1- _k, p(B)≥ 1- _o, where εk _k (which is 00 in the case of TK) and εo _o can be made arbitrarily small by reducing the attribution variances through increased sampling. Since A∩BA∩ B implies h(j)=h∗h^(j)=h^*, Lemma 1 gives: p0:=p(h(j)=h∗)≥1−εk−εo.p_0:=p(h^(j)=h^*)≥ 1- _k- _o. Now let X be the number of runs (out of N) for which h(j)=h∗h^(j)=h^*. By independence of runs, X∼Bin(N,p)X (N,p) for some p∈[0,1]p∈[0,1]. Our goal is to estimate the empirical posterior and, in particular, to characterize the probability of observing large values of the posterior probability. As we will show, by sufficiently controlling the variance, we can guarantee with high probability that the posterior exceeds an arbitrary threshold. It is immediate to observe that the empirical posterior in eq. 5 satisfies p^(h∗∣e)=X/N≥λ p(h^* e)=X/N≥λ if and only if X≥⌈λN⌉X≥ λ N . Lemma 3 Let X∼Bin(N,p)X (N,p) and k∈1,…,Nk∈\1,…,N\. The function T(p):=P(X≥k)T(p):=P(X≥ k) is strictly increasing in p∈(0,1)p∈(0,1). Proof By definition, T(p):=P(X≥k)=∑i=kN(Ni)pi(1−p)N−iT(p):=P(X≥ k)= _i=k^N Nip^i(1-p)^N-i. Differentiating: T′(p) T (p) =∑i=kN−1(Ni)[ipi−1(1−p)N−i−(N−i)pi(1−p)N−i−1]+NpN−1 = _i=k^N-1 Ni [ip^i-1(1-p)^N-i-(N-i)p^i(1-p)^N-i-1 ]+Np^N-1 =1p(1−p)∑i=kN(Ni)(i−Np)pi(1−p)N−i=1p(1−p)[(X−[X]) 1X≥k]. = 1p(1-p) _i=k^N Ni(i-Np)\,p^i(1-p)^N-i= 1p(1-p)\,E\! [(X-E[X])\,1_X≥ k ]. Since [(X−[X])]=0E[(X-E[X])]=0, we have: [(X−Np) 1X≥k]=−[(X−Np) 1X<k].E\! [(X-Np)\,1_X≥ k ]=-\,E\! [(X-Np)\,1_X<k ]. If k≤Npk≤ Np, then for all i<ki<k we have (i−Np)<0(i-Np)<0, hence [(X−Np) 1X<k]<0E[(X-Np)\,1_X<k]<0, and therefore [(X−Np) 1X≥k]>0E[(X-Np)\,1_X≥ k]>0. If instead k>Npk>Np, then for all i≥ki≥ k we have (i−Np)>0(i-Np)>0, hence [(X−Np) 1X≥k]>0E[(X-Np)\,1_X≥ k]>0. In both cases: ∂T∂p=1p(1−p)[(X−Np) 1X≥k]>0. ∂ T∂ p= 1p(1-p)\,E\! [(X-Np)\,1_X≥ k ]>0. ∎ Applying Lemma 3 with p≥1−εk−εo=:plp≥ 1- _k- _o=:p_l: p(X≥⌈λN⌉)≥∑i=⌈λN⌉N(Ni)pli(1−pl)N−i=:pλ.p(X≥ λ N )≥ _i= λ N ^N Nip_l^i(1-p_l)^N-i=:p_λ. Now, we observe that 1≥pλ≥plN1≥ p_λ≥ p_l^N where the second inequality is obtained by setting i=Ni=N and noting that the terms of the sum are all positive. Hence, as pl→1p_l→ 1, pλ→1p_λ→ 1 for any fixed λ∈(0,1)λ∈(0,1). Implication for WoE. Under sufficiently low attribution variability, p^(h∗∣e)≥λ p(h^* e)≥λ with probability pλp_λ. Now, considering the complementary hypothesis h¯∗ h^* as the alternative one, p^(h¯∗∣e)≤1−λ p( h^* e)≤ 1-λ. Since the prior odds term is bounded, it follows that, as λ→1λ→ 1: woe(h∗:e)→∞.woe(h^*:e)→∞. This establishes the following. Theorem 4.1 Let α be a stochastic mutually independent FIM with component variances σi2 _i^2. Fix an instance x and let h∗h^* be the hypothesis generated from the mean attribution α¯ α via AS or TK (with the only difference of ϵk=0) _k=0) and let h′h be the complementary hypothesis. Then, as maxiσi2→0 _i _i^2→ 0, woe(h∗:e)→+∞woe(h^*:e)→+∞ with probability approaching 11. 5 Experimental Setup This section is structured according to the origin of the reference hypothesis. (1) We test LIME and SHAP against a hypothesis h∗h^* derived from domain knowledge on the Titanic dataset [6], quantifying its alignment with FIMs. While a FIM produces a local explanation, the domain hypothesis is global. This reveals that WoE is affected by the global–local discrepancy (Section 5.1). (2) We consider a ground truth (GT) setting by generating a synthetic dataset and training a RF, thus having access to both data- and model-level relevance. This allows us to assess alignment (w.r.t. the data) and faithfulness (w.r.t. the model) via WoE (Section 5.2). 3) We test a third scenario where no external references are available. In this case, h∗h^* is defined as the empirical mean attribution across runs, so WoE captures internal stability (Section 5.3). The section concludes with an empirical validation of Theorem 4.1 and an analysis of the impact of the number of explainer runs (Section 5.4). Notably, to generate hjh_j (Section 3.3), we adopt top-k selection in the first two settings, as the exact set of relevant features is known; while in the third, we rely on the adaptive strategy, since no prior information on feature relevance is available. Due to space constraints, details on datasets, preprocessing, and model hyperparameters will be left for the final version. 5.1 Hypothesis Testing With Domain Knowledge In this section, we compute the WoE for 50 instances LIME and SHAP on the Titanic dataset using a RF model. Historical analyses of this setting identify sex (due to the “women first” policy) and pclass (a proxy for socio-economic status) as the most influential factors for survival, as also confirmed by prior empirical studies [11]. Taking this as our reference hypothesis, our results show that LIME produces 2727 instances with woe=+∞woe=+∞ (i.e., p(h∣e)=1p(h e)=1, where the reference hypothesis is always generated) and 2323 cases with finite woe>0woe>0. In contrast, SHAP yields deterministic explanations, with 2323 instances having woe=+∞woe=+∞ and 2727 having woe=−∞woe=-∞ (i.e., p(h∣e)=0p(h e)=0), indicating the selection of features beyond sexsex and pclasspclass. Figure 1: PCA Analysis for LIME and SHAP grouped according to WoE values. A PCA projection with 54%54\% of variance explained (Figure 1) further highlights geometric differences: for LIME, instances with finite and divergent WoE occupy distinct regions, while for SHAP the separation between +∞+∞ and −∞-∞ cases is even more pronounced. This suggests that in specific regions of the feature space, alternative variables (e.g., age) influence the explanations, contrary to the assumption of a single global explanation. Insight: through WoE, we can test alignment between a local explanation and an assumed global explanation, thereby identifying regions where the local behavior differs. 5.2 Hypothesis Testing with Synthetic Ground Truth Knowledge In this section, we employ the WoE framework to measure alignment with Ground Truth (GT) knowledge. To this end, we construct a synthetic classification dataset, using the make_classification function from scikit-learn—which generates a random n-class classification problem—, with ninf=3n_inf=3 informative features, whose indices 0,1,2\0,1,2\ constitute the known ground truth hGT∗h^*_GT by construction, and nred=7n_red=7 redundant features generated as: Xred=(1−λ)XinfW+λℰ,W∈ℝ3×7,ℰ∈ℝn×7,ℰ∼(0,I7),X_red=(1-λ)\,X_inf\,W+λ\,E, W ^3× 7,\;E ^n× 7,\,E (0,I_7), where λ∈[0,1]λ∈[0,1] interpolates between a perfect linear combination of the informative features (λ=0λ=0) and pure noise (λ=1λ=1) and n represents the amount of samples. We evaluate LIME and SHAP on a RF with N=50N=50 runs on 5050 instances using top-k selection (k=3k=3), and report: WoEGTWoE_GT, measuring alignment with hGT∗h^*_GT; and WoEModelWoE_Model, measuring alignment with the model’s feature ranking hModel∗h^*_Model (i.e., faithfulness). The results for LIME are summarized in Figure 2. Figure 2: LIME experiment with known GT. (a) Mean WoEGTWoE_GT and WoEModelWoE_Model ± 95% CI, with fraction of finite WoE values on the right vertical axis. (b) Fraction of divergent instances as a function of λ. (c) Sensitivity of WoEGTWoE_GT and WoEModelWoE_Model to LIME’s kernel-width factor (kw_factorkw\_factor) at λ=0.65λ=0.65, with the vertical line marking the default value. This allows us to measure the alignment FIM-Model and FIM-GT. Two opposite trends emerge: at low noise levels, LIME aligns much more with the model than with the GT, showing that the model relies more on redundant features than on the truly relevant ones. As noise increases (λ≥0.70λ≥ 0.70), WoEGTWoE_GT and WoEModelWoE_Model converge, meaning that redundant features become too noisy and the model is forced to rely on the truly relevant features. Counterintuitively, when λ∈(0.50,0.70)λ∈(0.50,0.70), the FIM appears more aligned with the GT than with the model. To better understand this phenomenon, we analyzed the case λ=0.65λ=0.65. As shown in Figure 2(c), reducing the kernel width by a multiplicative factor (kw_factorkw\_factor) causes WoEModelWoE_Model to progressively align with WoEGTWoE_GT. A possible explanation for this behavior is the high curvature of the local neighborhood around the sample to be explained, which LIME ignores when its sampling radius is too high, thus leading to unfaithful explanations. However, our WoE framework manages to diagnose this. SHAP plots are omitted because they are not very informative: since its values are almost deterministic, they lead to either complete alignment/disalignment w.r.t. the model and the GT, resulting in few finite instances to plot. However, the same trend is observed: as noise increases, the model converges to the GT behavior and the WoE values capture this pattern. Insight: WoE can localize the disalignment along the Data→\,→\,Model→\,→\,FIM pipeline, thereby assessing the overall reliability of the FIM. 5.3 Hypothesis Testing Without Prior Knowledge The experiments in this section focus on comparing WoE across four datasets: Diabetes [28], Heart Disease [14], Mobile [10], and Churn [27]. Here we opted for the AS strategy with τ=0.8τ=0.8. Figure 3 illustrates the distributions obtained for LIME and SHAP. For the latter, Diabetes and Churn are not presented because SHAP always produced the same hypotheses, yielding woe=+∞woe=+∞ (Table 1). Overall, there are no significant differences between the two distributions, demonstrating that the two methods are comparable. However, when analyzing Table 1 of divergent WoE, we observe that SHAP tends to produce the same explanation more consistently, a finding consistent with prior works [5, 23]. Interestingly, WoE generally presents a positive score, demonstrating that LIME and SHAP produce internally consistent explanations. However, this score decreases for more complex datasets, as is the case with Mobile, and where LIME even shows 0 positive divergent values and 5 negative divergent values. This is likely due to the greater interactions between the variables, which make it more difficult to obtain a stable ranking and thus generate the same hypothesis. Insight: when evaluating WoE on mean feature importance, we can capture the internal consistency of the FIM—i.e. its variability. Figure 3: Guassian KDE of the LIME and SHAP WoE values across datasets. Table 1: Number of divergent WoE cases out of 5050 instances for each method and dataset. Each entry is reported as (+∞ / -∞). Method Diabetes Heart Mobile Churn LIME 5 / 0 11 / 0 0 / 5 5 / 0 SHAP / 50/0 / 25/0 / 41/0 / 50/0 5.4 Empirical Validation of Theorem 4.1 We empirically validate that, when using the mean attribution as the reference hypothesis, lower variance in AMs leads to higher WoE. We consider two settings: (i) real data, using LIME on the Diabetes dataset while varying the number of background samples for 4 models: Random Forest (RF), Multilayer perceptron (N), Decision Tree (DT) and Support Vector Machine (SVM); and (i) a synthetic FIM with controlled variance. In the first setting (Figure 5), increasing num_samplesnum\_samples, which ranges in 100,200,300,500,1000,2000,3000,5000, reduces variance and results in higher WoE values. This is also reflected in the proportion of divergent cases: positively divergent WoE (+∞+∞) slightly increases, while negatively divergent cases (−∞-∞) decrease significantly. In the synthetic setting (Figure 5), we isolate the effect of variability by directly generating synthetic attribution vectors, without relying on any dataset or model. We define a reference importance vector with 1010 features evenly spaced in [0,1][0,1], and model the FIM as a stochastic generator centered around these values. For each configuration, we sample 5050 attribution vectors for each of 5050 simulated instances. Variability is controlled by adding Gaussian noise with standard deviation σ=(1/l)⋅0.1σ=(1/l)· 0.1, where l∈0.5,1,1.5,2l∈0.5,1,1.5,2. The results consistently show that lower variance leads to higher WoE, confirming the theoretical result. Figure 4: Impact of the variability on the WoE for LIME across 4 models on the Diabetes dataset. Figure 5: Analysis of the impact of variability in the WoE for a controlled synthetic setting. Impact of the Number of Runs Our estimate of the posterior distribution is based on the proportion of runs that support the reference hypothesis. In this section, we briefly analyze the impact of the number of runs N on WoE estimates. We report the RF results for Diabetes using LIME. We run LIME N times on 5050 instances of the test set with N∈5,10,20,30,40,50,60,70,80,90,100N∈\5,10,20,30,40,50,60,70,80,90,100\. We then analyzed the distributions in Figure 7 of the observed WoE values, noting that there is high consistency across the runs. Furthermore, we measured the dispersion of the WoE vectors (see Figure 7) for each instance using the coefficient of variation (CV), which is defined as σ/|μ|σ/|μ|. The distribution of CV values is concentrated around values close to 00 (with 80% of values between 00 and 0.30.3), indicating consistency in the weight of evidence values across the runs. Figure 6: Gaussian KDE of WoE distributions across different numbers of runs N. The strong overlap between the distributions highlights the stability of WoE values. Figure 7: Gaussian KDE of the CV of WoE values across different numbers of runs for 50 instances. The plot is concentrated around values close to 00, indicating consistency of WoE estimates. 6 Limitations The flexibility of the proposed framework comes at the cost of several design choices. In this work, we adopted intuitive choices without claiming their optimality. For instance, as no prior knowledge was available, we modeled the prior probability as a uniform distribution over hypotheses. However, alternative priors could be naturally incorporated [20]. Similarly, the posterior distribution is estimated via the empirical frequency of a reference hypothesis across multiple runs, requiring exact equality between feature sets. This choice might be too restrictive. Softer alternatives could instead consider partial matches (e.g., h(j)⊆h∗h^(j) h^*). More generally, a continuous formulation based on a distance measure ‖h(j)−h∗‖||h^(j)-h^*|| could provide a smoother notion of similarity. Another degree of freedom lies in the hypothesis generation process from feature importance scores. In this work, we opted for the TK and AS strategies but alternative options could be explored. Finally, the proposed approach requires running the explanatory method multiple times (even though we have shown that stable estimates of WoE can be obtained with just a few runs), making the overall procedure dependent on the computational cost of the FIM. 7 Conclusion and Future Work In this work, we proposed a framework that embeds FIMs within a hypothesis-testing perspective based on WoE. The key design choice is the source of the reference hypothesis h∗h^*, which determines what the WoE is actually measuring: alignment with domain knowledge, faithfulness to the intrinsic mechanism or internal stability of the FIM across repeated runs. In short, the framework not only allows one to evaluate a hypothesis against an alternative, but also to quantify key properties within the evaluation landscape of FIMs. On the theoretical side, we established a formal connection between the stability of a FIM and the WoE. The experimental setup led to several insights: WoE detects the local-global discrepancy; it identifies disalignment along the Data-Model-FIM pipeline; and it measures the internal consistency (variability, stability) of FIMs. Future work will focus on relaxing some of the current design choices, in particular by developing softer formulations of the posterior that allow partial hypothesis matches, and by exploring alternative prior distributions beyond the uniform assumption. We also plan to extend the evaluation to a broader range of FIMs, model types, and data modalities. Finally, we believe that the experiment described in Section 5.2 represents a promising avenue to explore, particularly with regards to the concept of faithfulness, which has always been studied exclusively in relation to the model while ignoring the potential disalignment between the model itself and the underlying data generating process. Acknowledgements Eddie Conti has been partially supported by the predoctoral grant FI-STEP (2025 STEP 00108) from the Research and University Department of the Generalitat de Catalunya and cofunded by the European Social Fund Plus. Álvaro Parafita acknowledges his AI4Science fellowship within the “Generacion D” initiative by Red.es, Ministerio para la Transformación Digital y de la Función Pública, for talent attraction (C005/24-ED CV1), funded by NextGenerationEU through PRTR. Axel Brando received funding from the Horizon Europe Programme under the AI4DEBUNK Project (https://w.ai4debunk.eu), grant agreement num. 101135757. Work partially supported by the Italian Ministry of Education and Research (MUR) in the framework of the FoReLab project (Departments of Excellence). References [1] C. Agarwal, S. Krishna, E. Saxena, M. Pawelczyk, N. Johnson, I. Puri, M. Zitnik, and H. Lakkaraju (2022) Openxai: towards a transparent evaluation of model explanations. Advances in neural information processing systems 35. Cited by: §2. [2] A. Alshehri, T. Miller, M. Vered, and H. Alamri (2021) Human centered explanation for goal recognition system.. In IJCAI-PRICAI Workshop On Explainable Artificial Intelligence (XAI) 2020, Cited by: §1. [3] A. Alshehri, T. Miller, and M. Vered (2023) Explainable goal recognition: a framework based on weight of evidence. In Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 33. Cited by: §1, §2. [4] N. Balasubramaniam, M. Kauppinen, A. Rannisto, K. Hiekkanen, and S. Kujala (2023) Transparency and explainability of AI systems: from ethical guidelines to requirements. Information and Software Technology. Cited by: §1. [5] T. A. Butt and M. Iqbal (2025) Towards reliable explainable ai: a novel stability metric for trustworthy interpretations. In International Conference on Multimedia Systems and Signal Processing, p. 198–210. Cited by: §5.3. [6] W. Cukierski (2012) Titanic-machine learning from disaster. Kaggle. available at: https://kaggle. com/competitions/titanic. Cited by: §5. [7] M. Du, N. Liu, and X. Hu (2019) Techniques for interpretable machine learning. Commun. ACM. Cited by: §1. [8] A. Esteva, A. Robicquet, B. Ramsundar, V. Kuleshov, M. DePristo, K. Chou, C. Cui, G. Corrado, S. Thrun, and J. Dean (2019) A guide to deep learning in healthcare. Nature medicine 25 (1). Cited by: §1. [9] Z. Geng, M. Schleich, and D. Suciu (2022) Computing rule-based explanations by leveraging counterfactuals. arXiv preprint arXiv:2210.17071. Cited by: §3.1. [10] N. Hu (2022) Classification of mobile phone price dataset using machine learning algorithms. In 2022 3rd International Conference on Pattern Recognition and Machine Learning (PRML), Cited by: §5.3. [11] C. Huang (2024) The prediction and feature importance investigation in titanic survival prediction. In Proceedings of the 2nd International Conference on Data Analysis and Machine Learning - Volume 1: DAML, Cited by: §5.1. [12] S. R. Islam, W. Eberle, S. K. Ghafoor, and M. Ahmed (2021) Explainable artificial intelligence approaches: a survey. arXiv preprint arXiv:2101.09429. Cited by: §1. [13] A. Jacovi, S. Swayamdipta, S. Ravfogel, Y. Elazar, Y. Choi, and Y. Goldberg (2021) Contrastive explanations for model interpretability. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Cited by: §1. [14] A. Janosi, W. Steinbrunn, M. Pfisterer, and R. Detrano (1989) Heart Disease. Note: UCI Machine Learning Repository Cited by: §5.3. [15] M. A. Kadir, A. Mosavi, and D. Sonntag (2023) Evaluation metrics for xai: a review, taxonomy, and practical applications. In 2023 IEEE 27th International Conference on Intelligent Engineering Systems (INES), Cited by: §1, §2. [16] T. Le, T. Miller, L. Sonenberg, R. Singh, and H. P. Soyer (2025) From evidence to decision: exploring evaluative ai. External Links: 2402.01292 Cited by: §2. [17] P. Linardatos, V. Papastefanopoulos, and S. Kotsiantis (2020) Explainable ai: a review of machine learning interpretability methods. Entropy. Cited by: §1. [18] Z. C. Lipton (2018) The mythos of model interpretability: in machine learning, the concept of interpretability is both important and slippery.. Queue 16 (3). Cited by: §1. [19] S. M. Lundberg and S. Lee (2017) A unified approach to interpreting model predictions. Advances in neural information processing systems 30. Cited by: §3.2. [20] D. Lunn, C. Jackson, N. Best, A. Thomas, and D. Spiegelhalter (2013) The bugs book. A practical introduction to Bayesian analysis, Chapman Hall, London. Cited by: §6. [21] D. A. Melis, H. Kaur, H. Daumé I, H. Wallach, and J. W. Vaughan (2021) From human explanation to model interpretability: a framework based on weight of evidence. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. Cited by: §2. [22] T. Miller (2019) Explanation in artificial intelligence: insights from the social sciences. Artificial intelligence 267. Cited by: §1. [23] A. Nayebi, S. Tipirneni, B. Foreman, C. K. Reddy, and V. Subbian (2023) An empirical comparison of explainable artificial intelligence methods for clinical data: a case study on traumatic brain injury. In AMIA annual symposium proceedings, Vol. 2022. Cited by: §5.3. [24] M. Parola, A. L. Alfeo, and M. G. Cimino (2026) Human-centered xai via a concept-informed prompt-based validation framework for saliency maps [ciprova]. Image and Vision Computing. Cited by: §2. [25] M. T. Ribeiro, S. Singh, and C. Guestrin (2016) "Why should i trust you?": explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, External Links: ISBN 9781450342322 Cited by: §3.2. [26] C. Rudin and Y. Shaposhnik (2023) Globally-consistent rule-based summary-explanations for machine learning models: application to credit-risk evaluation. Journal of Machine Learning Research 24 (16). Cited by: §3.1. [27] N. Sharma, A. Awasthi, D. S. Parmar, T. Chouhan, A. Singh, and D. Rawat (2025) Predictive analytics for customer retention: a catboost model for churn detection. In 2025 International Conference on Networks and Cryptology (NETCRYPT), Cited by: §5.3. [28] V. Sigillito (1990) Diabetes dataset. Note: Research Center, RMI Group Leader, Applied Physics Laboratory, The Johns Hopkins University, Laurel, MD, USA Cited by: §5.3. [29] I. Wod (1985) Weight of evidence: a brief survey. Bayesian statistics 2. Cited by: §1, §3.