Paper deep dive
From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier
Vincent Lemaire, Fabrice Clérot
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/9/2026, 7:47:15 AM
Summary
This paper establishes a formal geometric link between supervised distances in weighted Naive Bayes classifiers and Shapley values. It introduces a log-odds-based supervised distance that exactly coincides with the L1 distance between analytical Shapley value vectors, providing a rigorous explanatory interpretation of the model's geometry. The authors validate this equivalence theoretically for binary and multiclass cases and empirically demonstrate its utility using k-NN classifiers on UCI datasets.
Entities (6)
Relation Signals (4)
Log-Odds Distance → isequivalentto → Shapley Value Distance
confidence 98% · the l1 distance between vectors of analytical Shapley values... coincides exactly with the l1 distance between vectors of log-odds contributions
Weighted Naive Bayes Classifier → uses → Log-Odds
confidence 95% · introduce a discriminative reformulation based on log-odds... more directly related to the classification decision
Shapley values → providesinterpretationfor → Weighted Naive Bayes Classifier
confidence 93% · providing a formal explanatory interpretation of the geometry induced by the model
K-Nearest Neighbors → evaluates → Supervised Distances
confidence 90% · empirically compare several supervised distances derived from these representations using a k-nearest neighbors classifier
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper studies the construction of an explanatory space for a weighted naive Bayes classifier from the supervised representation induced by the model. We start from the classical supervised distance based on conditional log-likelihoods and introduce a discriminative reformulation based on log-odds, which is more directly related to the classification decision. We then show that this representation induces a distance that exactly coincides with the $\ell_1$ distance between vectors of analytical Shapley values, thereby providing a formal explanatory interpretation of the geometry induced by the model. Finally, we empirically compare several supervised distances derived from these representations using a $k$-nearest neighbors classifier. This work highlights a close link between supervised distance, local explanation, and predictive behavior, from a primarily methodological perspective.
Tags
Links
- Source: https://arxiv.org/abs/2610.10642v1
- Canonical: https://arxiv.org/abs/2610.10642v1
Trouble viewing inline? Open PDF directly →
Full Text
40,840 characters extracted from source content.
Expand or collapse full text
From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier Vincent Lemaire Affiliation: Orange Research, France, E-mail vincent.lemaire@orange.com Fabrice Clérot Affiliation: Orange Research, France, E-mail fabrice.clerot@orange.com Abstract This paper studies the construction of an explanatory space for a weighted naive Bayes classifier from the supervised representation induced by the model. We start from the classical supervised distance based on conditional log-likelihoods and introduce a discriminative reformulation based on log-odds, which is more directly related to the classification decision. We then show that this representation induces a distance that exactly coincides with the ℓ1 _1 distance between vectors of analytical Shapley values, thereby providing a formal explanatory interpretation of the geometry induced by the model. Finally, we empirically compare several supervised distances derived from these representations using a k-nearest neighbors classifier. This work highlights a close link between supervised distance, local explanation, and predictive behavior, from a primarily methodological perspective. 1 Context: Supervised distance induced by a classifier 1.1 Introduction In the context of a probabilistic classifier, it can be useful to have a representation of the observations that is consistent with how the model generates its prediction scores. Such a representation allows us to define a distance between individuals that is based not only on the raw geometry of the input variables, but also on their role in the classifier’s decision. In the case of the naive Bayes classifier, this idea naturally leads to constructing a supervised representation based on the conditional probabilities associated with the explanatory variables. Two observations that are close in this representation are then expected to be close in terms of their predictive behavior as well. This section reviews the principle of this classifier-induced representation, as well as the distance associated with it. This construction will then serve as a starting point for introducing a more directly discriminative representation based on log-odds in Section 2, and subsequently for establishing a connection with the analytical Shapley values of the naive Bayes classifier in Sections 3 and 4. Finally, before concluding, an experimental validation using a classifier based on the k-nearest-neighbor (k-N) algorithm is presented in Section 5. 1.2 Preprocessing The naive Bayes classifier used in the Khiops library [1] relies on a preliminary step to prepare the explanatory variables. Numeric variables are discretized, while categorical variables are grouped by category. Following this preprocessing, each variable m is recoded into a discrete variable with ImI_m possible values. An observation D can thus be represented as D=(D1i1,D2i2,…,DMiM),D=(D_1i_1,D_2i_2,…,D_Mi_M), where DmimD_mi_m denotes the recoded value taken by the variable m for this observation. For the sake of simplicity in the remainder of this article, D=(D1i1,D2i2,…,DMiM),D=(D_1i_1,D_2i_2,…,D_Mi_M), will be denoted by D=(D1,D2,…,DM)D=(D_1,D_2,…,D_M). This preprocessing enables us to estimate the conditional probabilities p(Dm|Cj)p(D_m|C_j) for each recoded variable and target class CjC_j. These probabilities form the basis of the naive Bayes classifier’s score. This provides a homogeneous representation of the variables, suitable for defining a supervised distance consistent with the model. In other words, an observation can be described not only by its raw values, but also by the probabilistic quantities used by the classifier to make a decision. This model-induced representation will be used to define a distance between observations in what follows. 1.3 Distance depending on target classes Lemaire et al. [9] proposed defining a supervised distance induced by a classifier—in this case, the Naive Bayes classifier—that is consistent with how the classifier computes its classification scores. Using the notation introduced in Section 1.2, we can rewrite, for each target class CjC_j, the logarithmic score of the naive Bayes classifier in the form logp(Cj∣D)=∑m=1Mlogp(Dm∣Cj)+logp(Cj)−logp(D), p(C_j D)= _m=1^M p(D_m C_j)+ p(C_j)- p(D), (1) where D=(Dm)m=1,…,MD=(D_m)_m=1,…,M denotes an instance and DmD_m the recoded value of variable m. The Bayesian decision is to choose the class CjC_j that maximizes this quantity. Lemaire et al. [9] then define the distance between two instances D and D′D as dNB1(D,D′)=∑m=1M∑j=1J|logp(Dm∣Cj)−logp(Dm′∣Cj)|.d_NB^1(D,D )= _m=1^M _j=1^J | p(D_m C_j)- p(D _m C_j) |. (2) Each instance is thus represented by a vector in a supervised space of dimension MJMJ, whose components are the quantities logp(Dm∣Cj) p(D_m C_j), indexed by the pairs (m,j)(m,j) where m∈1,…,Mm∈\1,…,M\ and j∈1,…,Jj∈\1,…,J\. The proposed distance corresponds to the ℓ1 _1 norm between these vectors. The distance between two instances is therefore defined in terms of their supervised recodings. Instances that are close in terms of their supervised recodings will exhibit similar behaviour with respect to the predicted class. Indeed, if we define the distance between the predicted class distributions as follows: Δ1(D,D′)=∑j=1J|log(p(Cj|D))−log(p(Cj|D′))| ^1(D,D )= _j=1^J |log(p(C_j|D))-log(p(C_j|D )) | (3) Thus we have the following majorization [9] Δ1(D,D′)≤[dNB1(D,D′)+J|log(p(D))−log(p(D′))|] ^1(D,D )≤ [d_NB^1(D,D )+J |log(p(D))-log(p(D )) | ] (4) This inequality shows that the distance d1NBd_1^NB controls the differences between posterior log-probabilities up to a normalization term. Thus, when two observations are close in the sense of d1NBd_1^NB and their normalization terms logp(D) p(D) et logp(D′) p(D ) are also close, their predictive score vectors are close in the sense of Δ1 _1. In the context of the generative naive Bayes model, the term logp(D) p(D) is itself determined by the class scores via a log-sum-exp operation. Therefore, despite equation (4) explicitly isolating its contribution, it is related to the conditional log-likelihoods used in the representation. In the case of a ‘weighted’ Naive Bayes classifier, the variables are assigned weights such that the original equation 2 becomes: dSNB1(D,D′)=∑m=1M∑j=1Jwm|logp(Dm∣Cj)−logp(Dm′∣Cj)|d_SNB^1(D,D )= _m=1^M _j=1^Jw_m | p(D_m C_j)- p(D _m C_j) | (5) where wmw_m is the weight of variable m, which is the same for all target classes. Each instance is then represented by an MJMJ-dimensional vector, whose components—indexed by the pairs (m,j)(m,j)—are weighted by the weight wmw_m associated with variable m. The distance (Equation 4), where dSNB1d_SNB^1 is replaced by dSNB1d_SNB^1, is thus a weighted version of the initial supervised distance, in which the relative influence of each variable is modulated by the weights derived from the classifier. 2 Proposal for a log-odds-based supervised discriminant distance 2.1 Motivation In [9], the authors define a supervised representation induced by the naive Bayes classifier, as presented in the previous section. The upper bound given in equation 4 relates the proximity of two observations in this representation to the differences between their log-posterior probabilities. However, in addition to the distance d1SNB(D,D′)d_1^SNB(D,D ), it involves a global normalization term, J|logp(D)−J| p(D)- logp(D′)|. p(D )|. This term is not independent of the conditional log-likelihoods, since p(D)p(D) is determined from the model’s scores. Equation 4 does not allow us to attribute the predictive proximity of the observations entirely to the similarity of their discriminant contributions alone. This motivates the construction of a new representation, based no longer on the conditional log-likelihoods taken separately, but on quantities directly related to the contrasts between classes, namely the log-odds. 2.2 A new supervised representation based on log-odds. The idea is to replace the representation (logp(Dm∣Cj))m,j ( p(D_m C_j) )_m,j with a representation based on the discriminant contrasts between a class and the set of all other classes. Let C=C1,…,CJC=\C_1,…,C_J\ be the J target classes. For each class CjC_j, we define the complementary class Cj¯=C∖Cj. C_j=C \C_j\. For an observation D=(Dm)m=1,…,MD=(D_m)_m=1,…,M, we define, for each variable m∈1,…,Mm∈\1,…,M\ and each class j∈1,…,Jj∈\1,…,J\, the ‘grow’ local contribution to the one-vs-rest log-odds11 1 Note—Here, an additional modeling assumption is made: the variables are conditionally independent given the metaclass (Cj¯ C_j). by ℓm(j)(D)=wmlogp(Dm∣Cj)p(Dm∣Cj¯), _m^(j)(D)=w_m p(D_m C_j)p(D_m C_j), (6) where p(Dm∣Cj¯)=∑r≠jp(Dm∣Cr)p(Cr∣Cj¯).p(D_m C_j)= _r≠ jp(D_m C_r)\,p(C_r C_j). (7) This quantity measures the contribution of variable m to discriminating between class CjC_j and all other classes. The one-vs-rest supervised discriminant representation is then defined as follows: Φ(D)=(ℓm(j)(D))m=1,…,Mj=1,…,J∈ℝM×J. (D)= ( _m^(j)(D) )_ subarraycm=1,…,M\\ j=1,…,J subarray ^M× J. (8) In other words, each observation is represented by the set of its local contributions to the set of J binary “one class versus the rest” problems. This representation therefore preserves, for each variable, detailed discriminating information on a class-by-class basis. A natural distance associated with this representation is the ℓ1 _1 distance: dodds1(D,D′)=∑j=1J∑m=1M|ℓm(j)(D)−ℓm(j)(D′)|.d^1_odds(D,D )= _j=1^J _m=1^M | _m^(j)(D)- _m^(j)(D ) |. (9) This distance compares two observations based on their sets of discriminating explanatory profiles. Two individuals will therefore be considered similar if, for each class CjC_j, the explanatory variables contribute similarly to the contrast between CjC_j and the other classes. This formulation has several advantages. Firstly, it is directly related to the discriminative nature of the decision, since each component measures a contribution to the log-odds of a binary “one-vs-rest” problem. Secondly, unlike a prior aggregation of the contributions, it preserves the directional information carried by each class. It naturally eliminates the normalization term p(D)p(D) from equation 4. We now have the following bound22 2 This is a unidirectional guarantee. The proof of this bound is given in the Appendix. Δ1(D,D′)≤dodds1(D,D′) ^1(D,D )≤ d_odds^1(D,D ) (10) 2.3 Discussion The distance initially proposed in [9] is consistent with the additive structure of the naive Bayes classifier. However, it is still based on conditional log-likelihoods considered separately. Consequently, it does not optimally capture the model’s discriminative power. The log-odds-based distance has several advantages: • it exactly eliminates the normalization term logp(D) p(D); • it is more directly related to the decision rule; • it allows for the grouping of individuals with similar predictive profiles, even when their ‘grow’ likelihoods differ. It therefore appears better suited to the supervised segmentation objective at hand, particularly when the goal is to form groups of individuals who are homogeneous in terms of their behavior with respect to the target class. The analysis presented here shows that it is possible to improve the supervised distance proposed in [9] by using a representation based on the log-odds between classes. This new representation remains compatible with the structure of the naive Bayes classifier but is better aligned with the problem’s discriminative objective. In the context of supervised clustering focused on score proximity, as suggested in [9], this distance therefore constitutes an alternative to the distance defined solely based on conditional log-likelihoods. The distances considered in this article are motivated not only by their representational interpretation, but also by their ability to control discrepancies between classifier outputs. Each induced geometry is linked to a notion of proximity that is relevant from a predictive standpoint. The initial supervised distance d1SNBd^SNB_1 is justified by the fact that it directly controls the differences between the posterior log-probabilities of the classes. Equation 4 bounds these differences by the sum of d1SNB(D,D′)d_1^SNB(D,D ) and an additional term associated with global normalization. Proximity based on d1SNBd_1^SNB alone is therefore not sufficient, based on this inequality, to guarantee a small value of Δ1(D,D′) _1(D,D ) if the normalization terms differ significantly. The discriminant distance d1oddsd^odds_1 has a stronger predictive justification, since it directly bounds the difference Δ1(D,D′) _1(D,D ) between vectors of logarithmic class scores, without involving the global normalization term. 3 Link to Shapley Values This section establishes a formal link between supervised representation based on log-odds contributions and representation constructed from the analytical Shapley values of the naive Bayes classifier [8]. We show that, in the binary case and under the chosen reference distribution for the Shapley values, these two representations induce exactly the same inter-individual geometry: they differ only by a fixed translation, independent of the individual. In particular, the (ℓ1 _1) distance between vectors of log-odds contributions is exactly equal to the (ℓ1 _1) distance between vectors of analytical Shapley values Shapley values were introduced in cooperative game theory to distribute the value generated by a coalition among its players fairly [14]. In the context of model explainability, they provide an additive framework for attributing the contribution of each variable to an individual prediction [12]. In the specific case of the weighted Naive Bayes model studied here, this contribution can be calculated analytically based on the model’s additive structure [8]. Theorem 3.1 (Equivalence Between Log-Odds Distance and Shapley Value Distance) Consider a weighted naive Bayes classifier in the binary case, with classes C1C_1 and C0C_0, and an observation D=(D1,…,DM)D=(D_1,…,D_M). For each variable m∈1,…,Mm∈\1,…,M\, we define the local contribution to the log-odds as ℓm(D)=wmlogp(Dm∣C1)p(Dm∣C0) _m(D)=w_m p(D_m C_1)p(D_m C_0) (11) We also consider the analytical Shapley value proposed for the Naive Bayes model: ϕm(D)=wm(logp(Dm∣C1)p(Dm∣C0)−m[logp(Dm∣C1)p(Dm∣C0)]) _m(D)=w_m ( p(D_m C_1)p(D_m C_0)-E_D_m [ p(D_m C_1)p(D_m C_0) ] ) (12) Thus, for two individuals D and D′D , the ℓ1 _1 distance between their Shapley value vectors is exactly equal to the ℓ1 _1 distance between their log-odds contribution vectors: ∑m=1M|ϕm(D)−ϕm(D′)|=∑m=1M|ℓm(D)−ℓm(D′)| _m=1^M| _m(D)- _m(D )|= _m=1^M| _m(D)- _m(D )| (13) In other words, the supervised distance based on log-odds coincides exactly with the L1L^1 distance between vectors of analytic Shapley values. Proof For each variable m, let μm=wmm[logp(Dm∣C1)p(Dm∣C0)] _m=w_m\,E_D_m [ p(D_m C_1)p(D_m C_0) ] (14) According to the definition (see (12)), the Shapley value can be rewritten as33 3 Where the expectation, μm _m, is taken with respect to the marginal distribution of the recoded variable (DmD_m) in the reference population — estimated from the training sample — such that μm _m is constant for all individuals. ϕm(D)=ℓm(D)−μm _m(D)= _m(D)- _m (15) In fact, substituting the expression for ℓm(D) _m(D) given in equation (11) immediately yields ϕm(D)=wmlogp(Dm∣C1)p(Dm∣C0)−wmm[logp(Dm∣C1)p(Dm∣C0)]=ℓm(D)−μm _m(D)=w_m p(D_m C_1)p(D_m C_0)-w_m\,E_D_m [ p(D_m C_1)p(D_m C_0) ]= _m(D)- _m Now, let us consider two individuals: D and D′D . For each variable m, we have ϕm(D)−ϕm(D′) _m(D)- _m(D ) =(ℓm(D)−μm)−(ℓm(D′)−μm) = ( _m(D)- _m )- ( _m(D )- _m ) =ℓm(D)−ℓm(D′). = _m(D)- _m(D ). (16) Taking the absolute value gives us: |ϕm(D)−ϕm(D′)|=|ℓm(D)−ℓm(D′)|.| _m(D)- _m(D )|=| _m(D)- _m(D )|. (17) Summing over all the variables, m=1,…,Mm=1,…,M, we obtain ∑m=1M|ϕm(D)−ϕm(D′)|=∑m=1M|ℓm(D)−ℓm(D′)|. _m=1^M| _m(D)- _m(D )|= _m=1^M| _m(D)- _m(D )|. (18) This proves the equality (13). Analytic Shapley value vectors and log-odds contribution vectors differ only by a fixed translation, coordinate by coordinate, given by (μm)m=1,…,M( _m)_m=1,…,M. However, a translation does not alter the distances between individuals. Consequently, the two distances44 4 The ℓ1 _1 norm is used in this article because it allows for an additive interpretation of the differences between individual contributions of the variables, but the equality proven above does not depend specifically on the choice of norm. coincide exactly. 4 Extension to the multiclass case In the multiclass case, the aggregation method proposed in the article on Shapley values [8], which involves summing the absolute values obtained from the various binary one-vs-rest problems for each variable, should not be used. Such a sum yields a scalar per variable, causing the loss of directional information carried by each class. However, equivalence with the supervised distance based on log-odds remains exact only if a detailed vector representation indexed by both the variable and the reference class is preserved. In other words, to preserve this equivalence, the Shapley contributions must not be summed across classes before calculating the distance. Instead, individuals must be compared component by component, and the differences obtained across all variables and classes must be aggregated using the ℓ1 _1 norm. Proposition 1 (Multiclass one-vs-rest cases: equivalence between the log-odds distance and the Shapley distance) Consider a weighted naive Bayes classifier with J classes =C1,…,CJC=\C_1,…,C_J\, and an observation D=(D1,…,DM)D=(D_1,…,D_M). For each class CjC_j, we define the complementary class Cj¯=∖Cj. C_j=C \C_j\. For each variable m∈1,…,Mm∈\1,…,M\ and each class j∈1,…,Jj∈\1,…,J\, we define the local contribution to the one-vs-rest log-odds as ℓm(j)(D)=wmlogp(Dm∣Cj)p(Dm∣Cj¯), _m^(j)(D)=w_m p(D_m C_j)p(D_m C_j), (19) where p(Dm∣Cj¯)=∑r≠jp(Dm∣Cr)p(Cr∣Cj¯).p(D_m C_j)= _r≠ jp(D_m C_r)\,p(C_r C_j). (20) We then define the analytical Shapley value associated with the one-vs-rest comparison of class CjC_j against Cj¯ C_j as ϕm(j)(D)=wm(logp(Dm∣Cj)p(Dm∣Cj¯)−X~m∼p(Xm)[logp(Dm∣Cj)p(Dm∣Cj¯)]). _m^(j)(D)=w_m ( p(D_m C_j)p(D_m C_j)-E_ X_m p(X_m) [ p(D_m C_j)p(D_m C_j) ] ). (21) If we store all the components ϕm(j)(D) _m^(j)(D) in a vector ΦShap(D)=(ϕm(j)(D))m=1,…,Mj=1,…,J∈ℝMJ, _Shap(D)= ( _m^(j)(D) )_ subarraycm=1,…,M\\ j=1,…,J subarray ^MJ, (22) and if we define the associated ℓ1 _1 norm by dShap1(D,D′)=∑j=1J∑m=1M|ϕm(j)(D)−ϕm(j)(D′)|,d_Shap^1(D,D )= _j=1^J _m=1^M | _m^(j)(D)- _m^(j)(D ) |, (23) so this distance coincides exactly with the one-vs-rest distance based on the log-odds contributions: dodds1(D,D′)=∑j=1J∑m=1M|ℓm(j)(D)−ℓm(j)(D′)|.d_odds^1(D,D )= _j=1^J _m=1^M | _m^(j)(D)- _m^(j)(D ) |. (24) In other words dShap1(D,D′)=dodds1(D,D′).d_Shap^1(D,D )=d_odds^1(D,D ). (25) Remark 1 This formulation is consistent with the multi-class extension proposed in the article on Shapley values [8] for the Naive Bayes classifier, where each class is successively treated as the positive class and all others as the negative class. 5 Experiments The purpose of this section is to empirically evaluate the relevance of the various distances introduced earlier for structuring a local neighborhood consistent with the classifier’s predictive behavior. To achieve this, we use these distances within the framework of a k-nearest-neighbor (k-N) classifier—a classification method based on the local neighborhood structure [4]—for several values of k. This protocol allows us to directly evaluate the discriminative quality of the geometries induced by the different distances directly. We compare three distances. The first is the initial weighted supervised distance defined by equation 5, based on the conditional log-likelihoods taken separately but with the global normalization term: dglobal1(D,D′)=[dSNB1(D,D′)+J|log(pw(D))−log(pw(D′))|]d_global^1(D,D )= [d_SNB^1(D,D )+J |log(p_w(D))-log(p_w(D )) | ] (26) The second is the discriminant distance based on the local contributions to the one-vs-rest log-odds, defined by equation 25: dodds1(D,D′)=dshap1(D,D′)=∑j=1J∑m=1Mwm|logp(Dm∣Cj)p(Dm∣Cj¯)−logp(Dm′∣Cj)p(Dm′∣Cj¯)|d_odds^1(D,D )=d_shap^1(D,D )= _j=1^J _m=1^Mw_m | p(D_m C_j)p(D_m C_j)- p(D _m C_j)p(D _m C_j) | (27) Finally, to evaluate the effect of the global normalization term, we consider the distance obtained by removing from equation (26) the term J|logpw(D)−J| p_w(D)- logpw(D′)| p_w(D )|. We then obtain exactly the weighted supervised distance defined in equation (5), dSNB1(D,D′)d_SNB^1(D,D ), used in [9]: dSNB1(D,D′)=∑m=1M∑j=1Jwm|logp(Dm∣Cj)−logp(Dm′∣Cj)|.d_SNB^1(D,D )= _m=1^M _j=1^Jw_m | p(D_m C_j)- p(D _m C_j) |. (28) 5.1 Datasets In what follows, the comparison of algorithms was performed on twelve different datasets from the UCI repository [11]. These datasets were selected for their diversity in terms of the number of classes, variables, and instances (see Table (1)55 5 The code provided on GitHub allows you to reproduce all of the experiments on other datasets.). Table 1: Datasets used in the experiments. VnV_n denotes the number of numerical variables, VcV_c the number of categorical variables, and C the number of classes. Dataset # examples VnV_n VcV_c C Dataset # examples VnV_n VcV_c C Adult 48 842 7 8 2 Pima 768 8 0 2 Heart 270 10 3 2 Segmentation 2 310 19 0 7 Letter 20 000 16 0 26 Shuttle 58 000 9 0 7 Mushroom 8 416 0 22 2 SickEuthyroid 3 163 7 18 2 Pendigits 10 992 16 0 10 Vehicle 846 18 0 4 Phoneme 5 404 5 0 2 Waveform 5 000 40 0 3 5.2 Implementation of distances The experiments were conducted using the Khiops library [1], which implements a supervised preprocessing and learning pipeline tailored for probabilistic classifiers. Numeric variables were automatically discretized using a supervised approach based on the MODL (Minimum Description Length) criterion, which seeks a partition into intervals that provides a Bayesian trade-off between descriptive accuracy and model complexity [2]. Categorical variables are processed using a supervised grouping of modalities, also based on the MODL criterion: modalities with similar conditional class distributions can be grouped together [3]. The classifier used (which assigns weights to the variables) is the Khiops weighted selective fractional Naive Bayes classifier, as described in [7]. Finally, Shapley values are calculated analytically based on the additive structure of the weighted naive Bayes classifier, as described in [8] and implemented in the library. All experiments can be reproduced using the code available on GitHub [10]. 5.3 Experimental Results 5.3.1 k-n Classification performances Table 2: Performance moyenne en Test sur 10 folds stratifiés par dataset Dataset dglobal1(D,D′)d_global^1(D,D ) dshap1(D,D′)d_shap^1(D,D ) dSNB1(D,D′)d_SNB^1(D,D ) ACC AUC ACC AUC ACC AUC Adult 0,8239 0,7541 0,8241 0,7547 0,8256 0,7557 Heart 0,7889 0,7867 0,8148 0,8117 0,8148 0,8117 Letter 0,9348 0,9659 0,9398 0,9686 0,9431 0,9703 Mushroom 1,0000 1,0000 1,0000 1,0000 1,0000 1,0000 Pendigits 0,9731 0,9852 0,9773 0,9874 0,9758 0,9867 Phoneme 0,8941 0,9297 0,8915 0,9277 0,8900 0,9271 Pima 0,6756 0,6486 0,6730 0,6545 0,6717 0,6455 Segmentation 0,9732 0,9843 0,9766 0,9864 0,9749 0,9854 Shuttle 0,9995 0,9425 0,9996 0,9507 0,9996 0,9507 SickEuthyroid 0,3639 0,6433 0,4145 0,6666 0,3636 0,6432 Vehicle 0,6833 0,7898 0,6915 0,7950 0,6892 0,7937 Waveform 0,7596 0,8196 0,7720 0,8289 0,7680 0,8259 Moyenne 0,8225 0,8541 0,8312 0,8610 0,8264 0,8580 Table (2) compares the performance of the 1-ppv classifier obtained using the three supervised distances under consideration. The initial results are presented for k=1 in order to isolate as much as possible the specific effect of the geometry induced by each of the distances. With a single nearest neighbor, the prediction depends directly on the closest observation in the supervised space under consideration, without any smoothing effect or majority voting among multiple neighbors. This choice therefore allows for a particularly direct assessment of the distances’ ability to group together individuals with similar predictive behavior. Across all twelve datasets, the distance dshap1d_shap^1, defined in Equation (27), achieves the best average performance in both ACC and AUC. These results show that the representation based on log-odds contributions allows us to construct a local neighborhood that is well aligned with the discriminative structure of the classifier. Figure 1 complements the analysis of average performance by comparing the ranks of the three distances on the one-versus-rest area under the curve (AUC) (ranks by dataset, followed by the non-parametric Friedman test and the Nemenyi post-hoc test, as described in [5]). Unlike mean values alone, the critical plot enables us to evaluate the consistency of methods across all datasets by minimising the influence of a few particularly favourable or unfavourable datasets. The diagram shows that dshap1d_shap^1 is the best method, with a mean rank of 1.4, followed by dSNB1d_SNB^1 with a mean rank of 2 and then dglobal1d_global^1 with a mean rank of 2.6. However, given the small performance differences in Table 1, which do not call the critical diagram into question, we also performed local pairwise comparisons on the same folds using a t-test corrected for cross-validation [13] with Holm’s adjustment for multiple comparisons [6]. This did not allow us to distinguish any pairs at the 1% significance level. Thus, in this protocol, the observed rank differences do not constitute statistical evidence of the superiority of d1shapd_1^shap, but suggest an empirical advantage, which remains interesting. Figure 1: Critical Diagram between the 3 distances (AUC OVR). 5.3.2 Predictive consistency In addition to the classification performance of the 11-N, we also evaluate the local predictive consistency of the neighborhoods induced by each distance. For a test observation D, let c^SNB(D)=argmaxjP(Cj∣D) c_SNB(D)= _jP(C_j D) be the class predicted by the weighted Naive Bayes classifier trained on the training fold. We then define LocAgreek(d)=1|T|∑D∈T1k∑D′∈Nkd(D)[c^SNB(D)=c^SNB(D′)].LocAgree_k(d)= 1|T| _D∈ T 1k _D ∈ N_k^d(D) 1\! [ c_SNB(D)= c_SNB(D ) ]. This quantity corresponds to the average proportion of neighbors that share the same class predicted by the SNB as the observation in question; a high value is therefore preferable. (a) Unweighted average across the twelve datasets. (b) Adult: average across the ten test folds, with the x-axis on a base-2 logarithmic scale. Figure 2: Evolution of LocAgreekLocAgree_k as a function of k. A high value indicates that the neighbors selected based on the given distance more frequently share the same class predicted by the SNB than the test observation. Subfigure (2(a)) shows the average results across the twelve datasets, while subfigure (2(b)) details this behavior for the Adult dataset over a wide range of k values. Figure 2 confirms that the considered distances result in neighborhoods that are generally homogeneous in terms of of the SNB’s decision, with the values of LocAgreekLocAgree_k remaining high for all values of k. The d1shapd_1^shap distance yields the highest values, indicating that the neighbors it selects are more frequently assigned the same predicted class as the test observation. As expected, LocAgreekLocAgree_k tends to decrease as k increases: expanding the neighborhood gradually leads to the inclusion of more distant observations and thus those potentially located on the other side of a decision boundary. The distance constructed from the Shapley values not only brings together observations with similar probabilistic profiles but also tends to better preserve the stability of the predicted class better, since LocAgreekLocAgree_k decreases more slowly for it than for the other distances. 5.3.3 Analysis of dispersion For a given distance d and for each test observation D, we denote by kd(D)N_k^d(D) the set of its k nearest neighbors in the training sample. We compute LocDeltak(d)=1|T|∑D∈T1k∑D′∈kd(D)Δ1(D,D′),LocDelta_k(d)= 1|T| _D∈ T 1k _D _k^d(D) _1(D,D ), (29) where T denotes the test set. This allows us to evaluate the local predictive consistency of the neighborhoods induced by each distance using an average local predictive deviation. For a test observation (D) and one of its neighbors (D’), the quantity (Δ1(D,D′) _1(D,D )) compares the complete vectors of posterior log-probabilities estimated by the SNB. It is zero if and only if the two observations have exactly the same vector of predicted probabilities. A low value of (LocDeltakLocDelta_k) therefore indicates that the selected neighbors have similar probabilistic profiles according to the reference model. However, this measure alone does not guarantee stable predictions; this property is examined separately using (LocAgreekLocAgree_k). Figure 3: Dispersion values as a function of k (average across the 12 datasets). Figure reffig:dispersion-versus-k2 shows that, on average, the neighbourhoods with the lowest predictive dispersion are those produced by dshap1d_shap^1 for all considered values of k. The general increase in variance as k increases is to be expected. As the neighborhood expands, the selected observations become progressively less similar to the observation under study in terms of their predictive score vectors. The advantage of dshap1d_shap^1 is that it limits this increase more effectively: for the same neighborhood size, its neighbors have, on average, class log-probabilities closer to those of the individual under consideration. 5.3.4 Analysis of the local classification margin Beyond local agreement with the SNB’s decisions, we evaluate the ability of each distance to locally separate the true classes within the 11-ppv classifier framework. For each test observation D, of true class y(D)y(D), we denote by A the training sample of the fold in question. We define the distance to the nearest neighbor of the same class as d+(D,d)=minD′∈Ay(D′)=y(D)d(D,D′),d^+(D;d)= _ subarraycD ∈ A\\ y(D )=y(D) subarrayd(D,D ), and the distance to the nearest neighbor of a different class by d−(D,d)=minD′∈Ay(D′)≠y(D)d(D,D′).d^-(D;d)= _ subarraycD ∈ A\\ y(D )≠ y(D) subarrayd(D,D ). We then define the normalized local margin as follows: Margin(D,d)=d−(D,d)−d+(D,d)d−(D,d)+d+(D,d).Margin(D;d)= d^-(D;d)-d^+(D;d)d^-(D;d)+d^+(D;d). Table 3: Average performance on test across 10 folds stratified by dataset dataset dglobal1(D,D′)d_global^1(D,D ) dshap1(D,D′)d_shap^1(D,D ) dSNB1(D,D′)d_SNB^1(D,D ) Adult 0.4838 0.5264 0.5015 Heart 0.4626 0.4645 0.4698 Letter 0.5931 0.6304 0.6210 Mushroom 0.9999 1.0000 1.0000 Pendigits 0.5276 0.5576 0.5443 Phoneme 0.3073 0.3004 0.3017 Pima 0.3057 0.3101 0.3091 Segmentation 0.7598 0.7948 0.7790 Shuttle 0.9977 0.9979 0.9979 SickEuthyroid 0.2703 0.2703 0.2703 Vehicle 0.2372 0.2489 0.2550 Waveform 0.0893 0.1033 0.0960 Average 0.5029 0.5170 0.5121 This quantity ranges from −1-1 to 11. A positive value means that D’s nearest neighbor in the training sample belongs to its true class. Conversely, a negative value indicates that an observation from another class is closer than any neighbor from the same class. A high margin therefore reflects better local separation of classes in the geometry induced by the distance under consideration. The results are presented in Table 3. 6 Conclusion This article proposes a reformulation of the supervised distance induced by the weighted Naive Bayes model, which is based on local contributions to the log-odds. This representation is more closely related to the discriminative nature of the decision rule than the traditional representation based on conditional log-likelihoods. We have demonstrated that when the contributions are preserved class by class in the multiclass case, the resulting distance coincides exactly with the distance between the vectors of analytical Shapley values. The proposed geometry thus has an interpretation that is both supervised and explanatory. Experiments on twelve datasets indicate that the distance d1shapd_1^shap achieves good predictive performance with a 11-ppv classifier. Furthermore, it tends to preserve the local homogeneity of the classes predicted by the SNB more effectively, as measured by LocAgreekLocAgree_k. These results suggest that this distance is a relevant alternative to the supervised distance used in [9]; this is achieved with an interpretation based on Shapley values. Future work will explore its use in supervised segmentation and in the context of conformal prediction. References [1] M. Boullé, N. Voisine, B. Guerraz, C. Hue, F. Olmos, V. Popescu, S. Gouache, S. Bouget, A. Bondu, L. Gauthier, Y. Nair Benrekia, F. Clérot, and V. Lemaire (2025) Khiops: an end-to-end,Frugal Automl and XAI Machine Learning Solution for Large, Multi-Table Databases. In Conference on Artificial Intelligence for Defense, External Links: 2508.20519, Link Cited by: §1.2, §5.2. [2] M. Boullé (2004) Khiops: a statistical discretization method of continuous attributes. Machine Learning 55 (1), p. 53–69. External Links: Document Cited by: §5.2. [3] M. Boullé (2005) A bayes optimal approach for partitioning the values of categorical attributes. Journal of Machine Learning Research 6 (49), p. 1431–1452. Cited by: §5.2. [4] T. M. Cover and P. E. Hart (1967) Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13 (1), p. 21–27. External Links: Document Cited by: §5. [5] J. Demšar (2006) Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, p. 1–30. Cited by: §5.3. [6] S. Holm (1979) A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6 (2), p. 65–70. Cited by: §5.3. [7] C. Hue and M. Boullé (2024) Fractional naive bayes (FNB): non-convex optimization for a parsimonious weighted selective naive bayes classifier. arXiv preprint arXiv:2409.11100. External Links: Document Cited by: §5.2. [8] V. Lemaire, F. Clérot, and M. Boullé (2023) An efficient shapley value computation for the naive bayes classifier. In Machine Learning and Knowledge Discovery in Databases: ECML PKDD 2023 Workshop on Advances in Interpretable Machine Learning and Artificial Intelligence (AIMLAI), p. 75–90. External Links: Document Cited by: §3, §3, §4, §5.2, Remark 1. [9] V. Lemaire, F. Clérot, and N. Creff (2015) K-means clustering on a classifier-induced representation space: application to customer contact personalization. In Real World Data Mining Applications, M. Abou-Nasr, S. Lessmann, R. Stahlbock, and G. M. Weiss (Eds.), Annals of Information Systems, Vol. 17, p. 139–153. External Links: Link Cited by: §1.3, §1.3, §1.3, §2.1, §2.3, §2.3, §5, §6. [10] V. Lemaire (2026) Github. Note: [Code Github …] Cited by: §5.2. [11] M. Lichman (2013) UCI machine learning repository. Cited by: §5.1. [12] S. M. Lundberg and S. Lee (2017) A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. von Luxburg, S. Bengio, and H. Wallach (Eds.), p. 4765–4774. Cited by: §3. [13] C. Nadeau and Y. Bengio (2003) Inference for the generalization error. Machine Learning 52 (3), p. 239–281. External Links: Document Cited by: §5.3. [14] L. S. Shapley (1953) A value for n-person games. In Contributions to the Theory of Games I, H. W. Kuhn and A. W. Tucker (Eds.), p. 307–317. Cited by: §3. Appendix 0.A Proof Equation 10 This representation naturally eliminates the global term of normalization when considering the log-odds between a class and its complement. Under the assumption that the variables are conditionally independent given the metaclass Cj¯ C_j, we can define, for each class CjC_j, the posterior log-odds rj(D)=logpw(Cj∣D)pw(Cj¯∣D).r_j(D)= p_w(C_j D)p_w( C_j D). Based on the additive structure of the weighted naive Bayes classifier, this score is expressed as rj(D)=logp(Cj)p(Cj¯)+∑m=1Mℓm(j)(D).r_j(D)= p(C_j)p( C_j)+ _m=1^M _m^(j)(D). Thus, for two observations D and D′D , the a priori term cancels out: rj(D)−rj(D′)=∑m=1M(ℓm(j)(D)−ℓm(j)(D′)).r_j(D)-r_j(D )= _m=1^M ( _m^(j)(D)- _m^(j)(D ) ). Or, pw(Cj∣D)=11+exp(−rj(D)).p_w(C_j D)= 11+ (-r_j(D)). The function h(t)=log(1/(1+exp(−t)))h(t)= (1/(1+ (-t))) is 11-Lipschitz. Therefore, |logpw(Cj∣D)−logpw(Cj∣D′)| | p_w(C_j D)- p_w(C_j D ) | ≤|rj(D)−rj(D′)| ≤ |r_j(D)-r_j(D ) | ≤∑m=1M|ℓm(j)(D)−ℓm(j)(D′)|. ≤ _m=1^M | _m^(j)(D)- _m^(j)(D ) |. Summing this inequality over all classes, we get: Δ1(D,D′) _1(D,D ) =∑j=1J|logpw(Cj∣D)−logpw(Cj∣D′)| = _j=1^J | p_w(C_j D)- p_w(C_j D ) | ≤∑j=1J∑m=1M|ℓm(j)(D)−ℓm(j)(D′)| ≤ _j=1^J _m=1^M | _m^(j)(D)- _m^(j)(D ) | =d1odds(D,D′). =d_1^odds(D,D ).