Paper deep dive
Informative Semi-Factuals for XAI: The Elaborated Explanations that People Prefer
Saugat Aryal, Mark T. Keane
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/22/2026, 5:54:49 AM
Summary
The paper introduces the Informative Semi-Factuals (ISF) method, a novel XAI algorithm that generates semi-factual explanations by identifying 'hidden features' that maintain a decision outcome despite significant changes in a key input feature. By leveraging a 'seesaw' pattern of marginal feature contributions, ISF provides more elaborated and human-preferred explanations compared to traditional semi-factual methods.
Entities (6)
Relation Signals (3)
Saugat Aryal → authored → Informative Semi-Factuals (ISF)
confidence 100% · In this work, we advance a new algorithm -- the informative semi-factuals (ISF) method
Informative Semi-Factuals (ISF) → generates → Semi-factuals
confidence 95% · the informative semi-factuals (ISF) method -- that generates more elaborated explanations supplementing semi-factuals
Informative Semi-Factuals (ISF) → outperforms → Traditional semi-factual methods
confidence 90% · a user study shows that people prefer these elaborated explanations over the simpler semi-factual explanations generated by current methods
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recently, in eXplainable AI (XAI), $\textit{even if}$ explanations -- so-called semi-factuals -- have emerged as a popular strategy that explains how a predicted outcome $\textit{can remain the same}$ even when certain input-features are altered. For example, in the commonly-used banking app scenario, a semi-factual explanation could inform customers about better options, other alternatives for their successful application, by saying "$\textit{Even if}$ you asked for double the loan amount, you would still be accepted". Most semi-factuals XAI algorithms focus on finding maximal value-changes to a single key-feature that do $\textit{not}$ alter the outcome (unlike counterfactual explanations that often find minimal value-changes to several features that alter the outcome). However, no current semi-factual method explains $\textit{why}$ these extreme value-changes do not alter outcomes; for example, a more informative semi-factual could tell the customer that it is their good credit score that allows them to borrow double their requested loan. In this work, we advance a new algorithm -- the $\textit{informative semi-factuals}$ (ISF) method -- that generates more elaborated explanations supplementing semi-factuals with information about additional $\textit{hidden features}$ that influence an automated decision. Experimental results on benchmark datasets show that this ISF method computes semi-factuals that are both informative and of high-quality on key metrics. Furthermore, a user study shows that people prefer these elaborated explanations over the simpler semi-factual explanations generated by current methods.
Tags
Links
- Source: https://arxiv.org/abs/2603.17534v1
- Canonical: https://arxiv.org/abs/2603.17534v1
Trouble viewing inline? Open PDF directly →
Full Text
64,735 characters extracted from source content.
Expand or collapse full text
11institutetext: School of Computer Science, University College Dublin, Dublin, Ireland 22institutetext: Insight Research Ireland Centre for Data Analytics, Dublin, Ireland 22email: saugat.aryal@ucdconnect.ie 22email: mark.keane@ucd.ie Informative Semi-Factuals for XAI: The Elaborated Explanations that People Prefer Saugat Aryal (✉) Mark T. Keane Abstract Recently, in eXplainable AI (XAI), even if explanations – so-called semi-factuals – have emerged as a popular strategy that explains how a predicted outcome can remain the same even when certain input-features are altered. For example, in the commonly-used banking app scenario, a semi-factual explanation could inform customers about better options, other alternatives for their successful application, by saying “Even if you asked for double the loan amount, you would still be accepted". Most semi-factuals XAI algorithms focus on finding maximal value-changes to a single key-feature that do not alter the outcome (unlike counterfactual explanations that often find minimal value-changes to several features that alter the outcome). However, no current semi-factual method explains why these extreme value-changes do not alter outcomes; for example, a more informative semi-factual could tell the customer that it is their good credit score that allows them to borrow double their requested loan. In this work, we advance a new algorithm – the informative semi-factuals (ISF) method – that generates more elaborated explanations supplementing semi-factuals with information about additional hidden features that influence an automated decision. Experimental results on benchmark datasets show that this ISF method computes semi-factuals that are both informative and of high-quality on key metrics. Furthermore, a user study shows that people prefer these elaborated explanations over the simpler semi-factual explanations generated by current methods. 1 Introduction In recent years, as Machine Learning models have become more complex and opaque, eXplainable Artificial Intelligence (XAI) has advanced methods to improve the transparency of these models, making their decisions more interpretable to humans. Indeed, in some jurisdictions explainability is a legal requirement (see e.g. the EU’s GDPR [wachter2017counterfactual] and AI Act [EUAIAct2024]). “If only" explanations using counterfactuals have been extensively explored in XAI, as they naturally show human end-users how a model’s predictions can change when input-features are altered (often minimally). For example, if I am turned down for a $70k loan and query the decision, I could be given a counterfactual (e.g., “If only you had asked for a slightly lower loan of $68k, you would have been approved”) giving me some recourse on the outcome. Even if or semi-factual explanations are related but differ as they show users how a model’s predictions can remain the same when input-features are altered. For example, on being refused a $70k loan, a semi-factual explanation could say “Even if you asked for a $10k loan, you would still be refused”; here, even though this explanation seems less helpful, it still tells me something about my negative outcome (namely, that the bank considers me to be a credit risk even for a much smaller loan). As we shall see, semi-factuals are not always unhelpful as they can often inform users about gainful alternatives to outcomes (e.g., telling a farmer that even if they halved their fertilizer use, they would still get the same high crop yield). Here, we present a novel algorithm for computing semi-factual explanations, one that aims to make them much more informative for people using AI decision-making systems. 1.1 How Counterfactuals & Semi-Factuals Differ Although semi-factuals are a special case of the counterfactual – as they tell us about counters to the facts, albeit ones that do not change outcomes – they have different computational constraints and psychological impacts than counterfactuals; semi-factuals appear to serve very different explanatory purposes. Typically, good counterfactuals are computed by finding minimal changes to features that alter negative outcomes; they inform us about alternative antecedents that can change a bad outcome (e.g., asking for a lower amount to get a loan). People spontaneously use counterfactuals to draw attention to enabling conditions leading to a negative event, that they may have overlooked ([byrne2019counterfactuals]; e.g.,“If only it hadn’t been raining, the accident wouldn’t have happened”). Furthermore, counterfactuals implicitly assume ceteris paribus (i.e.,“other things being equal”); that the altered outcome is solely due to the specified feature changes and is not due to any other hidden/unmentioned feature changes. This assumption has to hold for the counterfactual to be meaningful and informative. In contrast, good semi-factuals often hinge on finding maximal changes to a feature that lead to the same outcome (e.g., asking for a much lower loan and still being refused); they are often designed to rhetorically convince people that a particular feature is not as important as it first seemed. For example, if after asking for a $70k loan, I am told that I would still be unsuccessful in a $10k-loan application, the semi-factual is indicating that loan-amount may not be the main factor in the refusal (e.g., maybe credit score is more important?). Accordingly, semi-factuals can weaken people’s causal understanding of events ([mccloy2002semifactual]; e.g., changing a farmer’s belief that more fertilizer equals higher yields). Furthermore, unlike counterfactuals, semi-factuals do not seem to assume ceteris paribus. Rather, implicitly, they seem to invite a consideration of other factors, hidden features that are affecting the outcome. For instance, if loan-amount is not the main factor affecting my refusal, what is? Am I being rejected because of my age (I am too young) or income level (its too low) or maybe it’s my credit rating (its too low). This observation suggests that semi-factual explanations could be made much more informative, if we could identify such hidden features in the decision process and inform people about them. Figure 1 shows a loan scenario in which a semi-factual explanation could be made more informative by revealing information about other features. Imagine Mark has two friends – John and Mary – who have recently applied for loans to the same bank as Mark. John, who has a decent credit score (of 700), asked for a lowish loan ($17k) and was successful. Mary was unsuccessful, perhaps because she asks for a higher loan ($70k) and/or because she has a poor credit score (of 300). Mark has a low-to-fair credit score (of 550) and when he applies for a modest loan (of $20k), he is successful. But, he wants to know if he can get a better deal. Could he perhaps borrow more? Here, the semi-factual explanation could tell him, that even if he asked for a $65k loan he would be successful, as his credit score is good enough to make him less of a risk than Mary. At present, no existing semi-factual method considers this additional information and such explanations have not been user-tested. In this paper, we consider both. Figure 1: The decision space for three loans, with two features shown (i.e., Loan Amount in $s and Credit Score as CS), for three applicants: John, Mark and Mary. John and Mark have their loans accepted but Mary has been rejected. Mark asks for an explanation about how to get a better deal. The semi-factual (SF) tells him that with his 550 credit score, he can actually get a $65k loan. Mark previously thought that if he asked for more, he would end up being rejected like Mary. The semi-factual shows Mark the limits on his loan aplication given his credit score (n.b., if he asked for $70k, he would be rejected like Mary). 1.2 Computing More Informative Semi-Factuals Aryal & Keane [ijcai2023p732] found that most semi-factual methods compute explanations that have maximal value-changes, typically to a single key feature of a query instance, that leave the outcome unaltered. For example, their Most Distant Neighbor (MDN) method considers each feature-dimension of the query and tries to find another instance in the dataset that is furthest from it, while still being in the same class. However, none of these methods consider "why" it is possible to radically change a key-feature’s value without causing a class-change. So, none of these methods provide users with a really informative semi-factual, one that explains the other hidden features influencing the decision outcome (e.g., explaining to Mark that loan amount may not be the most important factor, but that credit score becomes more important). It is the computation of these more informative semi-factuals that is the main focus of this paper. Figure 2: The rightmost graph shows a decision space for Mark and his semi-factual explanation (SF), with a path between them based on two perturbation steps (Q’ and Q”) in which the key-feature loan amount is systematically increased from $20k to $65K, without changing credit-score (which stays at 550). The leftmost graphic shows the relative changes in the marginal contributions of these two features across these perturbed instances as they remain in the loan-accept class. As loan amount increases its marginal contribution to keeping instances in the loan-accept class decreases (see purple plot) and even though credit-score’s value does not change, it’s marginal contribution increases (see orange plot) revealing this seesaw pattern between the two features. Our hypothesis is that these more informative semi-factuals reflect local changes in the relative importance of different features across different instances in the data distribution. Imagine a perturbation path between Mark and his semi-factual (SF), created by systematically changing only the value of the loan amount in incremental steps (see right of Figure 2). For any given predictive model, we can assess the marginal contribution of these feature changes for a given instance with respect to it remaining in its current class or moving to another class. Over these perturbation steps, as loan amount increases, instances move closer to the decision boundary with the loan-reject class. But, these changes do not tip it into the loan-reject class because credit-score’s influence is simultaneously increasing (see left of Figure 2). Even though, inevitably, increasing loan amount will eventually flip the class, over these perturbations credit score is effectively blocking that class change, as its marginal contribution increases (n.b., even though credit-score’s value does not change). Hence, there is a characteristic seesaw pattern in the importance of these two features, as the loan-amount key-feature weakens and credit-score hidden-feature strengthens. Based on these ideas, we propose a novel explanation algorithm – the Informative Semi-factuals (ISF) method – that relies on computing these changing feature contribution patterns in different feature-types as a new constraint in a multi-objective XAI method (see Section 2). We then report some user studies to test whether people actually find these elaborated semi-factuals to be more useful as explanations of automated decisions (see Section 4). Our intuition was that these patterns of changing importance between key- and hidden-features occur in most good semi-factuals. To test this intuition we carried out an extensive test on a large sample of the best semi-factuals produced by an ensemble of the main semi-factual methods in the literature (see Appendix A.1). This dataset of explanations came from comprehensive tests of 8 different semi-factual methods applied to 7 representative tabular datasets (using 5-fold cross-validation) evaluated against 5 key metrics (i.e., L2-norm distance, plausibility, confusability, robustness, sparsity) [aryal2024even]. Based on a large sample of queries from these datasets, ∼ 10,000 of the best semi-factuals were identified irrespective of the explanation method that produced them (n.b., most were generated-instance explanations rather than existing data-points). This analysis showed that 89% of the best semi-factuals found show the seesaw pattern of changing marginal contributions between key- and hidden-features (akin to that shown in Fig. 2). So, it appears that this property of semi-factuals has always been present in good semi-factuals, and can, therefore, be used to compute more informative semi-factuals for end users. Hence, we proceeded to implemented a new semi-factual explanation method with these new constraints. 1.3 Outline of Paper & Contributions In the remainder of this paper, we begin by formally defining informative semi-factuals that include new constraints along with their traditional requirements (Section 2.1). Subsequently, we introduce a novel method that computes these informative semi-factuals based on the defined properties (Section 2.2). In Section 3, computational experiments are run to evaluate the performance of this new method in obtaining informative and good semi-factuals by comparison to existing methods. Next, we report two user studies to examine whether people find these elaborated explanations to be more useful (Section 4). Finally, we review the existing works in semi-factual literature (Section 5), before closing with some discussion (Section 6). As such, the paper makes three new contributions to the field of semi-factual explanations by (1) defining a new desideratum for semi-factuals, namely the necessity to surface hidden features to provide more informative explanations, (2) proposing a novel method for computing this new requirement, the Informative Semi-factuals (ISF) method, along with tests showing it produces the best semi-factuals, (3) reporting two novel user tests showing that people prefer these elaborated semi-factual explanations over ones lacking such elaboration. 2 Computing Informative Semi-factuals Earlier, we advanced the argument for computing more informative semi-factuals, ones that convey a better mental model of the feature contributions leading to various automated decisions. This argument hinged on the proposal that good semi-factual instances involve a seesaw pattern in which the marginal contribution of key-features weaken (i.e, key-feature weakening) as the contribution of hidden-features simultaneously strengthen (i.e, hidden-feature strengthening) relative to the presented query instance. In this section, we propose a new algorithm for computing these more informative semi-factuals - the Informative Semi-factual (ISF) method – based on implementing these ideas as new constraints on semi-factual generation. Accordingly, we need to extend the current desiderata for semi-factuals to formally specify these new constraints (see Definition 2). Then, armed with this new specification we set about defining a multi-objective optimization method for computing “better” semi-factuals, ones that give people more informative explanations (see Section 2.2). 2.1 Formalizing Informative Semi-factuals Informative Semi-factuals are formalized in the following definitions that are distinguished from previously-proposed semi-factuals by the addition of two new constraints about feature-contribution changes. Preliminaries. Let f^:D→ f:X^D be a prediction model, X the D-dimensional feature space and Y a set of desired outcomes. Let γ:[0,1]→γ:[0,1] be a path from xqx_q to xsfx_sf in the feature space, with γ(0)=xqγ(0)=x_q and γ(1)=xsfγ(1)=x_sf, where xqx_q is the query and xsfx_sf is the semi-factual explanation. For a feature j, let ϕj(t) _j(t) denote its marginal contribution to the prediction f^(γ(t)) f(γ(t)) at point t where t∈[0,1]t∈[0,1] and TϕjT_ _j is the trend strength of the marginal contribution measured by: Tϕj=∫01dtϕj(t)tT_ _j= _0^1 ddt _j(t)dt (1) Definition 1(Semi-factual Explanation) A semi-factual explanation, xsfx_sf for a query instance xqx_q is a data point that satisfies the following: (i) xsfx_sf has the same prediction as xqx_q, (i) xsfx_sf differs from xqx_q on a key-feature dimension, k (ideally 1), (i) xsfx_sf is close to xqx_q along all other features, and (iv) xsfx_sf is a plausible instance according to the probability distribution ℙP_X. Definition 2(Informative Semi-factual Explanation) A given semi-factual explanation, xsfx_sf (as defined in Definition 1), will be an informative semi-factual explanation if it additionally satisfies two properties: • Key-feature Weakening: The key-feature’s marginal contribution, ϕk(t) _k(t) has a decreasing trend when moving from xqx_q to xsfx_sf along γ: Tϕk<ϵT_ _k<ε (2) where ϵ<0ε<0 is the trend strength parameter. • Hidden-feature Strengthening: There exists a non-key hidden-feature, which simultaneously exhibits the maximum increasing trend in its marginal contribution ϕj(t) _j(t) when transitioning from xqx_q to xsfx_sf along γ: j∗=argmaxj∈1,…,D\kTϕj^*= _j∈\1,…,D\ \k\T_ _j (3) In the next sub-section, we show how these definitions can be implemented in the Informative Semi-factual method. 2.2 ISF: The Informative Semi-factual Method Taking the above specifications we implement this new Informative Semi-factual (ISF) method using a multi-objective optimization approach where the traditional requirements for a good semi-factual are pitted against the new constraints on patterns of marginal contribution changes. Stated simply, this method tries to balance the generation of semi-factuals with maximal value-change on key-features against the marginal contribution trends of key- and hidden-features. The semi-factual generation problem can be translated as following multi-objective minimization task: minxF(x):=minx(-o1(x,xq),o2(x,xq)) _xF(x)= _x(-o_1(x,x_q),o_2(x,x_q)) (4) subject to: g1(f^(x),f^(xq))≤0 and g2(x)≤0 g_1( f(x), f(x_q))≤ 0 and g_2(x)≤ 0 Here, the first objective o1o_1 ensures maximum distance between x and xqx_q along key-feature dimension, k: o1(x,xq)=|xk−xqk|o_1(x,x_q)=|x^k-x_q^k| (5) Since, the goal is to maximize this distance, we minimize -o1o_1. The second objective o2o_2 ensures minimizing the cumulative distance across all features except the key feature, encouraging similarity on non-key features: o2(x,xq)=∑i≠k|xi−xqi|∀i∈Do_2(x,x_q)= _i≠ k|x^i-x^i_q| ∀ i∈ D (6) This g1g_1 constraint ensures that x and xqx_q have the same class: g1(f^(x),f^(xq))=(f^(x)≠f^(xq))=0g_1( f(x), f(x_q))=I( f(x)≠ f(x_q))=0 (7) The second constraint, g2g_2 ensures that x remains within a plausible region of the data distribution. To do so, we introduce a constraint such that the Probability Density Function (PDF) of x lies within a dynamic threshold range. The range is based on the mean of the probability distribution of the observed data adjusted by an adaptive factor, which scales with the variability of the data: g2(x)=[(μ−Δδ)≤logℙ(x)≤(μ+Δδ)]g_2(x)=I[(μ- δ)≤ (x)≤(μ+ δ)] (8) where μ is the mean log-pdf of the data distribution, ℙP_X and Δδ δ is an adaptive threshold range, defined as: Δδ=θ∗σ δ=θ*σ (9) where σ is the standard-deviation of the log-pdf of the data distribution and θ is a scaling parameter which controls the strictness of the constraint. This optimization method produces a set of diverse and equally valid semi-factuals. Next, we analyze the marginal contribution of the features in these explanations to obtain informative semi-factuals. To do so, for each of the explanation-solutions, we linearly interpolate from xqx_q to xsfx_sf as: x(t)=(1−t)xq+txsfx(t)=(1-t)x_q+tx_sf (10) where t∈[0,1]t∈[0,1] and x(t)x(t) is an instance during interpolation at point t with t=0t=0 at xqx_q and t=1t=1 at xsfx_sf. During each step of interpolation, the marginal contribution of all the features (both key and non-key) on the outcome are noted. Specifically, their pure main effects, ϕj(t) _j(t), are determined to obtain a good estimation of how feature-influences are changing independently during interpolation. Trends in marginal contributions are identified using the Kendall’s tau of Mann-Kendall test [mann1945nonparametric]. Finally, we select those instances in which the key-feature’s contribution has the lowest decreasing trend as the best candidate semi-factual explanation. Concurrently, for this selected explanation, we identify the non-key feature with the highest increasing trend as the hidden-feature component, to form the most informative semi-factual explanation. For a given query, we run the method by treating each of its features as a key-feature to obtain multiple informative semi-factuals across each feature-dimension before selecting the best of the best as the final informative explanation that has the overall lowest key-feature-weakening trend. 3 Testing ISF Algorithm Two computational experiments compared ISF’s performance on representative datasets to the performance of an ensemble of leading semi-factual methods. The first experiment determined whether ISF’s semi-factual generation process correctly produced explanations with the requisite properties (i.e., the key-feature weakening and hidden-feature strengthening pattern) relative to ensemble methods that were not specifically designed to compute these properties. The second experiment evaluated the semi-factuals generated by ISF by comparison to the best semi-factuals generated by the ensemble methods to determine their relative goodness on the traditional metrics. In both experiments a leave-one-out cross-validation was performed in which each query instance was used to generate candidate semi-factuals for every dataset (for source code and data see here). Taken together these experiments determine whether (i) ISF really does what it claims to do in generating informative explanations, and (i) ISF produces quality semi-factual explanations that are competitive with the SOTA. Setup: Datasets & Benchmarks. Both experiments used five benchmark, publically-available tabular datasets, all of which were binary-classed: Adult Income, Blood Alcohol, PIMA Diabetes, German Credit, and HELOC. Following [van2021interpretable], all categorical features in these datasets were encoded into a numerical space using distance metrics. ISF’s performance was benchmarked against the best explanations produced by an ensemble of leading semi-factual methods: the CBR-based Local-Region method [nugent2009gaining], Knowledge-Light Explanation-Oriented Retrieval (KLEOR)[cummins2006kleor], Diverse Semi-factual Explanations of Reject (DSER) [artelt2022even], PIECE [kenny2021generating], C2C-VAE [zhao2022generating], MDN[ijcai2023p732], S-GEN[kenny2023sf], and DiCE [mothilal2020explaining]. PIECE and C2C-VAE had to be modified to work with tabular data. For DiCE, the desired class of the explanation was set to be the same as that of the query, to allow it to generate semi-factuals not counterfactuals. Appendices A.1, A.2 have full description of the models, the modifications made to them and parameters used. ISF Implementation. ISF implements the semi-factual generation as a multi-objective minimization task using Equations (4)-(10). We used Gaussian copula [nelsen2006introduction] to model the joint distribution of features and obtain ℙP_X. In Eq.(9) the threshold was set as θ=1.5θ=1.5. The Nondominated Sorting Genetic Algorithm I (NSGA-I) [deb2002fast] was used to solve the constrained multi-objective semi-factual problem. The algorithm efficiently approximates the Pareto front to produce a set of diverse and equally-valid semi-factual solutions. We used an initial population size of 50 for 100 generations (see Appendix A.3 for full details on parameters). A Random Forest Classifier model was used to determine class-membership (i.e., as in the g1g_1 constraint). The pure marginal contribution of features was computed using the diagonal entries of the SHAP interaction matrix [lundberg2017unified] as it isolates the main effects. Specifically, the TreeSHAP variant [lundberg2020local2global] was used to work with the Random Forest model. To compute the changes in marginal contributions, we created 10 intermediate instances interpolating from the query to the generated semi-factual. The trend in feature contributions occurring over these perturbed instances was analyzed using the Mann-Kendall trend test. The resulting Kendall’s tau (τ) coefficient was used to measure trend strength, where τ∈[−1,1]τ∈[-1,1] indicates the direction and strength of the trend. We set ϵ=−0.3ε=-0.3 in Eq.(2) as the trend strength for key-feature weakening. 3.1 Experiment 1: Informative Semi-factuals? Figure 3: From Expt.1, the percentage of semi-factuals, for five datasets, generated by the ISF and Ensemble-methods (N=38,233 in total), that manifested the seesaw pattern in key-feature versus hidden-feature contributions. (a) Distance (b) Sparsity (c) Plausibility (d) Trustworthiness Figure 4: From Expt.2, the mean goodness scores for semi-factuals generated by the ISF and Ensemble-methods for the five datasets (N=38,233 in total), in each of the four evaluation metrics (a-d). Experiment 1 was designed to determine whether ISF really does what it claims to do in generating informative explanations; namely, does it identify semi-factuals with the seesaw pattern showing a weakening of the key-feature alongside a strengthening hidden-feature. If these properties are properly identified then ISF will be able to provide informative explanations (e.g., as in “Mark, you can get a $65k loan, as you have a fair credit score”). Procedure. Across 5 datasets, a total of 38,223 queries were tested, recording the semi-factuals generated by ISF and the ensemble. For a given query, the best semi-factual from the ensemble models was selected to compare to that generated by ISF. Note, although none of the ensemble models explicitly compute patterns of marginal contributions, earlier tests showed that they tend to generate semi-factuals with these properties (though perhaps not optimally so). The measure used was the percentage of semi-factuals generated by ISF/ensemble that showed the seesaw pattern in the changing marginal contributions. Results. Figure 3 shows the percentage of semi-factuals generated by ISF and the ensemble that showed the requisite seesaw pattern needed to compute informative semi-factual explanations. It shows that across all datasets ISF is able to produce more informative semi-facutals than the ensemble of leading methods, at very high levels of almost 100% in 4 out of 5 datasets. 3.2 Experiment 2: Informative Semi-factuals Are Also Good? In Experiment 1, we saw that ISF regularly generates semi-factuals with the requisite seesaw pattern needed for more informative explanations. Using the same setup and procedure, Experiment 2 aimed to determine whether ISF’s semi-factuals were also the best semi-factuals relative to those produced by the ensemble of methods. So, it evaluated the quality of semi-factuals generated by ISF and the ensemble using standard metrics for assessing semi-factual goodness [ijcai2023p732, aryal2024even]. Four commonly-used evaluation metrics were applied to all semi-factuals: • Distance: measures the L2L_2-norm distance between a query and its semi-factual where higher value is preferred. • Sparsity: measures feature differences between the query and semi-factual as a ratio of the desired- (set to 1) to observed-difference, where a higher value is better. • Plausibility: is measured as the distance between a semi-factual and the nearest training datapoint, where smaller value is better. • Trustworthiness: measures the confidence for a semi-factual for being in the query class compared to the counterfactual class, measured as the ratio of their class distances (normalized), where higher score is better. Results. Figure 4 shows that ISF produces good semi-factuals, that are consistently better than those produced by the ensemble of leading methods on key evaluation metrics. It consistently outperforms the ensemble on Sparsity and Trustworthiness for all datasets. ISF’s semi-factuals are sparse as they focus on changing only one key-features and are trusted to be classified as being within the query class. These higher trust scores probably occur because of support from hidden features that keep the semi-factual in the query class. ISF also does better on Distance and Plausibility measures across most datasets. Overall, it is clear that ISF produces both informative and high-quality semi-factual explanations, ones that are generally better than the SOTA in the field. Figure 5: The percentage of people choosing the (a) Bare Semi-factual or Informative Semi-Factual in User Study 1, and (b) the Good or Bad Semi-factuals in User Study 2, for scenarios with the loan-accepted or loan-rejected outcomes 4 Testing People’s Preferences for Explanations The computational experiments showed that ISF can generate very good, informative semi-factuals, where we define an “informative” semi-factual to be one showing the key-feature and its supporting hidden feature (e.g., “Even if you asked for $65k you would be successful, as you have a fair credit score”). However, user studies are also required to determine whether people think ISF’s explanations are acceptable and useful. Two psychological experiments were carried out to assess this question, in which people assessed the utility of semi-factuals in a forced-choice task; that is, they were shown several loan-scenarios each of which had two semi-factual explanations and asked to select the one they thought was the most useful. Study 1 gave people scenarios with and without the hidden-feature added to the explanation. Study 2 gave people scenarios with semi-factual explanations lacking hidden-features, but which were deemed to be good or bad by the virtue of key-feature’s decreasing trend. 4.1 User Study 1: Do People Prefer Informative Explanations? To our knowledge, Kenny & Huang’s [kenny2023sf] study examining people’s preferences for counterfactuals and semi-factuals is the only XAI user test in this area; so, we adapted their design to test the ISF method. User Study 1 tested whether people prefer bare semi-factual explanations or the more elaborated ones generated by ISF (i.e., mentioning the key feature and the hidden feature). Participants were presented with loan-scenarios for different individuals, each of which had five features (Duration, Credit Amount, Age, Installment Percent, Existing Loans) with loan-accepted/loan-rejected outcomes. For each scenario, they were shown two explanations (counterbalanced in order of presentation across items), one with the semi-factual on its own (Bare-SF; e.g., “Even if you asked for $65k you would be successful”) and one showing the semi-factual with the added hidden feature (Informative-SF; e.g., “Even if you asked for $65k you would be successful, as you have a fair credit score”). They were then asked to select the explanation they thought was the most useful as feedback to the customer. Method: Participants, Design, Materials & Procedure. Participants (N=15N=15, based on a power analysis for the single-group design) were recruited from Prolific.com and were pre-screened to be native English speakers from Ireland/UK/USA/Australia/Canada/New Zealand who had not participated in previous related studies. They were paid £14/hr for their participation. The study employed a 2 (Explanation Type: Bare-SF v Informative-SF) X 2 (Loan Outcome: Loan Accepted v Loan Rejected) within-subject design. Each participant was presented with 32 different scenarios, half with loan-accepted and half with loan-rejected outcomes. Each scenario was presented with the two versions of the explanation, with and without the hidden feature. For each scenario the participants had to select the explanation they thought was the most useful. For analysis, the count for either choice for both loan outcome scenarios were counted, to be expressed as a percentage. Results. Figure 5(a) shows that people find the informative semi-factual explanations with the supporting hidden-feature to more useful than those without that feature (69% v. 31% in loan accept and 80% v. 20% in loan reject scenarios). Binomial tests further confirmed that these preference differences is statistically significant with p<.0001p<.0001 for both loan outcome scenarios. 4.2 User Study 2: Preferring Good or Bad Explanations? Most semi-factual algorithms can be tweaked to produce good or bad explanations, but we know of no user study that has explicitly tested whether people consider such explanations to be better or worse than one another. User Study 2 tested whether semi-factual explanations deemed to be good/bad by ISF were also deemed to be good/bad by people. Using the same loan materials, ISF’s parameters were modified to produce semi-factuals with high or low scores on key evaluation metrics. These semi-factuals were then presented as in User Study 1, with people being asked to select the one they thought was the most useful. To make this a simple test, the explanations were presented as unelaborated, bare semi-factuals (even though, according to ISF, they had different hidden-feature patterns). So, all of these explanation-pairs used the same key-feature and were classed as good or bad by having distant/close values for this key-feature and by having strong/weak see-saw patterns in their hidden features, respectively. Method: Participants, Design, Materials & Procedure. A new sample of participants (N=15N=15) were recruited using the same criteria and compensation as User Study 1. The experimental design and procedure remained the same, with the only difference being the Explanation Type. The study employed a 2 (Explanation Type: Good-SF v Bad-SF) X 2 (Loan Outcome: Load Accepted v Loan Rejected) within-subject design. Each participant was presented with 32 different scenarios, half with loan-accepted and half with loan-rejected outcomes. Furthermore, each scenario had “good” and “bad” semi-factuals from which the participants had to select the one they found the most useful. The good and bad semi-factuals were obtained from ISF by setting the trend parameter within −1≤ϵ≤−0.8-1≤ε≤-0.8 and −0.6≤ϵ≤−0.3-0.6≤ε≤-0.3 in Eq. (2) respectively. Results. Figure 5(b) shows that people find ISF’s good semi-factuals to be significantly more useful than its bad semi-factuals (69% v. 31% in loan accept and 81% v. 19% in loan reject), discriminating their relative goodness as explanations, even when their underlying hidden-features are not provided. So, even when people encounter semi-factual explanations without hidden-feature elaborations, at some level, they may “know” that there is another factor at work (and have some sense of the strength of that other factor). 5 Related Work While there are no previous studies which considers computing informative semi-factuals, we review some of the key works in semi-factual literature from computational and psychological standpoint (see [ijcai2023p732] for comprehensive review). Computational XAI Research. Research on semi-factual explanations in XAI originated in early 2000s in the field of Case-Based Reasoning (CBR) [nugent2005best, nugent2009gaining, cummins2006kleor]. This work characterized semi-factuals as "a fortiori" arguments to provide better and more convincing explanations. They used utility functions [doyle2004explanation], similarity methods [cummins2006kleor], and local-region based proxy models [nugent2009gaining] to compute such explanations. Kenny & Keane [kenny2021generating] revisited semi-factuals more systematically extending to generative AI, using GANs based on the exploitation of "exceptional features". This work was followed by Zhao et al. [zhao2022generating], who used class-to-class variational autoencoders (VAEs) to compute semi-factuals efficiently and Vats et al. [vats2022changes] who used the latent space of these generative models to obtain semi-factuals to explain the classification of medical images such as ulcers. Aryal & Keane [ijcai2023p732] surveyed the semi-factual literature to define the "requirements" for cognitively and computationally good semi-factuals. They proposed a novel benchmark method, Most Distant Neighbors (MDNs), which used a scoring function to select the most distant same-class instance as the semi-factual. In other work, Aryal & Keane [aryal2024even] demonstrated that semi-factuals are conceptually and computationally distinct from counterfactuals, possessing their own unique dynamics rather than being mere "by-products". Kenny & Huang [kenny2023sf] advanced the research further by introducing "gain" as a new constraint for semi-factual explanations. They argued that semi-factuals can provide better recourses for positive outcomes whereas counterfactuals are more useful for negative outcomes, which they validated through user studies. Other Computational Research. Semi-factuals have also been used for model auditing; specifically for identifying spurious patterns and bias [lu2022rationale] or for providing unified explanatory frameworks via Gaussian mixture models [xie2023joint]. Similarly, Artelt & Hammer[artelt2022even] utilized semi-factuals to justify "reject" decisions made by a model. Dandl et al. [dandl2023interpretable] proposed Interpretable Region Descriptors (IRDs) for semi-factuals which maps the stable region around the query through "hyperboxes". More recently, the scope of semi-factual explanations have expanded into dynamic environments. Gajcin et al. [gajcin2024semifactual] successfully adapted semi-factual desiderata to Reinforcement Learning (RL) agents using genetic algorithms to identify stable states. Jiang et al. [jiang2024interpreting] generated semi-factual explanations for Reward Models (RMs) to analyze their local and global behavior. Psychological Research. The research on the cognitive effects of semi-factuals were originally carried out in the field of psychology. The seminal work of McCloy & Byrne [mccloy2002semifactual] found that semi-factual can weaken the causal relation between an input and outcome, convincing people that the outcome would have occurred regardless. Similarly, they also showed that semi-factual thoughts decrease the emotion of regret. On the other hand, Green [green2008persuasion] showed that a semi-factual supports dissuasion convincing people to not take further action. 6 Discussion Semi-factual explanations employ Even if reasoning to explain how changes to certain input-features do not change the outcome. In XAI, all the existing works on semi-factuals have focused on computing the farthest instances from the query, within the same class, using various different techniques. However, to the best of our knowledge, none of these models examine "why" it is possible for a semi-factual to have such maximal value-changes and yet remain in the same class. This work advances the research on semi-factual explanations by introducing the notion of informative semi-factuals. These explanations complement a standard semi-factual with hidden features that further explain how the semi-factual can occur in the data distribution of a class. To compute these informative semi-factuals, we proposed a novel – the Informative Semi-factual (ISF) method – which performs better than the SOTA methods in computational experiments. We also conducted user studies which shows that people find these elaborated semi-factual explanations better and more useful than the simple standard ones. A notable limitation of our work is its computational efficiency when there are large number of features to consider in the multi-objective setting. However, this may be addressed with feature-selection techniques that identify the most relevant or locally-sparse features. As part of future work, it would also be interesting to explore how this concept can be formalized in other domains such as images, and time series which may require specific considerations. credits 6.0.1 Acknowledgements This work has emanated from research conducted with the financial support of Taighde Éireann – Research Ireland under Grant number: 12/RC/2289_P2. 6.0.2 The authors have no competing interests to declare that are relevant to the content of this article. References Appendix 0.A Appendix 0.A.1 Test for Seesaw Pattern in Best Semi-factuals In this test, we computed the presence of seesaw pattern of marginal feature contributions (key- and hidden-features) in best semi-factuals obtained from the ensemble of standard semi-factual methods. Methods. The following 8 SOTA semi-factual methods were considered for the ensemble. • Local-Region Model Nugent et al. [nugent2009gaining] proposed the Local-Region model, which analyses the local region around the query, using a surrogate model (akin to way LIME works), to select the nearest neighbor in the query class with the most marginal probability, as the semi-factual instance; as follows: Local-Region(q,C)=argminxϵCLR(x)Local-Region(q,C)= _xε CLR(x) (11) where, C is the set of candidate neighbors and LR() is the local logistic regression model providing the probability score. • Knowledge-Light Explanation-Oriented Retrieval (KLEOR) Cummins & Bridge [cummins2006kleor] identified query-class instances which lie between the query and its Nearest Unlike Neighbors (NUNs) (aka its counterfactuals) using distance measures and selected the best semi-factual as the one which is closest to the NUN and furthest from the query. They proposed three different variants out of which Attr-Sim was the most advanced. It enforces similarity across all of the features between the query and the candidate instance. Attr-Sim(q,nun,G)=argmaxxϵGSim(x,nun)+maxaϵFcount[Sim(qa,xa)>Sim(qa,nuna)] splitAttr-Sim(q,nun,G)&= _xε GSim(x,nun)\\ & + _aε Fcount[Sim(q_a,x_a)>Sim(q_a,nun_a)] split (12) where q is the query, x is a candidate instance, G is the set of query-class instances, nun is the NUN, Sim is Euclidean Distance and F is the feature set. • Diverse Semifactual Explanations of Reject (DSER) Artelt & Hammer [artelt2022even] proposed a semi-factual method based on optimization methods to explain reject decisions in machine learning; that is, to explain why a model should not make a prediction. It applies its loss function using four constraints for good semi-factuals (i.e., feasibility, sparsity, similarity, diversity): DSER(q)=argminqsfϵℝdℓ(qsf)DSER(q)= _q_sf ^d (q_sf) (13) where, qsfq_sf is the semi-factual of query q and ℓ() () represents the combined loss function such that, ℓ(qsf)=ℓfeasibile(qsf)+ℓsparse(qsf)+ℓsimilar(qsf)+ℓdiverse(qsf) (q_sf)= _feasibile(q_sf)+ _sparse(q_sf)+ _similar(q_sf)+ _diverse(q_sf) (14) where feasibility is cast as, ℓfeasible (qsf)=Cfeasible⋅max(r(qsf)−θ,0)+Csf⋅max(r(q)−r(qsf),0) _feasible (q_sf )=C_feasible· (r (q_sf )-θ,0 )+C_sf· (r(q)-r (q_sf ),0 ) (15) which ensures that the semi-factual is also predictively uncertain but more certain than the original query, q (to be convincing). Here, C represents the regularization parameters for each component, r()r() is the reject function based on the certainty of predictive function and θ is the reject threshold. ℓsparse(qsf)=Csparse⋅max(∑i=1d((qsf−q)i≠0)−μ,0) _sparse (q_sf )=C_sparse· ( _i=1^d 1 ( (q_sf-q )_i≠ 0 )-μ,0 ) (16) covers sparsity promoting candidates with fewer feature differences between the semi-factual and the query. Here d is the number of feature dimensions and μ≥1μ≥ 1 is a hyperparameter that controls the number of feature-differences. ℓsimilar(qsf)=−Csimilar ⋅‖qsf−q‖2 _similar (q_sf )=-C_similar · \|q_sf-q \|_2 (17) deals with similarity promoting greater distance between the query and the semi-factual in Euclidean space, and finally, ℓdiverse (qsf)=Cdiverse ⋅∑j∈ℱ((qsf−q)≠0) _diverse (q_sf )=C_diverse · _j 1 ( (q_sf-q )≠ 0 ) (18) handles diversity ensuring that several featurally-distinct semi-factuals are generated. Here, ℱF represents the set of features that have already been used to generate semi-factuals, feature-sets that should be avoided: ℱ=j∣∃i:(qsfi−q)j≠0F= \j ∃ i: (q_sf^i-q )_j≠ 0 \ (19) • PlausIble Exceptionality-based Contrastive Explanations (PIECE) This method proposed by Kenny & Keane [kenny2021generating] computes semi-factuals “on the way to” computing counterfactuals using statistical techniques and a Generative Adverserial Network (GAN) model. It identifies "exceptional" (i.e., probabilistically-low) features in a query with respect to its counterfactual class and iteratively modifies these features until they become "normal" (i.e., probabilistically-high). As these exceptional features are incrementally altered, the generated instances gradually move away from the query towards the counterfactual class, with the last instance just before the decision boundary being selected to be the semi-factual. So, the semi-factual is like a point on the trajectory from the query to the counterfactual. Exceptional features are identified using the statistical probabilities in the training distribution of the counterfactual class c′c . Specifically, a two-part hurdle process is used to model the latent features of the query (when it is an image) in the feature-extracted layer (X) of a Convolutional Neural Network (CNN) with an ReLU activation function. The first hurdle process is modelled as a Bernoulli distribution and the second as a probability density function (PDF) as: p(xi)=(1−θi)δ(xi)(0)+θifi(xi), s.t. xi≥0p (x_i )= (1- _i ) _ (x_i )(0)+ _if_i (x_i ), s.t. x_i≥ 0 (20) where p(xi)p(x_i) is the probability of the latent feature value xix_i for c′c , θi _i is the probability of the neuron in X activating for c′c (initial hurdle process), and fif_i is the subsequent PDF modelled (the second hurdle process). The constraint of xi≥0x_i≥ 0 refers to the ReLU activations, and δ(xi)(0) _ (x_i )(0) is the Kronecker delta function, returning 0 for xi>0x_i>0, and 1 for xi=0x_i=0. After modelling the distribution, a feature value xix_i is regarded as an exceptional feature for the query in situations where, xi=0∣p(1−θi)<αx_i=0 p (1- _i )<α (21) if the neuron XiX_i does not activate, given the probability of it not activating being less than α for c′c , and, xi>0∣p(θi)<αx_i>0 p ( _i )<α (22) if a neuron activates, given that the probability of it activating being less than α for c′c , where α is a threshold. Once the exceptional features are identified, the query’s features are adjusted to their expected values (x′x ) with generated instances being checked by the CNN to be in the query or counterfactual-class. The semi-factual is the last generated instance in the query-class before crossing into the counterfactual-class. Finally, a GAN is used to visualize the explanations by identifying a latent vector (z′z ) such that loss between x′x and C(G(z′))C(G(z )) is minimized as, z′=argmin‖C(G(z))−x′‖22z = z \|C(G(z))-x \|_2^2 (23) PIECE(q)=G(z′)PIECE(q)=G(z ) (24) where C is a CNN classifier and G is the GAN generator. • Class-to-Class Variational Autoencoder (C2C-VAE) Zhao et. al [zhao2022generating] proposed an efficient way to obtain semi-factuals based on autoencoders. They used a variational autoencoder (VAE) with an encoder (f)(f) and a decoder (f′)(f ) to learn the embedding space representing the differences between feature patterns in two classes. In the initial learning phase, given a pair of cases s and t from two classes, C2C-VAE encodes the feature difference, fΔf_ , where fΔ(s,t)=f(s)−f(t)f_ (s,t)=f(s)-f(t) using an encoder g, as g(<fΔ,Cs,Ct>)g (<f_ ,C_s,C_t> ) and decodes the embedding using a decoder g′g as fΔ′=g′(g(<fΔ,Cs,Ct>)f_ =g (g (<f_ ,C_s,C_t> ) . To derive an explanation for a query, q in class CqC_q, the method first generates a guide t in the counterfactual class CtC_t. This guide selection leverages the feature difference embedding space g. Specifically, the method randomly samples vectors from g, decodes them back to the original feature space, and selects the one with the least mean squared error compared to q. Finally, it interpolates between the extracted features of f(q)f(q) and f(t)f(t) in the VAE’s latent space to obtain semi-factuals as, C2C-VAE(q)=f′((1−λ)∗f(q)+λ∗f(t)),0≤λ≤1C2C-VAE(q)=f ((1-λ)*f(q)+λ*f(t)),0≤λ≤ 1 (25) where λ is a hyperparameter which determines the weight of interpolation between q and t and controls whether the output is more similar to q (for a semi-factual) or t (for a counterfactual). • Most Distant Neighbor (MDN) Aryal & Keane [aryal2024even] proposed and this method as a naïve, benchmark algorithm to find query-class instances that are most distant from the query on some dimension(s), while also sharing many common features. MDN scores all the query-class’s instances on the extremity of their feature-values, determining whether they are much higher or lower than the feature-values of the query, q, to find its most distant neighbor. Its custom distance function, Semi-Factual Scoring (sfs), prioritises instances that are sparse relative to the query (i.e., fewer feature differences), but have the highest value-differences in their non-matching features, as follows: sfs(q,S,F)=same(q,x)F+diff(qf,xf)diffmax(qf,Sf)sfs(q,S,F)= same(q,x)F+ diff(q_f,x_f)diff_max(q_f,S_f) (26) where S is Higher/Lower Set and x∈Sx∈ S, same() counts the number of features that are equal between q and x, F is the total number of features, diff() gives the difference-value of key-feature, f, and diff()max_max() is the maximum difference-value for that key-feature in the Higher/Lower Set. The best-feature-MDN is selected as the instance with the highest sfs score from the Higher/Lower set for each feature, independently. Finally, the best of the best-feature-MDNs across all dimensions is chosen as the overall semi-factual for the query, MDN(q,S)=argmaxx∈Ssfs(x)MDN(q,S)= _x∈ Ssfs(x) (27) • Explanations for Positive Outcomes (S-GEN) Kenny & Huang [kenny2023sf] proposed the novel concept of “gain" (akin to “cost" in counterfactuals) as a new constraint for semi-factual methods. They argue that semi-factuals best explain positive outcomes, whereas counterfactuals work best for negative outcomes. Their S-GEN method uses gain, along with traditional constraints (such as plausibility, robustness and diversity) to compute semi-factuals that inform users about better and positive recourses. It’s objective function is: S-GEN()=max1,…,m S-GEN(q)= _a_1,…,a_m 1m∑i=1mf(P(,i),G(,i))+γR(1′,…,m′) 1m _i=1^mf (P (q,a_i ),G (q,a_i ) )+γ R ( \q_1 ,…,q_m \ ) (28) s.t. ∀i,j:i′=Sℳ(,i),Hj(i′)≥0( or >) ∀ i,j:q_i =S_M (q,a_i ),H_j (q_i )≥ 0( or >) where, a_i represents an action taken on ithi^th feature-dimension, m is the desired number of semi-factuals to be generated. P(,i)=Pr(Sℳ(,i))P(q,a_i)=Pr (S_M(q,a_i) ) denotes the plausibility of explanation for q by taking action ia_i where PrPr is the distribution density. S-GEN uses a Structural Causal Model (SCM), SℳS_M to better capture the causal dependencies between the features and hence obtain feasible explanations. G(,i)=SF∘δ(,Sℳ(,i))G(q,a_i)=P_SF δ (q,S_M(q,a_i) ) (29) represents the gain function for q by taking action ia_i where δ()δ() is the distance function. Intuitively, it measures the difference between original state q and the new state obtained by the transition from q by taking action ia_i through an SCM, Sℳ(,i)S_M(q,a_i). Greater differences indicate higher “gains” (i.e., better explanations). R(ii=1m)=2m(m−1)∑i=1m∑j>imLp∘δ(i,j)R ( \q_i \_i=1^m )= 2m(m-1) _i=1^m _j>i^mL_p δ (q_i,q_j ) (30) shows the diversity function which is regularized by γ in Eq. (28). LpL_p is the LpL_p-norm and δ()δ() is the distance function used, so as many distinct explanations as possible are generated that are also far from each other. Finally, robustness is achieved in a post-hoc manner as a hard constraint defined by: H(,)=min′∈s(,)h(′)−ψH(q,a)= _q _s(q,a)h (q )-ψ (31) where H(,)H(q,a) denotes the post-robustness of an action a for a test instance q. The intuition is that any instances lying in the neighborhood sB_s of the generated semi-factual ′=Sℳ(,)q =S_M(q,a) after taking the action a also have a positive outcome. This function ensures that the output of a predictive model h for ′q is higher than a threshold ψ=0.5ψ=0.5 (in case of binary classes). • Diverse Counterfactual Explanations (DiCE) Mothilal et. al [mothilal2020explaining] used an optimization method based on various constraints such as distance, diversity (using determinantal point processes) and feasibility (using causal information from users) to compute counterfactuals which can be used to obtain semi-factuals by adjustments to the loss function: DiCE(q)=argmin1,…,ck1k∑i=1kyloss(f(ci),y)+λ1k∑i=1kdist(ci,q)−λ2dpp_diversity(c1,…,ck) splitDiCE(q)&= c_1,…,c_k 1k _i=1^kyloss (f (c_i ),y )+ _1k _i=1^kdist (c_i,q )\\ & - _2dpp\_diversity(c_1,...,c_k) split (32) where q is the query input, cic_i is a counterfactual explanation, k is the total number of diverse counterfactuals to be generated, f()f() is the black box ML model, yloss()yloss() is the metric that minimizes the distance between f()f()’s prediction for cic_i and the desired outcome y, dist()dist() is the distance measure between cic_i and q, and dpp_diversity() is the diversity metric. λ1 _1 and λ2 _2 are hyperparameters that balance the three components of the loss function. Metrics. The following 5 key metrics were used to assess the quality of semi-factuals generated from each method. • Distance. It measures the L2L_2-norm distance between query and the semi-factual where higher is better. • Sparsity. It measures the number of feature difference between query and the semi-factual as a ratio of desired (set to 1) to observed-difference where higher value is preferred. Sparsity=desireddiffobserveddiffSparsity= desired_diffobserved_diff (33) • Plausibility. It measures the distance between the semi-factual and the nearest training instance where lower value is better. • Trustworthiness. It measures the confidence for the semi-factual being in the query class compared to the counterfactual class measured as a ratio of their class distance where higher score is better. Trustworthiness(x)=d(x,CF)d(x,Q)Trustworthiness(x)= d(x,CF)d(x,Q) (34) where x is the semi-factual, CF is the counterfactual-class, Q is the query-class and d()d() measures the distance. • Robustness. It measures the difference in semi-factuals obtained through small perturbations of the query measured using Lipschitz continuity where lower value is better. Robustness(x)=argmaxxi∈Bϵ(x)‖f(x)−f(xi)‖2‖x−xi‖2Robustness(x)= x_i∈ B_ε (x )argmax \|f (x )-f (x_i ) \|_2 \|x-x_i \|_2 (35) where x is the input query, Bϵ(x)B_ε (x ) is the ball of radius ϵε centered at x, xix_i is a perturbed instance of x and f()f() is the explanation method. Datasets. The evaluation was carried out across 7 benchmark tabular datasets which are all binary-classed. • Adult Income (N=26,540, 12 features) • Blood Alcohol (N=2000, 5 features) • PIMA Diabetes (N=392, 8 features) • Default Credit Card (N=30,000, 23 features) • German Credit (N=1000, 20 features) • HELOC (N=8291, 20 features) • Lending Club (N=39,239, 8 features) Procedure. For a given query in a dataset, its corresponding semi-factual was computed using all the 8 semi-factual methods along with their respective values for each evaluation metric. The semi-factual which had the highest aggregated score combining all the metrics was determined to be the best semi-factual explanation for the query. This process was carried out across all datasets to obtain best semi-factuals for each query-instance. A sample of best semi-factuals from each dataset was then used to analyze the seesaw pattern. Results. The results in Table 1 show that out of 10,300 best semi-factuals sampled, 9175 (or 89%) of them revealed the seesaw pattern of changing marginal contributions between key- and hidden-features. This validates our intuition that this property exists in most good semi-factuals. Table 1: Table showing the number of best semi-factuals sampled from each dataset and how many of those had the seesaw pattern. Dataset Sample of Best Semi-factuals Best-Semi-factuals with Seesaw Pattern Adult Income 1200 1029 Blood Alcohol 400 360 PIMA Diabetes 100 92 Default Credit Card 3200 2826 German Credit 400 358 HELOC 2000 1820 Lending Club 3000 2690 Total 10,300 9,175 0.A.2 Baseline Implementation In this section, we discuss the implementation details and parameter specifications for the existing semi-factual methods. • Local Region Model The local surrogate model was trained with a minimum of 200 instances from each class. • KLEOR A k-N model with k=3 was used to compute the distances between the instances and obtain NUN. • DSER The model was implemented based on the publicly available library111https://github.com/HammerLabML/DiverseSemifactualsReject. A k-N classifier was used to fit the conformal predictor with k=5. The reject threshold θ was set to 0.4. The regularization parameter C for each component in the loss function was set to 1. The hyperparamter μ to control the number of feature-differences was set to 2. • PIECE The framework was modified to work with the tabular data inspired by Kenny & Huang’s [kenny2023sf] modification. The training data was partitioned into two subsets based on model predictions: instances predicted as the original class c and those predicted as the counterfactual class c′c . For each subset, feature distributions were modeled independently, using Beta distributions for continuous features and categorical distributions for discrete features. To construct a semi-factual predicted as c, we evaluate the probability of each query feature value under the distributions of class c′c . Features are then sequentially adjusted to their expected values under c′c , starting from the lowest-probability feature, until the next modification would cross the decision boundary. For continuous features, the probability of a value is defined as the minimum of the two integrals on either side of that value in the distribution. If an expected value lies outside the permitted actionability range, it is clipped to the nearest feasible value. • C2C-VAE To adapt to the tabular data, the encoder and decoder in both VAE and C2C-VAE were implemented using 3 and 2 fully connected layers, respectively, with the latent dimension z set to 4. The value of interpolation parameter λ was set to 0.2. • MDN In the sfs() function, the categorical features were determined to be similar by direct comparison of their values whereas the continuous features were considered to be the same if the values lie within ±20%± 20\% of standard deviation of the selected feature. • S-GEN We followed the publicly available implementation of S-GEN222https://github.com/EoinKenny/Semifactual_Recourse_Generation with default actionability constraints and hyperparameter specifications for causal and non-causal settings. • DiCE The DiCE method was implemented using the publicly available library333https://github.com/interpretml/DiCE. The random search method was used and all parameters were kept at their default settings except the desired_classdesired\_class was set to "same" in the objective function to ensure the generation of semi-factuals. 0.A.3 NSGA-I Implementation in ISF The NSGA-I algorithm used to solve the constrained multi-objective optimization in ISF was implemented via the pymoo444https://github.com/anyoptimization/pymoo framework. The algorithm was initialized using random float sampling and evolved using standard evolutionary operators. Parent selection was performed using tournament selection, while offspring were generated using simulated binary crossover (SBX) (crossover probability = 0.9, distribution index = 15) and polynomial mutation (mutation probability = 0.9, distribution index = 20). Environmental selection followed the standard NSGA-I rank-and-crowding survival strategy, which performs non-dominated sorting and maintains diversity using crowding distance. Duplicate solutions were eliminated during evolution. The algorithm was executed with a population size of 50 for 100 generations, while all other parameters were kept at their default values provided in the pymoo implementation. 0.A.4 User Studies The materials for the user studies were based on German Credit dataset. We used 5 important features and modified the dependent variable (class) to be loan accepted and rejected rather than the original good and bad credit. The semi-factual explanations in both user studies were generated using ISF method. Below we show the initial setup of both user studies before showing some sample scenarios for each study. 0.A.4.1 User Study 1: Do People Prefer Informative Explanations? Sample Scenario 1 Sample Scenario 2 0.A.4.2 User Study 2: Preferring Good or Bad Explanations? Sample Scenario 1 Sample Scenario 2