Paper deep dive

Taking the Whys Seriously: Limitations of Counterfactual Explanations in Justification and Recourse

Mattia Cerrato, Otto Sahlgren, Xenia Heilmann

Year: 2026Venue: arXiv preprintArea: cs.CYType: PreprintEmbeddings: 85

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Counterfactual explanations (CEs) are widely used in explainable artificial intelligence (AI) to show how a model's outputs would change if the input features were manipulated. This technique is used for a range of tasks such as debugging models, explaining predictions, justifying decisions, and providing algorithmic recourse. In this paper, we explore the normative legitimacy of employing counterfactuals in real-life model deployment settings. We discuss the different stakes involved in these different purposes for which CEs are commonly employed, and find stricter requirements for justification and recourse. In particular, we find that naive application of CEs for justification and recourse can lead to ignoring contestable choices made throughout the machine learning (ML) pipeline, thus obfuscating that decisions and counterfactuals for those decisions are also artifacts of an organization's materialized design and governance choices. We demonstrate this with four empirical experiments involving interventions at stages of the ML pipeline ``upstream" of the explanation itself, and show that these affect the generated counterfactuals. We find that an organization's choices on measurement models for feature and labels, business requirements, model validation, and the metric of model success have as much or more impact on the generated counterfactuals as the specifics of the generating method. Our findings underline the need to account for such choices upon providing justification and recourse, providing a stark reminder of the relational nature of these tasks. As putative justifications or recourse recommendations, CEs do not provide adequate answers to some important "why"-questions because they preclude consideration of whether the decision-maker ought to have acted differently.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

81,722 characters extracted from source content.

Expand or collapse full text

Taking the Whys Seriously: Limitations of Counterfactual Explanations in Justification and Recourse Mattia Cerrato Otto Sahlgren Xenia Heilmann Abstract Counterfactual explanations (CEs) are widely used in explainable artificial intelligence (AI) to show how a model’s outputs would change if the input features were manipulated. This technique is used for a range of tasks such as debugging models, explaining predictions, justifying decisions, and providing algorithmic recourse. In this paper, we explore the normative legitimacy of employing counterfactuals in real-life model deployment settings. We discuss the different stakes involved in these different purposes for which CEs are commonly employed, and find stricter requirements for justification and recourse. In particular, we find that naive application of CEs for justification and recourse can lead to ignoring contestable choices made throughout the machine learning (ML) pipeline, thus obfuscating that decisions and counterfactuals for those decisions are also artifacts of an organization’s materialized design and governance choices. We demonstrate this with four empirical experiments involving interventions at stages of the ML pipeline “upstream” of the explanation itself, and show that these affect the generated counterfactuals. We find that an organization’s choices on measurement models for feature and labels, business requirements, model validation, and the metric of model success have as much or more impact on the generated counterfactuals as the specifics of the generating method. Our findings underline the need to account for such choices upon providing justification and recourse, providing a stark reminder of the relational nature of these tasks. As putative justifications or recourse recommendations, CEs do not provide adequate answers to some important ”why”-questions because they preclude consideration of whether the decision-maker ought to have acted differently. 1Johannes Gutenberg University Mainz, Germany 2University of Tampere, Finland mcerrato@uni-mainz.de, otto.sahlgren@tuni.fi, xheilmann@uni-mainz.de Introduction The growing adoption of machine learning (ML) systems in high-stakes decision-making contexts has spurred research on the interrelated notions of transparency, explainability, and interpretability. Transparency, generally defined as “the availability of information about an actor that allows other actors to monitor the workings or performance of the first actor” (Meijer 2013, p.430), lacks a unique and well-defined equivalent in the context of ML systems. Instead, multiple notions of “algorithmic transparency” can be defined in relation to different objects and subcomponents as subject to transparency, where the transparency thereof can take different forms and gradations (Diakopoulos 2025, p.199). For example, “algorithmic transparency” can refer to the code, datasets, algorithms or their purposes and use-cases being disclosed in an understandable form to some audience. One of the most celebrated techniques to this end is the generation of counterfactual explanations (CEs) (Wachter et al. 2018; Kusner et al. 2017), which have been referred to as “de-facto standard in explainable artificial intelligence” (Sokol and Flach 2019). The basic concept of CEs is to provide explanation to the “why” question: why does the model return this decision in this situation? CEs propose modifications to the input data that would incur a different decision by the model (Verma et al. 2024). Thus, CEs relate to singular decisions rather than global approximation of a complex model, and may provide an explanation by contrast rather than by local approximation as done by other techniques such as local explanations (Ribeiro et al. 2016). CE is both attractive and flexible, seeing usage beyond the abstract opening of the “black box”, such as in model debugging (Abid et al. 2022), and the provision of justification and recourse for algorithmic decisions (Karimi et al. 2021; Wachter et al. 2018). Our contribution is motivated by critical research on CE and its limits in the aforementioned tasks. Though we acknowledge the potential of CEs, we are interested in gauging the extent to which they can reliably support justification, contestation, and reversal of decisions across real-world decision-making settings. Specifically, we believe that assessing CEs’ sensitivity to expectable circumstantial variation is crucial for determining whether CE can robustly realize values central to practices of public justification and recourse mechanisms, such as agency, trust, and accountability. For example, adequate recourse mechanisms will enable contestation, appeal, and reversal across a range of circumstances rather than only under narrowly specified conditions (Venkatasubramanian and Alfano 2020). It is therefore crucial to assess whether CEs can deliver stable and meaningful recourse under variation characteristic to real-world decision-making settings. Our paper provides both empirical and theoretical contributions. While prior research has explored the robustness of CEs under minor changes in the data, model, and generation method (Virgolin and Fracaros 2022; Slack et al. 2021; Brughmans et al. 2024a), we measure the sensitivity of CEs under systematic “upstream interventions” affecting the pipeline that starts from data collection, goes through model selection and fitting, and finally ends in the generation of CEs. Specifically, we start from explicitly modeling certain choices usually made on the “business side” of the ML pipeline as interventions on a structural causal model (SCM) representing the pipeline itself, such as the observation model of features, the labeling mechanism and the metric employed to operationalize the fitness of a model. For simplicity, we refer to these choices collectively as upstream choices which produce different specifications of the SCM and, by extension, models. We find that upstream interventions can dramatically affect the CEs generated downstream, producing differences comparable in magnitude to those observed when the explanation method itself is changed (Brughmans et al. 2024a). Drawing on these findings, we substantiate the theoretical claim that several assumptions need to hold—in both an empirical and normative sense—for CEs to represent adequate “whys” in many real-life cases. Upstream interventions represent meaningful, non-neutral, and potentially contestable choices made by model providers or decision-makers. However, they remain unaccounted for when CEs are presented as evidence to justify models or particular outputs or as recommendations for decision reversal. We argue that taking the “whys” seriously requires, besides mere explainability, broader transparency about upstream assumptions and choices, accordingly, and recognizing model providers and deployers as parties whose choices and policies require justification and potential revision. Background and Related Work The concepts explainability and interpretability, often used interchangeably, have received significant attention in the ML literature (Lipton 2018). Broadly, explainability refers to the “the ability to explain or to present [the algorithm’s prediction logic] in understandable terms to a human” (Doshi-Velez and Kim 2017, p.2), whereas interpretability refers more specifically to “the degree to which an observer can understand the cause of a decision” (Miller 2019, p.8) (see also Biran and Cotton (2017)). Technical approaches in these areas, which we describe below, pursue a range of overlapping but conceptually distinct aims. Examples include debugging, evaluating, and monitoring ML system performance; enabling decision-makers to explain and justify decisions to decision-subjects; ensuring that subjects and oversight bodies can understand, contest, trust those decisions; and providing subjects with actionable means of recourse in case of incorrect or unfavorable decisions (Diakopoulos 2025; Verma et al. 2024). These concerns are reflected in existing regulations and literature on the so-called right to explanation (Juliussen 2025). Article 22 of the General Data Protection Regulation (GDPR) states that “[t]he controller shall provide the data subject with […] meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject” (European Parliament and Council of the European Union 2016, Art. 21). In addition, article 86 of the AI act states that, with some exceptions, “[a]ny affected person subject to a decision which is taken by the deployer on the basis of the output from a high-risk AI system” as classified in the AI act (European Parliament and Council 2024, Art. 86) “shall have the right to obtain from the deployer clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken”. Technical solutions to these ends may be grouped roughly into two categories. First, global explanation methods seek to approximate a complex model in its entirety by identifying important input variables, for example, or by providing visual summaries (Lundberg and Lee 2017). Examples of global techniques include ones for computing feature importance and dependency plots. Second, local explanation methods aim to explain individual decisions, either by approximating the model’s decision surface (i.e., a boundary or the regression hyperplane) in a region close to the decision point or by providing an alternative scenario where the model’s decision would be different. Local approximation tools such as LIME fall into the former class (Ribeiro et al. 2016), while CEs, which comprise our focus, fall under the latter. A further alternative worth mentioning is to employ statistical models which are transparent “off-the-shelf” whenever possible, as proposed by Rudin (2019). Nonetheless, so-called “post-hoc” explainability has remained a central topic in AI research for over a decade (Guidotti et al. 2018). Counterfactual Explanations (CE) This paper focuses on counterfactual explanation (CE), a class of local explanations that describe how a prediction model’s prediction (e.g., a classification or a ranking) would change given a change in the input (Verma et al. 2024). In plain terms, CE answers “why” questions by computing “what ifs”, namely, by identifying and specifying minimal changes in input that would flip the output. How much job experience should a rejected job applicant have had in order to get hired? Would a change in an applicant’s salary have made a difference to their credit score? CEs are versatile and relatively easy to implement as they do not require opening “black box” models (Wachter et al. 2018). They have become the “de facto standard in explainable artificial intelligence” (Sokol and Flach 2019, p.2), accordingly. Subsequent sections discuss its numerous use-applications ranging from model debugging (Abid et al. 2022) and model evaluation and justification (e.g., Goethals et al. (2023); Wang et al. (2024); see also Kusner et al. (2017)) to the provision of recourse (Wachter et al. 2018; Verma et al. 2024). Formally, a counterfactual explanation x′x is an alternative input instance x,x′∈x,x which induces a different decision from a decision model F. Usually, the counterfactual instance x′x should be taken to be as similar as possible to x; sometimes, it is constrained to be an actual instance from the collected dataset =xi,yinD=\x_i,y_i\^n. Finding counterfactuals can then be operationalized as an optimization problem, of which various variants exist. For expository purposes, we consider the following problem: arg⁡minX′ℒ(F⁡(X),F⁡(X′))+λ⋅d⁡(X,X′) min_X L(F(X),F(X ))+λ· d(X,X ) (1) where ℒL is a loss (or cost) function modelling the similarity between F⁡(x)F(x) and F⁡(x′)F(x ), which should be minimized under the constraint that F⁡(x)≠F⁡(x′)F(x)≠ F(x ); and d⁡(x,x′)d(x,x ) is a dissimilarity function computed directly on the instances (for instance, a norm defined in mX^m where m is the number of features available for each data point). Finally, λ is a parameter available to the method user to balance out the two objectives. Methodologies for obtaining CEs differ in their underlying optimization techniques and their concern. Advancement in CE research and evaluation of methodologies may be performed over different dimensions and under various concerns. We summarize here four key concerns central to our present discussion. We refer readers interested in an exhaustive list of evaluation criteria to the surveys by Verma et al. (2024) and Bayrak and Bach (2024): • Validity: a counterfactual explanation is considered valid if the counterfactual x′x is predicted by the model f in a way considered desirable for counterfactual reasoning. As an example, if a user is wondering why their resumé was not selected after screening – that is, say, y=f⁡(x)=0y=f(x)=0, in the counterfactual explanation, y′=f⁡(x′)y =f(x ) should be equal to 1. That is, when considering f as a classifier, counterfactual explanations need to cross the decision boundary. • Minimality: a minimal counterfactual explanation x′x is as close as possible to the explanandum x. This property is sometimes referred to as sparsity, which implies the minimization of the L1-norm |x′−x|1 |x -x |_1 and mandates additionally that the least number of entries are changed. An appropriate norm choice is usually dependent on the feature types available. • Actionability: a counterfactual explanation is considered actionable if it does not include immutable, sensitive or otherwise hard-to-modify characteristics of the input data. Operationally, this may be implemented by a set of no-go features ||≤m |A |≤ m which should not be included in the counterfactual explanation. The choice of A may also be related to fairness concerns, broadly intended. As an example, under the constraint of actionability, an explanation in the resume’ screening classification should not include their gender or age. Under a cost-sensitive relaxation of actionability, it may be considered more actionable to suggest gaining some professional certification relevant to the job position rather than enrolling in a multi-year education program. • Robustness: a counterfactual explanation is considered robust if it remains valid under small perturbations to the model or its underlying data. Fokkema et al. (2022) show that naive counterfactual methods applied to ensemble models can have validity below 50% after model retraining, and that requiring higher prediction confidence for the counterfactual class mitigates this issue at the cost of increased distance from the original instance. Algorithmic Recourse (AR) This paper discusses CE specifically in connection to algorithmic recourse (AR). Commonplace definitions of AR include “an actionable set of changes a person can undertake in order to improve their outcome” (Joshi et al. 2019, p.1), “the ability of a person to obtain a desired outcome from a fixed model” (Ustun et al. 2019, p.10), and “the systematic process of reversing unfavorable decisions by algorithms and bureaucracies across a range of counterfactual scenarios” (Venkatasubramanian and Alfano 2020, p.284). Existing literature covers roughly two types of scenarios in which AR is typically sought. First, a decision subject may seek recourse for decisions that are correct yet unfavorable (e.g., a job applicant is rejected but wants to reapply in the future). Here, recourse might entail a specification of “actions that the agent would need to undertake to receive a more favorable decision from the algorithm” in the future (Venkatasubramanian and Alfano 2020, p.288). Second, a decision subject may seek recourse for decisions that are unfavorable but also incorrect (e.g., the job applicant is rejected on the basis of erroneous or incomplete data). Here, recourse could entail reversing the decision and, depending on the case, possibly retraining the algorithm or changing or supplementing its input data (Cerrato et al. 2023). Recourse is generally understood to facilitate values like personal autonomy and social trust: when people (know that they) can reverse unfavorable decisions if needed, they can exercise their agency and plan for the future (Venkatasubramanian and Alfano 2020). Our paper adopts the notion of AR as a modally robust good advanced by Venkatasubramanian and Alfano (2020) who appeal to Pettit’s view according to which a good is modally robust if it remains valuable across possible worlds (Pettit 2015). Recourse is valuable in a wide range range of possible situations and counterfactual circumstances, but arguably especially so in nonideal circumstances where decisions can be erroneous or normatively unfounded (e.g., unfair or arbitrary). Intuitively, the existence of effective mechanisms for appealing, contesting, and reversing decisions is particularly crucial for decision subjects where decision-makers may err or enact morally problematic decision policies. CE is widely regarded as a promising means of providing recourse to decision subjects (Wachter et al. 2018; Verma et al. 2024). As explanations, CEs help decision subjects understand the reasons behind the decisions they receive, enabling them to also appeal and contest those decisions where needed. They can also highlight how the subject could receive a favorable decision in the future. Here, an actionable and minimal CE can be thought of as a recommendation that highlights a feasible set of actions the subject should undertake to receive a different (favorable) decision from the model at a later time. Relatedly, CE is seen as a promising approach to implementing the contentious right to explanation (e.g., Wachter et al. (2018)): ideally, they will express in an understandable format what features about the individual should have changed for the decision to change: “had you done A instead of B, you would have received a favorable outcome”. The idea that CEs should facilitate recourse reliably in different circumstances appears to also motivate recent research that has developed methods to ensure that CEs remain robust to changes in the data (Dominguez-Olmedo et al. 2022) and the model (Upadhyay et al. 2021). Ensuring the technical robustness of CEs remains crucial in light of our findings. However, our results also demonstrate the need to attend to broader circumstantial variation—particularly, variation in upstream design choices and ML pipelines—to ensure that recourse mechanisms reliably facilitate their guiding values. Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations Our experiments are motivated by critical research examining the assumptions, challenges, and limitations of existing approaches to CE and AR. One key challenge for CE concerns the so-called “Rashomon effect” (Breiman 2001): typically multiple non-identical counterfactuals can be computed even for a fixed model and output, and these may differ with respect to their validity and minimality. Slack et al. (2021) find that popular CE methods are also vulnerable to manipulation, lacking robustness against minor perturbations in the model. Brughmans et al. (2024a) test 12 different methods for CE and find high levels of disagreement between their respective explanations, suggesting that a particular counterfactual does not capture all relevant information and instead represents a partial and “unique point of view on the involved prediction logic” (p. 455). The idea that CE provides a restricted “view from somewhere” aligns with critical studies emphasizing that CEs are colored by ontological, epistemic, semantic, normative and empirical assumptions. For example, Barocas et al. (2020) and Kasirzadeh and Smart (2021) observe how CEs can be premised on contestable and empirically questionable assumptions pertaining to the relationship between model features and real-world actions, the commensurability and manipulability of features highlighted in counterfactuals, and the similarity between actual and counterfactual scenarios. Our empirical findings reinforce these critiques, demonstrating that CEs are sensitive to upstream assumptions and choices conditioning the generated counterfactuals. Previously identified limitations of existing AR approaches represent form a second motivation for our study. Existing work highlights that standard AR frameworks adopt an individualistic and static perspective to recourse (Venkatasubramanian and Alfano 2020; Upadhyay et al. 2021; De Toni et al. 2025). On the one hand, prevailing approaches tend to place responsibility for contesting and reversing decisions on the decision subject. This idea that “recourse for a person is necessarily recourse by them” overlooks that recourse for the affected individual can require involving caretakers or legal representatives (Venkatasubramanian and Alfano 2020, p.288). On the other hand, standard AR approaches tend to neglect temporal dynamics and instabilities, missing how updates to datasets and models, and changes in the underlying causal structure, can undermine the actionability and validity of recourse recommendations (Upadhyay et al. 2021; De Toni et al. 2025). Actions that suffice for obtaining favorable outcomes at time t may fail to achieve the same effect at t+1, indicating that current approaches to AR fail to reliably facilitate decision reversal across changing circumstances. The challenges and limitations discussed above intersect most clearly where CEs are used as a basis for recourse. For example, Slack et al. (2021, p.10) note that the manipulability of CEs via minor data perturbations “bring[s] into the question the trustworthiness of counterfactual explanations as a tool to recommend recourse to algorithm stakeholders”. Similar concerns arise from changes due to distribution shifts and model updates as non-robust CEs may provide subjects seeking recourse with false hope of reversing their decision. These issues also problematize approaches to AR which view the decision subject as the locus of responsibility and the decision-maker as a “benevolent paternalist” recommending what the subject should do. Indeed, while research has developed promising methods to generate robust counterfactuals (Slack et al. 2021; De Toni et al. 2025; Upadhyay et al. 2021), recourse recommendations that are invariant to changes in data and models do not avoid the problems stemming from this framing. Especially given the contentious nature of assumptions preceding the generation of CEs, their potential misalignment with real-world facts, and potential disagreement between multiple CEs for the same output, it appears further critical scrutiny of “upstream” factors and choices is warranted. To this end, we analyze how upstream interventions at various stages of the ML pipeline affect CEs. Example: Resume Screening We begin by introducing a running application example used throughout the remainder of this paper. We investigate candidate and resumé screening as a step of the hiring process which companies have an interest in automating (Sinha et al. 2020; Lo et al. 2025) and which has been previously investigated critically in the literature due to its reliance on unvalidated psychometric measurements (Rhea et al. 2022). Commercial applications for screening we have reviewed appear to focus on computation of “fitness scores” of candidates to specific job positions, with a heavy emphasis on NLP and LLMs to generate both scores and ostensive explanations for these (PrescreenAI 2025). Some companies even propose automated interviews according to scripts that can be given by domain experts and HR professionals (Alex.com 2025). In recent academic work, Lo et al. (2025) propose the extraction and computation of five separate scores via LLM-based information extraction from resumes, dealing with i) self-evaluation of the candidate i) job-relevant skills i) previous work experience, iv) basic information and v) educational background, which can then be combined and employed to sort candidates. We design a hypothetical task of resume’ screening with the objective of adhering as much as possible to the characteristics of the tools and methodologies currently investigated and developed in this application context. However, we step back from LLM and RAG-based methodologies (Lo et al. 2025) as far as they are employed for candidate scoring directly. Our main interest is the production of CEs for screening decisions, and the production of counterfactuals in LLMs is not particularly mature and appears to suffer from issues of counterfactual validity (Youssef et al. 2024) in the sense discussed earlier in this paper. Instead, we operationalize the problem of resumé screening as one of supervised score-based ranking (Zehlike et al. 2022). We assume that a dataset =(xi,yi)i=1n⊂m×D=\(x_i,y_i)\_i=1^n ^m×Y is given. Here, mX^m represents a series of m characteristics of individuals extracted from CVs such as previous job experience and education, whereas Y is the label space. We assume that an underlying, continuous fit construct y∗∈[0,1]y^*∈[0,1] exists, representing the probability score that a recruiter would, in principle, assess a candidate as the best one, and that the recorded labels yiy_i are produced from it by an explicit business rule, so that the practical screening outcome is binary. Then, we define F:→F:X as a decision model trained on dataset D so to predict the potential fit of new unseen applicants which could be interviewed or not depending on the prediction of F. We then assume that individuals that are screened do not have access to F but rather to a counterfactual explanation x^=⁡(F,x) x=C(F,x). Compared to usual formalizations of ranking, we do not explicitly represent a query q as a feature vector and assume instead that the firm responsible to produce the model F has a way to produce datasets qD^q relevant to the specific query at hand. To clarify, imagine that the company implementing the resume’ screening service offers it to screen candidates for various positions, responsibilities, and competences. The modeling approach usually adopted in the Information Retrieval literature here would be to represent a specific position to be filled as a query q and assume that the model F may take a vector representation of q as an input. Here, we instead assume that depending on the position of interest, a relevant supervised dataset D is constructed which is then employed to fit the model F. Thus, the good being distributed here by model F is admittance to further interviewing, whether AI-based (Alex.com 2025) or not, which we deem here a binary outcome and a potential target for a “why” question from candidates that have not been selected. Thus, F may also be analyzed as a binary classifier wherever relevant or useful. We give the causal graph representing the assumed data generating process in Figure 1. The features X and labels Y in D depend on measurement models ϕX _X and ϕY _Y which are non-observable by the candidates and only implicitly defined by the model-producing firm. Figure 1: The structural causal model representing the data generating process. We assume that the world W is not observable, but that features X and labels Y may be obtained via measurement models. The trained model F similarly depends on certain design choices and produces labels Y Y. The counterfactual examples X X will ultimately depend on the interventions made by the developers regarding the SCM pictured here. We report structural equations and other variables in Equation 2. Before introducing the structural equations we fix notation. We use uppercase X,YX,Y for the random variables corresponding to the SCM nodes, and lowercase xi,yix_i,y_i for their realizations in the dataset; their value spaces are mX^m and Y respectively, so xi∈mx_i ^m and yi∈y_i . The dataset =(xi,yi)i=1nD=\(x_i,y_i)\_i=1^n is then an i.i.d. sample from the joint distribution that the SCM induces on (X,Y)(X,Y). The recorded label is the post-business-rule label Y, not the latent fit construct Y∗Y^*: only the projection Y=fY(Y∗,ϕY)Y=f_Y(Y^*, _Y) is ever observable, which is precisely why ϕY _Y is one of the design choices we will intervene on. The same logic applies to X: the recorded feature vector is the output of the measurement model ϕX _X applied to the unobservable world W, and ϕX _X is similarly an explicit design choice rather than a fact of the world. How exactly these measurement models are operationalized has a downstream causal effect on the trained model F, which we make explicit in the structural equations of the SCM in Figure 1: W W :=NW :=N_W (2) X X :=fX(W,ϕX,NX) :=f_X(W, _X,N_X) (3) Y∗ Y^* :=fY∗(W,NY∗) :=f_Y^*(W,N_Y^*) (4) Y Y :=fY(Y∗,ϕY) :=f_Y(Y^*, _Y) (5) F F :=fF(X,Y,M,C,K,NF) :=f_F(X,Y,M,C,K,N_F) (6) Y Y :=F⁡(X) :=F(X) (7) X X :=fX^(X,Y^,F,τ,NX^) :=f_ X(X, Y,F,τ,N_ X) (8) We use F throughout for the trained model itself, and reserve fFf_F for the training procedure that maps (X,Y,M,C,K,NF)(X,Y,M,C,K,N_F) to F; similarly fX,fX^,fY,fY∗f_X,f_ X,f_Y,f_Y^* denote the structural functions producing the corresponding nodes. To describe the process, we assume that the features X and raw labels Y∗Y^* are measured noisily from the state of the world W. This may be understood as the process of obtaining certain measurements of skill, experience and fit for the position; we will be more concrete about the data setup in the Experiments section. We note that here we assume that the raw labels may be post-processed into Y due to business requirements such as salary caps. The model training stage is represented by fFf_F and we assume it to be causally influenced by a metric of success M, a model selection strategy C and K the model family or families being considered. The trained model F then induces ranking labels Y Y which are used to generate counterfactuals X X via the generation function fX^f_ X. While counterfactual generation strategies have a plethora of hyperparameters, we focus here on τ, the probability threshold that should be crossed by F for a counterfactual x x to be considered. In our experiments, we consider both most-minimal and most-valid counterfactuals by varying the threshold τ. Experiments Data Sources. The screening application described in the prior section motivates our choice of variables, but, as previously discussed, our experiments are not based on textual resumes. We use the American Community Survey (ACS) microdata sample obtained via the folktables library, treating each respondent as a candidate. The ACS gives us actual population-level data for X, that is, the demographic, occupational and educational features that a screening firm would, on real resumes, have to reconstruct via parsing or LLM-based extraction. The latent fit construct Y∗Y^* is however not so easily accessed: no public dataset that we are aware of connects features of this kind to recruiter judgments, and we have no easy way to bridge that gap. We therefore approximate Y∗Y^* by an observed market signal: earnings, on the reading that wages reflect, imperfectly, the labour market’s valuation of the same features that a screening model would weigh. In terms of the features X, we take a total of seven columns, with two continuous features and five discrete ones. The features contain diverse information, ranging from the type of college degree obtained (if any) to the number of working hours per week. We report the complete variable definitions in Table 1 in the Appendix, due to space constraints. Experimental Design. Our experimental setup is centered around the comparison of CE sets generated as a result of an intervention on the causal graph of Figure 1 which we take to represent a ML pipeline. Thus, we are interested in observing whether upstream interventions on the SCM, for instance intervening on the feature definitions and labeling mechanisms, create significant variation on the generated CE sets. To understand whether a variation is significant, we take a comparison with a downstream intervention on the same graph, that is, a change of the validity threshold τ. At a basic level, this value can be set at τ=0.51τ=0.51 to ensure that the trained model F is confident “just enough” that a certain change in the feature values of an individual, resulting in the explanation or recommendation X X, will change its output Y Y. We then vary τ to higher values, and treat this downstream intervention as our baseline: if upstream design decisions produce comparable or larger effects than directly tuning the explanation method, this suggests that practitioners should attend to these decisions more carefully, and that such decisions should also be considered as possible answers to the “Why was this user not selected for screening?” question. Specifically, we analyze the effect of the following interventions: • Intervention 1: Label definition (do(ϕY=s)do( _Y=s) for s∈40k,50k,60k,70k,80ks∈\40k,50k,60k,70k,80k\). We vary the salary cap parameter that determines which individuals are labeled as positive. Specifically, we define Y=1Y=1 if an individual is both in the top 10% of earners and earns below the salary cap. This simulates a business constraint where the organization cannot afford candidates above a certain salary threshold. • Intervention 2: Feature measurement (do(ϕX=p)do( _X=p) for p∈for-profit,all-private,private-self,non-govp∈\for-profit,all-private,private-self,non-gov\). We vary the definition of “private sector worker” by including different combinations of employment classes: (i) private for-profit only, (i) all private employers (for-profit and non-profit), (i) private plus self-employed, and (iv) all non-government. This reflects ambiguity in how categorical concepts are operationalized from raw survey data. • Intervention 3: Success metric (do(M=m)do(M=m) for m∈accuracy,ROC-AUC,NDCG@100,precision,recall,F1m∈\accuracy,ROC-AUC,NDCG@100,precision,recall,F1\). We vary the metric used for model selection during cross-validation. This reflects choices made by ML practitioners when optimizing model performance. • Intervention 4: Validation strategy (do(C=c)do(C=c) for c∈holdout,5-fold,10-foldc∈\holdout,5-fold,10-fold\). We vary the selection strategy for validation of the model F from simple holdout (a 15% test split followed by a 20% validation split of the remainder, i.e. approximately 68/17/15 train/validation/test) to 5-fold and 10-fold cross-validation. Both this intervention and the preceding one tune the hyperparameters of a random forest classifier over a small grid that we report in the Appendix. • Intervention 5: Stopping threshold (do(τ=t)do(τ=t) for t∈0.51,0.55,0.60,0.65,0.70t∈\0.51,0.55,0.60,0.65,0.70\). We vary the confidence threshold used by the counterfactual generation algorithm11 1 This is available as a parameter in DICE-kdtree, but not in NICE. For NICE, we simply reject CEs that are below the confidence.. This serves as our baseline for comparison with the upstream interventions. We operationalize the “impact of an intervention” by measuring the sensitivity of counterfactuals across the separate causal graphs resulting from the intervention at hand: Definition (Explanation Sensitivity). Let S be a structural causal model with design parameters ξ=(ϕX,ϕY,M,C,K,τ)ξ=( _X, _Y,M,C,K,τ). For a query instance x with prediction y^=F⁡(x) y=F(x), let X^ξ=⁡(F,x,τ) X_ξ=C(F,x,τ) denote the set of counterfactual explanations generated under parameters ξ. The explanation sensitivity with respect to an intervention do(ξi=ξi′)do( _i= _i ) on a single parameter ξi _i, holding all other parameters fixed, is: S⁡(ξi,ξi′)=x[d⁡(X^ξ,X^ξ′)]S( _i, _i )=E_x [d( X_ξ, X_ξ ) ] (9) where ξ′ξ differs from ξ only in component i, d is a distance metric between CE sets on the feature space, and the expectation is taken over query instances x for which the model gave a negative prediction y^=0 y=0. In our experiments, we compute explanation sensitivity over sets of CEs generated by DiCE-kdtree (Mothilal et al. 2020) and NICE (Brughmans et al. 2024a). Both methods are retrieval-based: a counterfactual is selected from the training set rather than constructed by gradient-based optimization of a counterfactual loss. We exclude optimization-based methods deliberately, as we observed in pilot experiments that their seed-to-seed variability swamped the effect of both upstream and downstream interventions. We define the set distance d⁡(X^ξ,X^ξ′)d( X_ξ, X_ξ ) as a symmetric mean-of-nearest-neighbors distance, on which we further perform some rescaling to account for the different ranges of the features we employ. Continuous features are rescaled by their training-set median absolute deviation and categorical features contribute a 0/1 mismatch indicator. Full definitions are given in the Appendix. Lastly, we note that the “default setting” of the SCM is ξ0=(ϕXpriv,ϕY60k,Macc,C5,KRF,τ0.51) _0=( _X^priv, _Y^60k,M_acc,C_5,K_RF, _0.51): “private sector” restricted to private for-profit employment, a salary cap of $60\$60k, accuracy as the model selection metric, 55-fold cross-validated grid search, a random-forest hypothesis class, τ=0.51τ=0.51. Research Questions. Our research questions are the following: • RQ1 (Sensitivity). Are counterfactual explanations more sensitive to upstream interventions on the SCM than to the downstream τ intervention? • RQ2 (Consistency). Do upstream interventions produce more disagreement between explanations than τ does? RQ1: Mechanism Sensitivity to the Four Interventions We report the five sensitivity interventions together in Figure 2. Each panel shows, for one intervention, the mean symmetric set-distance between the strict counterfactual sets generated under the baseline level and each alternative level of that intervention, separately for DiCE-kdtree (circles) and NICE (triangles), with 95% Student-t CIs over ten seeds. The shaded band on each panel is the sensitivity of the generated CEs when intervening on τ, where we take the maximum sensitivity we observed over τ interventions. A method whose marker sits outside this band has been moved more by the upstream intervention than it would have been by changing the validity threshold; a marker inside the band has been moved by an amount the threshold could already account for. Figure 2: Mechanism sensitivity of counterfactual explanations to each of the five interventions on the SCM of Figure 1. We show the effefct of intervening on the CE generating method (τ) as a baseline (colored shaded band). Markers outside the band indicate that the upstream intervention perturbs the explanation more than the explanation procedure’s own validity threshold. Three of the four interventions move both methods clearly outside the τ band. In the salary-cap experiment (Figure 2, top left) sensitivity grows monotonically with the distance from the $40k baseline, with caps at $70k and $80k producing distances well above the band for both methods. We take this to indicate that business requirements such as salary cap are also viable answers to a “why” question. The same pattern holds for the operationalization of private-sector worker (Figure 2, top right): all three alternatives to the for-profit-only definition DICE-kdtree outside the band, while NICE displays comparable or higher sensitivity across three settings, but within-band. The success-metric experiment (Figure 2, bottom-left) is the most extreme of the four: switching from accuracy-based model selection to recall, ROC-AUC, NDCG@100, or F1 produces sensitivities that exceed the τ. The only metric whose explanations remain inside the band is precision, which on this pipeline selects classifiers that overlap heavily with the accuracy-optimal model. The cross-validation experiment (Figure 2, bottom right) is the only case in which the upstream intervention does not dominate the τ hyperparameter: moving from a held-out train/val/test split to 5- or 10-fold cross-validation produces sensitivities that overlap with the τ band for both methods. This upstream intervention has therefore a comparable effect on the generated CEs as the downstream intervention on the generating method itself. Taken together, Figure 2 answers RQ1. The choice of label definition (ϕY _Y), feature measurement (ϕX _X), and model-selection criterion (M) typically perturbs the explanation more than the explanation procedure’s own validity threshold, while the choice of cross-validation strategy (C) is roughly comparable. This phenomenon holds across both retrieval-based CE methods. RQ2: Consistency of Explanations The sensitivity analysis quantifies how far the CE set moves on average; a complementary question, addressing RQ2, is whether upstream interventions also produce more counterfactual disagreement than τ does. We follow (Brughmans et al. 2024b) on the set of features each CE changes relative to the query and report three per-query agreement scores: their Jaccard agreement (do the same features change?), plus two extensions: Sign Agreement on shared continuous features (same direction of change?) and Value Agreement on shared categorical features (same new value?). We report formal definitions and aggregation across multi-CE sets in the Appendix. For each experiment, seed, and query instance x with y^=0 y=0, we compute the three scores between the CE sets X^ξ(x) X_ξ(x) and X^ξ′(x) X_ξ (x) for every pair of intervention levels (ξ,ξ′)(ξ,ξ ), holding all other parameters at the default setting ξ0 _0. We then average per-query scores and report them in Figure 3 as means with 95% Student-t confidence intervals over the ten seeds. Figure 3: Within-method agreement under upstream interventions (top two rows of each panel) vs. the downstream τ baseline (bottom two rows). DiCE-kdtree in blue, NICE in orange. Markers are means with 95% Student-t CIs over per-query scores; metrics are Jaccard (∘ ), Sign Agreement on continuous features (△ ), Value Agreement on categorical features (□ ). Higher values indicate more agreement; an upstream marker to the left of its τ counterpart means the upstream intervention produced more disagreement than τ does. For three of four experiments, we observe DiCE-kdtree’s mean Value Agreement drops from 0.390.39 across different values of τ to between 0.180.18 and 0.260.26 under upstream interventions; NICE shows the same direction (from 1.001.00 to between 0.610.61 and 0.870.87). The Jaccard agreement is also slightly lower for upstream interventions, while Sign Agreement is roughly equal both in downstream and upstream interventions. The CV-folds panel (model selection, top right) is the exception, with upstream and τ at comparable agreement, which is consistent with the sensitivity result presented in the previous section. Our conclusion with regard to RQ2, is that upstream interventions may also impact the consistency of the explanation at the same level or slightly higher than intervening on τ. Discussion Summarizing our results, we find that upstream interventions have a significant effect on the resulting counterfactuals. Our operationalization of the sensitivity of CEs to interventions reveals that this effect is larger when compared to changing the validity threshold τ of the method, something that we observe in two separate retrieval-based CE generators (RQ1). Compared to recent work from Brughmans et al. (2024b), we show that disagreement in counterfactual explanations may also stem from upstream interventions (RQ2). We note here that we do not report this result intending to demonstrate a limitation of CEs per se: compared to previous work (such as e.g. Slack et al. (2021)) we do not necessarily criticize the (lack of) robustness and manipulability of CEs to the upstream interventions we designed. In fact, there exist methods to ameliorate the robustness of CEs to certain variations in the data and model (see e.g. Virgolin and Fracaros (2022); Jiang et al. (2024)), and we have no reason to believe that they could not be extended to account for some of all the upstream interventions we analyze in this paper. Rather, our purpose with the quantitative evidence presented in the previous section is to substantiate our theoretical argument that several assumptions need to hold for CEs to represent an adequate “why”, especially in applications such as justification and recourse. The central assumption that we have operationalized with our experiments is the fixed model assumption, that is, that there is a specific model that best fits the data. While this may be the case, this is conditional on several actions from the decision-maker which are then not provided as the underlying reason for a certain decision having being reached. However, we have shown that certain decision-maker choices such as the model success metric, the labeling mechanism and the measurement model for certain features have significant impact on the generated CEs. We now turn to the implications of our findings for real-life applications of CE, where they direct attention to the reasonableness of the fixed model assumption. Is the model stable over time? Is it legitimate in the sense that it is based on justifiable assumptions and design choices? The fixed model assumption can be assumed to be reasonable in some cases, where CE is used to explain outputs or debug models. Our findings are less directly relevant for such cases, accordingly, as the examined upstream interventions alter the model and its outputs. However, in many real-world settings data-generating processes evolve and models are updated, meaning pipeline stability cannot be assumed. Our findings have implications for such cases. Debugging signals produced with CE may fluctuate under upstream perturbations, for example, highlighting the need to distinguish issues arising from stable properties of the model from those stemming from data processing, labeling, and other contingent upstream factors. The observed sensitivity of CEs also underscores that CEs depend on upstream assumptions and choices that often remain open to normative and empirical contestation (Barocas et al. 2020; Kasirzadeh and Smart 2021). Intuitively, why a subject has received a particular decision depends not only on whether the subject meets certain criteria, but also on the deciding party’s choices that shape the criteria—for example, feature measurement, labeling, success metrics, and more. Our results thus reinforce the idea that CEs are contestable artifacts of ML pipeline configurations which may themselves require critical scrutiny, an idea we examine next in connection to the provision of justification and recourse. In brief, we argue that CEs cannot reliably facilitate the guiding goals and values of these practices unless the legitimacy of upstream assumptions and design choices has been established. Implications for Justification The dependence of CEs on contestable upstream design and measurement choices has significant implications for their use in justifying models and outputs in decision-making and auditing contexts. In practice, where models—and, by extension, assumptions and design choices shaping them—remain (or should remain) open to evaluation and revision, this dependence CEs marks a fundamental limit on their evidential and justificatory role. Because upstream choices also affect what counterfactuals are generated, CEs do not necessarily provide sufficient evidence for normative justification. Treating them as such inflates their evidential and justificatory role, and conflates conditional model behavior with adequate normative grounds for decisions. Meanwhile, it obfuscates agents’ responsibilities by precluding consideration of the possibility that the model provider or deployer should justify and potentially revise their underlying assumptions and upstream choices. Connecting to concerns of the manipulability of CEs (Slack et al. 2021), we note a related risk: CEs may be misinterpreted as model-independent evidence or used selectively to shirk responsibility and public accountability. To illustrate, consider a stylized example in which our hypothetical recruiter is suspected of indirect age discrimination, and the burden of proof has shifted onto them. To rebut the suspicion, the recruiter conducts a counterfactual fairness analysis by perturbing applicants’ age to measure differences in outcomes. Suppose the analysis shows quantitatively non-negligible age-based disparities. However, the recruiter argues that they reflect age-correlated differences in qualifications and skills, and that they are also proportional to those differences. This exemplifies the risk that CEs may be misinterpreted as evidence of causality (Karimi et al. 2021). But the recruiter’s CEs presuppose a causal structure over the data as opposed to providing evidence for a specific causal explanation of the disparities (e.g., that age causes differences in the screened features). The generated CEs cannot establish that the disparities are justified, accordingly, as supplementary evidence is required. The CEs may provide evidence of the disparities’ proportionality to base-rate differences, supporting the recruiter’s claim. Nonetheless, this remains insufficient for justification in the case at hand. The recruiter must also demonstrate that they have legitimate aim, and that the preferred model is necessary and appropriate for achieving that aim. Importantly, however, this cannot be demonstrated under the assumption of a fixed model. Rather, comparison to reasonable alternatives is required, including but not limited to alternative models produced under different specifications of the SCM. This point is echoed in recent works stating that decision-makers have a legal duty to search for and implement alternative models which exhibit comparable performance with less disparity (Black et al. 2024; Laufer et al. 2025). If CEs are to facilitate the required comparisons, they must notably be produced for multiple models trained on the same data. Our results show that, besides model selection, choices of labels and metrics can also affect CEs. Thus, they further underline the need to scrutinize upstream assumptions and choices, and to assess the comparability, commensurability, and reliability of the compared CEs (Barocas et al. 2020; Kasirzadeh and Smart 2021). Implications for Recourse Our findings have significant implications for using CEs as a basis for recourse, lending support to critiques described in prior sections. On the one hand, they raise concerns about the (long-term) robustness and actionability of CEs qua recourse recommendations beyond concerns of data and model drift (Upadhyay et al. 2021; De Toni et al. 2025). How decisions could be reversed depends not only on subjects’ features and the chosen decision threshold, but also on upstream elements of the ML pipeline. Accordingly, CEs produced under different assumptions and design choices are also likely to differ in properties such as minimality and actionability. Given the substantial changes observed under alternative feature measurements and model selection metrics, for instance, one can expect notable differences in the associated costs of achieving a favorable outcome. Where the stability of the ML pipeline is not guaranteed, technical robustness under a fixed model fails to ensure that CEs facilitate the values that recourse should facilitate in a modally robust manner. Rather, if CEs are to reliably recommend feasible pathways to favorable outcomes under circumstantial variation, their robustness to upstream perturbations must be ensured. Our results call into questions whether recourse recommendations could also remain normatively underdetermined by standard evaluation criteria. If CEs for models produced under distinct but equally defensible upstream choices and assumptions describe incompatible recommendations for recourse (e.g., changing different features), then there may be no uniquely identifiable action the decision subject should have taken to achieve a favorable outcome. This possibility underscores our previous point about the order of justification: upstream choices must be relatively settled, normatively speaking, prior to evaluating CEs for recourse. This also gives reason to expand the normative basis of evaluating recourse recommendations with additional desiderata such as fairness or diversity (Boxer and Neill 2025; Laugel et al. 2023). On the other hand, our findings also challenge the prevailing individualistic and unilateral framing of AR (Venkatasubramanian and Alfano 2020). Because CEs bracket upstream assumptions, choices, and their consequences, they represent partial and incomplete answers to why a subject received the decision but also to how that decision could be reversed. This abstraction from the actions and responsibilities of model developers and/or deployers implicitly forecloses consideration of the possibility that they ought to have acted otherwise, and that the normatively preferable path to favorable outcomes may be to revise the model or decision policy itself. In the language of modal robustness, CE as a recourse mechanism does not reliably facilitate trust or subjects’ autonomy across counterfactual cases where better design and measurement choices could have been made ( either in the empirical or normative sense). Thus, where the model’s stability or normative justification remains unsettled, CEs should not be taken at face value. Treating them as individualized advice for reversing decisions obscures their status as artifacts of organizations’ design and governance choices, and risks encouraging affected subjects to comply with potentially problematic policies. Taken together, these points highlight the need to move beyond the simplistic conception of AR as a unilateral process in which decision-makers benevolently recommend actions to affected decision-subjects. They serve as an important reminder that AR is an inherently relational practice involving both decision subjects and decision-makers as parties whose behavior may require revision. In practice, evaluating whether recourse mechanisms reliably facilitates goods like personal autonomy and peace of mind requires considering all parties to the institutional relation along with their respective possibilities and obligations for action. While design choices impact the CEs decision-subjects receive, it may be impossible for subjects to reverse their decisions through contesting such choices absent appropriate transparency. Beyond explainability, effective and robust recourse thus requires broader measures of “algorithmic transparency”—including but not limited to data-sharing structures and documentation trails (Gebru et al. 2021)—that enable critical scrutiny and informed appeal and contestation. The relational perspective also avoids insulating the deciding party’s design choices from critical scrutiny without undermining individual responsibility. While it recognizes that decision-subjects can exercise agency to alter their situation, it also does not unfairly place onto the subjects the burden of simply adapting to institutional discretion. Conclusions In this paper, we provided an empirical and theoretical analysis of the limits of CEs. Complementing existing work on their robustness, we measured the sensitivity of CEs to systematic “upstream interventions” across the ML pipeline and showed that CEs generated with the same method vary substantially in response to design choices made prior to the point of explanation. Our experiments illustrate that CEs track many factors in the ML pipeline which condition the model to which a CE method is applied. While our analysis focuses on a single CE method and metric (an extended version of Jaccard agreement), we expect similar patterns to hold more broadly. We encourage future research to test this with other local explanation methods (including alternative CE methods), alternative robustness measures, and recourse-relevant metrics such as diversity and actionability. Building on our findings and prior critical research on CE and AR, we argued that CE can play only a limited role in real-life practices of explanation, debugging, justification, and recourse provision. In particular, we argued that the potential temporal instability of key design choices, along with the contestability of their underlying assumptions, should inform the evaluation and practical application of CE methods. As model-dependent artifacts, CEs offer, at best, partial justification for models or decisions, and are prone to misinterpretation and strategic misuse. Our findings also challenge individualistic and unilateral understandings of AR. Recourse is relational and involves multiple parties, and thus effective recourse mechanisms must address the possibility that organizations’ design choices may be empirically or normatively untenable and in need of critical scrutiny and potential revision. Therefore, CEs alone, absent broader “algorithmic transparency”, remain insufficient to secure core values like public accountability, trust, and agency. Acknowledgements The authors gratefully acknowledge the support of the “TOPML: Trading Off Non-Functional Properties of Machine Learning” project funded by the Carl Zeiss Foundation, grant number P2021-02-014. References Abid et al. (2022) A. Abid, M. Yuksekgonul, and J. Zou Meaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, p. 66–88. Cited by: Introduction, Counterfactual Explanations (CE). Alex.com (2025) Alex.com Alex.com — automated ai interviews. Note: https://alex.comAccessed: 2026-20-05 Cited by: Example: Resume Screening, Example: Resume Screening. Barocas et al. (2020) S. Barocas, A. D. Selbst, and M. Raghavan The hidden assumptions behind counterfactual explanations and principal reasons. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT*), p. 80–89. External Links: Document Cited by: Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Implications for Justification, Discussion. Bayrak and Bach (2024) B. Bayrak and K. Bach Evaluation of instance-based explanations: an in-depth analysis of counterfactual evaluation metrics, challenges, and the CEval toolkit. IEEE Access 12, p. 137683–137695. Note: User cites as 2025 External Links: Document Cited by: Counterfactual Explanations (CE). Biran and Cotton (2017) O. Biran and C. Cotton Explanation and justification in machine learning: a survey. In IJCAI-17 Workshop on Explainable AI (XAI), p. 8–13. Cited by: Background and Related Work. Black et al. (2024) E. Black, J. L. Koepke, P. T. Kim, S. Barocas, and M. Hsu Less discriminatory algorithms. Georgetown Law Journal 113 (1), p. 53–115. Cited by: Implications for Justification. Boxer and Neill (2025) K. S. Boxer and D. B. Neill Realizing the promises of algorithmic recourse through reliability, accessibility, and fairness principles. In Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, p. 218–240. Cited by: Implications for Recourse. Breiman (2001) L. Breiman Statistical modeling: the two cultures (with comments and a rejoinder by the author). Statistical science 16 (3), p. 199–231. Cited by: Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations. Brughmans et al. (2024a) D. Brughmans, P. Leyman, and D. Martens NICE: an algorithm for nearest instance counterfactual explanations. Data Mining and Knowledge Discovery 38 (5), p. 2665–2703. External Links: Document Cited by: Introduction, Introduction, Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Experimental Design.. Brughmans et al. (2024b) D. Brughmans, L. Melis, and D. Martens Disagreement amongst counterfactual explanations: how transparency can be misleading. Top 32 (3), p. 429–462. Cited by: Pairwise metrics, RQ2: Consistency of Explanations, Discussion. Cerrato et al. (2023) M. Cerrato, A. Vallenas Coronel, and M. Köppel The case for correctability in fair machine learning. In European Workshop on Algorithmic Fairness (EWAF’23), CEUR Workshop Proceedings. Cited by: Algorithmic Recourse (AR). De Toni et al. (2025) G. De Toni, S. Teso, B. Lepri, and A. Passerini Time can invalidate algorithmic recourse. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT), Note: arXiv:2410.08007 Cited by: Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Implications for Recourse. Diakopoulos (2025) N. Diakopoulos Prospective algorithmic accountability and the role of the news media. In Computer Ethics Across Disciplines: Algorithmic Accountability and AI through Deborah G. Johnson’s Lens, M. Noorman and M. Verdicchio (Eds.), p. 125–143. Note: Preprint posted December 2024 Cited by: Introduction, Background and Related Work. Dominguez-Olmedo et al. (2022) R. Dominguez-Olmedo, A. Karimi, and B. Schölkopf On the adversarial robustness of causal algorithmic recourse. In Proceedings of the 39th International Conference on Machine Learning (ICML), Cited by: Algorithmic Recourse (AR). Doshi-Velez and Kim (2017) F. Doshi-Velez and B. Kim Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. Cited by: Background and Related Work. European Parliament and Council of the European Union (2016) European Parliament and Council of the European Union Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Vol. L 119. Cited by: Background and Related Work. European Parliament and Council (2024) European Parliament and Council Regulation (eu) 2024/1689 (artificial intelligence act). Vol. L 2024/1689. Note: Accessed: March 2026 External Links: Link Cited by: Background and Related Work. Fokkema et al. (2022) G. Fokkema, R. de Heide, and T. Philips Don’t explain noise: robust counterfactuals for randomized ensembles. In Proceedings of the 39th International Conference on Machine Learning, Cited by: 4th item. Gebru et al. (2021) T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. D. I, and K. Crawford Datasheets for datasets. Commun. ACM 64 (12), p. 86–92. External Links: ISSN 0001-0782, Link, Document Cited by: Implications for Recourse. Goethals et al. (2023) S. Goethals, D. Martens, and T. Calders PreCoF: counterfactual explanations for fairness. Machine Learning. External Links: Document Cited by: Counterfactual Explanations (CE). Guidotti et al. (2018) R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51 (5), p. 1–42. Cited by: Background and Related Work. Jiang et al. (2024) J. Jiang, F. Leofante, A. Rago, and F. Toni Robust counterfactual explanations in machine learning: a survey. In 33rd International Joint Conference on Artificial Intelligence, IJCAI 2024, p. 8086–8094. Cited by: Discussion. Joshi et al. (2019) S. Joshi, O. Koyejo, W. Vijitbenjaronk, B. Kim, and J. Ghosh REVISE: towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615. Cited by: Algorithmic Recourse (AR). Juliussen (2025) B. A. Juliussen The right to an explanation under the GDPR and the AI act. In MultiMedia Modeling (M 2025), Lecture Notes in Computer Science, Vol. 15523, p. 184–197. External Links: Document Cited by: Background and Related Work. Karimi et al. (2021) A. Karimi, B. Schölkopf, and I. Valera Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, p. 353–362. Cited by: Introduction, Implications for Justification. Kasirzadeh and Smart (2021) A. Kasirzadeh and A. Smart The use and misuse of counterfactuals in ethical machine learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT), p. 228–236. External Links: Document Cited by: Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Implications for Justification, Discussion. Kusner et al. (2017) M. J. Kusner, J. Loftus, C. Russell, and R. Silva Counterfactual fairness. In Advances in Neural Information Processing Systems, Vol. 30, p. 4067–4077. Cited by: Introduction, Counterfactual Explanations (CE). Laufer et al. (2025) B. Laufer, M. Raghavan, and S. Barocas What constitutes a less discriminatory algorithm?. In Proceedings of the 2025 Symposium on Computer Science and Law, p. 136–151. Cited by: Implications for Justification. Laugel et al. (2023) T. Laugel, A. Jeyasothy, M. Lesot, C. Marsala, and M. Detyniecki Achieving diversity in counterfactual explanations: a review and discussion. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, p. 1859–1869. Cited by: Implications for Recourse. Lipton (2018) Z. C. Lipton The mythos of model interpretability: in machine learning, the concept of interpretability is both important and slippery.. Queue 16 (3), p. 31–57. Cited by: Background and Related Work. Lo et al. (2025) F. P.-W. Lo, J. Qiu, Z. Wang, H. Yu, Y. Chen, G. Zhang, and B. Lo AI hiring with LLMs: a context-aware and explainable multi-agent framework for resume screening. arXiv preprint arXiv:2504.02870. Cited by: Example: Resume Screening. Lundberg and Lee (2017) S. M. Lundberg and S. Lee A unified approach to interpreting model predictions. Advances in neural information processing systems 30. Cited by: Background and Related Work. Meijer (2013) A. Meijer Understanding the complex dynamics of transparency. Public Administration Review 73 (3), p. 429–439. External Links: Document Cited by: Introduction. Miller (2019) T. Miller Explanation in artificial intelligence: insights from the social sciences. Artificial Intelligence 267, p. 1–38. External Links: Document Cited by: Background and Related Work. Mothilal et al. (2020) R. K. Mothilal, A. Sharma, and C. Tan Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 conference on fairness, accountability, and transparency, p. 607–617. Cited by: Experimental Design.. Pettit (2015) P. Pettit The robust demands of the good: ethics with attachment, virtue, and respect. Oxford University Press. Cited by: Algorithmic Recourse (AR). PrescreenAI (2025) PrescreenAI PrescreenAI — automated resume screening. Note: https://prescreenai.comAccessed: 2026-20-05 Cited by: Example: Resume Screening. Rhea et al. (2022) A. K. Rhea, K. Markey, L. D’Arinzo, H. Schellmann, M. Sloane, P. Squires, F. Arif Khan, and J. Stoyanovich An external stability audit framework to test the validity of personality prediction in ai hiring. Data Mining and Knowledge Discovery 36 (6), p. 2153–2193. Cited by: Example: Resume Screening. Ribeiro et al. (2016) M. T. Ribeiro, S. Singh, and C. Guestrin ” Why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, p. 1135–1144. Cited by: Introduction, Background and Related Work. Rudin (2019) C. Rudin Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1 (5), p. 206–215. Cited by: Background and Related Work. Sinha et al. (2020) A. K. Sinha, M. A. K. Akhtar, and A. Kumar Resume screening using natural language processing and machine learning: a systematic review. Machine Learning and Information Processing. Cited by: Example: Resume Screening. Slack et al. (2021) D. Slack, A. Hilgard, H. Lakkaraju, and S. Singh Counterfactual explanations can be manipulated. In Advances in Neural Information Processing Systems, Vol. 34. Cited by: Introduction, Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Discussion. Sokol and Flach (2019) K. Sokol and P. Flach Desiderata for interpretability: explaining decision tree predictions with counterfactuals. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, p. 10035–10036. Cited by: Introduction, Counterfactual Explanations (CE). Upadhyay et al. (2021) S. Upadhyay, S. Joshi, and H. Lakkaraju Towards robust and reliable algorithmic recourse. In Advances in Neural Information Processing Systems, Vol. 34, p. 16926–16937. Cited by: Algorithmic Recourse (AR), Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Implications for Recourse. Ustun et al. (2019) B. Ustun, A. Spangher, and Y. Liu Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), p. 10–19. External Links: Document Cited by: Algorithmic Recourse (AR). Venkatasubramanian and Alfano (2020) S. Venkatasubramanian and M. Alfano The philosophical basis of algorithmic recourse. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT*), p. 284–293. External Links: Document Cited by: Introduction, Algorithmic Recourse (AR), Algorithmic Recourse (AR), Counterfactual Explanations for Algorithmic Recourse: Challenges and Limitations, Implications for Recourse. Verma et al. (2024) S. Verma, V. Boonsanong, M. Hoang, K. Hines, J. Dickerson, and C. Shah Counterfactual explanations and algorithmic recourses for machine learning: a review. ACM Computing Surveys. External Links: Document Cited by: Introduction, Counterfactual Explanations (CE), Counterfactual Explanations (CE), Algorithmic Recourse (AR), Background and Related Work. Virgolin and Fracaros (2022) M. Virgolin and S. Fracaros On the robustness of sparse counterfactual explanations to adverse perturbations. Artificial Intelligence, p. 103840. External Links: ISSN 0004-3702, Document, Link Cited by: Introduction, Discussion. Wachter et al. (2018) S. Wachter, B. Mittelstadt, and C. Russell Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harvard Journal of Law & Technology 31 (2), p. 841–887. Note: Preprint 2017, arXiv:1711.00399 Cited by: Introduction, Counterfactual Explanations (CE), Algorithmic Recourse (AR). Wang et al. (2024) Z. Wang, Z. Chu, R. Blanco, Z. Chen, S. Chen, and W. Zhang Advancing graph counterfactual fairness through fair representation learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD), p. 40–58. Cited by: Counterfactual Explanations (CE). Youssef et al. (2024) P. Youssef, C. Seifert, J. Schlötterer, et al. LLMs for generating and evaluating counterfactuals: a comprehensive study. In Findings of the association for computational linguistics: EMNLP 2024, p. 14809–14824. Cited by: Example: Resume Screening. Zehlike et al. (2022) M. Zehlike, K. Yang, and J. Stoyanovich Fairness in ranking, part i: score-based ranking. ACM Computing Surveys 55 (6), p. 1–36. Cited by: Example: Resume Screening. Table 1: The ACS Microdata Sample columns we employed in our experiments. More detailed definitions are available at the official ACS documentation: https://w2.census.gov/programs-surveys/acs/tech˙docs/pums/data˙dict/PUMS˙Data˙Dictionary˙2018.pdf. Column Name Definition Type JWTR Means of transportation to work Categorical WKW Weeks worked during past 12 months Continuous ENG Ability to Speak English Categorical SCIENGP Achieved a Degree in Science and Engineering, by the NSF Definition Categorical COW Class of worker Categorical SOC Standard Occupational Classification code Categorical JWMNP Travel time to work Continuous Hyperparameter Selection We report the hyperparameter grids and defaults we employ in Interventions 3 and 4 (model selection strategy, model success metric) in Table 2. The “Default” column shows the non-interventional choice, that is, the SCM setting ξ0 _0. Table 2: Random-forest hyperparameter grid searched during model selection. Cross-validation uses stratified C-fold splits with the success metric M; the validation strategy C and metric M are themselves interventions in Section Experiments. All other RF parameters are left at their scikit-learn defaults. Hyperparameter Grid values Default n_estimators 50, 100, 200\50,\ 100,\ 200\ 100100 max_depth 5, 10,None\5,\ 10,\ None\ None min_samples_split 2, 5\2,\ 5\ 22 Counterfactual distance metrics We measure the sensitivity of CE sets to an intervention by averaging a set-level distance d⁡(X^ξ,X^ξ′)d( X_ξ, X_ξ ) over query instances. The set distance is built in two layers: a per-CE cost c⁡(x,x′)c(x,x ) on the feature space, and a symmetric aggregator over sets of CEs. Per-CE cost. For two feature vectors x,x′x,x over continuous features ℱcontF_cont and categorical features ℱcatF_cat, we use a mixed-type cost c(x,x′)=∑j∈ℱcont|xj−xj′|MADj⁡(Xtrain)+∑j∈ℱcat[xj≠xj′],c(x,x )= _j _cont x_j-x _j MAD_j(X_train)+ _j _cat1\! [x_j≠ x _j ], (10) where MADjMAD_j is the median absolute deviation of feature j estimated on the run’s training set. The rescaling makes continuous contributions unit-free and outlier-robust; categorical features contribute a 0/1 mismatch indicator. The MAD is fitted per seed on that seed’s training set, so c is run-specific. Set-level distance. For two CE sets A,BA,B associated with the same query, we use the symmetric mean-of-nearest-neighbours distance d⁡(A,B)=12(1|A|∑a∈Aminb∈B⁡c⁡(a,b)+1|B|∑b∈Bmina∈A⁡c⁡(a,b)).d(A,B)= 12\! ( 1|A| _a∈ A _b∈ Bc(a,b)+ 1|B| _b∈ B _a∈ Ac(a,b) ). (11) This is a softened, averaging variant of the Hausdorff distance: replacing each mean by a max recovers Hausdorff. We use the mean form because retrieval-based methods can return CE sets of mildly different sizes (KDiCE=5K_DiCE=5, KNICE=3K_NICE=3), and a worst-case aggregator would tend to over-weight the single most discordant CE in either set. If either set is empty, d is undefined and the corresponding query is dropped from the across-query average. Aggregation across queries and seeds. For each intervention pair (ξ,ξ′)(ξ,ξ ), method, and seed, we compute d⁡(X^ξ(x),X^ξ′(x))d( X_ξ(x), X_ξ (x)) at every query x with y^=0 y=0 and take the mean over queries. Reported sensitivities are means across seeds with 95% Student-t confidence intervals. Consistency metrics: full definitions This appendix provides the formal definitions, equation numbers, and design choices underlying the consistency analysis proposed in our experiments. Explanation set We propose a slightly different notation for queries and feature vectors here compared to our experimental section, for convenience and clarity. Let x be a query instance with feature vector x∈ℝ|ℱ|x ^|F| over the feature schema ℱF of dataset D, and let x x be a counterfactual produced for x. We define the explanation set of x x relative to x is E⁡(x^,x)=j∈ℱ:x^j≠xj,E( x,x)=\j : x_j≠ x_j\, (12) i.e. the subset of features that the counterfactual changes relative to the query. Continuous features are compared with an absolute tolerance of 10−910^-9; categorical features are compared as strings. Pairwise metrics For two counterfactuals x^a,x^b x_a, x_b of the same query x, with explanation sets Ea=E⁡(x^a,x)E_a=E( x_a,x) and Eb=E⁡(x^b,x)E_b=E( x_b,x), the paper defines several disagreement metrics. We use the symmetric Jaccard agreement: Jaccard(Ea,Eb) (E_a,E_b) =|Ea∩Eb||Ea∪Eb| = |E_a∩ E_b||E_a∪ E_b| (13) Jaccard is 11 when the two CEs change exactly the same set of features and 00 when they are disjoint. The formulation from (Brughmans et al. 2024b) treats two CEs as ”agreeing on feature j” whenever both change j, regardless of how. Their analysis focuses on understanding the possibility that a malicious actor would include an arbitrary feature into an explanation as a means of assuaging recourse. As a consequence, a CE that changes (for instance) occupation from cook to chef agrees with one that changes it from cook to lawyer only at the level of ”occupation was changed”. Here, we are instead interested in understanding the overall direction of change suggested by two CEs across different interventions, and whether the direction is in some sense similar. To recover the directional information for continuous features and the exact-value information for categorical features we define two complementary scalars, both intersected with Ea∩EbE_a∩ E_b so that they only score features both CEs changed. Let ℱcont,ℱcatF_cont,F_cat partition ℱF into continuous and categorical features. Sign Agreement (continuous). We compute the sign agreement of continuous variables in a CE as follows: SA⁡(x^a,x^b,x) ( x_a, x_b,x) =1|M|∑j∈M[sign(x^a,j−xj)=sign(x^b,j−xj)]. = 1|M| _j∈ M1\! [sign( x_a,j-x_j)=sign( x_b,j-x_j) ]. J J =Ea∩Eb∩ℱcont =E_a∩ E_b _cont SA=1SA=1 when the two CEs always push shared continuous features in the same direction. Value Agreement (categorical). Closes the ”agree-on-feature-but-not-value” gap for categorical features: VA⁡(x^a,x^b,x) ( x_a, x_b,x) =1|M|∑j∈M[x^a,j=x^b,j]. = 1|M| _j∈ M1\! [ x_a,j= x_b,j ]. (14) M M =Ea∩Eb∩ℱcat =E_a∩ E_b _cat (15) VA=1VA=1 when the two CEs always pick the same new value for shared categorical features. Both metrics are undefined when the corresponding intersection is empty (no continuous or no categorical feature changed by both CEs). In that case, the original query vector is dropped from the overall computation of the metric. Aggregation over multi-CE sets In our setup each (run, method, threshold, intervention, query) tuple yields a CE set of k≥1k≥ 1 counterfactuals (with k=4k=4 for DiCE-kdtree and k=2k=2 for NICE in our configuration). We note that reducing each set to a single representative CE (e.g. with the median for continuous features, mode for categoricals) and applying the single-CE metrics, would smooth out differences in the CEs. Instead, for two CE sets A=x^1A,…,x^kAAA=\ x^A_1,…, x^A_k_A\ and B=x^1B,…,x^kBBB=\ x^B_1,…, x^B_k_B\ generated for the same query x under different conditions, we define the set-level metric as the average over the Cartesian product: Jaccard⁡(A,B,x)=1kAkB∑i=1kA∑l=1kBJaccard⁡(E⁡(x^iA,x),E⁡(x^lB,x)),Jaccard(A,B,x)= 1k_Ak_B _i=1^k_A _l=1^k_BJaccard\! (E( x^A_i,x),E( x^B_l,x) ), (16) and analogously for FD, SA, and VA.

Instant research discovery