Paper deep dive

Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs

Ranit Debnath Akash, Ashish Kumar, Gang Tan, Saeid Tizpaz-Niari

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 123

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%

Last extracted: 8/28/2026, 3:22:03 AM

Summary

The paper introduces REMI, a framework for the automated localization, explanation, and mitigation of individual fairness bugs in data-driven software systems. REMI treats counterfactual fairness as a relational invariant discovery problem, using bidirectional constraints on paired examples (original and counterfactual) to identify regions where fairness is violated. It employs three data-alignment techniques to infer interpretable rule-based models (fairness invariants) that act as guardrails to block or relabel unfair predictions without retraining. Evaluation shows REMI localizes ground-truth fairness bugs in over 83% of cases and reduces discriminatory decisions in black-box models by up to 70%.

Entities (10)

REMI · framework · 98%Individual Discrimination · problem · 95%Fairness Invariants · concept · 92%Protected Attributes · attribute · 92%Counterfactual Fairness · concept · 90%Guardrails · mechanism · 90%Bidirectional Relational Explanation · method · 88%C4.5Tree · algorithm · 87%Data Curation with Horizontal Extension (DCHE) · technique · 85%Decision-Making Program under Test (DPuT) · entity · 85%

Relation Signals (7)

REMI → localizes → Individual Discrimination

confidence 95% · Our evaluation on symbolic and neural network programs demonstrates that REMI localizes ground-truth fairness bugs in over 83% of cases

REMI → mitigates → Individual Discrimination

confidence 95% · REMI... reduces discriminatory decisions in black-box models by up to 70%.

Fairness Invariants → actas → Guardrails

confidence 90% · These rules serve as guardrails to selectively block or relabel unfair predictions without requiring model retraining.

REMI → uses → Bidirectional Relational Explanation

confidence 90% · We introduce a bidirectional relational explanation framework that learns over paired examples (x, x') to identify regions of the input space where fairness is violated.

REMI → isinspiredby → Loop-invariant Synthesis

confidence 88% · Inspired by loop-invariant synthesis in formal methods, we treat counterfactual fairness as a relational invariant discovery problem.

C4.5Tree → outperforms → Other Interpretable Learners

confidence 85% · We found that a C4.5Tree (Quinlan, 1993) achieves superior performance and localizes 2,237 IDIs among the 7 tree-based and interpretable algorithms.

DCHE → outperforms → Other Data Curation Techniques

confidence 80% · We find that horizontal extension outperforms the baseline and other alignment methods

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Data-driven software systems are increasingly deployed in high-stakes socio-economic domains, from criminal justice to financial lending. However, these systems often exhibit individual discrimination---unjustified disparities in which a program yields different outcomes for similar individuals who differ only in their protected attributes (e.g., race, gender, age). While existing research has focused on detecting and quantifying these bugs, there remains a critical lack of principled mechanisms to explain and localize individual fairness bugs. Current explanation techniques are largely designed for single-input decisions rather than the relational nature of discrimination, which inherently involves a comparison between an original and a counterfactual pair. We present REMI, a framework for the automated localization, explanation, and mitigation of individual discrimination. Inspired by loop-invariant synthesis in formal methods, we treat counterfactual fairness as a relational invariant discovery problem. We introduce a bidirectional relational explanation framework that learns over paired examples $(x, x')$ to identify regions of the input space where fairness is violated. Unlike traditional one-way implication pairs used in invariant inference, our approach enforces bidirectional constraints: requiring identical outcomes for both original and counterfactual samples. REMI utilizes three data-alignment techniques to infer interpretable rule-based models that act as "fairness invariants." These rules serve as guardrails to selectively block or relabel unfair predictions without requiring model retraining. Our evaluation on symbolic and neural network programs demonstrates that REMI localizes ground-truth fairness bugs in over 83% of cases, significantly outperforming state-of-the-art baselines and reducing discriminatory decisions in black-box models by up to 70%.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

119,779 characters extracted from source content.

Expand or collapse full text

Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs DOI: 10.1145/3832110Journal: PACMSEVolume: 3ISSTAISSTA01910issta26main-p172-pCCS: Software and its engineering Software testing and debuggingCCS: Computing methodologies Machine learningCCS: Computing methodologies Rule learning Ranit Debnath Akash Affiliation: University of Illinois at Chicago, Chicago, USA email: rakas@uic.edu , Ashish Kumar Affiliation: Pennsylvania State University, University Park, USA email: azk640@psu.edu , Gang Tan Affiliation: Pennsylvania State University, University Park, USA email: gtan@psu.edu and Saeid Tizpaz-Niari Affiliation: University of Illinois at Chicago, Chicago, USA email: saeid@uic.edu Received 2026-06-25 Abstract. Data-driven software systems are increasingly deployed in high-stakes socio-economic domains, from criminal justice to financial lending. However, these systems often exhibit individual discrimination—unjustified disparities in which a program yields different outcomes for similar individuals who differ only in their protected attributes (e.g., race, gender, age). While existing research has focused on detecting and quantifying these bugs, there remains a critical lack of principled mechanisms to explain and localize individual fairness bugs. Current explanation techniques are largely designed for single-input decisions rather than the relational nature of discrimination, which inherently involves a comparison between an original and a counterfactual pair. We present Remi, a framework for the automated localization, explanation, and mitigation of individual discrimination. Inspired by loop-invariant synthesis in formal methods, we treat counterfactual fairness as a relational invariant discovery problem. We introduce a bidirectional relational explanation framework that learns over paired examples (x,x′)(x,x ) to identify regions of the input space where fairness is violated. Unlike traditional one-way implication pairs used in invariant inference, our approach enforces bidirectional constraints: requiring identical outcomes for both original and counterfactual samples. Remi utilizes three data-alignment techniques to infer interpretable rule-based models that act as "fairness invariants." These rules serve as guardrails to selectively block or relabel unfair predictions without requiring model retraining. Our evaluation on symbolic and neural network programs demonstrates that Remi localizes ground-truth fairness bugs in over 83% of cases, significantly outperforming state-of-the-art baselines and reducing discriminatory decisions in black-box models by up to 70%. Keywords: Machine Learning, Bias Mitigation, Fairness, AI Ethics, Interpretability †c-license: by 1. Introduction Automated decision-support software systems have become foundational to modern socio-economic infrastructure. They have been used to make critical decisions in domains such as criminal justice, healthcare, financial lending, and hiring. However, because these models often learn from historical datasets, they risk encoding and amplifying biases related to protected attributes like race, gender, or disability status. Hence, ensuring their fairness has emerged as a critical requirement. In high-stakes scenarios, this manifests as "fairness bugs"—unjustified disparities where the software yields different outcomes for individuals who are identical in all relevant qualifications, but differ in a protected characteristic. For instance, studies on FICO scoring found that black non-defaulters were often assigned higher risk scores than their white counterparts (Hardt et al., 2016). In response to these risks, the software engineering and machine learning communities have developed a diverse array of techniques to detect, explain, and mitigate bias through pre-processing data (Chakraborty et al., 2020; Chakraborty et al., 2019), in-processing algorithmic adjustments (Tizpaz-Niari et al., 2022; Gohar et al., 2023; Nguyen et al., 2023), and post-processing calibration (Hardt et al., 2016). While these efforts have significantly improved our ability to address fairness issues, a principled mechanism for explaining individual fairness bugs remains largely lacking. Current explanation techniques primarily focus on local interpretability, explaining why a model made a specific decision for a single input (Ribeiro et al., 2016). However, individual fairness is inherently relational: it cannot be violated by a single input, but requires an analysis between a specific individual x and their "similar" counterpart x′x who may differ only in their protected attributes. We identify two core limitations. (1) The Localization Insufficiency: Existing tools provide explanations for model decisions (e.g., "why was this loan denied?"), but they lack the framework to explain discriminatory outcomes (e.g., "why was this loan denied to x, but granted to x′x ?"). We currently lack a way to consider multiple similar inputs simultaneously to pinpoint the exact logic driving the disparity. (2) The Mitigation Problem. Developers require compact and precise characterizations of how the counterfactual unfairness occurs. Without a way to localize these "fairness bugs" to specific relational constraints, mitigation strategies are limited and ineffective. Intuition. Inspired by loop-invariant synthesis from the programming language literature (Solar-Lezama et al., 2006), we develop a relational explanation and mitigation technique for software fairness. In loop-invariant synthesis, the goal is to identify a region of the program state space that captures all reachable program states. The key is to enforce ’implication pairs’: if a state satisfies the head of an implication pair, then it must also satisfy the tail (a positive instance). Decision tree inference with implication pairs (Garg et al., 2016) formalizes this by classifying states into positive and negative examples. The learned decision tree must therefore respect these unidirectional relational constraints in addition to separating positive and negative instances. The relational fairness problem can be viewed as an analogous task. Key Observation. Instead of reasoning about reachable program states, we infer fairness invariants of automated decision-making software and identify regions of the input space where the original x (head) and counterfactual x′x (tail) pairs disagree. Each relational pair (x,x′)(x,x ) acts as a constraint, requiring that the program assign the same outcome to both original and counterfactual samples that may only differ in their protected attributes. Unlike the one-directional implication pairs used in invariant inference, these counterfactual constraints are bidirectional: if one element of the pair receives a favorable outcome, the other must as well for fairness to hold, and likewise for unfavorable outcomes. Hence, the approach for using decision trees to respect unidirectional relational constraints can be extended to bidirectional constraints, enabling us to use decision trees to solve the counterfactual fairness problem. Approach. We present Remi (Relational Explanation and MItigation), a framework to localize, explain, and mitigate individual fairness bugs. Remi first generates counterfactual instances to identify discriminatory pairs. It then constructs a relational dataset where pairs are labeled ’+’ if the model treats them identically (fair) and ’-’ if the outcomes differ (unfair). We propose three alignment techniques—original, horizontal, and vertical extension—to structure this relational data. From this, Remi infers interpretable rules that distinguish fair regions from unfair ones Finally, Remi extracts these discriminatory rules as guardrails to selectively block or relabel unfair predictions, mitigating bias without requiring model retraining. Experiments. We perform experiments both on symbolic (rule-based) and neural network (black-box) programs. Since the ground truth for the symbolic programs is known and verifiable, we first use Remi to answer four research questions about i) the performance of three relational dataset curation techniques; i) the performance of nine different rule inference algorithms; i) the characteristics and precision of extracted rules vs. the ground truth; and iv) the performance of guardrails technique for a mitigation, based on the extracted discriminatory rules. We find that horizontal extension outperforms the baseline and other alignment methods, and C4.5Tree, FIGS, and CART tree inferences outperform other rule-based inference methods. The alignment and inference techniques localize the ground truth in more than 83% of the cases, compared to the state-of-the-art AFT (Zhao et al., 2024) approach. Additionally, the rule-based guardrails reduce individual instances of discrimination in all cases. Finally, we study whether our experiences with Remi over symbolic programs generalize to black-box neural networks. We found that applying guardrail rules to neural networks reduces individual discriminatory decisions by at least 40%, up to 70%, significantly outperforming the retraining-based bias mitigation techniques (Zhao et al., 2024; Fan et al., 2022; Zhang et al., 2020; Udeshi et al., 2018) Contributions. The key contributions of this paper are: • Inspired by the loop invariant synthesis, we design a data-driven approach to infer fairness invariants for automated decision-making programs, • We develop a novel guardrail strategy to mitigate individual discrimination, • We put forward Remi, a framework that automatically detects, localizes, explains, and mitigates individual fairness bugs. • Our experiments on both rule-based symbolic and data-driven neural network programs show that Remi significantly outperformed the baseline techniques in terms of localizing and mitigating individual discrimination. 2. Overview Following our intuition that individual fairness is a relational property that cannot be violated over a single point x. But it requires a relational analysis between the original and counterfactual inputs (x,x′x,x ) that only differ in their protected attributes. In light of this individual fairness (Dwork et al., 2012), a decision by decision-making programs-under-test (DPuTs) is considered unfair (discriminatory) if only flipping the protected attributes (and any logically dependent non-protected attributes) changes the outcome of DPuTs. We demonstrate that explaining the DPuT’s decision for a single data point, e.g., local explanation methods (Ribeiro et al., 2016) is insufficient. Figure 1. Remi Framework. Remi Workflow Summary. Here we will give an overview of how Remi works. Remi is a tool that identifies and explains individual fairness violations within decision-making programs. It identifies the specific logic that causes a program to produce discriminatory outcomes based on protected attributes, such as race. Our workflow consists of four main stages, as illustrated in Figure 1. It has two elements: (1) the decision-making program under test (DPuT) and (2) a dataset containing protected and non-protected attributes. A. Counterfactual generation and finding discriminatory instances. The first stage is to build a relational data point. In doing so, Remi generates a counterfactual instance for any given input by changing the protected attributes. A relational data point becomes discriminatory (unfair) when the DPuT gives different outcomes for each input in the pair. B. Relational dataset construction and curation. Evaluating the DPuT on both x and x′x yields a relational label: ++ if both outcomes agree, and −- if they disagree (a fairness bug). Remi introduces three data curation schemes, which all keep the relational label ℒL "fair (++) vs unfair (−-)", but transform the features differently: • Data curation with no extension (DCNE). This approach keeps only the original point (x), but the label of the point is the corresponding relational label ℒL as derived by comparing the outcome of DPuT on x against the counterfactual point x′x . • Data Curation with vertical extension (DCVE). It augments the original data point x with its counterfactual (x′x ) as a separate instance. The labels of both instances are common and equal to the relational label ℒL. This doubles the number of training instances and exposes the tree learner to more examples of fair and discriminatory profiles. • Data Curation with horizontal extension (DCHE). This method concatenates original (x) and counterfactual (x′x ) features into a single extended feature vector, keeping only the features that are not identical in both (xi≠xi′x_i≠ x _i). This explicitly exposes interactions between original vs counterfactual values that correlate with unfairness. Across all the preprocessing techniques, the target is the relational label ℒ⁡(x,x′)L(x,x ), i.e., (++ or −-). The non-protected attributes (and, for DCHE, their counterfactual counterparts) are the only inputs to the interpretable learner, so that the learned rules describe where the DPuT is unfair. C. Interpretable learning and rule extraction. Given the relational dataset, Remi instantiates the explainer using a set of tree-based algorithms (e.g., CART, C4.5, TaoTree, RulesFit, FIGS). Our target is to retrieve a performant, interpretable learner that can explain what features distinguish fair relational samples from the unfair ones. The tree partitions the curated feature space into regions whose leaves are labeled with relational labels. From leaves with a high proportion of unfair relational labels, Remi extracts if–then rules that characterize discriminatory subspaces (e.g., “age ≤ 20.5 ∧ priors_count ≤ 3.5” or “college rank = 50 ∧ years-of-experience ≤ 1 year”). D. Guardrail-based mitigation. Finally, Remi uses the extracted rules as guardrails at the final debiased model during deployment time. When a new input falls into an unsafe region, the framework can deny automatic prediction and defer to human review. Because guardrails are enforced selectively on localized regions, Remi avoids heavy-handed global post-processing that might degrade accuracy or introduce new fairness violations elsewhere. Listing 1: FairSquare (Albarghouthi et al., 2017) populaiton generation ⬇ def popModel() ethnicity = gauss(0,10) colRank = gauss(25,10) yExp = gauss(10,5) if ethnicity > 10: colRank = colRank + 5 return (colRank, yExp) Listing 2: FairSquare (Albarghouthi et al., 2017) hiring decision ⬇ def dec(colRank, yExp): expRank = yExp - colRank if colRank <= 5: hire = 1 else if expRank > -5: hire = 1 else: hire = 0 return (expRank, hire) Overview Example: We use an example of a hiring program, taken from Albarghouthi et al. (Albarghouthi et al., 2017), to illustrate our approach (see Listings 1,2), where the ethnicity is a protected attribute, and the college rank of an individual is causally and directly affected by their ethnicity (e.g., being Hispanic and attending a Hispanic-Serving Institution). Specifically, a low college rank and more years of prior job experience are key to hiring. The program’s decision function appears fair, as it uses only college rank and years of experience to decide hiring, but an upstream generative model causally influences college rank. Our approach starts by generating the population (following Listing 1) and passing those samples to obtain the score and hiring decision from dec program 2. Remi first generates counterfactual applicants by changing ethnicity and adjusting affected attributes, then labels pairs where the hiring decision flips as unfair. For this example, Remi generates 30,000 applicant samples from the program and their corresponding counterfactual with a different ethnicity. Our sampling finds 2,250 individual discriminatory instances, applicant pairs that differ only in ethnicity, but receive different hiring decisions (labeled ‘-’). Note that the rest (27,750 samples) is fair w.r.t. ethnicity and labeled ‘+’. We are interested in explaining and mitigating the individual discriminatory bugs in this program. Which curation scheme best enables discrimination localization? Among the three relational dataset curation, we found that DCNE infers models with very high predictive metrics (accuracy around 0.99, Precision close to 0.99), leading to the localization of bugs in up to 2,234 out of the 2,250 individual discriminatory instances (IDIs). This convinces us that the relational labels in the training data are a key driver of root-cause localization quality. Which interpretable learner is the best at identifying discriminatory regions? We compare multiple interpretable learners (e.g., CART, C4.5, TaoTree, rule-based baselines), while fixing the data curation method to DCNE. We found that a C4.5Tree (Quinlan, 1993) achieves superior performance and localizes 2,237 IDIs among the 7 tree-based and interpretable algorithms. How does our approach perform to pinpoint the root cause of the fairness bug? Our relational decision trees yield compact rules that localize unfair regions of interest. In total, it retrieves 97 rules and covers 98% of the considered discriminatory instances. For example, one rule captures 301 of the 2250 IDIs (13.4% discrimination coverage) is the following: 14.6<col-rank ≤ 17.25 ∧ 11.99 < y-exp ≤ 14.84 ∧ cf-ethnicity > 10.5 ∧ 17.85< cf-col-rank ≤ 22.95 where it finds a narrow region of college rank and year of experience when the DPuT of PG1 becomes significantly discriminatory based on the ethnicity. Intuitively, the rule recovers the root cause of the discrimination: in a narrow mid-college rank, mid-experience range (14.6< col-rank ≤ 17.25, 11.99 < y-exp ≤ 14.84), switching to an unfavorable ethnicity increases the counterfactual college rank (17.85 < cf-col-rank ≤ 22.95) and flips the hiring decision. Because this rule explicitly captures the ethnicity-to-college rank-to-hire pathway, it demonstrates a very close match with the ground truth level root cause of the discrimination in this hiring program. How to Mitigate Unfairness: We treat the learned rules as guardrails around the original program. On this program, inserting discriminatory rules as guardrails reduces the number of discriminatory instances from 2,250 to just 2, while preserving other similar classification metrics. We note that naively replacing the program with an unconstrained decision tree trained on counterfactual labels actually increases the number of fairness bugs to 2,430 on unseen samples. 3. Relational Explanation and Mitigation Problem We consider any Decision-Making Program under Test (DPuT), which is essentially a function that maps an input describing an individual into a binary decision, such as favorable (e.g., loan approved) or unfavorable (e.g., loan denied). The input features for each individual can be grouped into two disjoint categories: Protected attributes (ApA_p) such as race or gender, which by fairness considerations should not affect the decision, and Non-protected attributes (AnpA_np) such as income, age, or education level. Formally, the DPuT can be represented as a function f⁡(Ap,Anp)→Y,Y∈favorable,unfavorable.f(A_p,A_np)\;→\;Y, Y∈\favorable,unfavorable\. Fairness Notion. We use an individual fairness definition that requires two similar individuals should receive similar outcomes irrespective of their protected attributes (Dwork et al., 2012). Following the counterfactual fairness definition (Kusner et al., 2017), a decision by DPuT is considered unfair (discriminatory) for an individual if only changing the protected attributes (and any logically dependent non-protected attributes) flips the decision, i.e., f⁡(xp,xnp)≠f⁡(xp′,xnp′)f(x_p,x_np)≠ f(x _p,x _np) where xp≠xp′∧xnp∼xnp′x_p≠ x _p x_np x _np. Relational Labeling for Discrimination. We are given a DPuT together with an input dataset D. To evaluate fairness, we transform D into a relational dataset, where pairs of datapoints are explicitly labeled as fair or unfair. We formalize this notion below: Definition 3.1 (Relational Dataset). A relational dataset is a pair R=(′,R)D_R=(D ,R) where ′=x1,…,xnD =\x_1,…,x_n\ is a set of datapoints and R:′×′→+,−R:D ×D →\+,-\ is a labeling function that assigns to each pair (xi,xj)(x_i,x_j) either ++ (indicating a fair pair) or −- (indicating an unfair pair). For each datapoint xi=(api,anpi)∈x_i=(a_p_i,a_np_i) , where api∈Apa_p_i∈ A_p are the protected attributes and anpi∈Anpa_np_i∈ A_np are the non-protected attributes, we construct a corresponding counterfactual datapoint xi′x _i. The counterfactual is obtained by modifying the protected attributes, denoted api′a _p_i, while keeping the non-protected attributes unchanged, except in cases where a change in the protected attribute necessitates an adjustment to some non-protected attributes (e.g., changing sex from male to female requires adjusting the relationship attribute from “husband” to “wife”). Formally, xi′=(api′,anpi^),where api′≠api,x _i=(a _p_i, a_np_i), a _p_i≠ a_p_i, and anpi a_np_i denotes the non-protected attributes of xix_i, possibly updated to maintain consistency with the change in api′a _p_i. We then evaluate the DPuT on both the original datapoint and its counterfactual, obtaining outcomes yi=f⁡(xi)y_i=f(x_i) and yi′=f⁡(xi′)y _i=f(x _i), where f is the function modelling the DPuT. This comparison induces a relational label on the pair (xi,xi′)(x_i,x _i): • Agreement (++): The outcomes agree, indicating fair treatment. L(xi,xi′)=+if yi=yi′L(x_i,x _i)=+ y_i=y _i • Disagreement (−-): The outcomes differ, indicating potential counterfactual unfairness. L(xi,xi′)=−if yi≠yi′L(x_i,x _i)=- y_i≠ y _i In this construction, the set ′D of the relatioanl dataset consists of all original datapoints together with their counterfactual counterparts, and the relation R is defined by the agreement or disagreement labels assigned to each pair. The key challenge is not to explain the overall decision function f⁡(x)f(x) where x∈x . Instead, our focus is on explaining the relational label L⁡(x,x′)L(x,x ), which captures whether a pair of datapoints is treated fairly or unfairly. Our goal is to learn a human-interpretable model E that, can characterize the conditions under which a pair receives a disagreement label (−-), i.e., when the Decision-Making Program under Test (DPuT) exhibits unfair behavior. In other words, we want to answer the question: Informal Problem Statement. For which regions of the input space, defined solely by non-protected attributes, does the DPuT exhibit counterfactual unfairness? Formal Problem Statement Let our relational dataset be R=((x1,x1′),L1),((x2,x2′),L2),…,((xn,xn′),Ln)D_R=\((x_1,x _1),L_1),((x_2,x _2),L_2),…,((x_n,x _n),L_n)\ where each xi=(api,anpi)x_i=(a_p_i,a_np_i), xi′=(api′,anpi^)x _i=(a _p_i, a_np_i), and the label Li∈+,−L_i∈\+,-\ indicates whether the outcomes of the DPuT on (xi,xi′)(x_i,x _i) agree or disagree. For convenience, we denote Zi=(xi,xi′)Z_i=(x_i,x _i). Explanation. Our goal is to learn an interpretable explanation model E, instantiated as a decision-tree based learner (e.g., CART, C4.5, FIGS, RulesFit, or similar variants). The model E predicts whether a pair ZiZ_i will receive an agreement (++) or disagreement (−-) label. In practice, decision tree training is framed as a binary classification problem, where the learning algorithm recursively partitions the feature space so as to minimize an impurity measure at each split. For classification, the impurity at a node is most commonly measured using the cross-entropy between the two possible classes - agreement and disagreement. If p+p_+ and p−p_- denote the empirical probabilities of the labels ++ and −-, respectively, among the datapoints at a node, then the cross-entropy is then given by H⁡(p+,p−)=−p+log⁡p+−p−log⁡p−H(p_+,p_-)=-p_+ p_+-p_- p_- Thus, the explanation model E is a set of human-interpretable rules that delineate regions of the feature space Z where the DPuT is most likely to exhibit counterfactual unfairness. Mitigation. Since the learned model E characterizes the discriminatory regions purely in terms of attributes, the rules describing its disagreement (−-) regions can be extracted as guardrails. Let ℛ−R^- denote this rule set, inducing a guardrail predicate g⁡(anp)=1g(a_np)=1 iff anpa_np satisfies some rule r∈ℛ−r ^-, i.e., the input lies in a region where f is likely to discriminate. We then deploy a guarded decision function fg(x)=f(x)f_g(x)=f(x) when g⁡(anp)=0g(a_np)=0, and fg(x)=⊥f_g(x)= when g⁡(anp)=1g(a_np)=1, where ⊥ denotes withholding the automatic outcome and deferring the case to human review (or, alternatively, relabeling it to restore a consistent outcome across the pair (x,x′)(x,x )). The mitigation objective is to choose ℛ−R^- so that fgf_g minimizes the residual counterfactual unfairness while keeping both the benign inputs it unnecessarily blocks and the loss in predictive utility small. 4. Approach We define a region as fair if the DPuT’s outcome remains invariant under changes to protected attributes; otherwise, the region is unfair, indicating the existence of Individual Discriminatory Instances (IDIs). To localize these bugs, our tool Remi, employs an interpretable decision tree to classify fair versus unfair regions. This model captures feature interactions and enables the extraction of precise decision rules that characterize the discriminatory input space. As illustrated in Figure 1 and Algorithm 1, our framework consists of four primary steps: • Section 4.1: Generating counterfactuals to detect IDIs. • Section 4.2: Curating relational datasets from identified pairs. • Section 4.3: Training an interpretable model to extract explanation rules for discrimination. • Section 4.4: Deploying these rules as guardrails to mitigate unfairness. 4.1. Counterfactual Generation and Finding Discriminatory Instances For each of our different relational dataset construction and training approaches and programs (DPuTs), this is a common step. We take as input a decision-making program under test DPuT, which has access to sensitive attributes from the original dataset (XorigX_orig), the program will be applied to. After applying the program DPuT on XorigX_orig, we can retrieve their target (favorable or unfavorable) outcome YorigY_orig (Line 1 in Algorithm 1). Then we create a counterfactual xi′x _i for each data point xi∈Xorigx_i∈ X_orig based on its protected attribute. For our symbolic programs, we just flip the protected attribute (and update accordingly related dependent non-protected attribute) of to get the corresponding counterfactual entry (xi′∈Xcfx _i∈ X_cf), and retrieve the target outcome for counterfactual (YcfY_cf) based on the decision of DPuT (Line 2-5 in Algorithm 1). Here xi′x _i is the corresponding counterfactual CF⁡(xi)CF(x_i) of xix_i (Line 3). Here, we consider a protected attribute in a binary setting, grouping it into privileged or unprivileged groups. Then we audit for fairness violations with respect to the protected attribute, checking whether the outcomes for the corresponding counterfactual pairs change. When it does, we call these fairness violation instances individual discriminatory instances (IDI). There are several methods (Angell et al., 2018; Udeshi et al., 2018; Aggarwal et al., 2019; Fan et al., 2022; Zhao et al., 2024) to find these discriminatory instances. For data-driven DPuTs, e.g., DNNs, we also use the Themis (Angell et al., 2018) implementation by Zhao et al. (Zhao et al., 2024), ExpGA (Fan et al., 2022), and LIMI (Xiao et al., 2023) to identify and merge these IDIs. Here, based on the XorigX_orig, YorigY_orig, DPuT, and the corresponding IDI search, we can get their corresponding counterfactual input/output XcfX_cf, YcfY_cf (Line 2-5 in Algorithm 1). 4.2. Curating Relational Dataset From the DPuT programs (symbolic or black-box), original instances, and counterfactuals (and IDIs), we get two types of pairs in combination with original (XorigX_orig, YorigY_orig) and counterfactual (XcfX_cf, YcfY_cf) data points. One set of inputs or data points, even when their protected attributes have been flipped (from privileged to unprivileged or vice versa), both get the same target outcome from the DPuT. We refer to them as fair (or agreement) pairs and assign the relational label ++ to them. Another set of inputs or data points, where their protected attributes have been flipped, and where both points do not receive the same target outcome (favorable vs unfavorable) from the predictive model or DPuT. We refer to them as discriminatory (or disagreeable) pairs, and assign the relational label −- to them. We store these relational labels (fair vs discriminatory) corresponding to each data in YdiscY_disc (Line 6-8 in Algorithm 1). And here we create our relational dataset RD_R. R=((x1,x1′),L1),((x2,x2′),L2),…,((xn,xn′),Ln)D_R=\((x_1,x _1),L_1),((x_2,x _2),L_2),…,((x_n,x _n),L_n)\ Remi transforms RD_R into a supervised training dataset mSD_m for the classification of discriminatory pairs from the fair pairs using one of the following curation operator m∈DCNE,DCVE,DCHEm∈\DCNE,DCVE,DCHE\. m←(ui,Li)i=1nSD_m←\(u_i,L_i)\_i=1^n 4.2.1. Approach 1: Dataset curation with no extension (DCNE) For DCNE, only the original instance xix_i from each counterfactual pair is retained, annotated with the relational label LiL_i encoding whether the pair (xi,xi′)(x_i,x_i ) produced a fair or discriminatory outcome: DCNE←(xi,Li)i=1nSD_DCNE←\(x_i,L_i)\_i=1^n DCNE preserves relational information through labeling rather than explicit pairing: LiL_i encodes the outcome consistency between xix_i and its counterfactual xi′x_i , so the learned model captures which non-protected feature values trigger sensitivity to the protected attribute, without storing counterfactual features directly. The classification label YorigY_orig is dropped. This avoids introducing potentially unsound feature combinations, which is particularly important for black-box DPuTs where counterfactual validity cannot be verified. Algorithm 1 Remi Discrimination identification and verification Input: Program under test DPuT, Dataset XorigX_orig, Curation apprappr: ‘DCNE’, ‘DCVE’, or ‘DCHE’ Output: mitigation rules ℛR, Deferred or fair prediction ypredy_pred 1 YorigY_orig ← getOutcome(XorigX_orig, DPuT) 2 for i←1i← 1 to |Xorig||X_orig| do 3 xi′←CF⁡(xi)x _i (x_i) ; // Get corresponding counterfactual input 4 5 yi′←f⁡(xi′)y _i← f(x _i) 6 XcfX_cf.add(xi′x _i), YcfY_cf.add(yi′y _i) 7 foreach (yi,yi′)∈(Yorig,Ycf)(y_i,y _i)∈(Y_orig,Y_cf) do 8 Li←[yi≠yi′]L_i 1[y_i≠ y _i] 9 YdiscY_disc.add(LiL_i) ; // Add aggreement/disagreement relational labels 10 11 if apprappr == DCNE then 12 DCNE←(xi,Li)i=1nSD_ [rgb]0,0,0DCNE←\(x_i,L_i)\_i=1^n 13 else if apprappr == DCVE then 14 DCVE←(xi,Li),(xi′,Li)i=1nSD_ [rgb]0,0,0DCVE←\(x_i,L_i),(x _i,L_i)\_i=1^n 15 else if apprappr == DCHE then 16 DCHE←(h⁡(xi,xi′),Li)i=1nSD_ [rgb]0,0,0DCHE←\(h(x_i,x _i),L_i)\_i=1^n 17 ℰ,M←ℐℳtrain(Sappr,depthmax,Leafmax)E,M _train(S_appr,depth_ ,Leaf_ ); 18 foreach leaf λ in ℰE do 19 if isDiscriminatoryLeaf(λ) then 20 π←extractRuleForLeaf(λ)π← extractRuleForLeaf(λ) 21 ℛ.⁡(π)R. add(π) ; 22 debugAndGuardRail(ℛR, M) ; // Using rules/explanation to guard rail the model 23 foreach xpredx_pred in XpredX_pred do 24 if doesFallUnderDiscriminatoryRule(ℛR,xpredx_pred) then 25 Deny output 26 else 27 ypred←DPuT(xpred)y_pred← DPuT(x_pred) 28 return (ℛR, ypredy_pred) 4.2.2. Approach 2: Dataset Curation With Vertical Extension (DCVE) In this curation technique, both sets of input from each pair (xi,xi′)(x_i,x _i) are used, and the classification label is determined based on their relational label LiL_i (whether they are from a fair or unfair pair). But each element of the pair (original xix_i and counterfactual i.e., xi′x _i) is treated as a separate data point, and their corresponding relational labels are LiL_i. So we have one pair ((xi,Li)(x_i,L_i)) and another pair ((xi′,Li)(x _i,L_i)) In this way our dataset size doubles in terms of rows. The intuition behind this form of training is that if we add more instances from both fair and unfair pairs, and a better pattern emerges in the classification boundary, it will help the interpretable model distinguish unfair pairs from fair ones. DCVE←(xi,Li),(xi′,Li)i=1nSD_DCVE←\(x_i,L_i),(x _i,L_i)\_i=1^n where size of the dataset is NDCVE=2×nN_DCVE=2× n 4.2.3. Approach 3: Dataset Curation With Horizontal Extension (DCHE) For this curation technique, we also take information from both set of input from each pair (xi,xi′)(x_i,x _i), but they are merged into a single row. We add columns from both sources (xi∈Xorigx_i∈ X_orig and xi′∈Xcfx _i∈ X_cf) as training features which has different values, dropping one of the columns that has the same values in both sources. Let C=|xi|C=|x_i|, where C is the number of features we have, for xix_i. We can write xi=(vi,1,…,vi,C)x_i=(v_i,1,...,v_i,C) and xi′=(vi,1′,…,vi,C′)x _i=(v _i,1,...,v _i,C), where vi,1v_i,1 represent the first feature of the xix_i instance. And we can have a mask mi,k=1[vi,k≠vi,k′]∈0,1m_i,k=1[v_i,k≠ v _i,k]∈\0,1\ to drop the features from xi′x _i which are the same as xi′x _i. If we denote the DCHE horizontal feature set as h⁡(⋅)h(·), then h⁡(xi,xi′)=(vi,1,…,vi,C,mi,1∗vi,1′,…,mi,C∗vi,C′)h(x_i,x _i)=(v_i,1,...,v_i,C,m_i,1*v _i,1,...,m_i,C*v _i,C) For mi,k=1m_i,k=1, it will add the counterfactual values as they differ, and for mi,k=0m_i,k=0, it will drop the identical features. DCHE←(h⁡(xi,xi′),Li)i=1nSD_DCHE←$h(x_i,x _i),L_i)\_i=1^n And the classification label for each instance is its corresponding relation label LiL_i. So, we get with relational dataset DCHESD_DCHE. The intuition behind this form of training is that for each point, we add more information, so that in terms of both original and counterfactual values, the models get more intuition about where the fairness bugs are mostly located. 4.3. Interpretable Learning and Synthesizing the explanation Given a curated relational dataset SDmSD_m (where m∈DCNE,DCVE,DCHEm∈\DCNE,DCVE,DCHE$, we train an interpretable model ℳM (e.g., a decision tree) to classify fair versus unfair relational pairs (Alg. 1, Line 15). Interpretability is essential to capture the feature interactions that localize discriminatory regions within the DPuT. Following AFT (Zhao et al., 2024), we constrain the model’s complexity by limiting the number of leaves. We evaluate ℳM using standard metrics—Accuracy, Precision, Recall, and ROC-AUC—where True Positives (TPTP) represent correctly identified IDIs, True Negatives TNTN stands for correctly identified non-IDI, and accuracy measures the fraction of all data points (both fair and unfair) that are correctly classified. Impurity metric depends on the specific tree learner e.g. ID3 and C4.5 algorithms use cross-entropy, whereas CART decision trees use Gini impurity. Once ℳM achieves sufficient discriminative performance, we extract the path predicates leading to leaves λ classified as discriminatory. These paths form a set of explanatory rules ℛ=ϕπR=\ _π\ that localize the specific attribute ranges where discrimination occurs (Alg. 1, Lines 16–19). We validate each rule ϕπ _π by subsetting the relational dataset using the rule’s conjunction of predicates: SDm[ϕπ]=(ui,Li)∈SDm∣ϕπ(ui)=trueSD_m[ _π]=\(u_i,L_i)∈ SD_m _π(u_i)=true\ For each rule, we compute a few key performance metrics, e.g., impurity, ConfidenceleafConfidence_leaf, and CoverageleafCoverage_leaf, to assess the strength of each rule. ConfidenceleafConfidence_leaf describes the precision of the corresponding rule, i.e., the fraction of instances captured by the rule π that are IDI cases. Confidenceleaf(π)=TPleafTPleaf+FPleafConfidence_leaf(π)= TP_leafTP_leaf+FP_leaf Here TPleafTP_leaf stands for the number of actual IDIs that fall under this leaf, and FPleafFP_leaf stands for the number of points that were classified as IDI, but originally was not IDIs. Coverageleaf=TPleafTotal #IDICoverage_leaf= TP_leafTotal \#IDI CoverageleafCoverage_leaf metric finds the fraction of all true discriminatory instances that fall within leaf-rule π. 4.4. Mitigation via Guardrail With the extracted rules we have, we add them beside our DPuTs as guardrails to deny output in those stages. When an input is sent to DpuT, instead of sending it directly for prediction, we at first see if the input falls under any of the extracted discriminatory rules (ℛR). If so we refuse to pass it to our DPuT for a prediction and deny output for those datapoints (Alg. 1, Lines 20–26). The idea is to abstain from making a decision that could lead to discriminatory behavior. 5. Experiments Benchmarks. We test our framework Remi on both symbolic and data-driven programs. Table 1. The dataset and benchmark (symbolic DPuT, left; DNN, right). DPuT Dataset Prot. Feat #Inst. #Feat. Source Pg1 (Albarghouthi et al., 2017) ethnicity 30000 4 (Albarghouthi et al., 2017) Pg2 (ProPublica, 2021; Zhong et al., 2023; Larson et al., 2016) sex 6908 4 (Rudin, 2019) Pg3 (Albarghouthi et al., 2017) ethnicity 30000 4 (Zhang et al., 2022) Pg4 (Albarghouthi et al., 2017) ethnicity 30000 4 (Zhang et al., 2022) Pg5 (ProPublica, 2021; Zhong et al., 2023; Larson et al., 2016) age 6908 4 (Rudin and Ustun, 2018) Pg6 (ProPublica, 2021; Zhong et al., 2023; Larson et al., 2016) age 6908 5 (Rudin and Ustun, 2018) Dataset Prot. Feat #Inst. #Feat. DNN Source #Layers #Neurons Acc (%) Adult Census 32,561 13 AC1 (, 2024) 4 45 85.24 AC2 (, 2024), (Fan et al., 2022; Udeshi et al., 2018) 3 121 84.70 AC3 (, 2024) 3 71 84.52 Sex AC4 (, 2024) 4 221 84.86 Race AC5 (, 2024) 4 149 85.19 Age AC6 (, 2024) 4 45 84.77 AC7 (Zhang et al., 2020) 7 145 84.85 AC8 (Urban et al., 2020; Mazzucato and Urban, 2021) 4 10 82.15 AC9 (Urban et al., 2020; Mazzucato and Urban, 2021) 6 12 81.22 AC10 (Urban et al., 2020; Mazzucato and Urban, 2021) 6 20 78.56 AC11 (Urban et al., 2020; Mazzucato and Urban, 2021) 6 40 79.25 AC12 (Urban et al., 2020; Mazzucato and Urban, 2021) 11 45 81.46 Bank Marketing Age 45,211 16 BM1 (, 2024) 4 97 89.20 BM2 (, 2024) 4 65 88.76 BM3 (, 2024), (Fan et al., 2022; Udeshi et al., 2018) 3 117 88.22 BM4 (, 2024) 5 318 89.55 BM5 (, 2024) 4 49 88.90 BM6 (, 2024) 4 35 88.94 BM7 (, 2024) 4 145 88.70 BM8 (Zhang et al., 2020) 7 141 89.20 Symbolic DPuT Benchmarks (PG1–PG6): We evaluate our approach on six symbolic DPuTs (PG1-PG6) where the internal decision logic is known, allowing us to validate root-cause explanations against ground truth. Datasets for PG1, PG3, and PG4 are created following the program listed in Listing 1, following the work of Albarghouthi et al. (Albarghouthi et al., 2017). For PG2, PG5, PG6, we use the Compas (ProPublica, 2021) dataset, we inherit the preprocessed version of the data from (Zhong et al., 2023; Larson et al., 2016). Table 1 (left) shows the details. We evaluate our approach using six symbolic programs (PG1–PG6) representing diverse discriminatory scenarios. PG1, PG3, and PG4 model hiring decisions; PG1 (Albarghouthi et al., 2017) involves indirect discrimination via a proxy (college rank), while PG3 and its supposedly repaired variant PG4 (Zhang et al., 2022) exhibit distinct decision logics. PG2 (Rudin, 2019) explicitly incorporates a protected attribute (sex) alongside societal bias proxies like arrest records. Finally, PG5 and PG6 (Rudin and Ustun, 2018) utilize additive scoring systems where age and prior arrests influence high-risk classification thresholds (see detailed code and logic in (Akash et al., 2026)). Neural DPuT Benchmarks. To assess the generalization of our approach to black-box models, we employ 20 Deep Neural Networks (DNNs) curated by Biswas et al. (Biswas and Rajan, 2023) from literature (Mazzucato and Urban, 2021; Urban et al., 2020; Zhang et al., 2020; Udeshi et al., 2018; Fan et al., 2022) and Kaggle (, 2024). These black-box models encompass various feed-forward architectures with ReLU activations. Table 1 (right) shows the characteristics of benchmarks. • AC1–AC12: Trained on the Adult Census dataset (32,561 records, 13 attributes) to predict if annual income exceeds $50,000 (Dua and Graff, 2017). • BM1–BM8: Trained on the Bank Marketing dataset (45,211 entries, 16 features) to predict term deposit subscriptions for a Portuguese bank (Moro et al., 2014). Technical Details. Remi was implemented in Python v3.8.20, tensorflow v2.13.0, and scikit-learn v1.3.1 We run all our experiments on an Ubuntu 22.04.5 LTS (jammy) served by an instance type of c5ad.2xlarge (4 cores, 8 vCPUs, 0 GPUs) in Amazon Web Services (AWS) Elastic Computing (EC2). We repeat our experiments 10 times and then report the mean and standard deviation. For RQ5, we run the Themis (Angell et al., 2018), ExpGA (Fan et al., 2022), etc., for 60 minutes to find ID instances in the DNN models. Hyperparameter Selection. For decision trees with CART, following AFT (Zhao et al., 2024), we also set the maximum leaf nodes parameter to 1000 for decision trees. For other models, we follow the same settings. Common Comparison Metrics: Throughout our experiment section, we use the following metrics. The column labeled AccAcc refers to the accuracy of the model, PrecPrec refers to the precision, RecRec refers to the recall, F1F1 refers to the F1 score for classification, which is a harmonic mean of precision and recall, ROCAUCROCAUC is the Area under the curve (AUC) of the receiver-operating characteristic (ROC) score. The other relevant metrics will described accordingly in the relevant RQs. Baseline Selection and Scope. We select baselines that represent distinct paradigms in fairness testing and mitigation. AFT (Zhao et al., 2024) is the most directly comparable baseline for the explanation task, as it also employs a surrogate decision tree to identify discriminatory input regions. AFT iteratively generates random inputs within valid bounds, trains decision trees on the DPuT’s classification labels, and extracts path pairs to guide IDI discovery. The key distinction is that AFT trains its trees on the original classification task, whereas Remi trains on relational (fair vs. unfair) labels derived from counterfactual pairs. We compare both tools on their ability to accurately localize discriminatory input regions, using both symbolic and DNN benchmarks. ExpGA (Fan et al., 2022) uses explanation-guided genetic algorithms to generate IDIs efficiently in a black-box setting. LIMI (Xiao et al., 2023) generates realistic IDIs via latent-space imitation learning. Both tools focus on point-wise IDI discovery and mitigate unfairness through counterfactual data augmentation and retraining — they do not extract interpretable regional invariants, making them unsuitable as explanation baselines. We therefore compare ExpGA and LIMI against Remi on the mitigation task only, using their discovered IDIs as input to Remi’s relational pipeline. We exclude Themis, ExpGA, and LIMI from symbolic benchmark comparisons, as these tools are designed for black-box models. In this research work, we approach the following research questions: RQ1 Which relational dataset curation technique leads to accurate and precise fairness invariants? RQ2 Which interpretable algorithms is the best at extracting discriminatory regions? RQ3 What are the characteristics of extracted fairness invariants? RQ4 Can we utilize the extracted discriminatory rules as guardrails for mitigating unfairness? RQ5 Does Remi generalize to address individual discrimination in black-box deep neural networks? 5.1. RQ1. Effectiveness of Relational Dataset Curation Table 2. Comparison between different data pre-processing training approaches Prog Appr Depth Acc Prec Rec F1 ROC AUC TrainT_Train #IDILoc\#IDI_Loc Pg1 (SMB) most-freq NA 0.92 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0.0 (± 0.0) AFT (Zhao et al., 2024) 11.2 (± 1.14) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.32 (± 0.02) 0.0 (± 0.0) DCNE 18.0 (± 0.0) 1.0 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 0.38 (± 0.01) 2233.5 (± 1.18) DCVE 30.0 (± 0.0) 0.99 (± 0.0) 0.95 (± 0.0) 0.96 (± 0.0) 0.95 (± 0.0) 0.98 (± 0.0) 4.17 (± 0.05) 4329.6 (± 2.32) DCHE 17.0 (± 0.0) 1.0 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 0.37 (± 0.01) 2231.5 (± 1.65) Pg2 (SMB) most-freq NA 0.97 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0.0 (± 0.0) AFT (Zhao et al., 2024) 4.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.07 (± 0.02) 100.0 (± 0.0) DCNE 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.02 (± 0.0) 206.0 (± 0.0) DCVE 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.02 (± 0.0) 412.0 (± 0.0) DCHE 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.02 (± 0.0) 206.0 (± 0.0) Pg3 (SMB) most-freq NA 0.97 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0.0 (± 0.0) AFT (Zhao et al., 2024) 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.25 (± 0.02) 0.0 (± 0.0) DCNE 13.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.07 (± 0.0) 965.5 (± 1.58) DCVE 24.0 (± 0.0) 1.0 (± 0.0) 0.97 (± 0.0) 0.97 (± 0.0) 0.97 (± 0.0) 0.98 (± 0.0) 0.94 (± 0.02) 1872.4 (± 2.27) DCHE 3.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.05 (± 0.0) 968.0 (± 0.0) Pg4 (SMB) most-freq NA 0.98 (±0.0± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0 (± 0.0) AFT (Zhao et al., 2024) 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.25 (± 0.02) 0.0 (± 0.0) DCNE 11.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.06 (± 0.0) 580.1 (± 1.45) DCVE 21.0 (± 0.0) 1.0 (± 0.0) 0.97 (± 0.0) 0.98 (± 0.0) 0.98 (± 0.0) 0.99 (± 0.0) 0.5 (± 0.01) 1141.8 (± 1.55) DCHE 3.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.04 (± 0.0) 582.0 (± 0.0) Pg5 (Scr) most-freq NA 0.52 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0 (± 0.0) AFT (Zhao et al., 2024) 26.3 (± 2.0) 0.75 (± 0.01) 0.77 (± 0.02) 0.76 (± 0.03) 0.77 (± 0.01) 0.75 (± 0.01) 3.85 (± 0.08) 1178.5 (± 152.92) DCNE 21.0 (± 0.0) 0.63 (± 0.0) 0.64 (± 0.0) 0.53 (± 0.0) 0.58 (± 0.0) 0.63 (± 0.0) 2.61 (± 0.02) 1732.0 (± 0.0) DCVE 21.0 (± 0.0) 0.62 (± 0.0) 0.61 (± 0.0) 0.52 (± 0.0) 0.56 (± 0.0) 0.61 (± 0.0) 3.14 (± 0.03) 3448.2 (± 1.14) DCHE 21.0 (± 0.0) 0.63 (± 0.0) 0.64 (± 0.0) 0.53 (± 0.0) 0.58 (± 0.0) 0.63 (± 0.0) 2.6 (± 0.03) 1732.0 (± 0.0) Pg6 (Scr) most-freq NA 0.68 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) NA 0 (± 0.0) AFT (Zhao et al., 2024) 3.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.08 (± 0.01) 100.0 (± 0.0) DCNE 6.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 2216.0 (± 0.0) DCVE 5.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 4432.0 (± 0.0) DCHE 6.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 2216.0 (± 0.0) Baseline. We consider two approaches: i) the most frequent labels and i) AFT (Zhao et al., 2024) as our baseline for localization of discrimination. We choose AFT (Zhao et al., 2024) because it is a state-of-the-art individual fairness testing technique that uses a decision tree as a surrogate to infer the individual discriminatory regions. Specifically, it uses decision tree path pairs to decide if a subspace is promising for generating individual discriminatory instances. The key difference remains that the AFT does not apply a relational input alignment in synthesizing decision trees. Also, for AFT, instead of multiple iterations of training the decision tree to extract paths for generating tests for individual discrimination (IDs), we stop after one, because our proposed approach trains the interpretable decision tree only once with the corresponding data-curation technique To determine which relational dataset curation approach is most effective in localizing discrimination, we run experiments with the CART (Breiman et al., 2000) (decision tree) interpretable model to compare their efficacy. The results are summarized in Table 2. On the left side of the table, ProgProg lists relevant DPuTs described in the benchmark. TrainT_Train refers to the time (in seconds) taken to complete the training, #IDILoc\#IDI_Loc refers to the number of individual discriminatory instances (IDI) localized by the approach. Due to the class imbalance, PrecPrec, RecRec, F1F1, ROC AUC, and #IDILoc\#IDI_Loc are terms that are more important to focus on rather than accuracy. The most important evaluation metric here is #IDLoc\#ID_Loc and PrecPrec. From Table 2, we find that our three proposed data-curation techniques localize the largest number of IDIs around 83% of the time, compared to the considered baselines. In more than 80% of cases, we achieve very good performance metrics for accuracy, precision, recall, F1, and ROC for localizing IDI instances. Although in one case (Pg5), we can see AFT (Zhao et al., 2024) performs better than the proposed curation techniques. AFT identifies more IDIs in PG5, but these AFT-identified IDIs do not necessarily overlap with those considered by the other data curation techniques, due to the nature of the AFT algorithm. At each iteration, AFT generates random data within the valid input bounds to find a possible region of interest to generate the IDIs, which is less deterministic and can also generate unrealistic input. In a few cases, AFT has a better F1 score, but it is worth noting that the decision trees of AFT were trained for an actual classification task of a favorable outcome, whereas our trees were trained for IDIIDI localization, instead of the actual classification tasks. In most cases, the training approaches with DCNE (no extension) and DCHE (horizontal extension) have superior performance in terms of #ID\#ID, accuracy, precision, recall, f1-score, all the time. In terms of the depth of the decision tree, DCHE training needed trees with fewer depths (50% of cases) to localize the discrimination, whereas the other approaches needed trees with higher depths to achieve similar performance. Answer RQ1: Our proposed approaches of creating a relation dataset via pair alignments and training decision tree inference outperform the baselines in 80% of cases. The horizontal and no-extension alignments are similarly effective at accurately explaining individual discrimination. Table 3. Comparison among different interpretable models with training approach with relational dataset. Prog Mod Acc Prec Rec F1 ROC AUC TrT_Tr #IDLoc\#ID_Loc Pg1 SMB CART 1.0 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 0.38 (± 0.01) 2233.5 (± 1.18) GBC 1.0 (± 0.0) 0.98 (± 0.0) 0.98 (± 0.0) 0.98 (± 0.0) 0.99 (± 0.0) 0.89 (± 0.02) 2196.1 (± 9.52) XGBoost 1.0 (± 0.0) 0.98 (± 0.0) 0.99 (± 0.0) 0.98 (± 0.0) 0.99 (± 0.0) 0.15 (± 0.19) 2233.0 (± 0.0) RuleFit 0.98 (± 0.0) 0.93 (± 0.01) 0.8 (± 0.01) 0.86 (± 0.01) 0.9 (± 0.01) 64.75 (± 1.45) 1807.7 (± 27.51) FIGS 0.95 (± 0.01) 0.61 (± 0.06) 0.84 (± 0.05) 0.7 (± 0.03) 0.9 (± 0.02) 0.61 (± 0.01) 1898.8 (± 110.81) C4.5Tree 1.0 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 20.36 (± 1.33) 2232.0 (± 4.06) TaoTree 0.96 (± 0.0) 0.69 (± 0.01) 0.82 (± 0.04) 0.75 (± 0.01) 0.89 (± 0.02) 2.35 (± 0.56) 1839.4 (± 93.1) OneR 0.93 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) 0.26 (± 0.01) 0.0 (± 0.0) GreedyRuleList 0.93 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) 0.14 (± 0.0) 0.0 (± 0.0) Pg2 SMB CART 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.02 (± 0.0) 206.0 (± 0.0) GBC 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.05 (± 0.0) 206.0 (± 0.0) XGBoost 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 206.0 (± 0.0) RuleFit 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 4.97 (± 1.38) 206.0 (± 0.0) FIGS 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.04 (± 0.01) 206.0 (± 0.0) C4.5Tree 1.0 (± 0.0) 1.0(± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.21 (± 0.02) 206.0 (± 0.0) TaoTree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.07 (± 0.0) 206.0 (± 0.0) OneR 1.0 (± 0.0) 0.94 (± 0.0) 1.0 (± 0.0) 0.97 (± 0.0) 1.0 (± 0.0) 0.05 (± 0.0) 206.0 (± 0.0) GreedyRuleList 1.0 (± 0.0) 0.94 (± 0.0) 1.0 (± 0.0) 0.97 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 206.0 (± 0.0) Pg3 SMB CART 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.07 (± 0.0) 965.5 (± 1.58) GBC 1.0 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.29 (± 0.0) 967.3 (± 0.67) XGBoost 1.0 (± 0.0) 0.98 (± 0.0) 1.0 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 0.09 (± 0.03) 967.0 (± 0.0) RuleFit 1.0 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 37.19 (± 2.15) 967.6 (± 1.26) FIGS 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.54 (± 0.08) 966.3 (± 1.25) C4.5Tree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 5.61 (± 0.02) 966.2 (± 1.48) TaoTree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.48 (± 0.02) 966.5 (± 1.08) OneR 0.97 (± 0.0) 0.53 (± 0.0) 1.0 (± 0.0) 0.69 (± 0.0) 0.98 (± 0.0) 0.16 (± 0.0) 968.0 (± 0.0) GreedyRuleList 0.97 (± 0.0) 0.53 (± 0.0) 1.0 (± 0.0) 0.69 (± 0.0) 0.98 (± 0.0) 0.08 (± 0.0) 968.0 (± 0.0) Pg4 SMB CART 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.06 (± 0.0) 580.1 (± 1.45) GBC 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.26 (± 0.01) 581.3 (± 0.82) XGBoost 1.0 (± 0.0) 0.97 (± 0.0) 1.0 (± 0.0) 0.98 (± 0.0) 1.0 (± 0.0) 0.08 (± 0.01) 580.0 (± 0.0) RuleFit 1.0 (± 0.0) 0.99 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 42.1 (± 3.0) 581.8 (± 0.42) FIGS 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.44 (± 0.05) 580.4 (± 1.17) C4.5Tree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 5.52 (± 0.03) 580.9 (± 1.29) TaoTree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.4 (± 0.02) 580.8 (± 1.81) OneR 0.98 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) 0.16 (± 0.0) 0.0 (± 0.0) GreedyRuleList 0.98 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 0.5 (± 0.0) 0.09 (± 0.01) 0.0 (± 0.0) Pg5 Scr CART 0.63 (± 0.0) 0.64 (± 0.0) 0.53 (± 0.0) 0.58 (± 0.0) 0.63 (± 0.0) 2.61 (± 0.02) 1732.0 (± 0.0) GBC 0.6 (± 0.0) 0.61 (± 0.01) 0.47 (± 0.03) 0.53 (± 0.01) 0.6 (± 0.0) 0.18 (± 0.0) 1531.6 (± 87.62) XGBoost 0.61 (± 0.0) 0.61 (± 0.0) 0.49 (± 0.0) 0.55 (± 0.0) 0.61 (± 0.0) 0.04 (± 0.01) 1624.0 (± 0.0) RuleFit 0.53 (± 0.0) 0.58 (± 0.21) 0.02 (± 0.01) 0.03 (± 0.02) 0.5 (± 0.0) 6.36 (± 0.64) 54.8 (± 43.07) FIGS 0.63 (± 0.0) 0.63 (± 0.01) 0.54 (± 0.03) 0.58 (± 0.01) 0.63 (± 0.0) 109.12 (± 8.8) 1785.7 (± 113.57) C4.5Tree 0.63 (± 0.0) 0.65 (± 0.01) 0.5 (± 0.02) 0.57 (± 0.01) 0.63 (± 0.0) 1.75 (± 0.06) 1646.2 (± 61.05) TaoTree 0.63 (± 0.0) 0.65 (± 0.01) 0.49 (± 0.02) 0.56 (± 0.01) 0.63 (± 0.0) 4.83 (± 0.14) 1617.5 (± 71.01) OneR 0.55 (± 0.0) 0.52 (± 0.0) 0.57 (± 0.01) 0.54 (± 0.01) 0.55 (± 0.0) 0.07 (± 0.0) 1857.6 (± 35.63) GreedyRuleList 0.55 (± 0.0) 0.52 (± 0.0) 0.52 (± 0.1) 0.52 (± 0.06) 0.54 (± 0.01) 0.04 (± 0.01) 1698.3 (± 317.8) Pg6 Scr CART 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.03 (± 0.0) 2216.0 (± 0.0) GBC 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.09 (± 0.0) 2215.6 (± 0.52) XGBoost 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.04 (± 0.01) 2215.0 (± 0.0) RuleFit 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 4.46 (± 0.38) 2215.7 (± 0.48) FIGS 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.14 (± 0.02) 2215.9 (± 0.32) C4.5Tree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.53 (± 0.03) 2215.8 (± 0.42) TaoTree 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 0.22 (± 0.01) 2215.7 (± 0.48) OneR 0.81 (± 0.0) 0.63 (± 0.0) 1.0 (± 0.0) 0.78 (± 0.0) 0.86 (± 0.0) 0.06 (± 0.0) 2216.0 (± 0.0) GreedyRuleList 0.81 (± 0.0) 0.63 (± 0.0) 1.0 (± 0.0) 0.78 (± 0.0) 0.86 (± 0.0) 0.04 (± 0.0) 2216.0 (± 0.0) 5.2. RQ2. Performance of Interpretable Algorithms in Inferring Fairness Invariants We evaluate nine tree-based algorithms to identify the optimal learner for fairness invariant synthesis: CART (Breiman et al., 2000), GBC (Friedman, 2001), XGB (Chen and Guestrin, 2016), C4.5 (Quinlan, 1993), TaoTree (Carreira-Perpinan and Tavallali, 2018), RuleFit (Friedman and Popescu, 2008), FIGS (Tan et al., 2025), OneR (Holte, 1993), and GreedyRuleList (, 2024). Table 3 summarizes the training time (TrT_Tr) and the number of localized IDIs (#IDILoc\#IDI_Loc) across symbolic DPuTs. For PG1, C4.5, CART, and XGB (using DCNE) localized over 2,230 IDIs. While XGB was the fastest, its ensemble nature limits direct interpretability. In PG3 and PG4, all major learners showed comparable performance; however, FIGS and TaoTree provided the shortest training times. RuleFit achieved slightly higher localization in PG3, but at the cost of training latency and precision. In PG5, which exhibits low separability due to its probabilistic Bernoulli logic, OneR localized the most IDIs (1,857) but with the lowest precision. FIGS and CART provided a superior balance of precision and localization. For PG2 and PG6, simpler decision logic and smaller datasets led to high performance across all algorithms. Overall, C4.5, FIGS, and CART emerged as the top performers, outperforming other methods in 83% of cases. While XGB and GBC are computationally efficient, they lack the inherent interpretability required for precise bug localization. Conversely, OneR and GreedyRuleList frequently underperformed, resulting in low precision and F1 scores. Answer RQ2: C4.5, FIGS, and CART are the most effective for precisely inferring fairness invariants, achieving top-tier performance in 83% of benchmarks. Table 4. Comparison between the strength of the extracted rules of using proposed relational dataset (RQ3) Prog Appr Depth Rule.LenRule.Len Imp Conf Tot.Disc.Cov Tot.#IDILoc\#IDI_Loc Bst.Lf.Rule.Len Bst.Lf.Disc.Cov Bst.Lf.Conf Bst.Lf.#IDILoc\#IDI_Loc Pg1 SMB DCNE 18.0 (± 0.0) 11.56 (± 2.77) 0.0 (± 0.0) 0.97 (± 0.09) 0.98 (± 0.0) 2233.5 (± 1.18) 16.0 (± 0.0) 0.13 (± 0.0) 1.0 (± 0.0) 301.0 (± 0.0) DCHE 17.0 (± 0.0) 10.67 (± 2.69) 0.0 (± 0.0) 0.98 (± 0.09) 0.98 (± 0.0) 2231.5 (± 1.65) 15.0 (± 0.0) 0.13 (± 0.0) 1.0 (± 0.0) 301.0 (± 0.0) AFT (Zhao et al., 2024) 11.0 (± 0.0) 7.23 (± 1.79) 0.0 (± 0.0) 1.0 (± 0.0) 0.0 (± 0.0) 0.0 (± 0.0) 3.4 (± 0.52) 0 (± 0.0) 1.0 (± 0.0) 0 (± 0.0) Pg2 SMB DCNE 2.0 (± 0.0) 2.0 (± 0.0) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 206.0 (± 0.0) 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 206.0 (± 0.0) DCHE 2.0 (± 0.0) 2.0 (± 0.0) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 206.0 (± 0.0) 2.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 206.0 (± 0.0) AFT (Zhao et al., 2024) 4.0 (± 0.0) 2.67 (± 1.53) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 100.0 (± 0.0) 4 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 100.0 (± 0.0) Pg3 SMB DCNE 13.0 (± 0.0) 9.4 (± 2.5) 0.0 (± 0.0) 0.99 (± 0.01) 0.99 (± 0.1) 965.5 (± 1.58) 5.0 (± 0.0) 0.89 (± 0.0) 1.0 (± 0.0) 863.0 (± 0.0) DCHE 3.0 (± 0.0) 3.0 (± 0.0) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 968.0 (± 0.0) 3.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 968.0 (± 0.0) AFT (Zhao et al., 2024) 2.0 (± 0.0) 1.5 (± 0.71) 0.0 (± 0.0) 1.0 (± 0.0) 0.0 (± 0.0) 0 (± 0.0) 1 (± 0.0) 0 (± 0.0) 1.0 (± 0.0) 0 (± 0.0) Pg4 (SMB) DCNE 11.0 (± 0.0) 8.43 (± 2.07) 0.0 (± 0.0) 1.0 (± 0.01) 0.99 (± 0.0) 580.1 (± 1.45) 5.0 (± 0.0) 0.89 (± 0.0) 1.0 (± 0.0) 516.0 (± 0.0) DCHE 3.0 (± 0.0) 3.0 (± 0.0) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 582.0 (± 0.0) 3.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 582.0 (± 0.0) AFT (Zhao et al., 2024) 2.0 (± 0.0) 1.5 (± 0.71) 0.0 (± 0.0) 1.0 (± 0.0) 0 (± 0.0) 0 (± 0.0) 1 (± 0.0) 0 (± 0.0) 1.0 (± 0.0) 0.0 (± 0.0) Pg5 (Scr) DCNE 21.0 (± 0.0) 12.75 (± 3.47) 0.2 (± 0.23) 0.77 (± 0.2) 0.54 (± 0.0) 1732.0 (± 0.0) 9.0 (± 0.0) 0.02 (± 0.0) 0.5 (± 0.0) 63.0 (± 0.0) DCHE 21.0 (± 0.0) 12.79 (± 3.49) 0.2 (± 0.23) 0.77 (± 0.2) 0.54 (± 0.0) 1732.0 (± 0.0) 9.0 (± 0.0) 0.02 (± 0.0) 0.5 (± 0.0) 63.0 (± 0.0) AFT (Zhao et al., 2024) 25.0 (± 0.0) 13.61 (± 3.7) 0.27 (± 0.41) 0.9 (± 0.15) 0.9 (± 0.0) 1843 (± 0.0) 4 (± 0.0) 0.49 (± 0.0) 1.0 (± 0.0) 999 (± 0.0) Pg6 (Scr) DCNE 6.0 (± 0.0) 4.6 (± 1.14) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 2216.0 (± 0.0) 3.0 (± 0.0) 0.62 (± 0.0) 1.0 (± 0.0) 1378.0 (± 0.0) DCHE 6.0 (± 0.0) 4.6 (± 1.14) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 2216.0 (± 0.0) 3.0 (± 0.0) 0.62 (± 0.0) 1.0 (± 0.0) 1378.0 (± 0.0) AFT (Zhao et al., 2024) 3.0 (± 0.0) 2.0 (± 0.0) 0.0 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 100.0 (± 0.0) 3 (± 0.0) 1.0 (± 0.0) 1.0 (± 0.0) 100.0 (± 0.0) 5.3. RQ3. Various Characteristics of the Relational Explanation Models. For symbolic programs where the root cause of discrimination is verifiable, we measure how closely Remi’s extracted predicates align with the actual discriminatory conditions. Table 4 summarizes these findings using the following metrics: • Rule Length (Rul.LenRul.Len): Average number of predicates in rules reaching a discriminatory leaf. • Impurity (ImpImp): Average node impurity across all discriminatory leaves. • Confidence (ConfConf): Average precision of rules in identifying IDIs. • Coverage (Tot.Disc.CovTot.Disc.Cov / Bst.Lf.Disc.CovBst.Lf.Disc.Cov): Total IDIs captured by all rules versus the single most representative leaf rule. In PG1 (Hiring), both DCNE and DCHE trees hit zero impurity and high confidence around 0.97, with Tot.Disc.CovTot.Disc.Cov around 0.98 and locates around 2233 IDIs. The single best leaf rule (out of a total 95 rules) covers around 13% of the discriminatory instances considered (301 out of 2235). Average rule lengths are longer, ranging from 10 to 12 literals, indicating that the unsafe region is a narrow area under several tight predicate conditions. AFT (Zhao et al., 2024) does not find any IDIs for PG1 in the current settings, so we do not have any effective rule to compare with. For PG1, our framework discovers a compact discriminatory rule that matches the discrimination encoded in the DPuT, where the outcomes are based on ethnicity. After simplification, the best leaf rule becomes 14.6 < col-rank ≤ 17.25 ∧ 11.99 < y-exp ≤ 14.84 ∧ cf-ethnicity > 10.5 ∧ 17.85< cf-col-rank ≤ 22.95 which isolates a narrow region of (col-rank, y-exp) space when the model becomes significantly discriminatory with respect to the protected attribute (ethnicity). The rule recovers the root cause of the discrimination lies in a narrow college rank, mid-experience band (14.6< col-rank ≤ 17.25, 11.99 < y-exp ≤ 14.84) for a favorable ethnicity. Now, flipping the ethnicity to an unfavorable group increases the counterfactual college rank (17.85 < cf-col-rank ≤ 22.95) and flips the hiring decision. The rule explicitly captures the ethnicity to college rank to hire pathway, demonstrating the match with the ground truth level root cause of the discrimination of the DPuT PG1. For PG2 (Recidivism), Remi achieves 1.0 confidence and 100% coverage (206 IDIs) with a simple two-literal rule: age ≤ 20.5 ∧ priors-count ≤ 3.5. This matches the ground truth logic regarding gender-based discrimination in young defendants. AFT (Zhao et al., 2024) localizes only 100 IDIs using a similar predicate structure. For PG3, Remi achieves 0.99 confidence and 100% coverage (968 IDIs), identifying unfair behavior when y-exp < 15 and col-rank differ between groups. For PG4 (the repaired variant of the hiring algorithm), Remi maintains 0.99 confidence and 100% coverage (582 IDIs), revealing that discriminatory behavior persists within specific col-rank and y-exp ranges. AFT (Zhao et al., 2024) failed to identify IDIs for either program. In PG5, Remi achieves 0.77 confidence and 53% total coverage (1,732 instances). The extracted rules correctly identify the age bracket (20.5 < age ≤ 21.5) as the primary discriminatory driver. In PG 6, Remi achieves 100% confidence and 100% coverage. A single extracted rule—incorporating age, priors-count, and juv-misd-count—localizes 62% (1,378) of total IDIs. Remi significantly outperforms AFT, which localizes only 100 instances. Overall, Remi precisely characterizes the root cause of discrimination in 83% of cases. In all cases, except PG5, Remi significantly outperforms the state-of-the-art baseline, AFT (Zhao et al., 2024). Specifically, in 50% of cases (PG1, PG2, PG4), AFT fails to find any ID instances, rendering its rules inapplicable. This underscores the necessity of relational explanation, as our approach provides greater applicability where standard IDI search methods fail. Answer RQ3: Remi outperformed the state-of-the-art technique AFT (Zhao et al., 2024) in 83% of cases. In more than 66% of cases, Remi identified all the discrimination regions from the symbolic programs. 5.4. RQ4. Performance of Bias Mitigation via Fairness Invariant Guardrails A common mitigation approach involves retraining the model with individual discrimination instances (IDIs) so it learns to avoid discrimination in local regions (Zhao et al., 2024; Fan et al., 2022; Zhang et al., 2020; Udeshi et al., 2018). However, our framework provides rules that define the discriminatory region and its root causes. These rules act as guardrails: when an input falls within a localized discriminatory region, the system can delay inference for human review. This ensures the model maintains relational consistency and reduces errors in real-time deployment. Table 5. Comparing performance during Mitigation Prog Appr Acc Prec Rec F1 ROC AUC #IDI\#IDI #FPIDI\#FP_IDI Pg1 SMB Ini. Pg 1.00 1.00 1.00 1.00 1.00 2250 - DT w IDI 0.99 0.99 0.99 0.99 0.99 2430 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 2 2670 Pg2 SMB Init. Pg 1.00 1.00 1.00 1.00 1.00 206 - DT w IDI 1.00 1.00 1.00 1.00 1.00 206 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 0 0 Pg3 SMB Init. Pg 1.00 1.00 1.00 1.00 1.00 968 - DT w IDI 1.00 1.00 1.00 1.00 1.00 968 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 0 8 Pg4 SMB Init. Pg 1.00 1.00 1.00 1.00 1.00 582 - DT w IDI 1.00 1.00 1.00 1.00 1.00 582 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 0 5 Pg5 SC Init. Pg 1.00 1.00 1.00 1.00 1.00 3286 - DT w IDI 0.68 0.61 0.79 0.69 0.69 2895 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 1338 1567 Pg6 SC Init. Pg 1.00 1.00 1.00 1.00 1.00 2216 - DT w IDI 1.00 1.00 1.00 1.00 1.00 2216 - Ini.Pg + Remi 1.00 1.00 1.00 1.00 1.00 0 32 Table 5 reports the mitigation results across three program versions, focusing on the number of remaining IDIs (#IDI\#IDI) and the number of non-discriminatory points incorrectly flagged as IDIs (FPIDIFP_IDI). Applying our rules as guardrails reduced #IDI\#IDI in 100% of cases, demonstrating the strong efficacy of the rule-based approach. Performance metrics remained consistently close to those of the original DPuTs. Remi identifies non-discriminatory points as IDIs in a few instances (FPIDI>0FP_IDI>0). This occurs when discriminatory and non-discriminatory points share similar attributes or reside across a narrow boundary that the decision tree’s hyper-rectangular separation struggles to classify. Answer RQ4: The rule-based guardrails reduced the number of individual discriminatory instances from 2,250 to 2 in one case and from hundreds/thousands of instances to 0 in four cases (out of 6). 5.5. RQ5. Generalization of Remi to Black-Box Neural Networks (DNNs). Once we establish the precision and usefulness of Remi relative to verifiable symbolic programs, we apply it to black-box deep neural networks to study the generalization. Specifically, we evaluate (a) whether Remi can perform rule inference, and (b) whether the extracted rules reduce discriminatory decisions while preserving the model’s original classification utility. Table 6. DNN rule extraction results using ExpGA (Fan et al., 2022) as the IDI source under DCNE curation strategy (RQ5). Model Depth Rule.LenRule.Len Tot.RulesTot.Rules Imp Conf Tot.Disc.Cov Tot.#IDILoc\#IDI_Loc Bst.Lf.Rule.Len Bst.Lf.Disc.Cov Bst.Lf.Conf Bst.Lf.#IDILoc\#IDI_Loc AC1 35.0 (± 0.0) 18.77 (± 7.08) 449 0.1 (± 0.17) 0.85 (± 0.18) 0.69 11496 4 0.5 1.0 7793 AC2 24.0 (± 0.0) 14.17 (± 3.98) 467 0.12 (± 0.18) 0.85 (± 0.18) 0.77 15583 7 0.43 1.0 8287 AC3 29.0 (± 0.0) 15.54 (± 4.73) 439 0.11 (± 0.18) 0.83 (± 0.19) 0.78 20531 5 0.66 1.0 16291 AC4 24.0 (± 0.0) 14.45 (± 3.91) 436 0.09 (± 0.16) 0.87 (± 0.17) 0.7 13864 2 0.58 1.0 10738 AC5 30.0 (± 0.0) 15.47 (± 4.51) 464 0.11 (± 0.18) 0.85 (± 0.19) 0.76 18223 5 0.64 1.0 14384 AC6 30.0 (± 0.0) 16.35 (± 4.83) 455 0.1 (± 0.17) 0.85 (± 0.19) 0.53 5333 2 0.23 1.0 2215 AC7 25.0 (± 0.0) 14.65 (± 3.98) 469 0.11 (± 0.17) 0.85 (± 0.18) 0.68 8942 2 0.17 0.99 2095 AC8 30.0 (± 0.0) 14.98 (± 5.05) 459 0.1 (± 0.18) 0.84 (± 0.19) 0.6 7231 9 0.34 1.0 3845 AC9 30.0 (± 0.0) 16.96 (± 4.81) 449 0.11 (± 0.18) 0.83 (± 0.19) 0.41 3729 2 0.1 1.0 851 AC10 24.0 (± 0.0) 14.39 (± 3.68) 441 0.11 (± 0.18) 0.84 (± 0.19) 0.84 23704 5 0.7 1.0 18997 AC11 29.0 (± 0.0) 16.17 (± 4.68) 457 0.11 (± 0.17) 0.85 (± 0.17) 0.8 20979 3 0.66 1.0 16368 AC12 33.0 (± 0.0) 17.37 (± 5.64) 443 0.11 (± 0.18) 0.84 (± 0.18) 0.75 14825 2 0.57 1.0 10649 Localization via Rule Extraction for DNNs. Table 6 reports rule extraction results for the 12 Adult Census DNN benchmarks using ExpGA-generated IDIs (Fan et al., 2022). Remi produces interpretable trees of moderate depth (24–35), yielding 430–500 rules with average lengths of 14–18 conditions. Extracted rules achieve high average confidence (0.83–0.88), indicating reliable mapping between input features and relational fairness labels. Discrimination coverage ranges from 41–84%, confirming that the rules capture a substantial majority of the discovered discriminatory input space. Table 7. Comparison of mitigation techniques using IDIs generated by black-box DNN fairness testing tools. Model Mitigation Acc F1 #IDIbef\#IDI_bef #IDIaft\#IDI_aft Red. (%) #FPIDI\#FP_IDI AC1 Themis (Angell et al., 2018) 0.85 0.66 7,173 7,571 -5.55 – Remi 0.85 0.64 7,173 4,111 42.69 1,323 Themis (Angell et al., 2018) + Remi 0.85 0.66 7,571 4,370 42.28 1,208 ExpGA (Fan et al., 2022) 0.84 0.61 26,647 15,687 41.13 – Remi 0.85 0.64 26,647 4,234 84.11 1,187 ExpGA (Fan et al., 2022) + Remi 0.84 0.61 15,687 4,191 73.28 1,310 LIMI (Xiao et al., 2023) 0.85 0.65 38,049 13,338 64.95 – Remi 0.85 0.64 38,049 4,294 87.84 1,901 LIMI (Xiao et al., 2023)+ Remi 0.85 0.65 13,338 4,515 66.15 1,468 AC2 Themis (Angell et al., 2018) 0.84 0.60 7,532 7,891 -4.77 – Remi 0.85 0.61 7,532 4,140 45.03 1,309 Themis (Angell et al., 2018) + Remi 0.84 0.60 7,891 4,236 46.32 1,259 ExpGA (Fan et al., 2022) 0.77 0.65 27,002 19,148 29.09 – Remi 0.85 0.61 27,002 4,101 84.81 1,377 ExpGA (Fan et al., 2022) + Remi 0.77 0.65 19,148 3,565 81.38 2,497 LIMI (Xiao et al., 2023) 0.82 0.42 29,132 16,940 41.85 – Remi 0.85 0.61 29,132 3,838 86.83 2,179 LIMI (Xiao et al., 2023)+ Remi 0.82 0.42 16,940 3,922 76.85 1,999 AC3 Themis (Angell et al., 2018) 0.85 0.64 7,525 7,094 5.73 – Remi 0.85 0.66 7,525 4,290 42.99 1,244 Themis (Angell et al., 2018) + Remi 0.85 0.64 7,094 4,754 32.99 988 ExpGA (Fan et al., 2022) 0.84 0.62 20,481 24,652 -20.37 – Remi 0.85 0.66 20,481 4,235 79.32 1,317 ExpGA (Fan et al., 2022) + Remi 0.84 0.62 24,652 4,121 83.28 1,495 LIMI (Xiao et al., 2023) 0.85 0.64 46,081 11,014 76.10 – Remi 0.85 0.66 46,081 4,233 90.81 2,478 LIMI (Xiao et al., 2023)+ Remi 0.85 0.64 11,014 4,120 62.59 1,749 AC4 Themis (Angell et al., 2018) 0.84 0.65 7,491 7,999 -6.78 – Remi 0.85 0.62 7,491 3,957 47.18 1,512 Themis (Angell et al., 2018) + Remi 0.84 0.65 7,999 4,151 48.11 1,363 ExpGA (Fan et al., 2022) 0.84 0.66 21,503 18,465 14.13 – Remi 0.85 0.62 21,503 4,094 80.96 1,420 ExpGA (Fan et al., 2022) + Remi 0.84 0.66 18,465 4,601 75.08 1,096 LIMI (Xiao et al., 2023) 0.83 0.67 38,043 34,463 9.41 – Remi 0.85 0.62 38,043 3,683 90.32 2,480 LIMI (Xiao et al., 2023)+ Remi 0.83 0.67 34,463 4,142 87.98 3,008 AC5 Themis (Angell et al., 2018) 0.84 0.65 7,206 7,202 0.06 – Remi 0.85 0.66 7,206 4,054 43.74 1,358 Themis (Angell et al., 2018) + Remi 0.84 0.65 7,202 4,331 39.86 1,229 ExpGA (Fan et al., 2022) 0.83 0.62 21,798 22,543 -3.42 – Remi 0.85 0.66 21,798 4,293 80.31 1,140 ExpGA (Fan et al., 2022) + Remi 0.83 0.62 22,543 4,320 80.84 1,518 LIMI (Xiao et al., 2023) 0.82 0.66 41,114 21,810 46.95 – Remi 0.85 0.66 41,114 4,066 90.11 2,433 LIMI (Xiao et al., 2023)+ Remi 0.82 0.66 21,810 4,244 80.54 2,351 AC6 Themis (Angell et al., 2018) 0.84 0.67 7,559 10,522 -39.20 – Remi 0.85 0.61 7,559 3,851 49.05 1,666 Themis (Angell et al., 2018) + Remi 0.84 0.67 10,522 3,042 71.09 2,495 ExpGA (Fan et al., 2022) 0.85 0.63 25,959 9,502 63.40 – Remi 0.85 0.61 25,959 3,803 85.35 1,665 ExpGA (Fan et al., 2022) + Remi 0.85 0.63 9,502 4,169 56.13 1,249 LIMI (Xiao et al., 2023) 0.85 0.67 34,822 15,302 56.06 – Remi 0.85 0.61 34,822 3,854 88.93 2,206 LIMI (Xiao et al., 2023)+ Remi 0.85 0.67 15,302 4,338 71.65 1,854 BM1 Themis (Angell et al., 2018) 0.88 0.00 3,136 3,788 -20.79 – Remi 0.89 0.48 3,136 1,076 65.69 479 Themis (Angell et al., 2018) + Remi 0.88 0.00 3,788 1,161 69.35 499 BM2 Themis (Angell et al., 2018) 0.88 0.00 3,291 3,638 -10.54 – Remi 0.89 0.54 3,291 1,063 67.70 489 Themis (Angell et al., 2018) + Remi 0.88 0.00 3,638 1,073 70.51 512 BM3 Themis (Angell et al., 2018) 0.88 0.00 3,506 3,573 -1.91 – Remi 0.88 0.58 3,506 1,082 69.14 513 Themis (Angell et al., 2018) + Remi 0.88 0.00 3,573 1,076 69.89 513 BM4 Themis (Angell et al., 2018) 0.23 0.23 3,121 3,445 -10.38 – Remi 0.90 0.53 3,121 1,060 66.04 470 Themis (Angell et al., 2018) + Remi 0.23 0.23 3,445 1,034 69.99 486 Model Mitigation Acc F1 #IDIbef\#IDI_bef #IDIaft\#IDI_aft Red. (%) #FPIDI\#FP_IDI AC7 Themis (Angell et al., 2018) 0.84 0.68 7,445 8,140 -9.34 – Remi 0.85 0.64 7,445 4,058 45.49 1,437 Themis (Angell et al., 2018) + Remi 0.84 0.68 8,140 3,996 50.91 1,826 ExpGA (Fan et al., 2022) 0.76 0.00 23,326 12,311 47.22 – Remi 0.85 0.64 23,326 4,105 82.40 1,425 ExpGA (Fan et al., 2022) + Remi 0.76 0.00 12,311 3,369 72.63 1,842 LIMI (Xiao et al., 2023) 0.76 0.00 37,705 11,208 70.27 – Remi 0.85 0.64 37,705 4,001 89.39 2,419 LIMI (Xiao et al., 2023)+ Remi 0.76 0.00 11,208 3,094 72.39 2,112 AC8 Themis (Angell et al., 2018) 0.84 0.56 7,566 7,689 -1.63 – Remi 0.83 0.66 7,566 4,302 43.14 1,277 Themis (Angell et al., 2018) + Remi 0.84 0.56 7,689 3,927 48.93 1,645 ExpGA (Fan et al., 2022) 0.84 0.64 20,053 11,306 43.62 – Remi 0.83 0.66 20,053 4,411 78.00 1,093 ExpGA (Fan et al., 2022) + Remi 0.84 0.64 11,306 4,075 63.96 1,451 LIMI (Xiao et al., 2023) 0.85 0.67 24,386 17,034 30.15 – Remi 0.83 0.66 24,386 4,188 82.83 2,346 LIMI (Xiao et al., 2023)+ Remi 0.85 0.67 17,034 4,362 74.39 1,733 AC9 Themis (Angell et al., 2018) 0.84 0.68 8,348 8,260 1.05 – Remi 0.83 0.56 8,348 3,642 56.37 1,927 Themis (Angell et al., 2018) + Remi 0.84 0.68 8,260 3,691 55.31 1,980 ExpGA (Fan et al., 2022) 0.84 0.65 20,006 8,120 59.41 – Remi 0.83 0.56 20,006 3,776 81.13 1,727 ExpGA (Fan et al., 2022) + Remi 0.84 0.65 8,120 4,391 45.92 1,168 LIMI (Xiao et al., 2023) 0.83 0.58 51,399 22,330 56.56 – Remi 0.83 0.56 51,399 3,696 92.81 2,379 LIMI (Xiao et al., 2023)+ Remi 0.83 0.58 22,330 4,160 81.37 2,054 AC10 Themis (Angell et al., 2018) 0.85 0.63 7,509 7,250 3.45 – Remi 0.78 0.33 7,509 4,357 41.98 1,164 Themis (Angell et al., 2018) + Remi 0.85 0.63 7,250 4,129 43.05 1,339 ExpGA (Fan et al., 2022) 0.84 0.57 28,342 27,314 3.63 – Remi 0.85 0.66 28,342 4,220 85.11 1,218 ExpGA (Fan et al., 2022) + Remi 0.84 0.57 27,314 3,610 86.78 1,803 LIMI (Xiao et al., 2023) 0.83 0.65 42,942 21,551 49.81 – Remi 0.85 0.66 42,942 4,141 90.36 2,175 LIMI (Xiao et al., 2023)+ Remi 0.83 0.65 21,551 4,512 79.06 1,932 AC11 Themis (Angell et al., 2018) 0.84 0.62 7,808 7,481 4.19 – Remi 0.81 0.67 7,808 4,529 42.00 1,171 Themis (Angell et al., 2018) + Remi 0.84 0.62 7,481 3,957 47.11 1,596 ExpGA (Fan et al., 2022) 0.80 0.63 16,445 24,718 -50.31 – Remi 0.81 0.67 16,445 4,746 71.14 931 ExpGA (Fan et al., 2022) + Remi 0.80 0.63 24,718 3,739 84.87 1,844 LIMI (Xiao et al., 2023) 0.76 0.00 65,598 11,209 82.91 – Remi 0.81 0.67 65,598 3,677 94.39 4,723 LIMI (Xiao et al., 2023)+ Remi 0.76 0.00 11,209 3,264 70.88 1,968 AC12 Themis (Angell et al., 2018) 0.76 0.00 7,667 11,208 -46.18 – Remi 0.84 0.65 7,667 3,972 48.19 1,580 Themis (Angell et al., 2018) + Remi 0.76 0.00 11,208 3,092 72.41 2,115 ExpGA (Fan et al., 2022) 0.83 0.61 23,401 18,645 20.32 – Remi 0.84 0.65 23,401 4,209 82.01 1,432 ExpGA (Fan et al., 2022) + Remi 0.83 0.61 18,645 3,820 79.51 1,642 LIMI (Xiao et al., 2023) 0.76 0.00 74,169 11,208 84.89 – Remi 0.84 0.65 74,169 3,927 94.71 2,898 LIMI (Xiao et al., 2023)+ Remi 0.76 0.00 11,208 3,086 72.47 2,120 BM5 Themis (Angell et al., 2018) 0.15 0.21 3,288 3,862 -17.46 – Remi 0.89 0.58 3,288 1,028 68.73 582 Themis (Angell et al., 2018) + Remi 0.15 0.21 3,862 908 76.49 578 BM6 Themis (Angell et al., 2018) 0.87 0.01 3,175 3,862 -21.64 – Remi 0.89 0.53 3,175 1,232 61.20 361 Themis (Angell et al., 2018) + Remi 0.87 0.01 3,862 1,032 73.28 572 BM7 Themis (Angell et al., 2018) 0.57 0.27 3,244 3,692 -13.81 – Remi 0.89 0.56 3,244 1,074 66.89 471 Themis (Angell et al., 2018) + Remi 0.57 0.27 3,692 1,058 71.34 516 BM8 Themis (Angell et al., 2018) 0.81 0.09 3,244 3,859 -18.96 – Remi 0.89 0.43 3,244 1,074 66.89 471 Themis (Angell et al., 2018) + Remi 0.81 0.09 3,859 1,006 73.93 528 Mitigation Results (Qualitative Differences between Remi vs. Baselines): We compare Remi against three baselines: Themis (Angell et al., 2018), ExpGA (Fan et al., 2022), and LIMI (Xiao et al., 2023). While ExpGA and LIMI are IDI-generation tools focused on discovering individual discrimination instances, Remi targets explanation and mitigation. Remi operates as a post-hoc wrapper that preserves original model utility, whereas counterfactual retraining modifies the model itself and risks degrading accuracy or collapsing when discovered IDIs are out of distribution. Crucially, Remi is agnostic to the upstream IDI-finding method: it takes IDIs discovered by Themis, ExpGA, or LIMI, aligns them into relational datasets, infers interpretable discriminatory regions, and deploys those regions as guardrails. Table 7 reports standard classification utility metrics (Accuracy, F1) and four fairness-specific columns: #IDIbef\#IDI_bef, the number of IDIs before mitigation; #IDIaft\#IDI_aft, the number of IDIs after mitigation; Red(%)Red~(\%), the percentage reduction between the two; and #FPIDI\#FP_IDI, the number of falsely rejected instances by the guardrail. • Data Augmentations vs. Remi. Across all AC models, Remi consistently reduces unfair outputs. Remi reduces IDIs by 42–56% on the Adult Census models for Themis IDIs, by 71–85% for ExpGA-based IDIs, and by 87–95% for LIMI-based IDIs. The larger reductions for ExpGA and LIMI stem from the greater volume (often 2-10x more vs Themis) of initial IDIs these tools discover ( 16,000–74,000 vs. Themis’s 7,000–8,000), which provides richer relational training data helping to create more precise rules. On the BM models, reductions are around 61–69%. For BM1, we get around 65.7% reduction from 3,136 to 1,076. For BM3, the reduction is from 3506 to 1082, or around 69.1%. • #FPIDI\#FP_IDI (false-positive guardrails). Remi incurs the cost of selectively denying some benign inputs. For AC models, #FPIDI\#FP_IDI typically ranges from 1,100–3,000 (e.g., 1,323 for AC1; 1,927 for AC9), but each denial is offset by 2.2–2.6× as many unfair cases prevented. For instance, AC1 prevents 3,062 unfair outputs against 1,323 benign blocks. • Data Augmentations with Guardrails. When Remi applied after retraining with the baseline techniques (tool ++ Remi), reliably lowers #IDI\#IDI well below the unguarded retrained (tool) models by 39-84% (with Themis AC3: reduced from 7094 to 4754, for AC6 reduced from 10522 to 3042, for AC7 reduced from 8140 to 3996). Also, when CF retraining reduce IDIs e.g., AC1 with ExpGA reduces from 26,647 to 15,687), adding Remi guardrails further reduces IDIs to 4,191 (73% beyond retrained ExpGA alone). Answer RQ5: Remi generalizes to improve individual fairness for black-box deep neural networks. The full pipeline of data curation, rule extraction, and guardrail deployment produces interpretable fairness invariants with high confidence (0.83–0.88) and substantial discrimination coverage (41–84%) on DNNs. Remi consistently reduces individual discriminatory instances by at least 40% and up to 84%, significantly outperforming the baseline mitigation techniques. 6. Discussions Runtime and Performance Overhead. For symbolic programs, the full Remi pipeline (IDI finding, data curation, interpretable model training, rule extraction, and guardrail application) completes in under 120 seconds per program. For DNN benchmarks, Stage 1 IDI finding is the primary overhead, capped at 60 minutes (3,600 seconds) per model for all evaluated tools. The remaining pipeline stages are completed in under 30 seconds per model. Component-Wise Ablation Analysis. Although Remi’s evaluation does not use a single ablation table, each RQ is structured to isolate the contribution of one pipeline component while holding others fixed. For example, RQ1 ablates data curation: fixing the learner (CART) and varying the alignment strategy (DCNE, DCVE, DCHE vs. AFT and most-frequent baselines) to measure the contribution of relational alignment to localization quality. Similarly, RQ2 ablates the interpretable learner: fixing the curation method and comparing tree-based algorithms to isolate the impact of learner choice. Limitation. Remi can be integrated into existing ML pipelines as a post-processing wrapper requiring no modification to the underlying model. After the DPuT produces a prediction, the guardrail module checks whether the input falls within a known discriminatory region and, if so, defers the decision to human review, making Remi compatible with any classifier. However, several limitations bound the current scope of applicability. First, Remi should be viewed as an auditing and guardrail-synthesis tool rather than a proof of global fairness: its rules describe unfair regions represented in the sampled relational dataset, and repeated audits may be needed as the DPuT or data distribution evolves. Second, rule quality is bounded by the coverage of the IDI-finding stage; input regions not represented during training—whether fair or discriminatory—will not be captured by the extracted rules. If the IDI-finding stage discovers no IDIs, Remi cannot proceed, though this outcome does not itself guarantee model fairness. Third, for natural language or vision systems, protected-attribute changes can affect semantics in subtle ways; applying Remi in such settings would require domain-specific counterfactual generators and semantic-validity filters. We therefore do not claim that the current implementation directly addresses fairness debugging for all large-scale systems. Threats to Validity. To ensure reproducibility and deterministic results, we repeat experiments multiple times, reporting averages and standard deviations. To mitigate the impact of randomness in fairness testing and decision tree initialization, we employ seed values, acknowledging the resulting seed dependency. Regarding generalizability, we evaluate our framework on 20 DNNs and diverse symbolic programs, including logic-based and scoring-based models. Internal validity concerns related to hyperparameter selection for IDI searching and interpretable model training are addressed by following established standards. Finally, to prevent variability from sampling symbolic datasets, we generate and fix our datasets once to maintain consistency across all experimental runs. Validity of Counterfactual Inputs. Counterfactual generation in Remi follows a two-step process to ensure semantic validity. First, the protected attribute is flipped within its valid domain (e.g., sex, ethnicity). Second, any non-protected attributes causally dependent on the protected attribute are updated according to domain-specific symbolic rules. For symbolic DPuTs, these constraints are derived directly from the benchmark program logic. For DNN benchmarks, we inherit input domains and preprocessing constraints from the corresponding dataset (e.g., flipping sex from male to female triggers an update of the relationship attribute). However, for DNN benchmarks, a potential limitation is that causal dependencies between features may not be fully known, meaning some generated counterfactuals may be invalid due to the limitations of preprocessing rules. Validity of Explanations.: Remi Explanations are quantitatively validated in different parts of the paper. In RQ3 and RQ5, we measure impurity, confidence, and coverage of the extracted rules against IDI localization and coverage metrics. Qualitative correctness is confirmed by benchmarking extracted invariants against ground-truth symbolic logic. In RQ4–RQ5, we demonstrate that applying these rules as guardrails effectively reduces IDIs, which would not be possible if the explanations were incorrect. A limitation is that for DNNs, where the ground-truth discriminatory logic is unknown, we can only validate explanations indirectly through their mitigation effectiveness. Interpretability in Symbolic Setting. Interpretability is not for identifying protected attributes, but for mapping discriminatory logic to regions of non-protected features. This enables generalization from point-wise violations to systematic patterns, which is critical for deploying automated mitigation guardrails, especially in a black-box setting. Besides, interpretability in a symbolic setting serves as a validation step to assess whether our approach accurately explains the ground-truth discrimination. Concrete Examples of Fairness Violations, Rules, and Mitigation. To illustrate Remi in a realistic fairness-debugging workflow, we consider a COMPAS-like recidivism risk assessment tool, which has been widely documented to produce racially and sexually disparate predictions (Julia Angwin and Kirchne, 2021). Our symbolic benchmark PG2 encodes a simplified recidivism scoring rule (PG2 Listing in (Akash et al., 2026) from (Rudin, 2019)). • Stage 1: IDI Discovery. From 6,908 individuals sampled from the preprocessed COMPAS dataset (ProPublica, 2021; Zhong et al., 2023; Larson et al., 2016), Remi find 206 IDIs by flipping sex (male ↔ female). • Stage 2: Relational Data Curation. Using DCNE, Remi constructs a relational dataset of 6,908 instances, each labeled fair (++) or discriminatory (−-) based on relational data points. • Stage 3: Rule Extraction. A CART decision tree trained on the relational labels extracts the rule age ≤ 20.5 ∧ priors_count ≤ 3.5, which captures all 206 IDIs with high confidence. • Stage 4: Guardrail Mitigation. Deploying this rule as a guardrail eliminates all 206 IDIs, achieving a full reduction in discriminatory behavior without changing the logic of the decision-making program. 7. Related Work Fairness Testing: When Galhotra et al. (Galhotra et al., 2017) proposed and popularized the causal fairness definition for individual fairness testing, a substantial line of work (Angell et al., 2018; Agarwal et al., 2018b; Aggarwal et al., 2019; Fan et al., 2022; Monjezi et al., 2025) focused on discovering discriminatory inputs (IDIs) through testing at scale following that fairness definition. Some of the tools e.g., Themis (Angell et al., 2018), AEQUITAS (Udeshi et al., 2018), SG (Aggarwal et al., 2019), ExpGA (Fan et al., 2022), LIMI (Xiao et al., 2023), AFT (Zhao et al., 2024), considered black-box settings, where the inner working knowledge of classifiers was not necessary for finding ID instances. Some other test generation algorithms ADF (Zhang et al., 2020), EIDIG (Zhang et al., 2021), NeuronFair (Zheng et al., 2022), DICE (Monjezi et al., 2023), MAFT (Wang et al., 2024) considers a white-box setting, which requires knowledge of the classifier being tested Many of these tools can be used in the first stage of our framework to find IDIs for DPuTs under test if the DPuT matches the target model for these tools. The problem of testing for group fairness has been explored extensively in existing literature (Bellamy et al., 2018; Chakraborty et al., 2020; Zhang and Harman, 2021; Chakraborty et al., 2021; Tizpaz-Niari et al., 2022; Chen et al., 2022). Formal Methods. Formal tools and techniques have been significantly studied in the literature (Kim et al., 2025; Akash et al., 2025; Li et al., 2023b; Kim et al., 2026; Monjezi et al., 2026) The FairSquare tool (Albarghouthi et al., 2017) targets probabilistic programs, employing volume-based computations to verify their fairness properties. A formal methodology for certifying individual fairness in standard machine learning architectures was established by John et al. (George John et al., 2020). Fairify (Biswas and Rajan, 2023) examines individual fairness and its variants in neural networks by translating pre-trained models into Satisfiability Modulo Theories (SMT) problems. Explainable-AI and Interpretability: Riberio et al.(Ribeiro et al., 2016) provide a local explanation model, LIME, to explain a locally faithful non-linear model via a sparse linear model Lundberg. et al., (Lundberg and Lee, 2017) propose a unified approach with SHAP and SHAPly values to interpret the global behavior of the model. Mothilal et al. (Mothilal et al., 2020) provide a tool called DiCE, which utilizes diverse counterfactual explanations to understand the decision boundary of the model, more specifically, changes in which feature will lead to the flip of the decision from one side of the boundary to the other side. FairLay-ML (Yu et al., 2025) is a debugging tool to explain the fairness implications of data-driven software. Parfait-ML (Tizpaz-Niari et al., 2022; Robles Herrera et al., 2024) explores the explanation behind hyperparameter configuration, which might lead to unfair models via decision trees. Instead, we used interpretable models to explain the root cause of unfairness. Bias Mitigation To address bias in machine learning outputs, several researchers have proposed different types of mitigation algorithms (Zhang et al., 2018; Kamiran et al., 2012; Agarwal et al., 2018a). Our work is closer to post-processing techniques (Dasu et al., 2024; Sun et al., 2022; Li et al., 2023a; Zhang and Sun, 2022; Gao et al., 2022; Dasu et al., 2026). While these works aim to modify the decision logic of black-box models, our approach leverages guardrails, derived from the explanation models, to improve the fairness of symbolic and data-driven black-box software. 8. Conclusion and Future Work In this paper, we presented Remi, a novel framework that treats individual fairness as a relational invariant discovery problem. By transforming counterfactual pairs into a relational dataset, Remi successfully localizes discriminatory regions using interpretable, rule-based explainers. These extracted fairness invariants provide both a precise explanation of the root cause of discrimination and a robust mitigation strategy via real-time guardrails. Our evaluation across symbolic, scoring, and deep neural network programs demonstrates that Remi is highly effective and outperforms the state-of-the-art baselines. Future work includes extending this relational framework to unstructured domains, such as natural language (Dahal et al., 2026). 9. Data Availability Our open-source tool Remi with all experimental subjects is available at Figshare (Akash et al., 2026) and Github. Acknowledgments This project has been partially supported by NSF under grants CCF-2536640 and CNS-2230061. References Agarwal et al. (2018a) A. Agarwal, A. Beygelzimer, M. Dudik, J. Langford, and H. Wallach A reductions approach to fair classification. In Proceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause (Eds.), Proceedings of Machine Learning Research, Vol. 80, p. 60–69. External Links: Link Cited by: §7. Agarwal et al. (2018b) A. Agarwal, P. Lohia, S. Nagar, K. Dey, and D. Saha Automated Test Generation to Detect Individual Discrimination in AI Models. arXiv e-prints, p. arXiv:1809.03260. External Links: Document, 1809.03260 Cited by: §7. Aggarwal et al. (2019) A. Aggarwal, P. Lohia, S. Nagar, K. Dey, and D. Saha Black box fairness testing of machine learning models. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019, p. 625–635. External Links: Link, Document Cited by: §4.1, §7. Akash et al. (2025) R. D. Akash, A. Kumar, V. Monjezi, A. Trivedi, G. Tan, and S. Tizpaz-Niari Uncovering discrimination clusters: quantifying and explaining systematic fairness violations. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE), Vol. , p. 1680–1692. External Links: Document Cited by: §7. Akash et al. (2026) R. D. Akash, A. Kumar, G. Tan, and S. Tizpaz Niari Supplementary materials of paper "fairness invariants: a relational approach to explaining and mitigating fairness bugs". figshare. Note: DOI: https://doi.org/10.6084/m9.figshare.33061115.v5 External Links: Link, Document Cited by: §5, §6, §9. Albarghouthi et al. (2017) A. Albarghouthi, L. D’Antoni, S. Drews, and A. V. Nori FairSquare: probabilistic verification of program fairness. Proc. ACM Program. Lang. 1 (OOPSLA). External Links: Link, Document Cited by: Listing 1, Listing 2, §2, Table 1, Table 1, Table 1, Table 1, §5, §5, §7. Angell et al. (2018) R. Angell, B. Johnson, Y. Brun, and A. Meliou Themis: automatically testing software for discrimination. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018, New York, NY, USA, p. 871–875. External Links: ISBN 9781450355735, Link, Document Cited by: §4.1, §5.5, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, §5, §7. Bellamy et al. (2018) R. K. E. Bellamy, K. Dey, M. Hind, S. C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilovic, S. Nagar, K. Natesan Ramamurthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, and Y. Zhang AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias. arXiv e-prints, p. arXiv:1810.01943. External Links: Document, 1810.01943 Cited by: §7. Biswas and Rajan (2023) S. Biswas and H. Rajan Fairify: fairness verification of neural networks. In Proceedings of the 45th International Conference on Software Engineering, ICSE ’23, p. 1546–1558. External Links: ISBN 9781665457019, Link, Document Cited by: §5, §7. Breiman et al. (2000) L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone Classification and regression trees. External Links: Link Cited by: §5.1, §5.2. Carreira-Perpinan and Tavallali (2018) M. A. Carreira-Perpinan and P. Tavallali Alternating optimization of decision trees, with application to learning sparse oblique trees. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31, p. . External Links: Link Cited by: §5.2. Chakraborty et al. (2021) J. Chakraborty, S. Majumder, and T. Menzies Bias in machine learning software: why? how? what to do?. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021, New York, NY, USA, p. 429–440. External Links: ISBN 9781450385626, Link, Document Cited by: §7. Chakraborty et al. (2020) J. Chakraborty, S. Majumder, Z. Yu, and T. Menzies Fairway: a way to build fair ml software. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, New York, NY, USA, p. 654–665. External Links: ISBN 9781450370431, Link, Document Cited by: §1, §7. Chakraborty et al. (2019) J. Chakraborty, T. Xia, F. M. Fahid, and T. Menzies Software Engineering for Fairness: A Case Study with Hyperparameter Optimization. arXiv e-prints, p. arXiv:1905.05786. External Links: Document, 1905.05786 Cited by: §1. Chen and Guestrin (2016) T. Chen and C. Guestrin XGBoost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, New York, NY, USA, p. 785–794. External Links: ISBN 978-1-4503-4232-2, Link, Document Cited by: §5.2. Chen et al. (2022) Z. Chen, J. M. Zhang, F. Sarro, and M. Harman MAAT: a novel ensemble approach to addressing fairness and performance bugs for machine learning software. ESEC/FSE 2022, p. 1122–1134. External Links: Link, Document Cited by: §7. Dahal et al. (2026) R. Dahal, P. Hossein Pour, P. Kamisetty, S. Pamulaparthy, S. Tizpaz-Niari, and N. Parde Investigating stigmatizing language in clinical documentation with open-source large language models. In BioNLP 2026, D. Demner-Fushman, S. Ananiadou, K. Roberts, and J. Tsujii (Eds.), San Diego, California, p. 490–501. External Links: Link, Document, ISBN 979-8-89176-434-7 Cited by: §8. Dasu et al. (2024) V. A. Dasu, A. Kumar, S. Tizpaz-Niari, and G. Tan NeuFair: neural network fairness repair with dropout. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, New York, NY, USA, p. 1541–1553. External Links: ISBN 9798400706127, Link, Document Cited by: §7. Dasu et al. (2026) V. A. Dasu, M. R. U. Rashid, V. G. S. Tizpaz-Niari, and G. Tan Attention pruning: automated fairness repair of language models via surrogate simulated annealing. In 48th International Conference on Software Engineering (ICSE), p. To appear. Cited by: §7. Dua and Graff (2017) D. Dua and C. Graff UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. External Links: Link Cited by: 1st item. Dwork et al. (2012) C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS ’12, New York, NY, USA, p. 214–226. External Links: ISBN 9781450311151, Link, Document Cited by: §2, §3. Fan et al. (2022) M. Fan, W. Wei, W. Jin, Z. Yang, and T. Liu Explanation-guided fairness testing through genetic algorithm. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, New York, NY, USA, p. 871–882. External Links: ISBN 9781450392211, Link, Document Cited by: §1, §4.1, §4.1, §5.4, §5.5, §5.5, Table 1, Table 1, Table 6, Table 6, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, §5, §5, §5, §7. Friedman and Popescu (2008) J. H. Friedman and B. E. Popescu Predictive learning via rule ensembles. The Annals of Applied Statistics 2 (3), p. 916–954. External Links: ISSN 19326157, Link Cited by: §5.2. Friedman (2001) J. H. Friedman Greedy function approximation: a gradient boosting machine. Annals of statistics, p. 1189–1232. External Links: Document Cited by: §5.2. Galhotra et al. (2017) S. Galhotra, Y. Brun, and A. Meliou Fairness testing: testing software for discrimination. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, New York, NY, USA, p. 498–510. External Links: ISBN 9781450351058, Link, Document Cited by: §7. Gao et al. (2022) X. Gao, J. Zhai, S. Ma, C. Shen, Y. Chen, and Q. Wang FairNeuron: improving deep neural network fairness with adversary games on selective neurons. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, New York, NY, USA, p. 921–933. External Links: ISBN 9781450392211, Link, Document Cited by: §7. Garg et al. (2016) P. Garg, D. Neider, P. Madhusudan, and D. Roth Learning invariants using decision trees and implication counterexamples. SIGPLAN Not. 51 (1), p. 499–512. External Links: ISSN 0362-1340, Link, Document Cited by: §1. George John et al. (2020) P. George John, D. Vijaykeerthy, and D. Saha Verifying individual fairness in machine learning models. In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), J. Peters and D. Sontag (Eds.), Proceedings of Machine Learning Research, Vol. 124, p. 749–758. External Links: Link Cited by: §7. Gohar et al. (2023) U. Gohar, S. Biswas, and H. Rajan Towards understanding fairness and its composition in ensemble machine learning. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), Vol. , p. 1533–1545. External Links: Document Cited by: §1. Hardt et al. (2016) M. Hardt, E. Price, and N. Srebro Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, p. 3323–3331. External Links: ISBN 9781510838819 Cited by: §1, §1. Holte (1993) R. C. Holte Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, p. 63–90. External Links: Link Cited by: §5.2. [32] (2024) Imodels greedy rule list. Note: https://csinva.io/imodels/rule_list/greedy_rule_list.htmlonline Cited by: §5.2. Julia Angwin and Kirchne (2021) S. M. Julia Angwin and L. Kirchne Machine bias. ProPublica. Note: https://w.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencingOnline Cited by: §6. [34] (2024) Kaggle. Note: https://w.kaggle.comonline Cited by: Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1, §5. Kamiran et al. (2012) F. Kamiran, A. Karim, and X. Zhang Decision theory for discrimination-aware classification. In 2012 IEEE 12th International Conference on Data Mining, Vol. , p. 924–929. External Links: Document Cited by: §7. Kim et al. (2026) B. H. Kim, J. L. Mitchell, and C. Wang Analyzing fairness of neural network prediction via counterfactual dataset generation. External Links: 2602.10457, Link Cited by: §7. Kim et al. (2025) B. H. Kim, J. Wang, and C. Wang FairQuant: certifying and quantifying fairness of deep neural networks. In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ICSE ’25, p. 527–539. External Links: ISBN 9798331505691, Link, Document Cited by: §7. Kusner et al. (2017) M. Kusner, J. Loftus, C. Russell, and R. Silva Counterfactual fairness. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Red Hook, NY, USA, p. 4069–4079. External Links: ISBN 9781510860964 Cited by: §3. Larson et al. (2016) J. Larson, S. Mattu, L. Kirchner, and J. Angwin How we analyzed the compas recidivism algorithm. ProPublica (5 2016) 9 (1), p. 3–3. Cited by: Table 1, Table 1, Table 1, §5, 1st item. Li et al. (2023a) T. Li, X. Xie, J. Wang, Q. Guo, A. Liu, L. Ma, and Y. Liu Faire: repairing fairness of neural networks via neuron condition synthesis. ACM Trans. Softw. Eng. Methodol. 33 (1). External Links: ISSN 1049-331X, Link, Document Cited by: §7. Li et al. (2023b) Y. Li, J. Wang, and C. Wang Certifying the fairness of knn in the presence of dataset bias. In Computer Aided Verification: 35th International Conference, CAV 2023, Paris, France, July 17–22, 2023, Proceedings, Part I, Berlin, Heidelberg, p. 335–357. External Links: ISBN 978-3-031-37702-0, Link, Document Cited by: §7. Lundberg and Lee (2017) S. M. Lundberg and S. Lee A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Red Hook, NY, USA, p. 4768–4777. External Links: ISBN 9781510860964 Cited by: §7. Mazzucato and Urban (2021) D. Mazzucato and C. Urban Reduced products of abstract domains for fairness certification of neural networks. In Static Analysis: 28th International Symposium, SAS 2021, Chicago, IL, USA, October 17–19, 2021, Proceedings, Berlin, Heidelberg, p. 308–322. External Links: ISBN 978-3-030-88805-3, Link, Document Cited by: Table 1, Table 1, Table 1, Table 1, Table 1, §5. Monjezi et al. (2026) V. Monjezi, A. Kumar, A. Trivedi, G. Tan, and S. Tizpaz-Niari On the robustness of fairness practices: a causal framework for systematic evaluation. In 48th International Conference on Software Engineering (ICSE), p. To appear. Cited by: §7. Monjezi et al. (2025) V. Monjezi, A. Trivedi, V. Kreinovich, and S. Tizpaz-Niari Fairness testing through extreme value theory. In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ICSE ’25, p. 1501–1513. External Links: ISBN 9798331505691, Link, Document Cited by: §7. Monjezi et al. (2023) V. Monjezi, A. Trivedi, G. Tan, and S. Tizpaz-Niari Information-theoretic testing and debugging of fairness defects in deep neural networks. 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), p. 1571–1582. External Links: Link Cited by: §7. Moro et al. (2014) S. Moro, P. Rita, and P. Cortez Bank Marketing. Note: UCI Machine Learning RepositoryDOI: https://doi.org/10.24432/C5K306 Cited by: 2nd item. Mothilal et al. (2020) R. K. Mothilal, A. Sharma, and C. Tan Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20, New York, NY, USA, p. 607–617. External Links: ISBN 9781450369367, Link, Document Cited by: §7. Nguyen et al. (2023) G. Nguyen, S. Biswas, and H. Rajan Fix fairness, don’t ruin accuracy: performance aware fairness repair using automl. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2023, New York, NY, USA, p. 502–514. External Links: ISBN 9798400703270, Link, Document Cited by: §1. ProPublica (2021) ProPublica Compas software ananlysis. Note: https://github.com/propublica/compas-analysisOnline Cited by: Table 1, Table 1, Table 1, §5, 1st item. Quinlan (1993) J. R. Quinlan C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. External Links: ISBN 1558602402 Cited by: §2, §5.2. Ribeiro et al. (2016) M. T. Ribeiro, S. Singh, and C. Guestrin "Why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, New York, NY, USA, p. 1135–1144. External Links: ISBN 9781450342322, Link, Document Cited by: §1, §2, §7. Robles Herrera et al. (2024) S. Robles Herrera, V. Monjezi, V. Kreinovich, A. Trivedi, and S. Tizpaz-Niari Predicting fairness of ml software configurations. In Proceedings of the 20th International Conference on Predictive Models and Data Analytics in Software Engineering, PROMISE 2024, New York, NY, USA, p. 56–65. External Links: ISBN 9798400706752, Link, Document Cited by: §7. Rudin and Ustun (2018) C. Rudin and B. Ustun Optimized scoring systems: toward trust in machine learning for healthcare and criminal justice. Interfaces 48 (5), p. 449–466. External Links: ISSN 0092-2102, Link, Document Cited by: Table 1, Table 1, §5. Rudin (2019) C. Rudin Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1 (5), p. 206–215. External Links: Document Cited by: Table 1, §5, §6. Solar-Lezama et al. (2006) A. Solar-Lezama, L. Tancau, R. Bodik, S. Seshia, and V. Saraswat Combinatorial sketching for finite programs. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XII, New York, NY, USA, p. 404–415. External Links: ISBN 1595934510, Link, Document Cited by: §1. Sun et al. (2022) B. Sun, J. Sun, L. H. Pham, and J. Shi Causality-based neural network repair. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, New York, NY, USA, p. 338–349. External Links: ISBN 9781450392211, Link, Document Cited by: §7. Tan et al. (2025) Y. S. Tan, C. Singh, K. Nasseri, A. Agarwal, J. Duncan, O. Ronen, M. Epland, A. Kornblith, and B. Yu Fast interpretable greedy-tree sums. Proceedings of the National Academy of Sciences 122 (7), p. e2310151122. External Links: Document, Link, https://w.pnas.org/doi/pdf/10.1073/pnas.2310151122 Cited by: §5.2. Tizpaz-Niari et al. (2022) S. Tizpaz-Niari, A. Kumar, G. Tan, and A. Trivedi Fairness-aware configuration of machine learning libraries. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, New York, NY, USA, p. 909–920. External Links: ISBN 9781450392211, Link, Document Cited by: §1, §7, §7. Udeshi et al. (2018) S. Udeshi, P. Arora, and S. Chattopadhyay Automated directed fairness testing. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE ’18, New York, NY, USA, p. 98–108. External Links: ISBN 9781450359375, Link, Document Cited by: §1, §4.1, §5.4, Table 1, Table 1, §5, §7. Urban et al. (2020) C. Urban, M. Christakis, V. Wüstholz, and F. Zhang Perfectly parallel fairness certification of neural networks. Proc. ACM Program. Lang. 4 (OOPSLA). External Links: Link, Document Cited by: Table 1, Table 1, Table 1, Table 1, Table 1, §5. Wang et al. (2024) Z. Wang, M. Zhang, J. Yang, B. Shao, and M. Zhang MAFT: efficient model-agnostic fairness testing for deep neural networks via zero-order gradient search. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, New York, NY, USA. External Links: ISBN 9798400702174, Link, Document Cited by: §7. Xiao et al. (2023) Y. Xiao, A. Liu, T. Li, and X. Liu Latent imitator: generating natural individual discriminatory instances for black-box fairness testing. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023, New York, NY, USA, p. 829–841. External Links: ISBN 9798400702211, Link, Document Cited by: §4.1, §5.5, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, Table 7, §5, §7. Yu et al. (2025) N. Yu, L. Carreon, G. Tan, and S. Tizpaz-Niari FairLay-ml: intuitive debugging of fairness in data-driven social-critical software. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), p. 25–28. External Links: Link, Document Cited by: §7. Zhang et al. (2018) B. H. Zhang, B. Lemoine, and M. Mitchell Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’18, New York, NY, USA, p. 335–340. External Links: ISBN 9781450360128, Link, Document Cited by: §7. Zhang et al. (2022) J. Zhang, I. Beschastnikh, S. Mechtaev, and A. Roychoudhury Fair decision making via automated repair of decision trees. In Proceedings of the 2nd International Workshop on Equitable Data and Technology, FairWare ’22, New York, NY, USA, p. 9–16. External Links: ISBN 9781450392921, Link, Document Cited by: Table 1, Table 1, §5. Zhang and Harman (2021) J. M. Zhang and M. Harman "Ignorance and prejudice" in software fairness. In 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22-30 May 2021, p. 1436–1447. External Links: Link, Document Cited by: §7. Zhang et al. (2021) L. Zhang, Y. Zhang, and M. Zhang Efficient white-box fairness testing through gradient search. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2021, New York, NY, USA, p. 103–114. External Links: ISBN 9781450384599, Link, Document Cited by: §7. Zhang and Sun (2022) M. Zhang and J. Sun Adaptive fairness improvement based on causality analysis. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, New York, NY, USA, p. 6–17. External Links: ISBN 9781450394130, Link, Document Cited by: §7. Zhang et al. (2020) P. Zhang, J. Wang, J. Sun, G. Dong, X. Wang, X. Wang, J. S. Dong, and T. Dai White-box fairness testing through adversarial sampling. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, ICSE ’20, New York, NY, USA, p. 949–960. External Links: ISBN 9781450371216, Link, Document Cited by: §1, §5.4, Table 1, Table 1, §5, §7. Zhao et al. (2024) Z. Zhao, T. Toda, and T. Kitamura Approximation-guided fairness testing through discriminatory space analysis. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE ’24, New York, NY, USA, p. 1007–1018. External Links: ISBN 9798400712487, Link, Document Cited by: §1, §4.1, §4.3, §5.1, §5.1, §5.3, §5.3, §5.3, §5.3, §5.3, §5.4, Table 2, Table 2, Table 2, Table 2, Table 2, Table 2, Table 4, Table 4, Table 4, Table 4, Table 4, Table 4, §5, §5, §7. Zheng et al. (2022) H. Zheng, Z. Chen, T. Du, X. Zhang, Y. Cheng, S. Ji, J. Wang, Y. Yu, and J. Chen NeuronFair: interpretable white-box fairness testing through biased neuron identification. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, New York, NY, USA, p. 1519–1531. External Links: ISBN 9781450392211, Link, Document Cited by: §7. Zhong et al. (2023) C. Zhong, Z. Chen, J. Liu, M. Seltzer, and C. Rudin Exploring and interacting with the set of good sparse generalized additive models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. External Links: Document Cited by: Table 1, Table 1, Table 1, §5, 1st item.

Instant research discovery