Paper deep dive
Actions Have Consequences: Detecting Outcome Performativity using Intervention Testing
Brandon Gower-Winter, Georg Krempl
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/4/2026, 11:19:08 AM
Summary
The paper introduces Outcome Performativity A/B Detection (OPAB), an offline method for detecting Outcome Performativity using intervention testing. It derives sample complexity bounds for various assumption classes (Simple, Model-based, Misclassification-based) and identifies 'regions of indistinguishability' where detection is statistically impossible due to insufficient samples. The method is validated on datasets like breast cancer and diabetes, showing that detection is feasible when performative effects are strong, but becomes prohibitive when effects are subtle.
Entities (10)
Relation Signals (9)
OPAB → detects → Outcome Performativity
confidence 98% · OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups
OPAB → uses → Intervention Testing
confidence 95% · we apply intervention testing [8] to detect Outcome Performativity
OPAB → usesstatisticaltest → Chi-Squared Test
confidence 95% · We use the Chi-Squared Test to detect Outcome Performativity
OPAB → hasmetric → Sample Complexity
confidence 92% · We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes
OPAB → evaluatedon → Breast Cancer Dataset
confidence 90% · The datasets used in this work are:breast cancer[16]... results reported in this section for every parameter combination are averaged
OPAB → evaluatedon → Diabetes Dataset
confidence 90% · The datasets used in this work are:breast cancer[16],diabetes[17]...
Outcome Performativity → hassubclass → Model-based Outcome Performativity
confidence 90% · Model-based Outcome Performativity [9]... assumes that p* can be represented by a linear model
Outcome Performativity → hassubclass → Simple Outcome Performativity
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Performativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performativity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits dataset, and provides directions for future work.
Tags
Links
- Source: https://arxiv.org/abs/2607.26908v1
- Canonical: https://arxiv.org/abs/2607.26908v1
Trouble viewing inline? Open PDF directly →
Full Text
86,106 characters extracted from source content.
Expand or collapse full text
ACTIONS HAVE CONSEQUENCES: DETECTING OUTCOME PERFORMATIVITY USING INTERVENTION TESTING - EXTENDED VERSION Brandon Gower-Winter and Georg Krempl Utrecht University 8 Hiedelberglaan, Utrecht, 3584 CS, NL b.gower-winter@u.nl, g.m.krempl@u.nl ABSTRACT In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Perfor- mativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performa- tivity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits dataset, and provides directions for future work. 1 Introduction It is often assumed that predictions made by Machine Learning (ML) models have no effect on the outcomes they observe, or the data distributions they encounter in the future. However, a growing body of research under the term Performative Prediction [1] has shown that this is not the case and that many problem domains such as Credit Assignment [2] or Patient Care prediction [3,4] may exhibit these performative phenomena. In the former, assigning high interest rates to high risk debtors may increase the likelihood that they default on their loans. In the latter, assigning palliative care to a patient (instead of curative care) may ultimately lead to their death. This work focuses on a specific type of Performativity known as Outcome Performativity [5] where predictions made by ML models causally affect the outcomes observed (Fig. 1a). The state-of-art often assumes the Outcome Performativeness of a setting is known a priori. However, it has been shown that ML algorithms can induce performative feedback loops inconspicuously and may produce unwanted phenomena such as an increased False Positive Rate (FPR) over time [3]. This motivates the need to detect such phenomena. If performativity is undetected, practitioners may unknowingly deploy models that exacerbate bias or suffer from performance decay. Despite the existence of online methods for identifying performative effects [2,6], there has been little work studying the identifiability of Outcome Performativity in an offline setting (i.e., during the data collection process). If such an offline detection method exists, it would allow practitioners to identify Outcome Performativity before a predictive model is deployed, which may be more ethically and financially beneficial to an online approach which requires a trained and deployed model. Taking inspiration from Randomized Controlled Trials (RCTs) [7] and work on A/B (Intervention) testing to detect performative feedback loops [6], we apply intervention testing [8] to detect Outcome Performativity (Fig. 1b). By arXiv:2607.26908v1 [cs.LG] 29 Jul 2026 Anonymous (a) do(Ŷ) YX (b) Figure 1: Causal diagrams of Outcome Performative with (a) no interventions and (b) OPAB (our method) applied. Here, the outcome (label) Y of a prediction is causally dependent on both the features X and prediction ˆ Y itself. observing the outcome distributions that arise from randomly assigning predictions to instances, we causally assess the dissimilarity of said distributions and, if significant, detect Outcome Performativity. This approach is called Outcome Performativity A/B Detection (OPAB). In contrast to contemporary methods, OPAB is an offline method which enables the detection of Outcome Performativity prior to a ML model’s training and deployment (i.e., detection occurs during the data labelling process). Given this, we are interested in evaluating the sample efficiency of OPAB. i.e., the ability to detect Outcome Performativity in as few samples as possible which is critical in settings where label acquisition is impractical. We investigate this by analysing the sample complexity of OPAB on several assumption classes of Outcome Performativity [3,5], for which we derive sample complexity bounds. This work can be summarized by the following research questions: 1. Under what conditions can intervention testing be used to detect Outcome Performativity in binary classification tasks? For this, we explore three assumption classes of Outcome Performativity in various configurations. 2. To what degree can the number of instances (sample complexity) needed to detect Outcome Performativity be estimated? This is important for domains where intervention testing is costly or has to be minimized for ethical reasons. 3. Using the derived sample complexity bounds, can we identify regions of indistinguishability? These are settings where required sample complexity is prohibitive for detecting Outcome Performativity in a practical setting. The rest of the paper follows: Sect. 2 introduces related work, Sect. 3 describes the implementation of OPAB, Sect. 4 derives the sample complexity bounds of OPAB under different Outcome Performativity assumption classes, and Sect. 6 reflects on the strengths and limitations of OPAB. Sect. 7 concludes the paper. The source code for this work can be found at: https://edu.nl/mhh9d 2 Background and Related Work Outcome Performativity was introduced by Mendler-Dünner et al. [9] and Kim and Perdomo [5]. Outcome Performativ- ity is an extension of Performativity [1] whereby the realised outcome for some prediction task is causally dependent on both the features of an instance and the prediction given to that instance by some predictor (e.g., human expert, or ML model). Outcome Performative tasks fall into subset of traditional supervised learning tasks where a prediction must be made before the true outcome is observed. For example, in a credit assignment task, one might want to predict whether an applicant would become insolvent in the near future. In this setting, predicting the likelihood of insolvency may affect whether that state of insolvency is realised. If a loan is granted, the applicant may have enough funds to stay operational in the near future. If the applicant is not granted a loan, that may increase the likelihood that they go bankrupt in that same time period. Formally, a binary classification Outcome Performative task is described as: p ∗ :X × b Y → [0, 1](1) Here, an outcomeyfrom outcome spaceY = 0, 1is determined by a Bernoulli distribution parametrized by p ∗ (x, ˆy)for every instancex ∈ Xand predictionˆy ∈ Ypair.p ∗ , also known as Nature [5], describes the true outcome distribution of the system. In practice, the dynamics ofp ∗ are unknown but some assumption classes exist in literature. Adam et al. [3,10] adopt misclassification-based dynamics where an outcomeyis only subject to Outcome Performativity if the instancexis misclassified. Their model requires that the notion of a desirable outcome can be described before a prediction is made. In the palliative care problem, it is desirable to offer palliative care to patients who would pass away even if curative interventions were administered because it maximises the quality of the patient’s 2 Detecting Outcome Performativity Using Intervention Testing end-of-life care. If a curative intervention would save a patient’s life, then it is the desirable outcome because life is preferable to the risk of death. In Mendler-Dünner et al. [9], they use model-based dynamics ofp ∗ which adopts the notion that the Outcome Performative behaviour of instances close together in the feature space should behave similarly when given the same prediction assuming no hidden confounders. Outcome Performative phenomena are highly relevant in many practical domains. A review from Pagan et al. [11] categorized Outcome Performativity under the term Outcome Feedback Loops and highlighted the prevalence of Outcome Performativity in both adversarial and non-adversarial domains. Liley et al. [12] showed that naive updating of ML models can cause performance degradation in Outcome Performative settings which is particularly relevant in healthcare settings where even an accurate model can produce unwanted phenomena [4]. Furthermore, Adam et al. [3,10] investigated the subtle consequences that arise if practitioners are not aware of the feedback loops induced by their ML models. They show that an over-reliance on the predictions made by a ML model results in an increased False Positive Rate if the model later trains on data it made predictions on. 3 Methodology Before introducing OPAB, we motivate why intervention testing is important, and useful, when detecting Outcome Performativity. In an binary classification Outcome Performative setting, an instancexis given a predictionˆywhich results in the observation of some outcomey. Formally, we say that outcomeyis sampled from a Bernoulli distribution parametrised byp ∗ :y ∼ Bernoulli(p ∗ (x, ˆy))wherep ∗ , Nature, is defined as a mapping of instances and predictions to some probability of realising an outcome y = 1: p ∗ :X × b Y → [0, 1](2) While the former definitions neatly capture the underlying mechanisms of Outcome Performativity, they do not describe how one should seek to identify if a setting is Outcome Performative. Given this, we can view Outcome Performativity through the lens of the Bayes Theorem: P (Y |X, b Y ) = P ( b Y |X, Y )P (Y |X) P ( b Y |X) (3) whereX, b Y, andYare the observed instances, predictions and outcomes resp. To detect Outcome Performative effects, we look for the following conditions: H(P (Y |X, b Y = 0),P (Y |X, b Y = 1))≤ δ(4) whereHis a hypothesis test that calculates the dissimilarity of the two distributions given some thresholdδwhich controls the type I error rate. Informally, Eq. 4 describes testing to see if the predictions b Y in some settingX, affect the realised outcomesY. However, this is not sufficient to detect Outcome Performativity as we first need to examine from where the predictions b Yoriginate. During outcome data acquisition, or the training of a predictive model, initial predictions about outcomes will come from either a different predictive model or some domain expertφ(i.e.,ˆy = φ(x)) Referring back to Eq. 4, this would mean that our naive test for Outcome Performativity would be: H(P (Y |X, φ(X) = 0),P (Y |X, φ(X) = 1))≤ δ(5) Herein lies the problem. Eq. 5, at most, allows one to tell if predictorφ’s predictions give rise to different outcome distributions, not if the setting itself is Outcome Performative. This is because the outputs ofφare themselves dependent on the instancesX(Fig. 1a). To remedy this and break the dependency of b YonX, we can take inspiration from A/B testing and randomised control trials (RCTs) and perform a do-intervention [8] on the predictions themselves. Note that this is equivalent to replacing the expert predictorφwith a random sampling strategy:φ(x) =U0, 1which decouples the dependency of b Y on X (Fig. 1b). This gives us the following: H(P (Y |X, do( b Y = 0)),P (Y |X, do( b Y = 1)))≤ δ(6) Eq. 6 is enough to allow one to detect Outcome Performativity, it however requires the ability to learn some reliable approximation ofP (Y |X , do( b Y ))using methods, such as T-learner [13] or Double ML [14], which are sensitive to 3 Anonymous intervention imbalances, hyper-parameter and model choices [15]. Given that one of the paper’s goals is to take the first steps towards estimating the sample complexity bounds for Outcome Performative settings, we make an assumption that the Outcome Performative effects in a given setting are uniform across the feature space. This reduces Eq. 6 to: H(P (Y | do( b Y = 0)),P (Y | do( b Y = 1)))≤ δ(7) which is achievable by simply performing interventions (i.e., random sampling predictions), and statistically evaluating the outcome distributionsYusing an appropriate statistical test such as a Chi-Squared or Fisher’s Exact Test. As we show later, the assumption that Outcome Performativity is uniform across the feature space is reasonable for several assumption classes. We do, however, elaborate on the limitations of this assumption in Sect. 6. Sect. A provided an additional perspective on why intervention testing is necessary to detect Outcome Performativity. 3.1 Outcome Performative A/B Testing This section describes the execution logic of Outcome Performative A/B testing (OPAB). Recall that in order test for Outcome Performativity, we need to perform do-interventions on the predictions b Yand observe if there are statistical differences in the outcome distributions of both interventions (Eq. 7). Formally, given a set ofTinstancesXfor some binary classification taskY =0, 1and a significance thresholdδfor controlling the type I error, OPAB will randomly assign predictionsˆy ∈ b Y =0, 1∀x∈ Xsampled from a Bernoulli distribution (ˆy ∼ Bernoulli(0.5)) [5] (i.e., do-intervention). This results in two groups of sizeN. For each group of instancesX b Y =0 andX b Y =1 (the A and B groups in A/B testing or the assigned groups in RCTs), OPAB observes the outcomey ∈ Yof each instance-prediction pair (x, ˆy), and then evaluates the dissimilarity of the label distributions P (Y | b Y = 0) and P (Y | b Y = 1) using Eq. 7. If H(P (Y | b Y = 0),P (Y | b Y = 1))≤ δ, then Outcome Performativity is detected. Alg. 1 provides the pseudocode. Algorithm 1 Pseudocode of OPAB. Here we assume that H is either a Chi-Squared or Fisher’s Exact Test. Require: Instances X , Function H , Threshold δ 1: A← [0, 0]▷ Construct frequency tables 2: B ← [0, 0]▷ for groups A and B 3: for x∈ X do▷ For each instance 4:ˆy ← Bernoulli(0.5)▷ Assign random prediction 5: y ← observe(x, ˆy)▷ Observe outcome of prediction ˆy on instance x 6:if ˆy = 0 then▷ Update frequency tables 7:A[y]← A[y] + 1 8:else 9:B[y]← B[y] + 1 10:end if 11: end for 12: return H(A,B)≤ δ▷ Return result of hypothesis test 4 Sample Complexity Analysis Recall that in an Outcome Performative setting, an outcomeyis sampled from a Bernoulli distribution parametrised byp ∗ : X × b Y → [0, 1]wherep ∗ is Nature (i.e.,y ∼ Bernoulli(p ∗ (x, ˆy))). Nature is the true conditional outcome distribution that describes the underlying dynamics of an Outcome Performative setting. In reality, the actual dynamics ofp ∗ are unknown, but several assumption classes describingp ∗ exist. Each subsequent subsection derives the sample complexity bounds for three of those assumption classes and empirically validates them using OPAB. Deriving sample complexity bounds for Outcome Performative settings is of practical importance because it allows practitioners to calculate the minimum sample size needed to reliably detect Outcome Performativity, which is vital in high-stakes domains like healthcare where the cost of intervention testing is either ethically ambiguous or prohibitively expensive. In order to empirically evaluate two of the assumption classes: Model-based (Sect. 4.2) [9], and Misclassification-based (Sect. 4.3) [3], real-data which is imputed with Outcome Performativity is required. The datasets used in this work are:breast cancer[16],diabetes[17],adult census[18],kickstarter[19],titanic[20], andloan[21]. They are all binary classification tasks. Unless stated otherwise, results reported in this section for every parameter combination are averaged over100replicates and a pseudorandom number generator is used to ensure reproducibility. We use the Chi-Squared Test to detect Outcome Performativity andδ = 0.05. A description of each dataset, the 4 Detecting Outcome Performativity Using Intervention Testing source code for all experiments, and additional experimental results such as using different statistical tests to determine significance or varying the parameter δ, are can be found in the Appendix. 4.1 Simple Outcome Performativity The first assumption class we investigate is a simplified model of Outcome Performativity. In this setting, we dropp ∗ ’s dependence on the instance spaceX describing it as follows: p ∗ = α 0 if ˆy = 0 α 1 if ˆy = 1 (8) whereα c = P (y = 1| ˆy = c)is the probability that the outcome isy = 1when the prediction isˆy = c∈0, 1(binary classification task). In this setting, a realized outcomeyis only dependent on the predictionˆy. We acknowledge that this assumption class is unrealistically simple, but we include it because it allows us identify key behavioural dynamics of Outcome Performativity as well as derive sample complexity bounds which are used as building blocks for deriving sample complexity bounds for more sophisticated Outcome Performative settings. In order to derive sample complexity bounds, we need to make assumptions about the hypothesis testHand the significance thresholdδ, AssumingHis a Chi-Squared Test, the estimated minimum number of interventions per prediction group to detect Outcome Performativity is given by the following Eq. (see Sect. B for proof): N = χ 2 δ 2(α 0 + α 1 )− (α 0 + α 1 ) 2 2(α 0 − α 1 ) 2 (9) whereχ 2 δ is the criticalχ 2 value for some significance thresholdδ(e.g., this work usesδ = 0.05, therefore theχ 2 δ=0.05 with one degree of freedom is3.841). Eq. 9 reveals several important findings: the first is that fewer instances are needed to detect Outcome Performativity as|α 0 − α 1 |→ 1.0. This makes sense as the resulting outcome distributions will be increasingly different as the performative response ofα 0 andα 1 differ greatly. Secondly, the number of instances required per interventionN → ∞whenα 0 ≈ α 1 . This means that for arbitrarily small differences in Outcome Performative responses per intervention, detecting Outcome Performativity becomes increasingly prohibitive. From a practical perspective, the derivation of sample complexity bounds for an Outcome Performative setting introduces another concept: regions of indistinguishability. Formally, given an allotted budget per intervention ̄ N, confidence thresholdδ, and assumption classA, the regions of indistinguishability are defined as the set of parametrisations of a∈Awhere the allotted interventions per group are less than the minimum required number of instances defined by the complexity bound N δ,a : a|a∈A and ̄ N < N δ,a (10) Note thatN δ,a is proxy notation for a sample complexity bound calculation. For Simple Outcome Performativity,N δ,a is Eq. 9 andais some 2-tuple that parametrises the setting (i.e.,a = (α 0 ,α 1 )). However, the concept of the regions of indistinguishability in an Outcome Performative setting are both assumption class, and sample complexity bound agnostic. 4.1.1 Results To empirically validate these findings, we run simulations of a Simple Outcome Performative setting over variousα 0 andα 1 ∈ [0, 1]using OPAB (δ = 0.05) to detect the presence of Outcome Performativity. Fig. 2 reports the detection rate of OPAB over values of ̄ N = 10, 100, 1000, 10000. The detection rate reported in each cell is the number of times OPAB was able to detect Outcome Performativity divided by the total number of repeated runs (25 in this case). These results confirm the presence of regions of indistinguishability (Fig. 2, dark blue regions): As ̄ Nincreases, the number of parameter combinations (α 0 ,α 1 ) decreases proportionally. In the simple Outcome Performativity setting, when|α 0 − α 1 |is large, few instances ( ̄ N = 10) are required to reliably detect Outcome Performativity. However, whenα 0 ≈ α 1 , even a large amount of instances ( ̄ N = 10 000 ) cannot detect Outcome Performativity. Interestingly, despite assuming a Chi-Squared test is used to detect Outcome Performativity, the estimatedNproduced by Eq. 9 is applicable when using other statistical tests to detect Outcome Performativity (Results using Fishers Exact Test and Mann-Whitney U Test are shown in Sect. D.3). 5 Anonymous 0.00.40.8 0 1.0 0.6 0.2 1 N = 10 0.00.40.8 0 1.0 0.6 0.2 N = 100 0.00.40.8 0 1.0 0.6 0.2 N = 1000 0.00.40.8 0 1.0 0.6 0.2 N = 10000 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 2: shows the detection rate of OPAB in the Simple Outcome Performativity setting as the allotted interventions per groupNincreases. Dark blue values indicate regions of indistinguishability where, for the givenN, Outcome performativity cannot be reliably detected. X X + β.θ f θ Feature 2 Feature 1 Figure 3: A visualization of the Model-based Outcome Performativity Model on a two dimensional feature space.f θ is a linear model trained on a labelled dataset and its decision boundary is represented by the dotted line. The red region indicates wheref θ (x) = 0and the green region indicates wheref θ (x) = 1. To simulate Outcome Performativity an instancexis moved in the feature space using the linear model’s weightsθand Outcome Performative strengthβ. The result off θ (x + β.θ)is then the observed outcomeY. In this example, the instance would initially have an observed outcome of Y = 0, but because of the simulated Outcome Performativity, the observed outcome is Y = 1 instead. 4.2 Model-based Outcome Performativity Recall that in Outcome Performative settings, the outcome of an instance-prediction pair is determined byp ∗ :X× b Y → [0, 1]. The previous assumption class abstracted awayX. This section reintroducesXwith Model-based Outcome Performativity. In Model-based Outcome Performativity [9], an instance’s position in the feature space is taken into consideration before applying any Outcome Performative Effects. More specifically, Model-based Outcome Performativity assumes thatp ∗ can be represented by a linear modelf θ →0, 1and its weightsθ. Usingf θ andθ, Outcome Performativity is modelled as: p ∗ = f θ (x + β ˆy .θ)(11) whereβ ˆy is the strength of the Outcome Performativity for predictionˆy = c. Ifβ c < 0.0, it biases instances towards an outcome ofy = 0. Ifβ c > 0.0, it biases instances towards an outcome ofy = 1. Ifβ c = 0.0, predictions are non- Outcome Performative. A visualization of Model-based Outcome Performativity is shown in Fig. 3. Sample complexity analysis shows that Model-based Outcome Performativity behaves similarly to Simple Outcome Performativity (Eq. 9,) despite the introduction of the feature spaceX (see Sect. B for proof): 6 Detecting Outcome Performativity Using Intervention Testing -1.0-0.5 0.00.51.0 0 1.0 0.5 0.0 -0.5 -1.0 1 Breast Cancer (N = 28) -1.0-0.5 0.00.51.0 0 1.0 0.5 0.0 -0.5 -1.0 Diabetes (N = 38) -1.0-0.5 0.00.51.0 0 1.0 0.5 0.0 -0.5 -1.0 Adult Census (N = 1628) -1.0-0.5 0.00.51.0 0 1.0 0.5 0.0 -0.5 -1.0 Kickstarter (N = 5406) 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 4: shows the detection rate of OPAB in Model-based Outcome Performativity settings for different parameter combinations (β 0 ,β 1 ) over several datasets. Dark blue parameter combinations reveal regions of indistinguishability for given datasets given the allotted interventions per group N . N = χ 2 δ 2(p β0 + p β1 )− (p β0 + p β1 ) 2 2(p β0 − p β1 ) 2 (12) wherep βc = P (Y = 1| b Y = c, θ)is the proportion of instances that would have an outcome ofy = 1given a prediction of classˆy = cwhen using modelf θ . When|β 0 − β 1 | →∞, the total number of instancesNneeded per intervention group→ 0.0. Conversely, when|β 0 − β 1 | → 0.0,N → ∞. The caveat is that the rate at which this happens depends on the parametrisation of f θ , which we show in the next section. 4.2.1 Results In order to simulate Model-based Outcome Performativity, a predictive modelf θ and its parametersθare needed. We achieve this by using non-Outcome Performative binary classification dataset (e.g.,Diabetes), and trainf θ on that dataset. This enables one to impute the non-Outcome Performative dataset with Model-based Outcome Performativity. This approach preserves class imbalances, and feature densities from the original non-Outcome Performative dataset. In order to simulate Outcome Performativity, one samples an instancexfrom the original dataset, assigns a predictionˆyand applies Eq. 11 to observe the final outcomey. To evaluate the sample complexity bounds derived in Eq. 12, we evaluate the detection rate of OPAB on the datasets described in Sect. 4 imputed with Model-based Outcome Performativity. When evaluating OPAB we randomly sample10%of the instances from each dataset without replacement. We do this because it is unreasonable in many cases to assume that some large percentage of the total available instances would undergo intervention testing (i.e., intervention testing could be cost prohibitive or unethical at large scales). Fig. 4 illustrates the results of this experiment where we find that asNincreases, the detection rate of OPAB increases. This is most clearly shown across thebreast cancerandadult censusdatasets where for the latter, OPAB reliably detects Outcome Performativity across a range ofβ 0 andβ 1 parameter values. For the former, the range ofβ 0 andβ 1 parameters that OPAB detects Outcome Performativity for decreases (see Sect. D.5). Results also confirm non-uniform Outcome Performativity responses across the datasets. Thebreast cancerdataset illustrates this where a class imbalance in the training dataset (bias towardsY = 1) causes non-uniformity in the number of interventions needed per classN. Asβ c → 1.0(the biased class),Nincreases as the outcome distributions rapidly converge. The original breast cancerdataset is imbalanced favouring benign samples (Y = 1) which means that asβ 0 andβ 1 → 1.0,f θ more rapidly produces identical distributions across both prediction groups. This effect is seen less intensely in the diabetessetting where the original dataset is slightly imbalanced favouringY = 0. These results further support the presence of regions of indistinguishability (Fig. 4, dark blue regions). Unlike the Simple Outcome Performative setting (Sect. 4.1), Model-based Outcome Performative settings reveal how imbalances in the Outcome Distributions and feature space density can significantly increase the number of instances required to detect Outcome Performativity. This is shown in Fig. 4 where certain combination of (β 0 ,β 1 ) are trivially easy to detect, but others are practically infeasible with the allotted interventions ( ̄ N ). 4.3 Misclassification-based Outcome Performativity The final assumption class we evaluate is Misclassification-based Outcome Performativity [3,10] wherep ∗ , Nature, is described as: 7 Anonymous 0.00.40.8 0 1.0 0.6 0.2 1 Breast Cancer (N = 28) 0.00.40.8 0 1.0 0.6 0.2 Diabetes (N = 38) 0.00.40.8 0 1.0 0.6 0.2 Adult Census (N = 1628) 0.00.40.8 0 1.0 0.6 0.2 Kickstarter (N = 5406) 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 5: shows the detection rate of OPAB in Misclassification-based Outcome Performativity settings. Black cells indicate parameter combinations (γ 0 ,γ 1 ) in the regions of indistinguishability for the allotted interventions per classN. p ∗ = 1.0 if ˆy = 1 and ̄y = 1 0.0 if ˆy = 0 and ̄y = 0 γ 1 if ˆy ̸= 1 and ̄y = 1 1− γ 0 if ˆy ̸= 0 and ̄y = 0 (13) Unlike the previous two assumption classes, Misclassification Outcome Performativity only occurs when some predictionˆyis not equal to some desirable outcome ̄y = c. What constitutes a desirable outcome is domain specific. In the palliative care problem, it is desirable to give curative care to patients who are likely to survive, whereas palliative is desirable when patients will pass away regardless of any type of medical intervention. In the credit assignment problem, it is desirable to grant credit to entities who will be able to pay the debt back. In Misclassification Outcome Performativity, when a mismatch between a predictionˆyand desirable outcome ̄yoccur, the probability the desired outcome is realized despite misclassification (i.e.,y = c) is defined by the probability γ c ∈ [0, 1]. Whenγ c = 0, desired outcomes are never realized upon misclassification and whenγ c = 1, the desired outcome is always realized despite misclassification (i.e. no Outcome Performativity). To derive sample complexity bounds for Misclassification Outcome Performativity, we again assumeHis the Chi- Squared Test. The estimated number of interventions needed per class N is: N =−χ 2 δ (γ 0 p 0 + γ 1 p 0 − γ 1 − 1)(γ 0 p 0 + γ 1 p 0 − γ 1 + 1) 2× (γ 0 p 0 − γ 1 p 0 + γ 1 − 1) 2 (14) Wherep 0 is the probability of encountering a desired outcome ̄y = 0when randomly selecting an instance and desired outcome pair(x, ̄y). See Sect. B for proof deriving Eq. 14 and visual aid for differentp 0 estimates. Eq. 14 shows that asγ 0 andγ 1 → 1.0,N → ∞meaning that weaker Outcome Performativity is harder to detect. This makes sense because asγ c → 0.0, more instances will not realize their desired outcomes when misclassified which creates increasingly different outcome distributions. Conversely, asγ c → 1.0fewer instances will not realize their desired outcomes resulting in increasingly similar outcome distributions. 4.3.1 Results We empirically validate Eq. 14, by applying OPAB to the datasets described before imputed with Misclassification Outcome Performativity. To achieve this, the set of desirable outcomesYis determined by the dataset’s original labels prior to being imputed with Outcome Performativity. If a prediction mismatches the desired outcome, we simulate Misclassification Outcome Performativity using Eq. 13. Fig. 5 shows the results of four datasets (others are included in the appendix). We allocate10%of the total dataset for detecting Outcome Performativity with each instance being randomly sampled without replacement. Empirical results confirm the theoretical findings of Eq. 14. WhenNis large, OPAB can detect Outcome Performativity over mostγ 0 andγ 1 values investigated. The exception being whenγ 0 = γ 1 = 1.0which is expected as this parameter combination is non-Outcome Performative. Interestingly, regions of indistinguishability (Fig. 5, black regions) seem less common in the Misclassification-based Outcome Performativity setting. This is expected because there is only one 8 Detecting Outcome Performativity Using Intervention Testing Table 1: Detection Rate of OP-AB across varying sample sizes (N) on the OBD datasets. These results show that OP-AB has a low false detection rate as shown on the non-Outcome performative women dataset and a increased efficacy as N increases as shown the Outcome Performative men dataset dataset / N1010 2 10 3 10 4 10 5 women0.00.0010.0420.0420.016 men0.00.00.050.0940.523 Table 2: Detection Rate of OP-AB across varying sample sizes (N) on balanced variants of the OBD datasets. The results show that OP-AB is more sample efficient when the outcome distribution is balanced. However, the detectability of Outcome Performativity in ultimately dependent on performative strength (effect size) which is negligible in these datasets. dataset / N10501005001000 women0.0460.0590.060.0510.032 men0.0440.0680.0740.1590.234 parameter combination (γ 0 = 1.0,γ 1 = 1.0) which is truly Outcome Performative as opposed to the infinite parameter combinations in the Simple (α 0 = α 1 ) and Model-based (β 0 = β 1 ) Outcome Performative settings. Given this, our analyses suggest that detecting Misclassification-based Outcome Performativity (using OPAB or otherwise) requires fewer interventions than the other assumption classes. 5 Case Study: Open Bandits Dataset To the best of our knowledge, no dataset exists which explicitly studies Outcome Performativity. However, Outcome Performativity can be inferred if there are instances for which (1) predictions are recorded and (2) these predictions are randomly assigned. The Open Bandits Dataset (OBD) [22] meets these criteria. OBD was constructed using multi-armed bandits on the fashion e-commerce platform ZOZOTOWN for the off-policy evaluation of recommender systems. Each instance represents a user impression containing feature values, item recommendations and their placement (clothing items to choose from and where they appear on the store webpage), and click indicators (was the recommended clothing item clicked on) as an outcome. For a subset of the data, the placement of each recommended item on the store page (left, centre, or right) was determined using random sampling. We use this subset and repurpose the placement of item recommendation as predictions b Y and click indications as outcomes Y . The dataset is also divided into sub-datasets for bothmenandwomen. Applying OP-AB to all the instances in both of these sub-datasets we find Outcome Performativity inmen(p = 0.014), but not inwomen(p = 0.87). We cannot speak to gender-based purchasing habits, but these results would indicate that the order in which recommended items appear on the ZOZOTOWN platform are not necessarily relevant for women (not outcome performative), but are relevant for men (outcome performative). Nevertheless, we use themendataset to study the efficacy of OP-AB and thewomen dataset to study the prevalence of false positives produced by OP-AB across variousN. Sect. C give a complete description of the dataset and all pre-processing steps. We first evaluated OP-AB (δ = 0.05)on the OBD datasets by taking random samples (without replacement) of size N =10, 10 2 , 10 3 , 10 4 , 10 5 over1000replicates. The detection rate reported in Table 1 shows that OP-AB has a low false positive of about[0.0, 0.05]in thewomendataset. Conversely, OP-AB seemingly struggles to detect Outcome Performativity in themendataset only being50%accurate at the largestN = 10 5 . This occurs for two reasons: (1) the Outcome Performative effect in this dataset is quite small and thus harder to detect which requires exponentially more instances with random predictions. This claim is supported by the low Phi coefficient (Φ = 0.004) which indicates a negligible effect size and is a limitation of this dataset (2) the class imbalance of the dataset makes it harder for Outcome Performative effects to be observed. The OBD datasets are heavily imbalanced with only about0.05%of all item recommendation resulting in a click in our setup. To investigate the potential class imbalance further, we created balanced datasets of both thewomenandmendatasets by randomly downsampling the majority class (without replacement) over1000replicates and repeating the experiment overN =10, 50, 100, 500, 1000(sample sizeNmust decrease as the total dataset decreases in size). The detection rates are reported in Table 2 and confirm that OP-AB is more effective in settings where the outcome distributionP (Y ) 9 Anonymous is balanced. For example, OP-AB was four times more effective at detecting Outcome Performativity whenN = 10 3 on the balanced men dataset. 6 Discussion This work formalised the act of performing intervention testing on predictions in order to detect Outcome Performativity. This method, OPAB, was evaluated on several theoretical models (assumption classes) of Outcome Performativity for which we also derived sample complexity bounds (i.e., the minimum number of interventions per class needed to detect Outcome Performativity). To the best of our knowledge, there exist no other works which explicitly derive sample complexity bounds for detecting Outcome Performativity. Given this, we believe our work provides a practical contribution to the state-of-the-art. The implications of these findings are threefold: First, by formally describing the process of performing intervention testing on predictions for binary classification tasks, OPAB enables practitioners to detect Outcome Performativity in real-world domains. This contrasts with related works which have only looked at the identifiability of Outcome Performativity in purely theoretical or semi-synthetic domains. To support the claim that OPAB can be used to detect Outcome Performativity in real-world settings, we conducted a case study on the Open Bandits Dataset [22]. Second, OPAB is an offline algorithm, This enables practitioners to detect Outcome Performativity before a model is deployed which can be more cost effective and prevent unwanted performative phenomena from arising once a model is deployed (Appendix includes a demonstration of why training a classifier in an Outcome Performative setting is different than in the classical setting). The complexity analysis of Eq.s 9, 12, and 14 provide guidance on the number of instances needed per intervention group. Lastly, the derivation of sample complexity bounds provides practical insight into the general applicability of intervention testing in various domains. In particular, the introduction of regions of indistinguishability which describe Outcome Performative settings for which the allotted number of interventions cannot reliably detect Outcome Performativity. Unsurprisingly, more interventions are needed to detect subtler Outcome Performative effects. This suggests that there may be settings for which detecting Outcome Performativity is practically infeasible. The consequences of such a finding are domain-specific. If the Outcome Performative effects of some setting are subtle (requiring a large number of intervention), but interventions are costly or unethical, it may be inappropriate to deploy ML models in that domain unless other, non-interventional methods, can be used to detect Outcome Performativity. The caveat being that non-interventional methods tend to be less sample efficient than intervention tests [15] which poses additional challenges around data availability. This work has several limitations. First we did not consider the verification latency [23] that might occur before an outcome is realized. In such settings, one would either have to wait until all the instances given to OPAB are realized (which could take a considerable amount of time) or the hypothesis testing would need to be done with the subset of instances whose outcomes have been realized. Our work also assumes that the labels recorded when observing an outcome are reliable (i.e. no noisy labels [24]). If both of these assumptions do not hold, it may increase the chance that OPAB produces a type I or type I error. Lastly, this work assumes that detecting Outcome Performativity is reliably achievable by observing the difference in outcome distributions of each group of interventions (i.e., OPAB tests ifP (Y| do( b Y = 0))̸= P (Y| do( b Y = 1)) ). If this assumption does not hold for a given setting where Outcome Performativity is only detectable throughP (Y|X, do( b Y = 0))̸= P (Y|X, do( b Y = 1)), OPAB is able to identify if a setting is Outcome Performative, but it cannot identify is a setting is not Outcome Performative. OPAB is sensitive to non-uniform Outcome Performative behaviour. However, this work has shown that OPAB and more specifically analysis of the outcome distributions under prediction interventions can reliably detect Outcome Performativity in a myriad of settings. Future work will address this limitation by relaxing the uniformity assumption. 7 Conclusions and Future Work In this work we formalise the process of detecting Outcome Performativity using prediction interventions. This method: Outcome Performativity A/B Detection (OPAB) is empirically evaluated across several models of Outcome Performativity for which we also derive sample complexity bounds. This work also introduces the concept of regions of indistinguishability which describe Outcome Performative settings where given an allotted number of interventions, Outcome Performativity cannot be reliably detected. Broadly speaking, maximising the number of interventions minimises regions of indistinguishability. These results have implications for settings where interventions are cost prohibitive or potential unethical. Future work will look to address the limitations of this paper. Namely, settings with 10 Detecting Outcome Performativity Using Intervention Testing verification latency (when outcomes take some amount of time to be observed) and imperfect / noisy labelling will be considered. A OP-AB Motivation Recall that OP-AB uses intervention testing to detect Outcome Performativity. The motivations for this decision are described using Figure 6. Naively, it might be tempting to simply compare whether the outcome distributions conditioned on a classifier’s predictions ˆy are dissimilar: H(P (Y | ˆy = 0),P (Y | ˆy = 1))≤ δ(15) Here,His a function that calculates the dissimilarity of the two distributions (e.g. a hypothesis test) 1 andδis the threshold that needs to be reached in order to determine if that dissimilarity is significant. However, this is not sufficient, as these results are only meaningful if the instances that make up the two groups come from the same distribution (i.e. P (X| ˆy = 0) ≈ P (X| ˆy = 1)). In contradiction, the very aim of fitting a classifier is to maximize the dissimilarity betweenP (X| ˆy = 0)andP (X| ˆy = 1). This is shown in the second row of Figure 6. Using these predictions will often falsely detect Outcome Performativity. To combat this phenomena, OP-AB takes inspiration from Randomized Controlled Trials by using random sampling to assign predictions, ensuringP (X| ˆy = 0)≈ P (X| ˆy = 1), removing the effects of confounding variables, and preventing false detections in settings with no Outcome Performativity. This is shown in row 3 of Figure 6. B Proofs In this section we provide proofs for Equations 9, 12, and 14. All proofs start with same assumption about estimating the number of samples needed per intervention group N : Assumption: The task is a binary classification task such that an outcome y ∈ Y =0, 1. Assumption: Predictions ˆy ∈ b Y =0, 1 are assigned to instances x∈ X at random. Assumption: A Chi-Squared Test will be used to determine if two outcome distributions are significantly dissimilar. If these assumptions hold, the χ 2 value of the calculated using the following Equation: χ 2 = X (O i − E i ) 2 E i (16) whereE i is the expected outcome andO i is the observed outcome of a prediction-outcome pairi = (y, ˆy). These values are calculated by creating a contingency / frequency table from the prediction-outcome pairs that are observed. B.1 Simple Model of Outcome Performativity In the Simple Model of Outcome Performativity (Sect. 4.1), outcomes are determined by: p∗ = α 0 if ˆy = 0 α 1 if ˆy = 1 (17) whereα c = P (y = 1| ˆy = c)is the probability that the outcome isy = 1for when the prediction isˆy = c∈0, 1 (binary classification task). Using Equation 17, we can populate a contingency table that would be generated (on average) for a given number of samples per intervention group N and α 0 ,α 1 ∈ [0.0, 1.0] as shown in Table 3 We can then calculate the Expected Contingency Table that would occur if the Null Hypothesis of the Chi-Squared Test was true as shown in Table 4. We then plug the values from Tables 3 and 4 into Equation 16 which will give us: χ 2 = 2N (α 0 − α 1 ) 2 2(α 0 + α 1 )− (α 0 + α 1 ) 2 (18) 1 If a distance metric is used, the equation will then instead be: H(P(Y | ˆy = 0),P(Y | ˆy = 1))≥ δ. 11 Anonymous 2.01.51.00.50.00.51.01.52.0 X 0.0 0.2 0.4 0.6 0.8 1.0 Density Visualization of the True Distribution P(X) P(X) - Decision Boundary P(X | Y = 0) P(X | Y = 1) 2.01.51.00.50.00.51.01.52.0 X 0.0 0.2 0.4 0.6 0.8 1.0 Density Visualization of the Distributions produced by P(X) - Decision Boundary P(X | Y = 0) (Group A) P(X | Y = 1) (Group B) 2.01.51.00.50.00.51.01.52.0 X 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 Density Visualization of the Distributions produced by Intervention Testing P(X) P(X | Y = 0) (Group A) P(X | Y = 1) (Group B) Figure 6: A visualization of the motivation behind OP-AB. The first row shows the true distributionP (X)of an example binary classification datasetX. The dataset is then further split per class (P (X|Y ∈0, 1)).θis the model trained on that dataset. In the second row, we show the distributions that would be produced (P (X| ˆ Y ∈0, 1)) ifθ were used to determine the intervention where ˆ Y ∈0, 1 . As shown, the two distributions (Group A and B) are very different, and thus cannot be used to determine if the label / outcome distributions (P (Y )) are significantly different. In the third row, we show the distributions that would be produced (P (X| ˆ Y ∈0, 1)) if intervention testing (randomized predictions) were used to determine the intervention. As shown, the two distributions are nearly identical and thus can be used to determine if the outcome / label distributions (P (Y )) are significantly different (and therefore Outcome Performative). 12 Detecting Outcome Performativity Using Intervention Testing ˆy = 0ˆy = 1Totals y = 0(1− α 0 )N(1− α 1 )N (2− α 0 − α 1 ) y = 1α 0 Nα 1 N (α 0 + α 1 ) TotalsNN2N Table 3: Average observed Contingency Table in the Simple Model of Outcome Performativity given N , α 0 and α 1 . ˆy = 0ˆy = 1 y = 0 N 2 (2− α 0 − α 1 ) N 2 (2− α 0 − α 1 ) y = 1 N 2 (α 0 + α 1 ) N 2 (α 0 + α 1 ) Table 4: Expected Contingency Table in the Simple Model of Outcome Performativity if the Null Hypothesis for the Chi-Squared Test is true for a given N , α 0 and α 1 . which we can rearrange to estimate N : N = χ 2 2(α 0 + α 1 )− (α 0 + α 1 ) 2 2(α 0 − α 1 ) 2 (19) Hereχ 2 is the Chi-Squared statistic that you’d want to achieve. In the work, we setχ 2 = 3.841which is the threshold required to get a significant result forp = 0.05with one degree of freedom. Figure 7 shows visualization of Equation 19. B.2 Model-based Outcome Performativity The model-based model [9] takes into account an instance’s position in the feature space. Given a set of labelled instances(X, ̄ Y ), a linear modelf θ → 0, 1is trained and its weightsθobserved. Usingθ, we simulate Outcome Performativity using: y = f θ (x + β ˆy .θ)(20) whereβ ˆy is the strength of the Outcome Performativity for predictionˆy = c. Ifβ c < 0.0, it biases instances towards an outcome ofy = 0. Ifβ c > 0.0, it biases instances towards an outcome ofy = 1. Ifβ c = 0.0, predictions are non-Outcome Performative. Using Equation 20, we can populate a contingency table that would be generated (on average) for a given number of samples per intervention group N and β 0 ,β 1 ∈ [−1.01.0] as shown in Table 5 ˆy = 0ˆy = 1Totals y = 0(1− p β0 )N(1− p β1 )N (2− p β0 − p β1 ) y = 1p β0 Np β1 N (p β0 + p β1 ) TotalsNN2N Table 5: Average observed Contingency Table in the Model-based assumption class of Outcome Performativity given N , β 0 and β 1 . In Table 5, the number of instances that would have an outcome ofy = 1given a prediction of classˆy = cwhen using modelf θ is given byp βc = P (Y = 1| b Y = c, θ). This contingency table has the same form as Table 3. The derivation for which we know is: N = χ 2 2(p β0 + p β1 )− (p β0 + p β1 ) 2 2(p β0 − p β1 ) 2 (21) The visualization (which looks the same as Figure 7) is shown in Figure 8 where the two main findings, as is the case in the Simple Model of Outcome Performativity presented are that Outcome Performativity is undetectable when p β0 ≈ p β1 and only a few samples are needed to detect Outcome Performativity when|p β0 − p β1 |→ 1.0. Of course, this derivation is not necessarily useful unless you knowp β0 andp β1 . This can be done for any arbitrary set of features Xand training labelsY ′ wheref θ trains onX, andY ′ , and then, for the desiredβ 0 andβ 1 pair, you calculatep β0 and p β1 . 13 Anonymous 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 10 1 10 2 10 3 10 4 Samples Needed per Group(N) Figure 7: The estimated number of instances needed per group (N) to detect Outcome Performativity (Equation 9) across varyingα 0 andα 1 values. The two main findings presented in this figure are that Outcome Performativity is undetectable whenα 0 ≈ α 1 and only a few samples are needed to detect Outcome Performativity when|α 0 −α 1 |→ 1.0. B.3 Misclassification-based Outcome Performativity Recall the definition of misclassification-based Outcome Performativity: Given a set ofTinstances and desirable outcomes(X,Y ), the outcomey ∈ Y =0, 1of an instancex∈ X, given desirable outcome ̄y ∈Y =0, 1and prediction ˆy ∈ b Y =0, 1 is defined as: p ∗ = 1.0 if ˆy = 1 and ̄y = 1 0.0 if ˆy = 0 and ̄y = 0 γ 1 if ˆy ̸= 1 and ̄y = 1 1− γ 0 if ˆy ̸= 0 and ̄y = 0 (22) where λ c is the likelihood that a desired outcome is realized despite being misclassified: λ c = P (y = c| ̄y = c and ˆy ̸= c)(23) Applying the same process as before, we can populate a contingency table that would be generated (on average) for a given number of samples per intervention group N and γ 0 ,γ 1 ∈ [0.0, 1.0] as shown in Table 6 ˆy = 0ˆy = 1Totals y = 0N (γ 1 p 0 − γ 1 + 1)Nγ 0 p 0 N (γ 0 p 0 + γ 1 p 0 − γ 1 + 1) y = 1Nγ 1 (1− p 0 )N (−γ 0 p 0 + 1)N (−γ 0 p 0 − γ 1 (p 0 − 1) + 1) TotalsNN2N Table 6: Average observed Contingency Table in misclassification-based Outcome Performativity givenN,p0,γ 0 and γ 1 . 14 Detecting Outcome Performativity Using Intervention Testing 0.00.20.40.60.8 p 0 1.0 0.8 0.6 0.4 0.2 p 1 10 1 10 2 10 3 10 4 Samples Needed per Group (N) Figure 8: The estimated number of instances needed per group (N) to detect Outcome Performativity (Equation 21) across varyingp β0 andp β1 values. This figure is shows that model-based Outcome Performativity behaves similarly to Simple Model Outcome Performativity. Note that in this model we have to also includep 0 which describes the likelihood that a randomly drawn samplex∈ X will have a desirable outcome ̄y = 0. We can then calculate the Expected Contingency Table that would occur if the Null Hypothesis of the Chi-Squared Test was true as shown in Table 7. ˆy = 0ˆy = 1 y = 0 N 2 (γ 0 p 0 + γ 1 p 0 − γ 1 + 1) N 2 (γ 0 p 0 + γ 1 p 0 − γ 1 + 1) y = 1 N 2 (−γ 0 p 0 − γ 1 p 0 + γ 1 + 1) N 2 (−γ 0 p 0 − γ 1 p 0 + γ 1 + 1) Table 7: Expected Contingency Table in a misclassification-based Outcome Performative settings if the Null Hypothesis for the Chi-Squared Test is true for a given N , p 0 , γ 0 and γ 1 . We then plug the values from Tables 6 and 7 into Equation 16 which will give us: χ 2 =− 2N × (γ 0 p 0 − γ 1 p 0 + γ 1 − 1) 2 (γ 0 p 0 + γ 1 p 0 − γ 1 − 1)(γ 0 p 0 + γ 1 p 0 − γ 1 + 1) (24) which we can rearrange to estimate N : N =− χ 2 × (γ 0 p 0 + γ 1 p 0 − γ 1 − 1)(γ 0 p 0 + γ 1 p 0 − γ 1 + 1) 2× (γ 0 p 0 − γ 1 p 0 + γ 1 − 1) 2 (25) Note that getting to Equation 14 required quite a few steps which we had to validate using Sympy. Figure 9 shows visualization of Equation 14. 15 Anonymous 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 10 1 10 2 10 3 10 4 Samples Needed per Group(N) Figure 9: The estimated number of instances needed per group (N) to detect Outcome Performativity (Equation 14, p 0 = 0.5) across varyingγ 0 andγ 1 values. This figure shows that it is increasingly difficult to detect Misclassification- based Outcome Performativity as γ 0 and γ 1 → 1.0. 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 (p 0 = 0.01) 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 (p 0 = 0.25) 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 (p 0 = 0.5) 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 (p 0 = 0.75) 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 (p 0 = 0.99) 10 1 10 2 10 3 10 4 10 5 10 6 Samples Needed Per Group (N) 10 1 10 2 10 3 10 4 10 5 Samples Needed Per Group (N) 10 1 10 2 10 3 10 4 Samples Needed Per Group (N) 10 1 10 2 10 3 10 4 10 5 Samples Needed Per Group (N) 10 1 10 2 10 3 10 4 10 5 10 6 Samples Needed Per Group (N) Figure 10: Visualizations of Equation 14 across varying values ofp 0 . The values on these heatmaps report the estimated number of samples needed per intervention group (N ) to detect Outcome Performativity when χ 2 = 3.841. B.3.1 Visualizations of Varying Initial Label Distributions Given that Equation 14 requires thatp 0 be stipulated to know the estimated number of samples per intervention neededNto detect Outcome Performativity, we include plots for various valuesp 0 as shown in Figure 10. Here we have assumed aχ 2 = 3.841. The plots clearly show that asp 0 → 0.0, the number of instancesNrequired per intervention increases exponentially predominantly dependent on the misclassification response of class 0 (γ 0 ). Conversely, asp 0 → 1.0, the number of instancesNrequired per intervention increases exponentially predominantly dependent on the misclassification response of class 1 (γ 1 ). These results make sense given that the detecting Outcome Performativity in these settings becomes increasingly dependent on the prediction response of the minority class. This is also supported by the two orders of magnitude more instances that are required to detect subtle (γ 0 ,γ 1 close to 1.0) Outcome Performativity. C Experimental Design In this work, we made of the following datasets for our misclassification and model-based assumption classes of Outcome Performativity. Note that in order to get the datasets to work with the model-based Outcome Performativity Model, we could only make use of numerical features which are listed in the description below: 16 Detecting Outcome Performativity Using Intervention Testing 1. breast cancer[16]: The Breast Cancer Wisconsin (Diagnostic) Data Set. It consists of 569 instances of which 357 are benign and 212 malignant. We use all 30 features. 2. diabetes [17]: The Pima Indians Diabetes Database. It consists of 768 instances of which 268 belong to Class 1 (The others Class 0). We use all 8 features. 3. adult census[18]: The 1994 Adult Census Income dataset. It has≈ 32 000instances of which76%are of Class≤ 50kwhich the others are of class> 50k. We make use of theage,fnlwgt,education.num, capital.gain, capital.loss, and hours.per.week features. 4. kickstarter [19]: The Funding Successful Kickstarter Projects dataset. It contains108 129instances of which 73 568 are Class 0. The others are Class 1. We make use of the goal and backers\_count features. 5. titanic[20]: The Titanic Survival Prediction dataset. It consists of891instances of which549are Class0 while the others are Class 1. We make use of the Age, SibSp, Parch, and Fare features. 6. loan[21]: A Loan Approval Dataset. It consists of 4269 instances of which62%are labelled Approved. The others are labelled Rejected. We use all 9 numerical features. For all of the Experiments conducted in sections 4.3 and 4.2, each parameter set was evaluated over100replicates. The hypothesis testHgiven to OP-AB was the Chi-Squared test. In the case where the outcome contingency table contained a0, Fisher’s Exact test was used instead. A successful detection meant that thepvalue returned by the hypothesis test was< δ = 0.05. A pseudorandom number generator was used to ensure reproducibility. All experiments were run in Google Colab using the Python 3 Google Compute Engine Backend. C.1 Open Bandits Dataset For our real-world case study, we made use of the Open Bandits Dataset (OBD [22]). OBD was constructed using multi- armed bandit policies on the fashion e-commerce platform ZOZOTOWN for the off-policy evaluation of recommender systems. Each instance represents a user impression containing feature values, item recommendations and their placement (clothing items to choose from and where they appear on the store webpage), and click indicators (was the recommended clothing item clicked on) as an outcome. For a subset of the data, the placement of each recommended item on the store page (left, centre, or right) was determined using random sampling. We use this subset and repurpose the placement of item recommendation as predictions and click indications as outcomes. For preprocessing, we take all of the randomly sampled impressions and extract thepositionandclickvalues. The former is our set of predictions ˆ Y and the latter the set of outcomesY. To ensure that both variables are binary, we filter out any recommendations withposition == 2which corresponds tocentreon the webpage. The rest of the preprocessing proceeds as described in the Sect. 4, we first test to see if Outcome Performativity is present in each sub-dataset (men, women). We then perform random subsampling procedures over 1000 replicates to evaluate OP-AB. D Additional Results In this section we report on the results for theTitanicandLoandatasets for both the misclassification and feature- based Models of Outcome Performativity. Additionally, we report on the additional experiments that we conducted. Namely: The efficacy of OP-AB when using different statistical tests, relationship between sample complexityNandδ, and demonstrating the importanceNplays in determining the detection capabilities of OP-AB. The section concludes with a comparison of OP-AB and the Online Adam detection method described in [3]. D.1 Misclassification-based Outcome Performativity Figure 11 reports on the results of applying OP-AB to detect misclassification-based Outcome Performativity on the TitanicandLoandatasets. These results are in keeping with our main findings. OP-AB exhibits a low false detection rate (i.e. γ 0 = γ 1 = 1.0). If N is larger, OP-AB can detect subtler Outcome Performative Effects. D.2 Model-based Outcome Performativity Figure 12 reports on the results of applying OP-AB to detect model-based Outcome Performativity on theTitanicand Loandatasets. These results are in keeping with our main findings. OP-AB exhibits a low false detection rate when β 0 = β 1 ) because the outcome distributions per intervention are identical and thus non-Outcome Performative. IfNis larger, OP-AB can detect subtler Outcome Performative Effects. Figure 12 also shows the limitation of the model-based assumption class of Outcome Performativity as features with extremely skewed Gaussian or Bimodal distributions (as 17 Anonymous 0.00.10.20.30.40.50.60.70.80.91.0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 Titanic (N = 35) 0.00.10.20.30.40.50.60.70.80.91.0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 Loan (N = 213) 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 11: Results of applying OP-AB to detect misclassification-based Outcome Performativity. These results are in keeping with our main findings. OP-AB exhibits a low false detection rate (i.e.γ 0 = γ 1 = 1.0) and ifNis larger, OP-AB can detect subtler Outcome Performative Effects. in the case of theLoandataset) can cause significant portions of the features to change outcomes at smallβ n values. This is not a limitation of OP-AB, but rather of the assumption class of Outcome Performativity. -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Titanic (N = 35) -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Loan (N = 213) 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 12: Results of applying OP-AB to detect model-based Outcome Performativity. These results are in keeping with our main findings. OP-AB exhibits a low false detection rate whenβ 0 = β 1 ) and ifNis larger, OP-AB can detect subtler Outcome Performative Effects. D.3 Using OP-AB with Other Statistical Tests In this work we primarily used the Chi-Squared Test to determine if the outcome distributions per intervention were significant. OP-AB is hypothesis test agnostic so we wanted to test the efficacy of OP-AB over different Statistical Tests. Figure 13 reports the detection rate of OP-AB applied to the Simple Model of Outcome Performativity using the Chi-Squared, Fisher’s Exact of Mann-Whitney U tests. Each cell is averaged over100replicates. We conducted these experiments withN = 50. The results reported in Figure 13 clearly show that across the statistical tests evaluated, the detection rate of OP-AB remains approximately the same. The Mann-Whitney U test seems slightly more unreliable 18 Detecting Outcome Performativity Using Intervention Testing 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 Chi-Squared Test 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 Fisher's Exact Test 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 Mann-Whitney U test 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 13: Results of using various Hypothesis Tests with OP-AB to detect Outcome Performativity. These results show that OP-AB is agnostic to the H used to detect Outcome Performativity (as long as H is appropriately applied). when the Outcome Performativity response is low (i.e. small|α 0 − α 1 |. These results reveal two additional insights: First,Nis without a doubt the most important parameter in OP-AB. If too few instances are used in the intervention testing, OP-AB will not detect Outcome Performativity. Second, the trends estimated by Eq. 9, 14 and 12 in the main paper capture the statistical test-agnostic dynamics of the Outcome Performativity Models in this work. D.4 Varying Detection Threshold δ In OP-AB, the detection thresholdδcorresponds to the necessary p-value required by Hypothesis testHto raise a detection event. Naturally, one would assume that asδincreases, the OP-AB becomes less restrictive and will more likely flag Outcome Performativity. This should also be doable with fewer random samples. Conversely, asδdecreases, OP-AB will become more restrictive and less likely to flag Outcome Performativity. A detection event, on overage, will require more samples asδdecreases. The trade-off when choosingδis between sample complexity (number of samples Nneeded per group) and the reliability of a detection event (a higherδwill produce more false positives). Figure 14 showcases the sample complexity dynamics across various commonly used statistical thresholds. These results confirm our suspicions, as δ increases, sample complexity N decreases. D.5 Increased N on Breast Cancer Dataset For the final set of additional experiments, we wanted to empirically demonstrate the importance thatNplays in being able to detect Outcome Performativity. Figure 15 shows these results where we apply OP-AB to theBreast Cancer dataset imputed with Feature-based Outcome Performativity over an increasingN. The results clearly show that as Nincreases, OP-AB is able to detect Outcome Performativity across a wider range ofβ 0 andβ 1 values. Again, we want to note that our results suggest that as the difference in Outcome Performativity responses tend to non-Outcome Performativity (|β 0 − β 1 |→ 0.0), the number of instances required per intervention N →∞. D.6 Comparison with Adam Detection To the best of our knowledge, the only other explicit Outcome Performativity detection method is described in [3]. In short, this method (which we will call Adam Detection) monitors the change in True Positive Rate (TPR) of a deployed model compared to its TPR that it achieved on its historical (training) data. If the change in TPR is determined to be statistically significant, Outcome Performativity is detected. Figures 16 summarizes the results of Adam Detection on several datasets imputed with the Misclassification Outcome Performativity described earlier. In all figures we can see that the detection rates of Adam Detection are lower and more inconsistent when compared to the results obtained by OP-AB. This occurs because Adam Detection requires a deployed model. Under Misclassification Outcome Performativity, the more accurate the deployed model classifier, the harder it is to detect said Outcome Performativity as only misclassified instances are susceptible to having their observed outcomes altered. This is most evident in Figure 16a where the deployed model achieves high accuracy, rarely misclassifying instances. Adam Detection is also less sample efficient because it requires historical labelled data to initially train a model, and labelled data after the model has been deployed. This is shown in Figures 16b and 16c where Adam Detection exhibits the same detection rate 19 Anonymous 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 100 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 1000 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10000 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate (a) δ = 0.01 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 100 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 1000 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10000 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate (b) δ = 0.05 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 100 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 1000 0.00.10.20.30.40.50.60.70.80.9 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 N = 10000 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate (c) δ = 0.1 Figure 14: Results from exploring various detection thresholdsδon the Simple Outcome Performativity Model. These results show that asδdecreases, sample complexityNincreases. The trade-off being that detection events at a smaller δ is more reliably than a detection event at a larger δ. 20 Detecting Outcome Performativity Using Intervention Testing -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Breast Cancer (N = 28) -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Breast Cancer (N = 71) -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Breast Cancer (N = 142) -1.0-0.8-0.6-0.4-0.2 0.00.20.40.60.81.0 0 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 1 Breast Cancer (N = 213) 0.2 0.4 0.6 0.8 1.0 Detection Rate Figure 15: Results demonstrating the importance the number of instances per intervention (N) plays on the efficacy of OP-AB. A higher N allows OP-AB to detect Outcome Performativity across a wider range of β 0 and β 1 values . 0.00.10.20.30.40.50.60.70.80.91.0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 0.120.240.240.320.40.160.280.240.160.20.12 0.160.320.160.20.240.280.280.240.360.280.24 0.20.320.20.160.160.240.120.40.160.160.24 0.40.320.20.160.160.120.240.280.080.240.12 0.160.240.360.120.240.120.320.160.20.280.28 0.20.280.360.280.320.40.20.20.080.240.2 0.360.280.320.280.240.280.20.280.120.240.16 0.320.080.280.20.280.120.160.20.240.320.28 0.320.120.160.120.160.240.240.120.280.20.28 0.20.240.360.240.280.20.20.280.440.320.36 0.440.280.240.40.280.560.120.320.360.40.12 Breast Cancer Dataset 0.1 0.2 0.3 0.4 0.5 Detection Rate (a) 0.00.10.20.30.40.50.60.70.80.91.0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 0.960.880.520.640.40.360.120.120.080.160.04 0.840.640.680.440.20.240.440.160.120.080.08 0.840.640.640.440.320.20.120.00.080.00.04 0.640.560.40.360.40.160.160.040.080.00.04 0.720.520.760.40.240.20.120.080.160.160.04 0.640.280.240.360.240.080.160.160.120.20.36 0.520.320.320.320.240.280.120.080.160.280.24 0.40.360.440.240.160.120.20.40.240.440.32 0.560.360.20.320.120.20.360.480.360.480.68 0.320.120.320.280.320.280.360.520.680.60.72 0.40.40.360.320.280.560.480.520.480.760.76 Adult Census Dataset 0.0 0.2 0.4 0.6 0.8 Detection Rate (b) 0.00.10.20.30.40.50.60.70.80.91.0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 1.01.01.00.960.960.920.680.520.40.040.0 1.01.00.960.920.920.840.720.320.160.00.08 1.00.920.960.920.840.840.440.160.040.080.12 0.921.00.880.920.640.680.360.160.080.080.24 0.960.880.760.80.560.520.40.080.080.080.32 0.880.760.80.720.760.520.120.080.160.20.28 0.920.840.720.560.440.280.080.120.120.240.6 0.920.760.640.640.480.080.160.240.320.360.48 0.720.680.560.560.280.20.080.160.360.520.56 0.520.360.560.320.240.240.240.440.560.520.68 0.640.520.20.20.20.20.360.280.40.60.6 Kickstarter Dataset 0.0 0.2 0.4 0.6 0.8 1.0 Detection Rate (c) Figure 16: Adam Detection performed on the Breast Cancer (a), Adult Census (b) and Kickstarter (c) datasets. Overall, the detection rate of Adam Detection is inconsistent. This is partly due to the accuracy of the deployed classifiers which when high, rarely produce misclassifications making detection difficult. The number of instances in each dataset also play a role with more instances increasing the likelihood of detection such as in the Kickstarter dataset. Our method, OP-AB, achieves higher and more consistent detection rates with only 10% of the samples used by Adam Detection. trends as OP-AB (although less consistent), but it required the whole of theAdult CensusandKickstarterdatasets whereas OP-AB achieved better and more consistent detection rates with only10%of the datasets’ instances. Adam Detection is an online algorithm, which is susceptible to both Intrinsic [25] and non-Outcome Performative [6] Concept Drift phenomena. These results put the primary advantages of OP-AB into perspective. Because OP-AB is an offline algorithm, it is (1) not susceptible to Concept Drift, and (2) it can be used already during the labelling process of a ML model’s development lifecycle. This has the added benefit of requiring fewer instances (i.e. better sample efficiency) and it eliminates any potential negative impacts that might occur from a deployed model that is used before the Outcome Performativeness of a problem domain is detected. D.6.1 Implementing Adam Detection Unfortunately, we were not able to obtain the source code for Adam detection so we had to remake it using the text-only description provided in the referenced paper [3]. This was slightly problematic as the paper did not describe the Online Updating procedure for the deployed model which made it challenging to know if we replicated Adam detection exactly. In short, our implementation of Adam will take in an entire static dataset, shuffle it and partition it into11approximately equal sized chunks. The 11th chunk is used as the test set. The first5of the remaining sets are used as historical data to determine the deployed model’s True Positive Rate (TPR) before deployment and the last final5sets are used to iteratively update the deployed model in an Online fashion. These Online sets are susceptible to the Adam Outcome Performativity described in Sect. 4.3. At the end of each update, the model’s new TPR is recorded. Once training is complete, a Mann-Whitney U test is used to determine if the difference in TPR achieved on the historical datasets are statistically different from the TPR achieved on the iterative (deployed) sets. For these experiments, reported results are the average over 25 replicates and a δ = 0.05 was used to determine statistical significance. 21 Anonymous 0.00.10.20.30.40.50.60.70.80.91.0 P(Y = 0 | Y = 0) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 P ( Y = 1 | Y = 1) 1.00.990.990.960.950.940.920.920.940.961.0 0.90.90.890.880.860.840.830.840.870.910.94 0.80.790.790.790.770.750.750.770.810.840.88 0.680.690.70.690.690.670.680.710.750.780.85 0.540.550.570.570.550.570.60.650.70.770.84 0.370.390.40.460.470.50.550.640.720.80.85 0.240.250.290.330.40.490.560.680.770.870.92 0.140.20.250.290.360.470.560.680.790.890.98 0.090.140.190.280.350.450.560.70.80.880.99 0.040.110.180.240.330.430.560.680.790.911.0 0.00.060.140.220.320.440.540.670.780.891.0 Heatmap of Average Accuracy obtained by Linear Model exploring different P(Y | Y) using Standard Supervised train-test loop 0.0 0.2 0.4 0.6 0.8 1.0 Average Accuracy (a) 0.00.10.20.30.40.50.60.70.80.91.0 P(Y = 0 | Y = 0) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 P ( Y = 1 | Y = 1) 1.01.01.01.01.01.00.990.990.980.981.0 0.90.90.890.890.890.890.880.880.870.910.95 0.810.790.790.790.790.780.770.780.810.870.96 0.70.70.710.690.690.670.680.710.770.860.98 0.60.60.590.570.550.570.60.670.760.880.99 0.50.480.460.480.470.50.570.670.770.91.0 0.390.370.340.350.40.480.560.680.780.91.0 0.290.280.270.290.370.460.570.670.80.91.0 0.180.170.190.280.370.470.570.70.80.881.0 0.060.110.180.270.370.470.580.690.780.911.0 0.00.080.190.290.390.50.590.70.80.91.0 Heatmap of Average Accuracy obtained by Linear Model exploring different P(Y | Y) using Prediction Conditioning 0.0 0.2 0.4 0.6 0.8 1.0 Average Accuracy (b) 0.00.10.20.30.40.50.60.70.80.91.0 P(Y = 0 | Y = 0) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 P ( Y = 1 | Y = 1) 0.00.010.010.020.040.070.070.060.040.010.0 0.0-0.0-0.00.010.020.040.040.030.010.00.01 0.010.0-0.0-0.00.010.020.010.010.00.030.08 0.010.00.010.01-0.00.00.00.00.020.080.13 0.090.040.010.01-0.0-0.010.00.020.060.110.16 0.140.090.060.030.010.00.00.040.060.10.14 0.170.120.050.020.0-0.00.0-0.00.010.030.08 0.140.090.030.00.0-0.010.01-0.00.010.010.03 0.090.030.00.010.020.020.010.00.00.00.0 0.020.00.010.030.040.040.030.01-0.00.0-0.0 0.00.020.040.070.080.060.050.030.010.00.0 Difference in Average Accuracy obtained by Linear Model with and without Prediction Conditioning over different P(Y = 0 | Y) 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Average Accuracy (c) Figure 17: Figure (a) shows the average accuracy achieved by a Linear Model (trained using standard Supervised Learning) over various parameters in the Simple Model of Outcome Performativity. Figure (b) plots the same using prediction conditioning whereby the Linear Model can predict the outcomes (Y) that will be observed given the prediction it makes( ˆ Y). Figure (c) shows the difference in accuracy achieved by each model ((b)− (a)) with prediction conditioning performing better, particularly around areas of label imbalance in the original dataset. E Training Machine Learning Models in Outcome Performative Settings Although it is not the focus of this paper, we do want demonstrate why training models in an Outcome Performative setting is different than in the traditional Supervised setting. For this, we will use thebreast cancer[16] dataset and the Simple Model of Outcome Performativity described in the paper. All results shown are average of 25 runs. The first challenge in the Outcome Performative setting is gathering data. This has to come from historical data where a prediction ˆ Yhas already been made and an outcomeYobserved. To simulate this in these demonstrations, we train a base classifier on thebreast canncerdataset, and use it’s predictions to simulate outcomes for our real modelθ to learn from. Assuming that we just use a standard Supervised training-test process, we would get results shown in Figure 17a. Overall, the results are about what we expected, the accuracy of the model increases when the probability of encountering a labelP (Y | ˆ Y )→ 1.0. WhenP (Y | ˆ Y )→ 0.0the accuracy decreases as the model will never make a correct prediction. These results are unsurprising, but the modelθis far from achieving the maximum achievable accuracy ofmax(P (Y | ˆ Y = 0),P (Y | ˆ Y = 1)). We can somewhat solve this problem by introducing prediction conditioning. That is when the model can be conditioned on the prediction it would make in order to predict the final outcome it will observe. In practice, this is just adding an additional feature to the model ˆ Y =0, 1. This introduces a new problem where we must now choose between the probability scores produced for each class (e.g. θ ˆy=0 (x) vs. θ ˆy=1 (x) for a binary classification task). If we want to maximize accuracy as we did in the standard Supervised setting, we add a decision rule to determine our final prediction ˆy: ˆy = 0 if P θ (Y = 0|θ ˆy=0 (x) = 0) > P θ (Y = 1|θ ˆy=1 (x) = 1) 1 else (26) whereP θ is the estimated probability score of outcomeYwhenθis conditioned on predictionˆy. In short, this decision rule will predictˆy = 0if the estimated probability of observing outcomey = 0(givenˆy = 0) is greater than observing outcomey = 1(givenˆy = 1). The results of using a model with prediction conditioning is shown in Figure 17b. Overall, the results look similar to that of the model without prediction conditioning, but if we observe Figure 17c which plots the difference in accuracy of the model with prediction conditioning−the accuracy of the model without, we can see that that prediction conditioning can improve the accuracy of the model dramatically. For thebreast cancer dataset, the greatest benefits are at parameter ranges around that correlate with the data imbalances in the original dataset. Naively, these results imply that prediction conditioning is good, and sufficient for solving Outcome Performative problems. This may not always be the case. Yes, prediction conditioning does allow one to gain insight into how their predictions may affect observed outcomes, but it relies heavily on the decision rule you use. Is accuracy all you really care about, then the aforementioned decision rule works well. Consider the palliative care problem, we don’t necessarily just want want to be accurate, we actually want to maximize an outcome (a patient’s quality or life). If our deployed model is maximizing accuracy, it may over or under-prescribe interventional care. Fortunately, changing the model’s behaviour is (in this abstract context) quite simple, we just need to change the decision-rule. We demonstrate that in Figure 18 where we show how the same prediction conditioned model can be used to maximize different outcomes (Y = 1 in this case) just by changing the decision rule to: 22 Detecting Outcome Performativity Using Intervention Testing 0.00.10.20.30.40.50.60.70.80.91.0 P(Y = 0 | Y = 0) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 P ( Y = 1 | Y = 1) 1.01.01.01.01.01.01.01.01.01.01.0 1.00.90.880.90.890.90.890.90.90.910.9 1.00.890.790.780.80.80.80.80.810.810.8 1.00.90.790.70.70.70.710.710.710.680.7 1.00.910.790.680.590.60.60.590.60.610.61 1.00.90.810.680.590.480.490.480.490.50.49 1.00.910.820.710.590.460.390.380.40.40.4 1.00.890.80.710.610.490.40.310.270.30.3 1.00.90.810.690.590.50.410.290.210.190.21 1.00.890.790.70.620.520.410.290.20.090.11 1.00.90.80.70.60.490.410.30.20.10.0 Average Likelihood of achieving an Outcome (Y = 1) using a Linear Model exploring different P(Y | Y) 0.0 0.2 0.4 0.6 0.8 1.0 Likelihood of achieving an Outcome ( Y = 1) Figure 18: Figure showing how the same model used in Figure 17b can be used to produce varying results over the same parameters. In this case, a simple change in the decision rule (from maximizing accuracy to maximizing observing an outcome ofy = 1) produced the following figure. This is meant to highlight the difficulty and potential danger of naively deploying predictive models in settings that are Outcome Performative. ˆy = 0 if P θ (Y = 1|θ ˆy=0 (x) = 1) > P θ (Y = 1|θ ˆy=1 (x) = 1) 1 else (27) The results clearly show that in the Outcome Performative setting, how you use your deployed model can have massive effect on the types of outputs you get. Before our model struggled to accurately predict outcomes in regions it is now achieving perfect scores in. This further acts as motivation for using OP-AB. Being aware of Outcome Performativity is the first step in understanding how to deploy a model in an Outcome Performative setting. These results also show how using prediction conditioning and different decision rules can produce vary different results, which further leads into ethical concerns that could arise given that these models will be deliberately steering outcomes towards results specified by their desired outcomes (i.e. decision rule). F Applying OP-AB to Monitoring Tasks We introduced OP-AB as a "one-and-done" algorithm. That is, you give OP-ABTinstances to perform intervention testing on and the output of the algorithm is a "yes/no" answer to whether the setting is Outcome Performative. Practically, you may want to repeatedly evaluate a setting (perhaps the Outcome Performativeness of setting itself is dynamic or subject to Concept Drift). This would then be a monitoring task we you would have to repeatedly have to re-evaluate the Outcome Performativeness of your problem domain. Algorithm 2 details how you might do this. Simply put, everyZinstances, you would dedicateTinstances to OP-AB and use the result to inform future decision making. References [1] Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. Performative prediction. In Hal Daumé I and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 7599–7609. PMLR, 2020. [2]Gary Cheng, Moritz Hardt, and Celestine Mendler-Dünner. Causal inference out of control: estimating per- formativity without treatment randomization. In Forty-first International Conference on Machine Learning, 2024. [3]George Alexandru Adam, Chun-Hao Kingsley Chang, Benjamin Haibe-Kains, and Anna Goldenberg. Hidden risks of machine learning applied to healthcare: unintended feedback loops between models and future data causing model degradation. In Machine Learning for Healthcare Conference, pages 710–731. PMLR, 2020. 23 Anonymous Algorithm 2 Simple Pseudocode for using OP-AB in a monitoring task. The code is intentionally vague about what should be done when a positive test is found, and what is done with data batch X . This will be task specific. Require: Data stream X , Test Frequency Z, Test Size T 1: x = X.next_batch() 2: T = 0 3: while x̸=∅ do 4: condition = T mod Z = 0 and OPAB(x,T ) 5:if condition then 6:Report Results 7:end if 8:Perform operations on x 9: x = X.next() 10: T + = 1 11: end while [4]Wouter AC van Amsterdam, Nan van Geloven, Jesse H Krijthe, Rajesh Ranganath, and Giovanni Ciná. When accurate prediction models yield harmful self-fulfilling prophecies. Patterns, 6(4), 2025. [5]Michael P Kim and Juan C Perdomo. Making decisions under outcome performativity. In 14th Innovations in Theoretical Computer Science Conference (ITCS 2023), pages 79–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2023. [6] Brandon Gower-Winter, Georg Krempl, Sergey Dragomiretskiy, Tineke Jelsma, and Arno Siebes. Identifying predictions that influence the future: Detecting performative concept drift in data streams. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025. [7] Nancy Cartwright. What are randomised controlled trials good for? Philosophical studies, 147(1):59–70, 2010. [8] Judea Pearl. Causality. Cambridge university press, 2009. [9] Celestine Mendler-Dünner, Frances Ding, and Yixin Wang. Anticipating performativity by predicting from predictions. Advances in neural information processing systems, 35:31171–31185, 2022. [10] George Alexandru Adam, Chun-Hao Kingsley Chang, Benjamin Haibe-Kains, and Anna Goldenberg. Error amplification when updating deployed machine learning models. In Machine Learning for Healthcare Conference, pages 715–740. PMLR, 2022. [11]Nicolò Pagan, Joachim Baumann, Ezzat Elokda, Giulia De Pasquale, Saverio Bolognani, and Anikó Hannák. A classification of feedback loops and their relation to biases in automated decision-making systems. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1–14, 2023. [12]James Liley, Samuel Emerson, Bilal Mateen, Catalina Vallejos, Louis Aslett, and Sebastian Vollmer. Model updating after interventions paradoxically introduces bias. In International Conference on Artificial Intelligence and Statistics, pages 3916–3924. PMLR, 2021. [13]Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, 116(10):4156–4165, 2019. [14]Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), 2018. [15] Yingrong Wang, Haoxuan Li, Minqin Zhu, Anpeng Wu, Baohong Li, Keting Yin, Ruoxuan Xiong, Fei Wu, and Kun Kuang. Causal inference with complex treatments: A survey. ACM Computing Surveys, 58(9):1–36, 2026. [16] Matjaz Zwitter and Milan Soklic. Breast Cancer, 1988. Accessed: April 2025. [17] Jack W Smith, James E Everhart, William C Dickson, William C Knowler, and Robert Scott Johannes. Using the adap learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the annual symposium on computer application in medical care, page 261, 1988. Accessed: April 2025. [18] Barry Becker and Ronny Kohavi. Adult census income, 1996. Accessed: April 2025. [19] Lathwal. Funding successful projects on kickstarter, 2017. Accessed: April 2025. [20] Will Cukierski. Titanic - machine learning from disaster, 2012. Accessed: April 2025. 24 Detecting Outcome Performativity Using Intervention Testing [21] Kai. Loan approval prediction dataset, 2023. Accessed: August 2024. [22]Yuta Saito, Aihara Shunsuke, Matsutani Megumi, and Narita Yusuke. Large-scale open dataset, pipeline, and benchmark for bandit algorithms. arXiv preprint arXiv:2008.07146, 2020. [23]Georg Krempl, Indre Žliobaite, Dariusz Brzezi ́ nski, Eyke Hüllermeier, Mark Last, Vincent Lemaire, Tino Noack, Ammar Shaker, Sonja Sievi, Myra Spiliopoulou, et al. Open challenges for data stream mining research. ACM SIGKDD Explorations Newsletter, 16(1):1–10, 2014. [24]Zeyu Ren, Shuihua Wang, and Yudong Zhang. Weakly supervised machine learning. CAAI Transactions on Intelligence Technology, 8(3):549–580, 2023. [25]João Gama, Indr ̇ e Žliobait ̇ e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM computing surveys (CSUR), 46(4):1–37, 2014. 25