Paper deep dive

SR4-Fit: A Unified Interpretable Rule-Based Machine Learning Framework for Informative and Trustworthy Decision-Making

Shyam Sundar Murali Krishnan, Dean Frederick Hougen

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 237

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:In many high-stakes applications, machine learning is dominated by black-box models that require post hoc explanations to justify their predictions. These explanations are often unreliable because they do not reflect the model's actual computations, limiting accountability and trust. A natural alternative is to use models that are interpretable by design. However, existing rule-based approaches, such as RuleFit and decision trees, while transparent, often lack stability and predictive strength, reinforcing a perceived trade-off between traditional performance measures and model understandability. To address this, we propose Sparse Relaxed Regularized Regression Rule-Fit (SR4-Fit), an intrinsically interpretable algorithm for both classification and regression that produces compact and stable rule sets without sacrificing performance. Using demographic data from the U.S. Census Bureau's American Community Survey, SR4-Fit predicts U.S. House election outcomes with high accuracy and interpretability while uncovering demographic interactions missed by black-box models. We further validate SR4-Fit across fourteen benchmark datasets (six classification and eight regression), where it outperforms existing rule-based methods, including RuleFit and decision trees in terms of accuracy, stability, and compactness while remaining competitive with black-box models in predictivity. These results demonstrate that interpretability and predictive reliability need not be mutually exclusive, offering a practical and transparent alternative for high-stakes decision-making.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

233,211 characters extracted from source content.

Expand or collapse full text

Highlights SR4-Fit: A unified interpretable rule-based machine learning framework for informative and trustworthy decision-making Shyam Sundar Murali Krishnan, Dean Frederick Hougen • Proposed SR4-Fit to resolve the predictivity-interpretability trade-off for both classification and regression via sparse rule sets. • Introduced Rule Understandability Score and Pareto analysis show SR4-Fit dominates rule-based baselines. • Enhanced Interpretability Score incorporates rule complexity for fairer comparison. • SR4-Fit interpreted demographic interactions in House of Representatives elections via human-readable rules. • Validated across 14 benchmark datasets, confirming generalizability and robustness. arXiv:2609.34019v1 [cs.LG] 27 Sep 2026 SR4-Fit: A unified interpretable rule-based machine learning framework for informative and trustworthy decision-making Shyam Sundar Murali Krishnan a , Dean Frederick Hougen a a School of Computer Science, University of Oklahoma, Norman, Oklahoma, USA A R T I C L E I N F O Keywords: SR4-Fit Interpretable AI Rule-Based Algorithms Intrinsic Interpretability Predictivity-Interpretability Trade- off Electoral Forecasting. A B S T R A C T In many high-stakes applications, machine learning is dominated by black-box models that require post hoc explanations to justify their predictions. These explanations are often unreliable because they do not reflect the model’s actual computations, limiting accountability and trust. A natural alternative is to use models that are interpretable by design. However, existing rule- based approaches, such as RuleFit and decision trees, while transparent, often lack stability and predictive strength, reinforcing a perceived trade-off between traditional performance measures and model understandability. To address this, we propose Sparse Relaxed Regularized Regression Rule-Fit (SR4-Fit), an intrinsically interpretable algorithm for both classification and regression that produces compact and stable rule sets without sacrificing performance. Using demographic data from the U.S. Census Bureau’s American Community Survey, SR4-Fit predicts U.S. House election outcomes with high accuracy and interpretability while uncovering demographic interactions missed by black-box models. We further validate SR4-Fit across fourteen benchmark datasets (six classification and eight regression), where it outperforms existing rule-based methods, including RuleFit and decision trees in terms of accuracy, stability, and compactness while remaining competitive with black-box models in predictivity. These results demonstrate that interpretability and predictive reliability need not be mutually exclusive, offering a practical and transparent alternative for high-stakes decision-making. 1. Introduction Machine learning (ML) models such as random forests [1], Support Vector Machines (SVMs) [2], XGBoost[3], and neural networks [4] have transformed how data-driven problems are approached across science and society. Despite their success, these models are often criticized as black boxes that capture complex, nonlinear patterns but provide little clarity on how inputs influence predictions. In many domains, this opacity limits the ability to verify, explain, or trust automated decisions, especially when those decisions have real-world consequences [5, 6]. These concerns have motivated a growing interest in interpretable machine learning which aims to design models that not only perform well but can also be understood and validated by humans [7]. Interpretability becomes particularly important in areas where transparency and accountability are essential, such as the study of U.S. House of Representatives elections. Understanding how demographic and structural factors shape political outcomes is central to evaluating fair representation in a democracy. Electoral behavior depends not only on individual choices but also on how districts are drawn, which can amplify or diminish the influence of certain groups. Demographic characteristics such as race, education, income, and age strongly correlate with voting patterns and party preference [8]. Modeling these relationships helps estimate district security—the likelihood that a political party will retain control in future elections—and can also provide insight into practices such as gerrymandering—the deliberate manipulation of electoral district boundaries to grant a political party, incumbent, or group an unfair advantage [9]. In this setting, models that balance predictivity with interpretability can offer more than just forecasts; they can help explain why elections turn out the way they do, providing models that are informative regarding the domain to which they are applied [10]. Black-box models have proven capable of predicting congressional election results with high accuracy [10]. Yet the inner workings of these models often remain obscure, making it difficult to identify which demographic patterns drive the predictions. This reflects a well-known tension between predictive accuracy and interpretability [11]: complex models tend to predict well but are hard to understand, whereas simpler models may be easier to interpret but often lose predictive strength. Finding methods that achieve both remains an active and important challenge. ORCID(s): 0000-0001-9239-4390 (S.S.M. Krishnan); 0000-0001-5393-1480 (D.F. Hougen) S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 1 of 77 Predictivity Understandability Deep Learning Ideal Support Vector Machines Random Forests Decision Trees Classification Rules Linear Regression Graphical Models RuleFit SR4-Fit Figure 1: Conceptual illustration of the predictivity–understandability trade-off across common machine learning models. SR4-Fit is positioned closer to the ideal region of high predictivity and high understandability, reflecting its aim to balance both criteria more effectively than existing rule-based approaches. One common strategy has been to use post-hoc explanation tools such as SHapley Additive exPlanations (SHAP) [12] and Local Interpretable Model-agnostic Explanations (LIME) [13]. These approaches provide approx- imate insights into a model’s behavior, such as feature importance or local decision boundaries. While valuable for interpretive analysis, post-hoc explanations can diverge from what the model is actually computing, particularly for non-linear systems. As a result, they can aid understanding but do not fully solve the transparency problem. An alternative direction focuses on intrinsically interpretable model algorithms whose structure makes their reasoning transparent by design [14]. These include decision trees [15], sparse linear models [16], and rule-based systems [17]. Because the logic behind each prediction is directly encoded in the model, such approaches make it easier to verify results and reason about causal patterns. While intrinsic interpretability does not automatically guarantee higher accuracy, it offers a solid basis for building models that are both reliable and understandable. In contexts like electoral forecasting, this combination supports both empirical rigor and public accountability. Among intrinsically interpretable methods, rule-based models are particularly appealing because they express decisions as if–then statements that can be read and understood directly. These models align closely with human reasoning and lend themselves naturally to policy and social analysis. However, existing rule-based techniques vary widely in their design and scope. Some focus only on binary classification [18, 19, 20], while others prioritize simplicity to the point of losing predictive depth or are tailored to a single task type, either classification or regression [21]. These limitations reinforce the perception that interpretability often comes at the expense of performance. The RuleFit algorithm [22] represented a key attempt to balance this trade-off by combining rule extraction from tree ensembles with linear modeling. While RuleFit offers a useful middle ground, it can struggle with stability, consistency, and predictivity, especially when applied to high-dimensional or noisy data [23]. The rules it generates can overlap or vary across runs, complicating interpretation and limiting trust in the results. Further refinements are therefore needed to make rule-based modeling more robust and versatile. To address these issues, we introduce Sparse Relaxed Regularized Regression Rule Fit (SR4-Fit), an algorithm for generating intrinsically interpretable rulesets that extends the RuleFit framework using the sparse optimization principles of Sparse Relaxed Regularized Regression (SR3) [24]. SR4-Fit is designed to produce compact and stable rule sets while maintaining competitive predictive accuracy. Its formulation allows fine-grained control over sparsity, offering a single framework that works for both classification (binary and multi-class) and regression tasks. Over the past decade, the literature on understandable AI has become replete with notional diagrams similar to Figure 1, showing greater understandability (or a related term such as explainability, interpretability, etc.) on one S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 2 of 77 axis and greater predictivity (accuracy, learning performance, etc.) on the other, along with a sampling of ML types scattered across the plot to indicate a tradeoff between understandability and predictivity, with complex ML types such as deep learning and random forests shown as featuring high predictivity but low understandability; simple types such as decision rules and linear regression shown as featuring high understandability but low predictivity; and approaches of intermediate complexity such as decision trees, graphical models, and support vector machines shown as more moderate in both understandability and predictivity. Such diagrams have their origins in DARPA’s Explainable Artificial Intelligence Program [25, 26], and have been included in papers on explainable AI itself [27] as well as in applications across a wide array of fields, including modeling human decision-making [28], wireless communications [29], fraud detection [6], natural language processing [30], healthcare [31, 32, 33], ecology [34], energy [35, 36, 37], and finance [38]. Figure 1 also illustrates the intuition that RuleFit [22], which first produces highly predictive random forests and then uses those to generate understandable decision rules, sacrifices less predictivity and less understandability than the intermediate complexity methods, and that SR4-Fit, by improving on the rule creation process of RuleFit, takes us further toward the ideal of high predictivity and high understandability. Using demographic data from the U.S. Census Bureau’s American Community Survey, SR4-Fit predicts U.S. House election outcomes with strong accuracy while generating interpretable rules that highlight meaningful demo- graphic interactions such as the influence of education, racial composition, and age distribution on party alignment. In the regression setting, we further examine the model’s ability to capture continuous electoral variation by using district-level Democratic and Republican vote percentages as target variables. This allows SR4-Fit to quantify how demographic gradients correspond to shifts in partisan support, offering a more detailed view of electoral behavior. Beyond elections, the model is tested on a suite of benchmark datasets, six for classification and eight for regression, where it performs competitively with both black-box and existing interpretable methods. Together, these results suggest that intrinsic interpretability and predictive reliability can coexist for tree-based algorithms, offering a practical path toward models that are transparent, stable, and empirically strong across different problem settings. Beyond the quantitative analysis of this data, we provide extensive discussion of the rules inferred by SR4-Fit. 2. Contributions The main contributions of this paper are as follows: 1. We propose SR4-Fit, an intrinsically interpretable rule-based algorithm that directly addresses the traditional trade-off between interpretability and predictivity by producing compact, stable, and human-readable rule sets without sacrificing performance. 2. SR4-Fit provides a unified framework for both classification and regression tasks, extending the capabilities of existing rule-based methods such as RuleFit, which are limited in predictive performance, and decision trees, which are limited in both stability and performance. 3. We introduce the Rule Understandability Score (RUS), a three-component metric combining rule stability, compactness, and complexity that deliberately excludes predictive performance, enabling an independent assessment of interpretability. We employ RUS in a Pareto analysis against predictivity, providing a model- selection perspective that does not depend on a particular understandability-predictivity weighting scheme. 4. We enhance the Interpretability Score (IPS) proposed by Margot and Luta [39] by incorporating rule complexity as an additional component alongside predictive accuracy, rule stability, and model compactness. Our enhanced interpretability score (E-IPS) provides a more comprehensive and principled basis for comparing rule-based models. SR4-Fit achieves consistently high E-IPS scores across both classification and regression tasks, demonstrating that it produces stable, compact, and interpretable rule sets while maintaining competitive predictive performance. 5. We validate SR4-Fit on U.S. House election demographic data, where interpretable rules uncover meaningful demographic interactions that black-box models obscure, demonstrating the practical value of intrinsic inter- pretability in high-stakes decision-making. 6. We further validate SR4-Fit across fourteen publicly available benchmark datasets (six classification and eight regression), confirming its generalizability and robustness across diverse domains. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 3 of 77 One-vs-Rest Forests (Classification) One-vs-Rest Forests (Classification) One-vs-Rest Forests (Classification) Construct One-vs-Rest Forests (Classification) Xy Repeat entire pipeline once per class 푐 Construct Single Forest (Regression) Extract Rules Root-to-leaf paths, bounded by 푇 푚 Construct Features 푍=[푋|푅] Initialize 훽,푤 Update 훽 Gradient step (cls.) or closed-form (reg.) Update 푤 Soft thresholding Converged? Sparse Optimization (SR3) No Ye s Prune Rule Set Drop 푤 푗 =0 Predict Sigmoid (cls.) or linear (reg.) Output Sparse ruleset + �푦 Data (1)(2)(3) (4) (5) Figure 2: Overview of the SR4-Fit pipeline (including classification and regression). 3. Methodology SR4-Fit is a rule-based learning algorithm that combines the interpretability approach of RuleFit with the sparse optimization capabilities of SR3 to produce compact, stable, and human-readable predictive models for both classification and regression tasks. Like RuleFit, SR4-Fit extracts decision rules from an ensemble of decision trees to capture non-linear patterns in the data and then fits a sparse linear model over both the extracted rules and the original input features. To support classification, the SR3 optimization objective is modified by replacing the squared loss with logistic loss (see Algorithm 1), allowing the model to output class probabilities rather than continuous values [40]. For regression, the squared-error loss is retained within the same optimization framework (see Algorithm 2), enabling direct prediction of continuous target variables. In both settings, SR3’s structured regularization balances predictive accuracy with sparsity, yielding interpretable models whose predictions can be traced through a compact set of human-readable rules. The algorithm proceeds in five stages: (1) random forest generation, (2) rule extraction, (3) feature construction, (4) sparse optimization, and (5) model pruning, as illustrated in Figure 2. 3.1. Random Forest Generation and Rule Extraction SR4-Fit begins by extracting interpretable decision rules from an ensemble of decision trees. In the classification setting, a separate forest is trained for each class using a one-versus-rest formulation [41], where each forest is trained to distinguish the class of interest from all others. In the regression setting, a single forest is trained to predict the continuous target variable directly, enabling it to capture nonlinear dependencies between input features and the output. To control model complexity and ensure interpretability, the maximum total number of rules extracted is bounded by the hyperparameter 푟 max . From each tree in the forest, rules are extracted by traversing every path from the root to a terminal node. Each path defines a rule as a conjunction of logical conditions on the input features. These rules are encoded as binary indicators: a rule evaluates to 1 if all its conditions are satisfied for a given input, and to 0 otherwise. For example, the rule 푋 1 ≤ 2.7 and 푋 3 > 1.4 evaluates to 1 if and only if 푋 1 ≤ 2.7 and 푋 3 > 1.4 hold simultaneously and to 0 otherwise. Here, 푋 are the features. Because rules are drawn from every tree across the ensemble, the resulting rule set captures diverse and complementary patterns across the input space, reflecting the ensemble’s collective coverage of the data. 3.2. Feature Construction and Optimization Objective Following rule extraction, SR4-Fit builds an extended feature matrix by combining the original input features with the binary rule indicators. For each data point, the rule evaluations are concatenated with the original feature values to form a comprehensive representation 푍 = [푋 | 푅] where 푋 ∈ℝ 푛×푑 where 푛 is the number of observations and 푑 is the number of features and 푅 ∈ 0, 1 푛×푚 . This expanded feature matrix allows the model to utilize both raw features and data-driven patterns captured by the trees. To learn a sparse and accurate classifier, the algorithm minimizes the loss function 퐿(훽,푤) with parameters 훽, the model’s predictive weight vector, and 푤, a vector enforcing sparsity, defined by: 퐿(훽,푤) = 푛 ∑ 푖=1 log(1 + exp (−푦 푖 푍 ⊤ 푖 훽) ) + 휆‖푤‖ 1 + 휅 2 ‖훽 − 푤‖ 2 2 (1) where 푦 푖 is the label. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 4 of 77 For regression tasks, the same structure is retained, but the logistic loss is replaced by the squared-error loss: 퐿(훽,푤) = 1 2 ‖푦 − 푍훽‖ 2 2 + 휆‖푤‖ 1 + 휅 2 ‖훽 − 푤‖ 2 2 .(2) In these objective functions, 휆 controls the sparsity and 휅 controls how closely 훽 must follow 푤. The algorithm minimizes these objective functions by alternating between the two update steps. First, with 푤 fixed, 훽 is updated using gradient-based optimization on the logistic loss for classification and a closed-form ridge-type solution on the squared error loss for regression, both balancing fitting accuracy with quadratic regularization that enforces 훽 ≈ 푤. Second, with 훽 fixed, 푤 is updated via soft-thresholding, which is key for achieving sparsity [42], using 푤 푗 = sign(훽 푗 )⋅ max ( |훽 푗 | − 휆 휅 , 0 ) (3) where sign(푧) = ⎧ ⎪ ⎨ ⎪ ⎩ +1, if 푧 > 0 0, if 푧 = 0 −1, if 푧 < 0. This iterative process continues until the relative change in the objective function falls below a small tolerance 휀 (typically 10 −4 ). The result is a compact, interpretable model in which the nonzero components of 푤 identify the features and rules that contribute most strongly to prediction accuracy. Algorithm 1 SR4-Fit Classification Algorithm 1: Input: Dataset (푋,푦), parameters 휆, 휅, 휀 and 푟 max . 2: Output: Predictive weights 훽. 3: for each class 푐 = 0, 1, ...,퐶 do 4: Train a random forest using one-vs-rest labeling for class 푐 [1]. 5: Extract decision rules from each path in each tree until 푟 max [3]. 6: Encode rules as binary indicators 푅 ∈ 0, 1 푛×푚 [43]. 7: Form an extended feature matrix 푍 = [푋 | 푅]. 8: Initialize 훽, 푤. 9: while not converged do 10:Minimize 퐿(훽,푤) = 푛 ∑ 푖=1 log(1 + exp (−푦 푖 푍 ⊤ 푖 훽)) +휆‖푤‖ 1 + 휅 2 ‖훽 − 푤‖ 2 2 11:Update 푤, 푤 푗 = sign(훽 푗 )⋅ max ( |훽 푗 | − 휆 휅 , 0 ) 12: end while 13: Prune rules where 푤 푗 = 0 14: end for 15: Prediction: 16: Compute 푠(푥) = 푑+푚 ∑ 푗=1 훽 푗 푍 푗 (푥) 17: Compute predicted probability ̂푦(푥) = 1 1+푒 (−푠(푥)) 3.3. Model Pruning and Rule Selection After optimization, SR4-Fit performs model pruning to retain only the most informative features and rules. The pruning step relies on the sparse auxiliary vector 푤, which reflects the importance of each feature and rule learned during training. Any component 푤 푗 = 0 indicates that the corresponding predictor contributes negligibly to the model and can be safely removed. The remaining nonzero elements of 푤 identify the subset of features and rules that form the final interpretable model. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 5 of 77 Algorithm 2 SR4-Fit Regression Algorithm 1: Input: Dataset (푋,푦), parameters 휆, 휅, maximum rules 푟 max , and tolerance 휀. 2: Output: Regression coefficients 훽. 3: Train a random forest of depth 푑 on (푋,푦). 4: Extract decision rules from all root-to-leaf paths up to 푟 max . 5: Encode each rule as a binary feature; form an extended matrix 푍 = [푋 | 푅]. 6: Initialize coefficients 훽, 푤. 7: while not converged do 8: Update 훽 by solving 훽 = arg min 훽 1 2 ‖푦 − 푍훽‖ 2 2 + 휅 2 ‖훽 − 푤‖ 2 2 9: Update 푤 using soft-thresholding: 푤 푗 = sign(훽 푗 )⋅ max(|훽 푗 | − 휆 휅 , 0) 10: Check convergence: |퐿 푡−1 −퐿 푡 | 퐿 푡−1 < 휀, where 퐿(훽,푤) = 1 2 ‖푦 − 푍훽‖ 2 2 + 휆‖푤‖ 1 + 휅 2 ‖훽 − 푤‖ 2 2 11: end while 12: prune rules with |푤 푗 | = 0. 13: Prediction: For a new instance 푥, compute ̂푦(푥) = ∑ 푑+푚 푗=1 훽 푗 푍 푗 (푥). This separation of roles between 훽 and 푤, where 훽 captures predictive strength and 푤 enforces sparsity, provides a more stable and interpretable selection process than methods that apply sparsity constraints directly to the predictive coefficients [24]. Because 푤 is updated through soft-thresholding, irrelevant or weakly correlated rules are automatically pruned, leading to a compact and transparent rule set. The pruning mechanism is identical for both classification and regression. By discarding inactive predictors and retaining only those associated with nonzero 푤 푗 values, SR4-Fit yields a sparse model that is both easy to interpret and efficient to evaluate. This final rule set highlights the dominant feature interactions uncovered during training, offering clear and human-readable insights into how the model forms its predictions. 3.4. Classification and Regression Prediction In the final stage, SR4-Fit uses the optimized coefficients훽 to make predictions for both classification and regression tasks. For a new input instance 푥, the model evaluates the active rules and features and computes a linear score 푠(푥) = 푑+푚 ∑ 푗=1 훽 푗 푍 푗 (푥), where 푍 푗 (푥) represents the 푗 푡ℎ component of the extended feature vector that combines both raw features and rule indicators. The value 푠(푥) serves as the aggregated evidence derived from all selected predictors. For classification tasks, SR4-Fit transforms this score into a probability through the sigmoid function: ̂푦(푥) = 휎(푠(푥)) = 1 1 + 푒 (−푠(푥)) , mapping the real-valued score to the interval (0, 1) and providing a probabilistic interpretation of the model’s confidence in the positive class. This probabilistic formulation not only supports threshold-based classification but also conveys uncertainty in predictions. For multiclass problems, SR4-Fit employs a one-vs-rest approach, training S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 6 of 77 one binary classifier per class; the final class prediction corresponds to the one with the highest estimated probability. Because each classifier is constructed from sparse and interpretable rules, the decision path for every class prediction can be directly traced through the contributing rules and coefficients. For regression tasks, SR4-Fit uses the same linear prediction structure without the sigmoid transformation, which provides a direct estimate of the target variable. Each nonzero coefficient 훽 푗 quantifies the contribution of the corresponding rule or feature to the predicted outcome, allowing clear interpretation of how specific variables influence the continuous target. Overall, SR4-Fit delivers a unified prediction framework that supports both probabilistic classification and continuous regression while preserving interpretability through its sparse, rule-based structure. 4. Experimental Setup To thoroughly evaluate SR4-Fit quantitatively, we conducted extensive experiments on a total of 15 datasets (Section 4.1) and introduce customized analysis methods appropriate for evaluation of interpretable ML (Section 4.2). Following the results and analysis (Section 5), we provide detailed discussion of the rules discovered (Section 6). 4.1. Datasets Our evaluation features in-depth analysis on a dataset that explores connections between demographic features and election outcomes (Section 4.1.1) considering both classification and regression problems using various configurations of this data, and a careful examination of the rules found, as well as broader numerical comparisons using 14 standard benchmark datasets (Section 4.1.2). 4.1.1. Election Data The demographic data covers U.S. House of Representatives elections from 2006 to 2016, with the period 2006– 2013 used for training and 2014–2016 for testing [44]. District-level election outcomes were obtained from the MIT Election Data and Science Lab’s U.S. House 1976–2018 dataset [45]. Following prior work [10], four dataset configurations were constructed based on the demographic features included. The minimum dataset consists of median age, male percentage, white percentage, and bachelor’s degree attainment. The standard dataset adds breakdowns of age, race, income, and education categories. The expanded dataset further adds language spoken at home, marital status, and poverty indicators. The previous dataset supplements the standard attributes with the party that won the district in the prior election. For classification, experiments were conducted under two feature configurations: (i) including both Democratic (DEM) and Republican (REP) party voting percentages as features for each year in the training datasets, and (i) excluding both party percentage variables to isolate model behavior under purely demographic inputs. For regression, two prediction targets were considered: DEM vote percentage and REP vote percentage. Each target was evaluated under two feature conditions, with and without the opposing party’s vote percentage included as a feature, yielding four regression experiment configurations in total. These conditions were designed to assess model sensitivity to the availability of politically informative features, since party percentage variables can be highly predictive and can dominate model behavior when present. 1 Previous research on U.S. demographic data [10] had shown that Random Forest and Support Vector Machine (SVM) models are strong predictors. Along with those models, XGBoost is included as an additional black-box baseline because it represents a current state-of-the-art in gradient boosting and consistently achieves top performance on structured tabular data [3], making it a strong and relevant benchmark against which to assess the predictive competitiveness of SR4-Fit. Together, Random Forest, SVM, and XGBoost provide a rigorous and representative set of high-performing black-box baselines, enabling a direct comparison between black-box models and the interpretable SR4-Fit rule-based model. Additionally, SR4-Fit was compared with the traditional RuleFit algorithm, as SR4-Fit was designed with a similar interpretability objective, ensuring a fair evaluation setting. Along with RuleFit, decision trees and linear regression (logistic for classification problems and LASSO for regression problems) were also included to provide comparisons across interpretable and compact models. The hyperparameters (푟 max , 휆 , 휅 and 휀) were selected 1 While party voting percentages from previous elections are often highly predictive, they may also be misleading in some cases because they may reflect not only party loyalty, but also particulars of the candidates (or lack thereof) in a given election. For example, in Alabama District 5, the Democratic candidate won with greater than 98% of the vote in 2006, and the Republican percentage that year was zero. This was because there was no Republican House candidate on the ballot that year in that district. In 2008, the vote was roughly 52% Democratic to 48% Republican, and in 2010 the vote was roughly 42% Democratic to 58% Republican. This shows that voting percentages can change rapidly from election to election. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 7 of 77 via grid search [46] across a range of values. Each setting was evaluated over multiple trials, and the combination yielding higher accuracy was chosen. All experiments on the U.S. demographic datasets were conducted over 30 independent trials for each model to provide statistically meaningful results. For classification tasks, predictive performance was assessed using standard classification metrics, including accuracy, precision, recall [47], and F1-score [48], to provide a quantitative evaluation. For regression tasks, the predictive performance was assessed using Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) [49], and Coefficient of Determination (푅 2 ). The Dice-Sorensen index [50] was also employed for both classification and regression tasks to assess the structural stability of the rule sets across trials, measuring the overlap between rule sets produced in independent runs of the same model. 4.1.2. Standard Benchmark Datasets In addition, to demonstrate the broad applicability of the SR4-Fit algorithm in classification settings, six publicly available datasets were selected for experimentation. These include both binary and multiclass classification tasks, as well as imbalanced datasets, ensuring comprehensive evaluation. The Wisconsin Breast Cancer dataset [51] provides diagnostic features from digitized images of breast masses to classify tumors as benign or malignant. The Ecoli dataset [52] comprises protein localization sites across different cellular compartments. The Page Blocks dataset [53] includes layout and geometric features of document blocks from scanned pages for classifying each block’s type. The Pima Indians Diabetes dataset [54] contains medical and demographic attributes of female Pima Indian individuals to predict the onset of type 2 diabetes. The Vehicle Silhouette dataset [55] involves shape descriptors extracted from vehicle silhouettes to classify them into categories based on geometric properties. Lastly, the Yeast dataset [56] includes sequence-based features of yeast proteins to predict their cellular localization. In this collection of datasets, the breast cancer and Pima Indians diabetes datasets are binary whereas the others are multi-class datasets. Similarly, for regression settings, eight distinct benchmark datasets were used to demonstrate broad applicability. The Ozone dataset [57] predicts daily average ozone measurements in Los Angeles in 1976. The MPG dataset [58] predicts fuel consumption for city cycles. The Machine dataset [59] predicts relative CPU performance. The Abalone dataset [60] predicts ages of abalone. The Diabetes dataset [61] predicts how the disease will progress one year from the baseline. The bone mineral density of 261 adolescents in North America is predicted by the Bones dataset [62]. The Housing dataset [63] predicts Boston home prices. Prostate-specific antigen levels in males slated for radical prostatectomy are predicted by the Prostate dataset [64]. 4.2. Analysis Methods To assess whether SR4-Fit succeeds with regard to the standard predictivity-understandability trade-off con- ceptually visualized in Figure 1, we conduct a Pareto analysis that intentionally separates the predictivity and understandability axes through the introduction of a novel measure of understandability for rule-based ML systems (Section 4.2.1). This is complemented by an extensive numeric analysis using a single metric value that combines predictivity and understandability (Section 4.2.2), extending a method used previously for interpretable ML [39], as well as standard statistical analysis (Section 4.2.3) and visualizations (Section 4.2.4). 4.2.1. Pareto Analysis and the Rule Understandability Score To provide a clear perspective on the trade-off between predictive performance and rule understandability, Pareto analysis [65] is employed. Pareto analysis examines whether one model simultaneously outperforms another along multiple independent axes, and our employment of it offers a model-selection perspective that does not depend on any particular predictivity-understandability weighting scheme. For this analysis, we introduce the Rule Understandability Score (RUS) as the measure of understandability. RUS is defined as a weighted combination of three components, each of which ranges from 0 to 1: (1) Dice-Sørensen stability, (2) normalized number of rules (inverted, so that fewer rules score higher), and (3) normalized rule complexity (inverted, so that simpler rules score higher): RUS(퐴) = 훾 푆 푆 푛 (퐴) + 훾 푠푖푚푝 (1 − 푅 simp 푛 (퐴)) + 훾 푐표푚푝 (1 − 푅 comp 푛 (퐴))(4) where 푆 푛 (퐴) represents the stability score of model 퐴, 푅 simp 푛 (퐴) corresponds to the rule simplicity score of the model, and 푅 comp 푛 (퐴) corresponds to the rule complexity of the model. The rule simplicity is computed through rule count, which is the average number of rules produced by the model per trial. The rule complexity is computed as the average number of conditions per rule, averaged across all rules a model produced in a trial. The weights 훾 푃 , 훾 푆 , 훾 푠푖푚푝 and 훾 푐표푚푝 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 8 of 77 allow flexibility in emphasizing different aspects of understandability. In the present work, all three RUS components are equally weighted (훾 푆 = 훾 푠푖푚푝 = 훾 푐표푚푝 = 1∕3). The components are normalized using MinMaxScaler in Python [66], which maps each component to the range [0, 1]. Rule count and rule complexity are normalized because stability already fall on that scale, and keeping all the components on a common range ensures each contributes proportionally when combined into RUS. Unlike the interpretability score (IPS, see Section 4.2.2), which has been employed in prior work for measuring interpretability [39], RUS deliberately excludes predictive performance when measuring model understandability so that the two axes used in the Pareto analysis remain independent. For the predictive performance axis in the Pareto analysis, we use mean F1 score as the measure for classification, while for regression tasks we use the mean푅 2 score as the predictive performance measure. For classification tasks, the F1 score is used as the predictive performance axis because it balances precision and recall simultaneously, providing a more robust measure than accuracy on imbalanced datasets or precision and recall individually, which each capture only one dimension of predictive performance. For regression tasks, the coefficient of Determination (푅 2 ) is used because it is bounded between 0 and 1, making it directly comparable across datasets with different scales and consistent with the classification setting where higher values always indicate better performance. This setup ensures that higher values indicate better performance on both axes. 4.2.2. Extended Interpretability Score (E-IPS) To further quantify the interpretability of the model, and recognizing the desirability of having a single score for each model for comparative purposes, we introduce an Enhanced Interpretability Score (E-IPS) as an extension of the original IPS proposed by Margot and Luta [39]. E-IPS is calculated based on a weighted combination of (1) the F-1 score (for classification) or 푅 2 (for regression); (2) the consistency or robustness of the model (Dice-Sørensen); (3) the simplicity of the model (normalized number of rules); and (4) the rule complexity (normalized number of components in a rule): E-IPS(퐴) = 훾 푃 푃 푛 (퐴) + 훾 푆 푆 푛 (퐴) + 훾 푠푖푚푝 (1 − 푅 simp 푛 (퐴)) + 훾 푐표푚푝 (1 − 푅 comp 푛 (퐴)).(5) In the present work, each E-IPS component is equally weighted (훾 푃 = 훾 푆 = 훾 푠푖푚푝 = 훾 푐표푚푝 = 0.25). 푅 simp 푛 (퐴) and 푅 comp 푛 (퐴) are inverted to account for lower values, which would contribute to higher interpretability. E-IPS is computed and compared exclusively among rule-based models, specifically RuleFit, SR4-Fit, Random Forest, and Decision Tree, because interpretability, as captured by the E-IPS, is inherently a property of rule-based representations. Black-box models such as SVM and XGBoost and interpretable models such as logistic and LASSO regression do not produce rule sets. They therefore cannot be meaningfully evaluated on metrics that depend on rule structure, stability, and complexity. Restricting the E-IPS comparison to rule-based models ensures that the evaluation is both fair and methodologically consistent. 4.2.3. Statistical Analysis To assess whether the performance differences between SR4-Fit and RuleFit are statistically meaningful, the Wilcoxon signed-rank test [67] is applied to compare the two models across all performance metrics. A significance threshold 푝 < 0.05 is applied throughout. The comparison is conducted exclusively between SR4-Fit and RuleFit, as these are the two primary rule-based models sharing a similar interpretability objective, making a direct statistical comparison between them the most informative and methodologically appropriate. The full Wilcoxon test results, including W-statistics and p-values across all metrics and demographic dataset configurations, are provided in Supplemental Material Section A. 4.2.4. Visualizations To provide a high-level visual summary, aggregate Pareto figures are produced by averaging results across all dataset variants within each experimental condition, giving a big-picture view of how the models compare overall. To facilitate more extensive visual comparative analysis, violin plots and line plots were also used [68]. These visualizations help to illustrate the variability, central tendency, and distribution of each model’s performance in aggregate and across trials, making it easier to assess both consistency and predictive strength [69]. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 9 of 77 0.00.20.40.60.8 F1 Score (Predictivity) → 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Rule Understandability Score (RUS) → Random Forest F1=0.9443 RUS=0.4891 RuleFit F1=0.8870 RUS=0.6285 SR4-Fit F1=0.9149 RUS=0.6985 Decision Tree F1=0.9114 RUS=0.2512 0.00.10.20.30.40.50.60.7 R² Score (Predictivity) → 0.0 0.2 0.4 0.6 0.8 Rule Understandability Score (RUS) → Random Forest R²=0.7579 RUS=0.0800 RuleFit R²=0.6719 RUS=0.9101 SR4-Fit R²=0.6733 RUS=0.9359 Decision Tree R²=0.6456 RUS=0.5952 Figure 3: Aggregate Pareto plots for the U.S. House election classification and regression tasks, showing mean predictivity (F1 score for classification, left; 푅 2 score for regression, right) against Rule Understandability Score (RUS). The classification panel averages results across all four demographic datasets and both feature configurations (with and without party percentages). The regression panel averages results across all four datasets and all four experimental configurations (DEM as label with and without REP percentage; REP as label with and without DEM percentage). 5. Results SR4-Fit is evaluated across four experimental settings: (1) U.S. House election classification, (2) Democratic and Republican party voting percentage regression, (3) a collection of six benchmark classification datasets, and (4) a collection of eight benchmark regression datasets. In each setting, predictive performance is benchmarked against Random Forest, SVM, XGBoost, Decision Tree, RuleFit, and linear baselines (logistic regression for classification and LASSO regression for regression) (see Supplemental Material in the Appendices), while interpretability is assessed only for rule-based models, RuleFit, Decision Tree, and Random Forest, using the Rule Understandability Score (RUS) and the Enhanced Interpretability Score (E-IPS). All experiments are conducted over 30 independent trials. For the demographic datasets, experiments are conducted under both full-information settings, where party vote percentage features are included, and reduced-information settings, where these features are removed, enabling a systematic assessment of SR4-Fit’s robustness and generalization under varying degrees of feature availability. We first present the aggregate Pareto analysis, which summarizes the predictivity–interpretability trade-off across all four experimental settings, followed by the detailed per-dataset E-IPS results in the subsections that follow. 5.1. Pareto Analysis Figure 3 presents the aggregate Pareto plots for the U.S. House election classification and regression tasks. The left panel shows the classification results, averaged across all four demographic datasets and both feature configurations (with and without party percentages). SR4-Fit achieves the highest RUS (0.6985) among all models while maintaining a competitive mean F1 score (0.9149), Pareto-dominating both RuleFit (F1=0.8870, RUS=0.6285) and Decision Tree (F1=0.9114, RUS=0.2512), as SR4-Fit exceeds both models in predictivity and interpretability (RUS) simultaneously. Random Forest attains the highest mean F1 score (0.9443) but a substantially lower RUS (0.4891) than SR4-Fit, reflecting a predictivity-interpretability trade-off rather than outright dominance. The right panel of Figure 3 shows the aggregate Pareto plot for the demographic regression tasks, averaged across all four datasets and all four experimental configurations (DEM as label with and without REP percentage; REP as label with and without DEM percentage). SR4-Fit Pareto-dominates both RuleFit (푅 2 =0.6719, RUS=0.9101) and Decision Tree (푅 2 =0.6456, RUS=0.5952), attaining a higher mean 푅 2 (0.6733) and higher RUS (0.9359) than either baseline. Random Forest attains the highest mean 푅 2 (0.7579) but the lowest RUS (0.0800) among all four models. Figure 4 presents the aggregate Pareto plots for the public benchmark classification and regression tasks. The left panel shows the results for the six public classification benchmark datasets. SR4-Fit and RuleFit achieve an identical RUS (0.7350), the highest among all four models, with SR4-Fit attaining a marginally higher mean F1 score S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 10 of 77 0.00.10.20.30.40.50.60.70.8 F1 Score (Predictivity) → 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Rule Understandability Score (RUS) → Random Forest F1=0.8316 RUS=0.4769 RuleFit F1=0.8119 RUS=0.7350 SR4-Fit F1=0.8164 RUS=0.7350 Decision Tree F1=0.7754 RUS=0.3203 0.00.10.20.30.40.50.6 R² Score (Predictivity) → 0.0 0.2 0.4 0.6 0.8 Rule Understandability Score (RUS) → Random Forest R²=0.6441 RUS=0.0800 RuleFit R²=0.5960 RUS=0.8999 SR4-Fit R²=0.6369 RUS=0.9312 Decision Tree R²=0.5481 RUS=0.7138 Figure 4: Aggregate Pareto plots for the public benchmark classification and regression tasks, showing mean predictivity (F1 score for classification, left; 푅 2 score for regression, right) against Rule Understandability Score (RUS). The classification panel averages results across the six public classification benchmark datasets. The regression panel averages results across the eight public regression benchmark datasets. (0.8164) than RuleFit (0.8119). SR4-Fit also Pareto-dominates Decision Tree (F1=0.7754, RUS=0.3203) on both axes. Random Forest attains the highest mean F1 score (0.8316) but a lower RUS (0.4769) than SR4-Fit, again reflecting a predictivity–interpretability trade-off. The right panel of Figure 4 shows the aggregate Pareto plot for the eight public regression benchmark datasets. SR4-Fit Pareto-dominates both RuleFit (푅 2 =0.5960, RUS=0.8999) and Decision Tree (푅 2 =0.5481, RUS=0.7138), attaining a higher mean 푅 2 (0.6369) and higher RUS (0.9312) than either baseline. Random Forest attains the highest mean 푅 2 (0.6441) but the lowest RUS (0.0800) among all four models, reflecting a predictivity–interpretability trade- off. The RUS values for the rule-based interpretable models are consistently higher in the regression settings (approximately 0.90–0.94 for SR4-Fit and RuleFit) than in the classification settings (approximately 0.63–0.74), a pattern driven by Random Forest’s markedly weaker performance on all three RUS components in regression relative to classification. In classification, Random Forest attains the lowest average rule complexity and comparatively lower rule count (when compared with regression data rule count) among all four models (approximately 1.9–2.0 conditions per rule; see Supplemental Material Section B and Section E) and near-zero stability (see Supplemental Section E). In regression, by contrast, Random Forest shows no such offsetting strength: its rule complexity is inconsistent and, in several datasets, the highest among all four models (approximately 5–6 conditions per rule; see Supplemental Material Section C, Section D, and Section F), while its rule counts remain on the order of 8000–9000, substantially larger than the rule counts produced by SR4-Fit and RuleFit (see Supplemental Material Section C, Section D, and Section F), and its stability remains near zero. Because Random Forest underperforms simultaneously on rule count, rule complexity, and stability in regression, whereas its favorable rule complexity partially compensates for its other weaknesses in classification, the gap between Random Forest and the three rule-based models (RuleFit, SR4-Fit, and Decision Tree) is proportionally larger in regression than in classification. This widened gap is reflected directly in the higher RUS values attained by SR4-Fit and RuleFit in the regression settings. Overall, across all four aggregate views, SR4-Fit consistently Pareto-dominates RuleFit and Decision Tree, achieving higher predictivity and higher RUS simultaneously in every setting. Relative to Random Forest, the same trade-off pattern holds throughout: Random Forest attains marginally higher predictivity, while SR4-Fit attains substantially higher RUS in each of the four settings. This consistent pattern reflects SR4-Fit’s balance between predictive performance and interpretability across both election and public benchmark tasks. 5.2. Classification Results As shown in Table 1, SR4-Fit achieves the highest E-IPS values in three of the four demographic datasets when party percentages are included as features, with scores of 0.6423, 0.6584, and 0.6728 for the Minimum, Standard, and S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 11 of 77 Table 1 Average enhanced interpretability score (E-IPS) ± standard deviation across 30 trials for demographic classification datasets. Bold indicates the best-performing method per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.4991±0.1478 0.6313±0.1657 0.6423±0.1678 0.5246±0.1035 Standard 0.5023±0.1512 0.6447±0.2353 0.6584±0.1649 0.4635±0.1165 Expanded 0.5030±0.1294 0.6542±0.2184 0.6728±0.2064 0.5738±0.1231 Previous 0.5267±0.1249 0.6544±0.1711 0.6541±0.2264 0.5045±0.1062 (a) With party percentages. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.3646±0.1111 0.4114±0.1635 0.4792±0.1996 0.5047±0.1214 Standard 0.4888±0.1261 0.5851±0.1095 0.5497±0.0981 0.3772±0.1584 Expanded 0.3978±0.1157 0.3050±0.1551 0.5230±0.1319 0.3982±0.1215 Previous 0.5128±0.1230 0.3240±0.1977 0.3000±0.1938 0.5329±0.1683 (b) Without party percentages. Expanded datasets, respectively; in the Previous dataset, RuleFit attains a marginally higher E-IPS (0.6544) than SR4- Fit (0.6541). The violin plots in Figure 5 confirm that SR4-Fit produces the tightest E-IPS distributions across trials, indicating greater interpretability stability than all competing rule-based models. SR4-Fit also achieves the highest predictive performance across all four datasets under this setting, outperforming Random Forest, SVM, RuleFit, logistic regression, decision tree, and XGBoost across accuracy, precision, recall, and F1 score. SR4-Fit achieves the highest Dice-Sorensen Index among rule-based models in three of the four datasets, with RuleFit leading in the standard dataset but closely followed by SR4-Fit. In contrast, Random Forest and Decision Tree produce near-zero stability scores across all datasets, indicating structurally inconsistent rule sets across trials. Regarding compactness, the decision tree produces the fewest rules when party percentages are included, followed closely by SR4-Fit and RuleFit, while Random Forest generates substantially larger rule sets. Full predictive line plots, stability tables, and rule compactness results are provided in Supplemental Material Section E. When party percentages are removed, E-IPS values in Table 1 decline across all models, reflecting the removal of highly informative features. Under this reduced-information setting, XGBoost and SVM maintain the highest predictive accuracy, while SR4-Fit produces smoother and more stable performance trajectories than RuleFit and Decision Tree. SR4-Fit achieves the highest E-IPS in the expanded dataset, while the decision tree leads in the minimum and previous datasets. However, this advantage comes at the cost of structural instability, the Decision Tree’s Dice-Sørensen scores remain near zero across trials, indicating that its rule sets change substantially between runs and cannot be relied upon for consistent interpretation. SR4-Fit and RuleFit maintain more consistent rule structures under feature reduction. Detailed predictive line plots and stability analyses for the reduced-information setting are provided in Supplementary Material Section E. Overall, SR4-Fit delivers the strongest and most consistent combination of predictive performance, interpretability, stability, and compactness when politically informative features are available and remains among the most stable rule-based models when they are not. 5.3. DEM Regression Results As shown in Table 2, when REP percentage is included as a feature, RuleFit achieves the highest E-IPS in the standard and expanded datasets, while SR4-Fit leads in the minimum and previous datasets with scores of 0.4521 and 0.5278, respectively. RuleFit’s higher E-IPS is accompanied by lower Dice-Sørensen stability and higher rule complexity than SR4-Fit. The violin plots in Figure 6 confirm that SR4-Fit maintains tighter and more stable E- IPS distributions than the baselines when REP percentage is included. In terms of predictive performance, XGBoost attains the lowest average error across most datasets, while SR4-Fit produces comparable RMSE, MAE, and 푅 2 values with notably smoother and more stable trajectories across trials. Full predictivity line plots, stability tables, and rule compactness results for this setting are provided in the Supplemental Material Section C). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 12 of 77 MinimumStandardExpandedPrevious 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree MinimumStandardExpandedPrevious 0.0 0.2 0.4 0.6 0.8 E-IPS Score Random Forest RuleFit SR4 Decision Tree Figure 5: Distribution of enhanced interpretability scores (E-IPS) across demographic classification datasets. The left panel reports results obtained using feature sets that include party percentage information, whereas the right panel shows results with party percentages removed. Table 2 Average enhanced interpretability score (E-IPS) ± standard deviation across 30 trials for demographic regression datasets. Results are reported for predicting Democratic vote percentage, with and without Republican percentage included as a feature. Bold indicates the best-performing method per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.4520±0.1848 0.4006±0.0643 0.4521±0.1708 0.3853±0.1440 Standard 0.4946±0.1977 0.5227±0.0697 0.4635±0.1110 0.4866±0.1393 Expanded 0.4297±0.1844 0.6524±0.0667 0.4737±0.1204 0.4874±0.1315 Previous 0.4971±0.2036 0.5219±0.0731 0.5278±0.1410 0.4274±0.1620 (a) With the Republican percentage as a feature. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.4541±0.1836 0.4448±0.2087 0.4822±0.1790 0.5006±0.0546 Standard 0.5223±0.1855 0.5309±0.1786 0.4338±0.2211 0.3786±0.1643 Expanded 0.4735±0.1587 0.2022±0.2001 0.6190±0.1787 0.6397±0.1637 Previous 0.5571±0.1550 0.5809±0.0433 0.8427±0.2512 0.3805±0.1844 (b) Without the Republican percentage as a feature. When REP percentage is removed, SR4-Fit achieves the highest E-IPS in the previous dataset, with a score of 0.8427, outperforming RuleFit, Random Forest, and Decision Tree. In the minimum and expanded datasets, Decision Tree attains the highest E-IPS of 0.5006 and 0.6397, narrowly ahead of SR4-Fit 0.4822 and 0.6190, respectively. The violin plots in Figure 6 confirm that SR4-Fit maintains tighter and more stable E-IPS distributions than competing models across both feature configurations, while all models show more concentrated distributions when REP percentage is excluded. Error values increase across all models under this reduced-information setting, with XGBoost still attaining the lowest average error, while SR4-Fit maintains comparatively smooth performance trends and more concentrated error distributions than the baselines. Detailed predictive line plots and stability analyses for the reduced-information setting are provided in Supplemental Material Section C. 5.4. REP Regression Results As shown in Table 3, SR4-Fit achieves the highest E-IPS in the Minimum, Standard, and Previous datasets when DEM percentage is included as a feature, with scores of 0.6182, 0.5654, and 0.5647, respectively, while RuleFit performs strongest in the Expanded dataset. The violin plots in Figure 7 confirm that SR4-Fit maintains tighter and more stable E-IPS distributions than the baselines when DEM percentage is included. In terms of predictive performance, XGBoost attains the lowest average error across most datasets, while SR4-Fit produces comparable RMSE, MAE, S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 13 of 77 MinimumStandardExpandedPrevious 0.2 0.4 0.6 0.8 E-IPS Score Random Forest RuleFit SR4 Decision Tree MinimumStandardExpandedPrevious 0.0 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree Figure 6: Distribution of enhanced interpretability scores (E-IPS) across demographic regression datasets when predicting Democratic vote percentage. Results are shown for models trained with all party-related features (left) and for models trained with Republican percentage features removed (right). Table 3 Average enhanced interpretability score (E-IPS) ± standard deviation across 30 trials for demographic regression datasets. Results are reported for predicting Republican vote percentage, with and without Democratic percentage included as a feature. Bold indicates the best-performing method per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.5281±0.1753 0.3858±0.0685 0.6182±0.1214 0.4583±0.1298 Standard 0.5166±0.1958 0.5221±0.0665 0.5654±0.2094 0.4830±0.1370 Expanded 0.5026±0.1753 0.6448±0.0646 0.5555±0.1658 0.5733±0.1400 Previous 0.4986±0.1640 0.5270±0.0701 0.5647±0.1853 0.4516±0.1421 (a) With the Democratic percentage as a feature. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.5271±0.1315 0.1610±0.1393 0.4290±0.1635 0.4478±0.1320 Standard 0.4754±0.1578 0.5016±0.0639 0.1536±0.1413 0.3551±0.1499 Expanded 0.4504±0.1578 0.5199±0.0562 0.2738±0.1583 0.6477±0.1725 Previous 0.5291±0.1660 0.5780±0.0422 0.6639±0.2393 0.3645±0.1469 (b) Without the Democratic percentage as a feature. and 푅 2 values with smoother and more stable trajectories across trials. Despite RuleFit’s advantage in the expanded dataset, SR4-Fit achieves higher Dice-Sørensen stability and lower rule complexity than RuleFit across most dataset configurations, indicating that SR4-Fit’s interpretability advantage is accompanied by greater structural consistency and more compact rule sets. Full line plots, stability tables, and rule compactness results for this setting are provided in Supplemental Material Section D. When DEM percentage is removed, E-IPS values decline across all models, but SR4-Fit retains the highest interpretability in the previous dataset with a score of 0.6639. The violin plots in Figure 7 confirm that SR4-Fit maintains tighter and more stable E-IPS distributions than competing models across both feature configurations, whereas RuleFit exhibits widening variance and Random Forest produces near-zero stability scores throughout. Error values increase across all models under this reduced-information setting, with XGBoost still achieving the lowest average error, while SR4-Fit maintains comparatively smooth performance trends and more concentrated error distributions than the baselines. Detailed line plots and stability analyses for the reduced-information setting are provided in Supplemental Material Section D. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 14 of 77 MinimumStandardExpandedPrevious 0.0 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree MinimumStandardExpandedPrevious 0.0 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree Figure 7: Distribution of enhanced interpretability scores (E-IPS) across demographic regression datasets when predicting Republican vote percentage. Results are shown for models trained with all party-related features (left) and for models trained with Democratic percentage features removed (right). Table 4 Average enhanced interpretability score (E-IPS) ± standard deviation across 30 trials for public classification datasets. Bold indicates the highest average E-IPS per dataset. DatasetRandom ForestRuleFitSR4-FitDecision Tree Breast Cancer 0.4350±0.1433 0.4793±0.1851 0.4931±0.1875 0.4477±0.2044 E. coli0.4505±0.1475 0.6085±0.1087 0.6141±0.1093 0.3624±0.1494 Page Blocks 0.5386±0.1296 0.4813±0.1322 0.5308±0.1381 0.5405±0.1391 Pima Indians 0.4644±0.1330 0.3596±0.3061 0.3208±0.3011 0.3862±0.1781 Vehicle0.4958±0.1642 0.4175±0.1314 0.4099±0.1262 0.5338±0.1570 Yeast0.4927±0.1426 0.6313±0.1234 0.6512±0.1281 0.4017±0.1596 5.5. Results on Standard Public Classification Datasets Across the six public classification benchmarks, SR4-Fit demonstrates a consistently strong balance between predictive performance, interpretability, and stability. As shown in Table 4, SR4-Fit achieves the highest E-IPS in three of the six datasets, including breast cancer (0.4931), E. coli (0.6141), and Yeast (0.6512), while remaining competitive in the remaining datasets. The violin plots in Figure 8 confirm that SR4-Fit’s predictive distributions remain tightly clustered across all 30 trials, while RuleFit’s distributions widen, and Random Forest exhibits the broadest and least consistent spread. The Dice–Sørensen stability results confirm that SR4-Fit and RuleFit achieve the highest stability across all six datasets, substantially outperforming Random Forest and Decision Tree, which produce near-zero scores. Although Random Forest occasionally achieves slightly higher accuracy on individual datasets, its trial-to-trial variability is substantially higher, and its interpretability remains near zero due to large, unstable rule sets. RuleFit performs competitively in accuracy but shows moderate instability and produces larger, more variable rule sets than SR4-Fit across most datasets. Overall, the public classification experiments confirm that SR4-Fit reliably produces interpretable and stable rule sets while maintaining competitive predictive performance across diverse classification tasks. Full stability results, line plots, and violin plots for this setting are provided in Supplemental Material Section E. 5.6. Results on Standard Public Regression Datasets Across the eight public regression benchmarks, SR4-Fit maintains competitive predictive accuracy while achieving stronger interpretability and rule stability than the baselines in most datasets. As shown in Table 5 and Figure 9, SR4-Fit attains the highest E-IPS in two of the eight datasets, Housing (0.5425) and Ozone (0.6303), while Decision Tree leads in four datasets: Abalone, Bone, MPG, and Prostate. RuleFit and Random Forest each lead in one dataset, Diabetes and Machine, respectively. The line plots in Supplemental Material Section E show that SR4-Fit produces smooth RMSE, MAE, and 푅 2 curves with minimal variance across trials, in contrast to Random Forest’s pronounced fluctuations and RuleFit’s moderate instability. The Dice-Sørensen stability results confirm that SR4-Fit achieves the highest stability in six of the eight datasets, while Random Forest remains near zero and RuleFit performs inconsistently S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 15 of 77 Breast Ecoli Pages PimaVehicle Yeast 0.0 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree Figure 8: Violin plot comparison of enhanced interpretability scores for all rule-based models across standard public classification datasets. Table 5 Average enhanced interpretability score (E-IPS) ± standard deviation across 30 trials for public regression datasets. Bold indicates the highest average E-IPS per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Abalone 0.4890±0.1166 0.4824±0.0649 0.1444±0.1525 0.5524±0.1113 Bone0.4723±0.1565 0.5756±0.0549 0.3263±0.3564 0.6124±0.2261 Diabetes 0.4019±0.1627 0.6474±0.0588 0.6176±0.2149 0.5338±0.0878 Housing 0.4965±0.1320 0.5330±0.0613 0.5425±0.1730 0.4301±0.1264 Machine 0.5615±0.1073 0.5410±0.1868 0.5588±0.1688 0.4165±0.1849 MPG0.4738±0.1402 0.5391±0.0615 0.5393±0.2905 0.5655±0.0507 Ozone 0.5450±0.1397 0.5230±0.0548 0.6303±0.2125 0.3638±0.1400 Prostate 0.4787±0.0951 0.2265±0.2401 0.4687±0.1499 0.4791±0.1833 Abalone Bone Diabetes Housing Machine MPG Ozone Prostate 0.0 0.2 0.4 0.6 0.8 1.0 E-IPS Score Random Forest RuleFit SR4 Decision Tree Figure 9: Violin plot comparison of enhanced interpretability scores for all rule-based models across standard public regression datasets. across datasets. Regarding compactness, SR4-Fit generates substantially fewer rules than Random Forest across all datasets and produces rule sets comparable in size to RuleFit but with markedly lower rule complexity in most cases. These findings demonstrate that SR4-Fit performs robustly across diverse regression benchmarks while providing more stable and concise rule representations than existing rule-based and ensemble baselines. Full line plots, stability results, and rule compactness results for this setting are provided in Supplemental Material Section F. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 16 of 77 6. Discussion The following analysis presents representative rules extracted from each dataset and interprets the associated demographic and structural characteristics of the regions these rules classify. In addition to classification-oriented rules, we also examine rules derived from regression-based demographic modeling, where SR4-Fit identifies structured combinations of predictors associated with continuous outcomes. In addition to this, detailed interpretations for the public classification and regression datasets are provided as well. 6.1. Including Prior Party Voting Percentages (Classification) In the minimum dataset, SR4-Fit builds a concise, interpretable model primarily driven by the Democratic and Republican voting percentages in prior elections. The most consistently important rule indicates that districts where the prior Democratic voting percentage is greater than about 48.7% while the prior Republican voting percentage is at or below about 45.5% are likely to elect Democrats. This pattern is not surprising, since districts where votes for Democratic candidates previously outnumbered votes for Republican candidates are likely to elect Democratic candidates in the future. Similarly, districts where the prior Democratic voting percentage is at or below about 48.7% and the prior Republican voting percentage is greater than about 45.5% typically signal Republican elections. At first glance this may seem surprising because it could include districts in which Democrats slightly outnumber Republicans. However, several explanations are possible: In the predicted years (2014 and 2016), Republican turnout may be higher than Democratic turnout, independent voters may favor Republican candidates, or some Democrats may cross over to vote for Republican candidates in those districts and years. 2 Interestingly, this latter pattern occasionally received negative coefficients in several trials, meaning it predicted Democratic outcomes in some cases. In those instances, the districts had a median age of 39.24 years compared to the overall average of 37.49 years, a White non-Hispanic population share of 69.46% compared to 63.64%, and a bachelor’s degree attainment rate of 19.06% compared to 17.91%. In other words, although districts without a clear Democratic majority often elect Republicans, districts that are older, whiter, and more highly educated sometimes elect Democrats. These demographic characteristics are widely known to influence voting patterns. In the standard dataset, SR4-Fit builds an interpretable model shaped largely by age distribution, party support levels, and socioeconomic indicators. Age between 45 and 54 emerges as the most influential feature and consistently receives high positive coefficients, indicating Republican-leaning tendencies in regions where middle-aged populations are prominent. Similarly, the age groups 65–74 and 18–24 also show strong positive associations with Republican classification, highlighting the predictive importance of both older and younger demographic groups. A more complex rule involving closely balanced prior Democratic and Republican voting percentages combined with relatively low median income (around $23,717 or lower) has only a small positive influence, suggesting that lower income levels may slightly shift predictions toward Republican classification in closely contested regions. Another narrow-range rule involving modest Democratic support, moderate Republican support, and bachelor’s degree attainment above about 21.7% also receives a small positive coefficient. This suggests that relatively high educational attainment may sometimes push moderately divided districts toward Republican classification. In the expanded dataset, SR4-Fit captures more nuanced voting behavior through interpretable patterns primarily driven by prior Democratic and Republican support percentages along with demographic factors. One pattern involving moderately split Democratic and Republican support tends to produce Democratic classifications. Districts satisfying this pattern tend to have a slightly older population, with a median age of about 38.97 years compared to the overall average of 37.49 years. These districts also show higher Asian population shares (8.48% versus 4.86%) and higher multiracial population shares (3.95% versus 2.68%). In addition, they have somewhat higher White non-Hispanic population shares, higher bachelor’s degree attainment rates (19.38% compared to 17.91%), higher median incomes (about $30,764 compared to $26,873), and lower poverty levels (11.22% compared to 14.63%). Together, these characteristics suggest that moderate diversity combined with higher socioeconomic status tends to correspond with Democratic outcomes in these districts. In the previous dataset, SR4-Fit identifies patterns where both Democratic and Republican support levels are relatively high but Republican support is slightly stronger, resulting in Republican classification. Districts following this 2 Nationally, both higher Republican voter turnout and independents more frequently voting Republican were found using traditional post- election analyses of the 2014 and 2016 elections, whereas crossover voting was rare in both of these years [70, 71, 72, 73, 74, 75, 76]. This suggests that SR4-Fit rules find relevant patterns in the data. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 17 of 77 pattern tend to have slightly older populations, with a median age of about 37.70 years compared to the overall average of 37.49 years. These districts are also less diverse, with higher White non-Hispanic population shares (71.58% versus 63.64%) and lower representation of Black, Asian, Hispanic, and multiracial populations. Educational attainment is also somewhat higher in these districts, with about 19.37 percent holding bachelor’s degrees and 11.68 percent holding graduate or professional degrees compared to 17.91% and 10.65% overall. 6.2. Without Prior Party Voting Percentages (Classification) For the minimum dataset, several demographic patterns emerge that help explain the predicted electoral outcomes. Districts with White population between approximately 26.8% and 42.25% tend to predict Democratic outcomes, sug- gesting that districts with greater racial diversity are more likely to support Democratic candidates. Similarly, districts with moderate White population levels between approximately 41.55% and 54.65%, combined with bachelor’s degree attainment above 12.95%, also tend to predict Democratic outcomes, indicating that moderate educational attainment, together with demographic diversity, may contribute to Democratic support. In contrast, districts characterized by very high White population shares exceeding approximately 77.15%, male population proportions above about 48.35%, and bachelor’s degree attainment greater than approximately 17.35% more likely to predict Republican outcomes. Age structure also appears to influence predictions: districts with lower White population shares (≤ 42.7) combined with older median ages above about 41.7 years, as well as districts with lower White population shares (≤ 41.55%) and younger median ages below approximately 40.05 years, tend to predict Republican outcomes. Additionally, temporal factors appear to play a role, as districts with White population shares greater than about 53.85%, male population proportions above roughly 48.55%, and median ages above approximately 35.55 years were associated with Democratic outcomes. Across the extracted rules in the standard dataset, districts with lower White population shares (generally ≤42.7%) combined with moderate working-age populations aged 25–44 (approximately 23.65%–24.55%) and younger median ages (≤41.7 years) frequently predict Republican outcomes, indicating that districts with younger working- age demographics may lean Republican in the rule set. Several rules also associate Republican predictions with districts where middle-aged populations (Age 45–54:≥12.65%) or pre-retirement populations (Age 55–64:≥10.45%) are present, while older populations such as Age 65–74 remain relatively small (≤8–9%), suggesting that certain age structures influence voting behavior. In addition, Republican predictions appear in districts with low minority population shares, including Black population≤1.95% and Asian population≤0.85%, as well as moderate Hispanic population shares≤38.2%. Some rules further associate Republican outcomes with family-structured populations containing both youth (Age 5–17≥13.45%) and elderly populations (Age 75+≥8.2%), suggesting the presence of multi-generational demographic structures. In contrast, Democratic predictions are observed in districts with lower White population shares (≤42.25%) combined with higher median income levels (≥ $30,298.5), indicating that economically stronger and demographically diverse districts may favor Democratic candidates. Additional Democratic outcomes appear in districts with higher proportions of young adults (Age 18–24≥11%) and moderate educational attainment levels, such as some college education exceeding 27.45% and high school graduation rates above 30.3%, suggesting that education and younger populations may contribute to Democratic voting patterns. Across the rules extracted from the expanded dataset, many rules involve districts where the proportion of individ- uals who have never been married is≤32.95% and the proportion of married individuals is≤54.05–54.45%, suggesting that marital status composition is a common structural feature in the model. Democratic predictions frequently occur in districts with greater demographic diversity, including Black population shares between approximately 20.75% and 29.0% or above 4.85%, Hispanic population shares greater than about 1.85%, and Asian population shares above roughly 1.45%, combined with higher educational attainment such as graduate or professional degree levels exceeding about 4.85–10.3% and median income levels above roughly $19,394. These districts may also contain younger adult populations (Age 18–24 above approximately 8.65–11.7%), indicating the influence of demographic diversity, education, and younger age groups in Democratic predictions. In contrast, Republican predictions appear in districts with moderate educational attainment, including bachelor’s degree levels above about 10.9% and relatively low proportions of individuals without a high school diploma (≤12.15%), along with lower Black population shares (≤8.9%), moderate language diversity where households speaking languages other than English remain≤34.15%, and moderate income groups in the $65,000–$74,999 range exceeding about 2.25%. Across the extracted rules in the previous dataset, Republican predictions frequently occur in districts with moderate educational attainment, such as populations with some college or associate degrees exceeding approximately 31.65%, combined with low Asian population shares (≤1.15%) and moderate representation of other racial groups (≥1.95%). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 18 of 77 Additional Republican outcomes appear in districts with large working-age populations (Age 25–44 above roughly 24.45–24.55%) and higher White population shares exceeding approximately 54.75%, particularly when family-age populations are present, such as youth populations (Age 5–17≥18.15%). Republican predictions also appear in districts with moderate racial diversity, including Asian population shares greater than about 2.05% and multiracial populations above approximately 0.95%, as well as districts where Black population shares remain below roughly 38.45%. In contrast, Democratic predictions tend to occur in districts with very high Hispanic population shares (≥67.2%), as well as districts where lower educational attainment levels are present, such as less than high school completion exceeding approximately 19.15%, combined with greater representation of other racial groups (≥2.30%). 6.3. Including REP Percentage and DEM Percentage as Label (Regression) In the minimum dataset, the rules primarily highlight the strong influence of prior Republican vote share (REP percentage) on predicting Democratic vote percentage. When Republican support is extremely low (REP percentage≤−2.0186), the model increases the predicted Democratic percentage by 23.72, indicating that districts with very weak Republican presence strongly favor Democratic outcomes. When Republican support remains low but not extremely low (−2.0186<REP Percentage≤−0.7020) and the share of the white non-Hispanic population is relatively smaller (≤−0.4248), the prediction increases by 18.12, suggesting that demographic composition further reinforces Democratic vote share in these districts. Additionally, in districts with very low Republican support (REP percentage≤−2.0338) and moderate levels of bachelor’s degree attainment (≤0.2639) increase the predicted Democratic percentage by 7.56. In the standard dataset, the rules again emphasize the strong role of prior Republican vote share (REP percentage) in predicting Democratic vote percentage. When Republican support is extremely low (REP percentage≤−2.0308), the model increases the predicted Democratic vote share by 16.96, indicating that districts with very weak Republican pres- ence strongly favor Democratic outcomes. Additionally, when Republican support is low (REP percentage≤−0.7970) and a relatively high Black population share (Black>1.6788), the predicted Democratic percentage increases by 15.74. In the expanded dataset, when Republican support is relatively low (REP percentage≤−0.8104) and the share of the Black or African American population is high (>1.5405), the model increases the predicted Democratic vote percentage by 17.12. This suggests that districts with weaker Republican presence and a higher proportion of Black residents tend to exhibit substantially stronger Democratic electoral support. In the previous dataset, the rules highlight the influence of prior Republican vote share, demographic characteristics, and historical voting patterns in predicting Democratic vote percentage. When Republican support falls within a moderately low range (−0.7020<REP percentage≤−0.3607) and the share of the population aged 25–44 years is relatively higher (>−0.4926), the predicted Democratic percentage increases by 9.19, suggesting that districts with larger working-age populations tend to favor Democratic outcomes when Republican support is moderate. When prior Republican vote share remains moderately low (−0.8065<REP percentage≤−0.1539) but the 25–44 age population is smaller, combined with higher proportions of residents with less than a high school education (>−0.4735), the prediction increases by 6.84. Finally, when Republican support is very low (REP percentage≤ −0.8128), the model increases the predicted Democratic vote percentage by 3.91, even when the Black population share remains below the specified threshold. 6.4. Without REP Percentage and DEM Percentage as Label (Regression) In the minimum dataset without including prior Republican vote share as a feature, the rules highlight the influence of racial composition, education, and demographic characteristics in predicting Democratic vote percentage. When the share of the White non-Hispanic population is relatively low (≤−1.0747), the predicted Democratic percentage increases by 17.82. A similar pattern appears when the White non-Hispanic population share remains low (≤−0.7819) combined with specific ranges of male population share, increasing the prediction by 18.73, indicating strong Democratic support in districts with a lower White population share. Additionally, districts with higher educational attainment (0.4004<Bachelor’s degree≤1.3966) and higher White population share (>0.5837) still increase the predicted Democratic percentage by 14.04, suggesting that education plays an important role in shaping voting behavior. In the standard dataset without including prior Republican vote share as a feature, the rules emphasize the impor- tance of racial composition, educational attainment, and age distribution in predicting Democratic vote percentage. Districts with low White population share (White≤−1.0973) and very high Black population share (Black>2.9733) produce the strongest increase in Democratic vote share, contributing 19.53 to the prediction. Similarly, districts with S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 19 of 77 higher Asian population share (>0.0735) and lower proportions of younger residents increase the predicted Democratic percentage by 18.08. Other rules highlight the role of age distribution, particularly larger shares of the 25–44 age group (>0.6508), which increases predictions by 15.06 to 15.21 when combined with lower White population share and specific education levels. Additionally, districts with higher educational attainment (graduate degree>0.0250) or higher proportions of high school graduates (>0.8662) also contribute positively to Democratic vote predictions. In the expanded dataset without including prior Republican vote share as a feature, the rules highlight the influence of marital status, racial composition, education, income, and age distribution in predicting Democratic vote percentage. Districts with lower shares of the White non-Hispanic population and higher shares of Black residents show strong increases in Democratic vote share, contributing 17.14 to the prediction. Similarly, districts characterized by high proportions of high school graduates combined with lower median income produce the strongest effect, increasing predicted Democratic vote share by 27.95. Other rules highlight the importance of marital status, particularly districts with higher shares of individuals who have never married, which increase Democratic vote predictions by 13.17 to 15.14 when combined with specific age distributions and demographic characteristics. Additionally, racial diversity and language composition further refine predictions, contributing 12.27 to 13.49 to the predicted Democratic percentage. In the previous dataset without including prior Republican vote share as a feature, when the previous party indicator reflects Democratic alignment and the share of the White non-Hispanic population is relatively higher (>−0.7791), the predicted Democratic vote percentage increases by 17.10. Similarly, when previous electoral alignment remains Democratic and the White non-Hispanic population share is relatively lower (≤−0.7882), the model increases the predicted Democratic percentage by 14.84. 6.5. Including DEM Percentage and REP Percentage as Label (Regression) In the minimum dataset where Republican vote percentage is the target variable and prior Democratic vote percentage is included as a feature, the rule highlights the inverse relationship between the two party vote shares. Specifically, when prior Democratic vote share is relatively low (−2.1595<DEM percentage≤−0.5436), the model increases the predicted Republican vote percentage by 6.47. This indicates that districts with weaker Democratic support tend to exhibit higher Republican vote percentages, reflecting the expected competitive relationship between the two major parties in electoral outcomes. In the standard dataset where Republican vote percentage is the label and prior Democratic vote percentage is included as a feature, when Democratic vote share is extremely low (DEM percentage≤−2.1753), the predicted Republican vote percentage increases substantially by up to 15.11 units, representing the strongest contribution among the rules. When Democratic support remains relatively low (−2.1629<DEM percentage≤−0.5604), the predicted Republican vote share increases by 8.00 to 10.16, particularly in districts with higher White population share or lower educational attainment. Additional rules show that moderately low prior Democratic vote share (−0.5604<DEM percentage≤−0.1930) contributes smaller increases ranging from 5.43 to 5.73, depending on demographic and socioeconomic characteristics such as lower Hispanic population share and lower Black population share and higher median income. In the expanded dataset where Republican vote percentage is the label and prior Democratic vote percentage is included as a feature, when prior Democratic vote share is extremely low (DEM percentage≤−2.1629), the predicted Republican vote percentage increases substantially by 16.18, particularly in districts where the income group earning $35,000–$49,999 remains below the specified threshold. When Democratic support remains relatively low (−2.1629<DEM percentage≤−0.5604) and the share of married individuals is relatively higher, the model increases the predicted Republican vote share by 9.95. Finally, when prior Democratic vote share falls within a moderate low range (−0.5604<DEM percentage≤−0.2231) and the share of individuals who have never married remains below the specified threshold, the predicted Republican percentage increases by 6.24. In the previous dataset where Republican vote percentage is the label and prior Democratic vote percentage is included as a feature, when prior Democratic vote share is relatively low (−2.1323<DEM percentage≤−0.5254), the predicted Republican vote percentage increases significantly by up to 12.92, especially in districts with lower Black population share. Similarly, districts with low Democratic support (−2.1851<DEM percentage≤−0.4622) and specific educational characteristics contribute 11.97 to Republican vote share. When prior Democratic vote share falls within moderate low ranges (approximately −0.57 to −0.21), the predicted Republican percentage increases more modestly by 6.05 to 6.85, depending on demographic factors such as higher White population share, lower Hispanic population share, and racial composition. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 20 of 77 6.6. Without DEM Percentage and REP Percentage as Label (Regression) In the minimum dataset where Republican vote percentage is the target variable and prior Democratic vote share is not included as a feature, the rules highlight the importance of demographic composition, education, age structure, and geographic context in explaining Republican vote share. Districts with higher shares of White non-Hispanic population (>−0.9646) and higher male population share tend to increase predicted Republican vote percentages. The strongest contribution occurs when the proportion of individuals with bachelor’s degrees is relatively low (≤−0.3805) and the median population age is lower (≤−0.1370), increasing the predicted Republican vote share by 20.05. Other rules show that districts with higher educational attainment (> 0.2185 bachelor’s degree share) it contribute 12.90, while similar demographic conditions contribute 7.72. In the standard dataset where Republican vote percentage is the label and prior Democratic vote share is not included as a feature, the rules highlight several demographic factors associated with higher Republican vote share. Districts with higher White population share (>−0.9312) and lower proportions of graduate-level educational attainment (≤0.3989) increase the predicted Republican vote percentage by 9.21. Additionally, individual demographic features such as White population share (+10.20) and population aged 65–74 (+11.57) contribute strongly to Republican vote predictions, suggesting that districts with larger White and older populations tend to exhibit higher Republican vote percentages. Educational attainment and ethnic composition also contribute smaller but meaningful increases, with bachelor’s degree share adding 6.47 and Hispanic or Latino population share contributing 5.41. In the expanded dataset where Republican vote percentage is the label and prior Democratic vote share is not included as a feature, the rules highlight the influence of marital status, age distribution, education, income, and economic status on Republican vote share. Districts with lower proportions of individuals who have never married (≤−0.2858) and higher shares of individuals with less than a high school education (>−0.3833) increase the predicted Republican vote percentage by 9.69, particularly during later election periods. Additional rules show that moderate levels of never-married individuals combined with higher median age groups such as 45–54 contribute 7.64 to Republican vote predictions. Demographic features such as population aged 65–74 (+4.24) and bachelor’s degree attainment (+4.45) also contribute positively, while Hispanic or Latino population share adds 9.04 to the prediction. Socioeconomic characteristics including median income levels and poverty thresholds further influence predictions, contributing 4.74 to 6.80 depending on the combination of conditions. In the previous political dataset where Republican vote percentage is the label and prior Democratic vote share is not included as a feature, the rules highlight the importance of historical electoral alignment, demographic composition, age distribution, and educational attainment in explaining Republican vote share. Districts where the previous party indicator suggests prior Republican alignment and the share of individuals with graduate or professional degrees exceeds −0.2243 increase the predicted Republican vote percentage by 14.06, representing the strongest contribution among the rules. Additional demographic factors also contribute positively, including population aged 65–74 (+6.40) and White non-Hispanic population share (+5.74), suggesting that districts with larger older and White populations tend to favor Republican electoral outcomes. Educational attainment further contributes to the prediction, with bachelor’s degree share adding 3.84. 6.7. Interpretation SR4-Fit Rules for Public Classification Datasets Across the extracted rules for the breast cancer dataset, SR4-Fit identifies several cellular morphology patterns that distinguish malignant from benign tumors. Malignant classifications frequently occur when features indicating cellular irregularity and abnormal nuclei are elevated. Malignant outcomes appear when clump thickness exceeds approximately 0.7301 and bare nuclei values are greater than about −0.4242, even when uniformity of cell size remains at or below 0.1140. Malignancy is also associated with high variability in cell size (uniformity cell size > 0.4405) combined with marginal adhesion values greater than about −0.4647, indicating irregular cell growth and stronger cell clustering. Additional malignant rules involve nuclear structure features such as bland chromatin values above approximately −0.3861 and bare nuclei values greater than about −0.0123, as well as rules where uniformity of cell shape remains below about 0.0954 with normal nucleoli values below approximately 0.2066 and mitoses below roughly 0.5179, highlighting the importance of chromatin structure and nuclear activity in identifying malignant cells. In contrast, benign classifications typically occur when cellular structures remain more regular. For instance, benign outcomes appear when uniformity of cell shape is relatively low (≤−0.2395), bare nuclei values remain within moderate ranges (between about −0.0123 and 0.5370), and uniformity of cell size remains above approximately −0.2125. Additional benign patterns occur when clump thickness remains small (≤0.3753) and single epithelial cell size remains above about −0.3305, indicating thinner and more stable cell clusters. Other benign rules involve lower S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 21 of 77 marginal adhesion values (≤−0.4647), even when uniformity of cell size exceeds 0.4405, suggesting that larger cell sizes without strong adhesion may still correspond to non-invasive tissue behavior. Across the extracted rules for the E. coli dataset, most rules predicting the cp (cytoplasmic) protein class involve moderate or low values of alm1 (≤0.6955) combined with strongly negative mcg values (≤−1.6469 or≤−1.7241) and moderate ranges of alm2 and aac, suggesting weaker signal characteristics associated with cytoplasmic proteins. In contrast, rules predicting the im (inner membrane) protein class consistently involve higher feature values, including alm1 greater than approximately 0.3473 to 0.6258, mcg greater than about 0.5914, and alm2 exceeding approximately 1.5317, indicating stronger signal patterns associated with inner membrane protein localization. Across the extracted rules for the page blocks dataset, SR4-Fit identifies patterns primarily driven by geometric properties of document blocks, including height, eccentricity, block area, length, pixel composition, and white-to-black transitions. Most rules predict class 1, which reflects the dominant category in the dataset and typically corresponds to standard document regions such as text blocks or simple rectangular areas. These rules commonly involve moderate or small block heights (≤0.8981), low eccentricity values (≤−0.4422), and moderate pixel composition measures, indicating regular block shapes and consistent document textures. Some rules rely solely on geometric attributes such as area or length, suggesting that the physical dimensions of blocks are strong predictors of common document structures. In contrast, class 2 predictions occur when blocks become larger or taller (height > 0.8453) while maintaining specific eccentricity values, indicating structural elements that differ from typical text regions. Across the extracted rules for the Pima Indians Diabetes dataset, diabetic predictions generally occur when glucose levels exceed approximately 0.96 to 1.15, often combined with moderate or high BMI values (> −1.1286) or additional factors such as pregnancy count, insulin levels, or blood pressure. These conditions reflect well-known clinical indicators of diabetes risk, particularly elevated blood glucose and body mass index. In contrast, non-diabetic classifications occur when glucose levels remain relatively low (≤ 0.3945 or≤ −0.2001), often accompanied by lower BMI values and younger age groups, suggesting lower metabolic risk. Some rules also indicate that genetic risk factors, represented by the Diabetes Pedigree Function, may play a secondary role when glucose levels remain low. Across the extracted rules for the vehicle dataset, SR4-Fit identifies interpretable patterns based on vehicle shape and geometric descriptors, including aspect ratios, elongatedness, circularity, rectangularity, variance, and scatter measures derived from vehicle silhouettes. Bus classifications occur when vehicle shapes exhibit larger elongated structures or compact rectangular silhouettes, often associated with moderate or low aspect ratios and specific variance ranges. Car classifications tend to appear when vehicle shapes are more compact, balanced, and circular, often characterized by moderate circularity and reduced scatter or hollow ratios. In contrast, van classifications are typically associated with box-like and rectangular silhouettes, reflected by moderate variance values, lower circularity, and specific rectangularity thresholds. Across the extracted rules for the yeast dataset, the rules that predict the CYT (cytosolic) class indicate that proteins belonging to this class exhibit distinctive combinations of signal intensities across these attributes. Several rules involve very low vac values (≤ −2.3346) combined with moderate or low mcg and gvh values, suggesting that weak localization signals for certain cellular compartments correspond to cytosolic proteins. Other rules predict CYT when mcg and gvh values are relatively high (mcg > 1.7113 or gvh > 2.1397), indicating alternative signal patterns associated with cytosolic localization. Additional rules rely on very low alm values (≤ −1.3277) or low mitochondrial indicators (mit ≤ 0.4656) combined with moderate nuclear signal values, suggesting that the absence of strong compartment-specific signals can also indicate cytosolic proteins. 6.8. Interpretation SR4-Fit Rules for Public Regression Datasets In the Abalone dataset, the regression rules primarily emphasize the role of shell weight in determining the predicted outcome. When the shell weight is relatively high (shell weight>0.8448) and the shucked weight remains moderate (shucked weight≤0.8860), the model adds 0.56 to the prediction, indicating the strongest positive contri- bution. Abalones with moderate shell weight (0.1150<shell weight≤0.8448) also increase the prediction by 0.48, suggesting that mid-range shell weight still has a strong positive relationship with the target variable. In contrast, when shell weight is lower (−1.3939<shell weight≤−0.6147) and the encoded sex value is less than 0.6568, the model contributes only 0.12, indicating a comparatively weaker influence. In the Bone dataset, the regression rules primarily highlight the influence of age on the predicted outcome. When age is relatively low (age≤−0.4755) and the encoded gender value is greater than −0.0568, the model contributes 0.01 to the prediction, indicating a small positive effect. Similarly, when age falls within the range −1.0279<age≤−0.5504, S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 22 of 77 the prediction increases by 0.01, suggesting that individuals in this younger age interval also contribute positively to the outcome. In the Diabetes dataset, when BMI is very high (BMI>1.5267) and S2 is relatively low (≤0.5068), the prediction increases substantially by 63.8, while a similar condition with S5>−0.0210, BMI>1.5237, and S2≤0.5107 results in an even stronger contribution of 68.7. When BMI remains high but S2 becomes larger, the contribution drops to 16.76 or 21.97, suggesting that S2 moderates the impact of high BMI. For moderate BMI (0.0745< BMI≤1.5267), elevated S6 (>0.6737) increases the prediction by 31.04, while lower S6 reduces the effect to 4.56. Additional rules show that high S5 combined with elevated blood pressure (BP>1.3623) contributes 55.17, indicating strong metabolic interactions. Even when BMI is relatively low, combinations involving S3≤−0.7763, S4>1.9298, or S1≤0.9309 still produce contributions around 24, demonstrating that biomarker interactions beyond BMI can also significantly influence predicted diabetes progression. In the Housing dataset, the rules highlight the strong influence of the average number of rooms (RM) on predicted housing prices. When room counts are very high (RM>1.6312 or RM>1.7026) and favorable neighborhood characteristics such as low pupil–teacher ratios (PTRATIO≤0.6006 or≤−0.0796), moderate tax rates (TAX≤0.7703), or reasonable environmental conditions (NOX≤0.8502) are present, the predicted housing value increases substantially by 12.12 to 15.15. For moderately high room counts (0.9091<RM≤1.6312) combined with low lower-status population percentages (LSTAT≤0.4702) and reasonable distances to employment centers (DIS>−0.9158), the prediction increases more modestly by 3.46. Even when room counts are smaller, combinations involving controlled crime rates, housing age, and socio-economic factors still contribute small positive effects of 0.58 to 2.04. In the Machine dataset, the rules emphasize the importance of hardware capacity and baseline performance metrics in predicting machine performance. When PRP values are extremely high (PRP>3.3872 or PRP>3.9534) and large memory capacity (MMAX), the model increases the predicted performance by 20.19 to 27.50, while combinations of high PRP and specific model categories can contribute as much as 47.75. The strongest contribution occurs when both memory capacity (MMAX>2.9077) and maximum channel capacity (CHMAX>1.2501) are high, resulting in an increase of 49.42. Additional rules show that model and vendor identifiers also influence predictions, indicating that certain machine configurations or manufacturers correspond to higher predicted performance. In the MPG dataset, the rules highlight the importance of engine size, horsepower, vehicle weight, and model year in determining fuel efficiency. Vehicles with very small engine displacement (≤−0.9550), low horsepower (≤−0.7266), and newer model years (model year>0.3881) increase the predicted MPG by 5.32, while low-cylinder engines with extremely low horsepower (≤−1.2563) in newer vehicles produces an even stronger effect of 7.45. Lightweight vehicles also contribute positively, as seen when weight≤−0.8996 combined with small displacement and newer model years increases MPG by 4.05. Additional rules show that moderate horsepower combined with low displacement and light vehicle weight adds 3.39 to the prediction. In contrast, vehicles with larger engines and higher horsepower, particularly from older model years, produce smaller effects such as 1.16, indicating weaker contributions to fuel efficiency. In the Ozone dataset, the rules emphasize the importance of temperature, atmospheric pressure conditions, humidity, and visibility in determining ozone levels. When temperatures at Sandburg and El Monte are very high (Temp Sandburg>1.2484 or Temp El Monte>1.3626) and IBT values are elevated, the predicted ozone concentration increases substantially by 6.21 to 7.07. The strongest increase occurs when Sandburg temperature exceeds 0.7464, IBT>0.9268, and visibility is extremely low (≤−0.9002), resulting in an increase of 8.23, indicating that hot and stagnant atmospheric conditions strongly promote ozone formation. Additional rules show that low visibility combined with high temperatures and low wind speeds contributes between 5.04 and 5.71, reflecting the accumulation of pollutants when atmospheric dispersion is limited. Even when IBT values are lower, combinations of moderate temperature, reduced visibility, and higher humidity still increase ozone predictions by 2.36. In the prostate dataset, the rules emphasize the role of tumor volume (lcavol), seminal vesicle invasion (svi), prostate weight (lweight), and other clinical markers in predicting PSA levels. When tumor volume is very high (lcavol>1.2430) and benign hyperplasia measurements are elevated (lbph>−0.4522), the predicted outcome increases by 0.48, indicating a strong association between tumor size and PSA levels. Similarly, the presence of seminal vesicle invasion (svi>0.6291) combined with high tumor volume (lcavol>1.2786) increases the prediction by 0.43, highlighting the influence of tumor spread. Additional rules show that high tumor volume combined with capsular penetration (lcp>2.1708) or moderate tumor volume with controlled Gleason scores (pgg45≤2.1094) contributes smaller increases ranging from 0.17 to 0.23. Even when tumor volume is lower, combinations involving larger prostate weight and older age still increase predictions by 0.19. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 23 of 77 7. Conclusions This work presented SR4-Fit, an interpretable machine learning framework designed to generate compact, stable, and rule-based predictive models while maintaining competitive predictive performance. The primary application examined in this study is the U.S. House of Representatives election datasets, where the objective is to explain variations in Democratic and Republican vote percentages using demographic and socioeconomic characteristics derived from the American Community Survey (ACS). By extracting interpretable decision rules, SR4-Fit enables transparent analysis of how demographic patterns across congressional districts relate to electoral outcomes. The rule sets generated for the U.S. House election data reveal interpretable relationships between demographic and socioeconomic variables, such as race, age distribution, education levels, income, marital status, and historical voting behavior. These rule-based explanations provide insight into how combinations of district-level characteristics influence Democratic and Republican vote shares, illustrating the usefulness of interpretable models for political and policy analysis, where transparency is critical. To support and validate the proposed approach beyond the primary electoral application, SR4-Fit was also evaluated on several public regression and classification benchmark datasets. These additional experiments demonstrate that the method performs consistently across diverse domains while producing interpretable rule structures. Interpretability in this study is evaluated using our newly introduced rule understandability score (RUS) which combines combines key aspects of interpretable modeling: rule stability, number of extracted rules, and rule complexity. RUS is utilized in a Pareto analysis placing RUS on one axis and predictive accuracy (measured using the F1 score for classification and the coefficient of determination, 푅 2 , for regression) on the other. We further introduce and utilize an enhanced interpretability score (E-IPS) that combines understandability and predictivity into a single numeric value. In both RUS and E-IPS, stability is assessed using the Dice-Sørensen similarity index across repeated trials, reflecting the consistency of rule structures. The number of rules measures model compactness, while rule complexity captures the number of conditions required to represent each rule. The experimental results show that SR4-Fit achieves a strong balance between predictive performance and interpretability. Compared with existing rule-based baselines, the framework consistently produces more stable rule sets with lower rule complexity while maintaining competitive F1 scores in classification and competitive 푅 2 values in regression, Pareto-dominating RuleFit and Decision Tree across all four experimental settings: election classification, election regression, public classification, and public regression. Relative to Random Forest, SR4-Fit consistently trades a modest amount of predictivity for substantially greater rule understandability in every setting, reflecting a single, coherent predictivity-interpretability trade-off rather than divergent behavior across tasks. Overall, the results demonstrate that SR4-Fit provides a practical and reliable approach for interpretable machine learning in complex real-world settings. By generating stable, compact, and human-readable rule sets, the framework enables transparent understanding of predictive relationships without sacrificing F1 or 푅 2 performance relative to existing rule-based methods. The analysis of the U.S. House election dataset illustrates how interpretable rules can reveal meaningful connections between demographic characteristics and electoral outcomes, highlighting the broader potential of interpretable models for social and policy-related research. At the same time, strong F1 and푅 2 performance across multiple public benchmark datasets confirm the general applicability of the proposed method. These findings suggest that SR4-Fit can serve as an effective tool for domains where understandability, reliability, and trust in model outputs are essential, contributing to the growing need for interpretable machine learning approaches in data-driven decision-making. 8. Acknowledgements This work was supported in part by funding from the Institute for Community and Society Transformation (ICAST) at the University of Oklahoma, which is an interdisciplinary initiative that supports collaborative research and fosters partnerships across disciplines to address complex societal challenges. The authors gratefully acknowledge ICAST for its support of this project. References [1] L. Breiman, Random forests, Machine Learning 45 (2001) 5–32. [2] M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, B. Scholkopf, Support vector machines, IEEE Intelligent Systems and Their Applications 13 (4) (1998) 18–28. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 24 of 77 [3] J. H. Friedman, Greedy function approximation: A gradient boosting machine, Annals of Statistics (2001) 1189–1232. [4] W. S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity, The Bulletin of Mathematical Biophysics 5 (1943) 115–133. [5] R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, N. Elhadad, Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission, in: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, p. 1721–1730. [6] A. Nesvijevskaia, S. Ouillade, P. Guilmin, J.-D. Zucker, The accuracy versus interpretability trade-off in fraud detection model, Data & Policy 3 (2021) e12. [7] C. Molnar, Interpretable Machine Learning: A Guide for Making Black Box Models Explainable, Lulu Press, Morrisville, NC, 2020, available at https://christophm.github.io/interpretable-ml-book/. [8] T. S. Arrington, Affirmative districting and four decades of redistricting: The seats/votes relationship 1972-2008, Politics & Policy 38 (2) (2010) 223–253. [9] J. N. Friedman, R. T. Holden, The rising incumbent reelection rate: What’s gerrymandering got to do with it?, The Journal of Politics 71 (2) (2009) 593–611. [10] B. Richardson, D. F. Hougen, Districts by demographics: Predicting us house of representative elections using machine learning and demographic data, in: 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, 2020, p. 833–838. [11] W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, B. Yu, Definitions, methods, and applications in interpretable machine learning, Proceedings of the National Academy of Sciences 116 (44) (2019) 22071–22080. [12] S. M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, Advances in Neural Information Processing Systems 30 (2017). [13] M. T. Ribeiro, S. Singh, C. Guestrin, “Why should I trust you?” explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, p. 1135–1144. [14] C. Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence 1 (5) (2019) 206–215. [15] J. R. Quinlan, Induction of decision trees, Machine learning 1 (1) (1986) 81–106. [16] D. Vidaurre, C. Bielza, P. Larranaga, A survey of l1 regression, International Statistical Review 81 (3) (2013) 361–387. [17] J. Fürnkranz, D. Gamberger, N. Lavrač, Foundations of rule learning, Springer Science & Business Media, 2012. [18] J. Obregon, J.-Y. Jung, RuleCOSI+: Rule extraction for interpreting classification tree ensembles, Information Fusion 89 (2023) 355–381. [19] B. Letham, C. Rudin, T. H. McCormick, D. Madigan, Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model, The Annals of Applied Statistics 9 (3) (2015) 1350 – 1371. doi:10.1214/15-AOAS848. URL https://doi.org/10.1214/15-AOAS848 [20] E. Angelino, N. Larus-Stone, D. Alabi, M. Seltzer, C. Rudin, Learning certifiably optimal rule lists for categorical data, Journal of Machine Learning Research 18 (234) (2018) 1–78. [21] C. Bénard, G. Biau, S. Da Veiga, E. Scornet, Interpretable random forests via rule extraction, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2021, p. 937–945. [22] J. H. Friedman, B. E. Popescu, Predictive learning via rule ensembles, The Annals of Applied Statistics 2 (3) (2008) 916–954. [23] H. Yang, C. Rudin, M. Seltzer, Scalable Bayesian rule lists, in: International Conference on Machine Learning, PMLR, 2017, p. 3921–3930. [24] P. Zheng, T. Askham, S. L. Brunton, J. N. Kutz, A. Y. Aravkin, A unified framework for sparse relaxed regularized regression: SR3, IEEE Access 7 (2018) 1404–1423. [25] D. Gunning, et al., Explainable artificial intelligence (xai), Defense Advanced Research Projects Agency (DARPA), nd Web 2 (2) (2017) 1. [26] D. Gunning, D. W. Aha, Darpa’s explainable artificial intelligence program, AI magazine 40 (2) (2019) 44–58. [27] P. P. Angelov, E. A. Soares, R. Jiang, N. I. Arnold, P. M. Atkinson, Explainable artificial intelligence: an analytical review, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 11 (5) (2021) e1424. [28] M. Guo, Q. Zhang, X. Liao, Y. Chen, An interpretable machine learning framework for modelling human decision behavior, arXiv preprint arXiv:1906.01233 (2019). [29] M. E. Morocho-Cayamcela, H. Lee, W. Lim, Machine learning for 5g/b5g mobile and wireless communications: potential, limitations, and future directions, IEEE access 7 (2019) 137184–137206. [30] A. Kumar, S. Dikshit, V. H. C. Albuquerque, Explainable artificial intelligence for sarcasm detection in dialogues, Wireless Communications and Mobile Computing 2021 (1) (2021) 2939334. [31] T. A. Abdullah, M. S. M. Zahid, W. Ali, A review of interpretable ml in healthcare: taxonomy, applications, challenges, and future directions, Symmetry 13 (12) (2021) 2439. [32] S. Nazir, D. M. Dickson, M. U. Akram, Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks, Computers in Biology and Medicine 156 (2023) 106668. [33] V. Viswan, N. Shaffi, M. Mahmud, K. Subramanian, F. Hajamohideen, Explainable artificial intelligence in alzheimer’s disease classification: A systematic review, Cognitive Computation 16 (1) (2024) 1–44. [34] M. Pichler, F. Hartig, Machine learning and deep learning—a review for ecologists, Methods in Ecology and Evolution 14 (4) (2023) 994–1016. [35] Z. Chen, F. Xiao, F. Guo, J. Yan, Interpretable machine learning for building energy management: A state-of-the-art review, Advances in Applied Energy 9 (2023) 100123. [36] X. Liu, T. Zhang, H. Yang, S. Qian, Z. Dong, W. Li, L. Zou, Z. Liu, Z. Wang, T. Zhang, et al., Explainable machine learning-based method for fracturing prediction of horizontal shale oil wells, Processes 11 (9) (2023) 2520. [37] L. Gugliermetti, F. Cumo, S. Agostinelli, A future direction of machine learning for building energy management: Interpretable models, Energies 17 (3) (2024) 700. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 25 of 77 [38] L. Sailer, Explainable artificial intelligence in finance: Mapping business problems and technological solutions, Finance Research Open (2026) 100128. [39] V. Margot, G. Luta, A new method to compare the interpretability of rule-based algorithms, AI 2 (4) (2021) 621–635. [40] S. S. Murali Krishnan, D. F. Hougen, SR4-Fit: An interpretable and informative classification algorithm applied to prediction of u.s. house of representatives elections, in: 2025 International Conference on Machine Learning and Applications (ICMLA), 2025, p. 382–389. doi:10.1109/ICMLA66185.2025.00058. [41] R. Rifkin, A. Klautau, In defense of one-vs-all classification, Journal of Machine Learning Research 5 (Jan) (2004) 101–141. [42] R. Tibshirani, Regression shrinkage and selection via the LASSO, Journal of the Royal Statistical Society Series B: Statistical Methodology 58 (1) (1996) 267–288. [43] W. W. Cohen, et al., Fast effective rule induction, in: Proceedings of the 12th International Conference on Machine Learning, 1995, p. 115–123. [44] U.S. Census Bureau, 2006–2016 American Community Survey 1-Year Estimates of Selected Characteristics of the Total and Native Populations in the United States, Table S0601, https://data.census.gov/cedsci/table?tid=ACSST5Y2015.S0601 (2020). [45] MIT Election Data and Science Lab, U.S. House 1976–2018, https://doi.org/10.7910/DVN/IG0UN2 (2017). [46] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, É. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825–2830. [47] J. A. Swets, Effectiveness of information retrieval methods, American Documentation 20 (1) (1969) 72–89. [48] C. J. V. Rijsbergen, Information Retrieval, 2nd Edition, Butterworth-Heinemann, USA, 1979. [49] T. O. Hodson, Root mean square error (rmse) or mean absolute error (mae): When to use them or not, Geoscientific Model Development Discussions 2022 (2022) 1–10. [50] A. Chao, R. L. Chazdon, R. K. Colwell, T.-J. Shen, Abundance-based similarity indices and their estimation when there are unseen species in samples, Biometrics 62 (2) (2006) 361–371. [51] W. Wolberg, Breast Cancer Wisconsin (Original), UCI Machine Learning Repository, DOI: https://doi.org/10.24432/C5HP4Z (1990). [52] K. Nakai, Ecoli, UCI Machine Learning Repository, DOI: https://doi.org/10.24432/C5388M (1996). [53] D. Malerba, Page Blocks Classification, UCI Machine Learning Repository, DOI: https://doi.org/10.24432/C5J590 (1994). [54] Smith, J. W. and Everhart, J. E. and Dickson, W. C. and Knowler, W. C. and Johannes, R. S., Using the ADAP learning algorithm to forecast the onset of diabetes mellitus, https://archive.ics.uci.edu/ml/datasets/pima+indians+diabetes (1988). [55] P. Mowforth, B. Shepherd, Statlog (Vehicle Silhouettes), UCI Machine Learning Repository, DOI: https://doi.org/10.24432/C5HG6N. [56] K. Nakai, Yeast, UCI Machine Learning Repository, DOI: https://doi.org/10.24432/C5KG68 (1991). [57] J. Friedman, T. Hastie, R. Tibshirani, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, NY, 2001. [58] J. R. Quinlan, Auto mpg data set, https://archive.ics.uci.edu/ml/datasets/Auto+MPG, uCI Machine Learning Repository (1993). [59] J. R. Quinlan, Computer hardware data set, https://archive.ics.uci.edu/ml/datasets/Computer+Hardware, uCI Machine Learning Repository (1993). [60] W. J. Nash, T. L. Sellers, S. R. Talbot, A. J. Cawthorn, W. B. Ford, Abalone data set, https://archive.ics.uci.edu/ml/datasets/Ab alone, uCI Machine Learning Repository (1994). [61] B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression, The Annals of Statistics 32 (2) (2004) 407–499. doi:10.1214/0090 53604000000067. [62] J. Friedman, T. Hastie, R. Tibshirani, Bone mineral density (bmd) data set, https://web.stanford.edu/~hastie/ElemStatLearn/, from The Elements of Statistical Learning (2001). [63] A. Asuncion, D. J. Newman, Uci machine learning repository, https://archive.ics.uci.edu/ml, contains Boston Housing dataset (2003). [64] J. Friedman, T. Hastie, R. Tibshirani, Prostate cancer data set, https://web.stanford.edu/~hastie/ElemStatLearn/, from The Elements of Statistical Learning (2001). [65] Y. Jin, B. Sendhoff, Pareto-based multiobjective machine learning: An overview and case studies, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38 (3) (2008) 397–415. [66] J. Hao, T. K. Ho, Machine learning made easy: a review of scikit-learn package in python programming language, Journal of educational and behavioral statistics 44 (3) (2019) 348–361. [67] G. E. Noether, Introduction to wilcoxon (1945) individual comparisons by ranking methods, in: Breakthroughs in Statistics: Methodology and Distribution, Springer, 1992, p. 191–195. [68] J. W. Tukey, Exploratory Data Analysis, Addison-Wesley, Reading, MA, 1977. [69] A. Suh, G. Appleby, E. W. Anderson, L. Finelli, R. Chang, D. Cashman, Are metrics enough? guidelines for communicating and visualizing predictive models to subject matter experts, IEEE Transactions on Visualization and Computer Graphics (2023). [70] Pew Research Center, The 2014 midterm: Congressional vote, top issues, in: Midterm Election Indicators Daunting for Democrats, Pew Research Center, Washington, D.C., 2014, Ch. 1, full report available at https://w.pewresearch.org/wp-content/uploads/sit es/4/legacy-pdf/5-5-14-Political-Release.pdf. URL https://w.pewresearch.org/politics/2014/05/05/the-2014-midterm-congressional-vote-top-issues/ [71] S. Keeter, R. Igielnik, Polls and votes: The 2014 elections by the numbers, in: Can Likely Voter Models Be Improved, Pew Research Center, Washington, D.C., 2016, Ch. 1, full report available at https://w.pewresearch.org/methods/2016/01/07/can-likely-voter -models-be-improved/. URL https://w.pewresearch.org/methods/2016/01/07/polls-and-votes-the-2014-elections-by-the-numbers/ S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 26 of 77 [72] J. A. Barnes, How turnout impacts presidential and congressional elections (Oct. 2016). URL https://ballotpedia.org/How_turnout_impacts_presidential_and_congressional_elections [73] Roper Center for Public Opinion Research, How groups voted in 2016 (Nov. 2016). URL https://ropercenter.cornell.edu/how-groups-voted-2016 [74] S. Clement, E. Guskin, 2016 election exit polls: How the vote has shifted, Washington Post (Nov. 2016). URL https://w.washingtonpost.com/graphics/politics/2016-election/exit-polls/ [75] Center for American Progress, Release: New CAP report examines turnout and voting patterns that led to 2016 election result (Nov. 2017). URL https://w.americanprogress.org/press/release-new-cap-report-examines-turnout-and-voting-patterns-t hat-led-to-2016-election-result/ [76] Pew Research Center, An examination of the 2016 electorate, based on validated voters (Aug. 2018). URL https://w.pewresearch.org/politics/2018/08/09/an-examination-of-the-2016-electorate-based-on-valid ated-voters/ Appendices (Supplemental Material) A Statistical Significance Test (Wilcoxon Test)27 B Results of U.S House of Representative Elections Classification29 C Results of U.S House of Representative Elections Regression (DEM)42 D Results of U.S House of Representative Elections Regression (REP)51 E Results on Standard Public Datasets Classification60 F Results on Standard Public Datasets Regression69 A. Statistical Significance Test (Wilcoxon Test) To assess whether the differences in predictive performance between SR4-Fit and RuleFit are statistically meaningful, one-sided Wilcoxon signed-rank tests were conducted across all performance metrics for each dataset and feature configuration, with a significance threshold of 푝 < 0.05. Table 6 reports the results for the classification task. When prior party voting percentages are included as features, SR4-Fit’s advantage over RuleFit is statistically significant across all four metrics (accuracy, precision, recall, and F1 score) in the Minimum, Standard, and Expanded datasets. In the previous dataset, none of the four metrics reach significance, consistent with the closely matched E-IPS and predictive performance observed between the two models on this dataset. When prior party voting percentages are removed, the differences become significant across all four metrics in every dataset, with most 푝-values falling below 0.0001, indicating that SR4-Fit’s advantage over RuleFit is most pronounced and most consistent in the reduced-information setting. Table 7 reports the results for the DEM regression task. The pattern here is more mixed than in classification. When the prior Republican voting percentage is included as a feature, significant differences favoring SR4-Fit appear for RMSE and푅 2 in the Standard and Expanded datasets and for MAE in the Minimum dataset, while no metric reaches significance in the Previous dataset. When the prior Republican voting percentage is removed, significant differences emerge for RMSE and 푅 2 in the expanded and previous datasets, while the minimum and standard datasets show no significant differences. Across both feature configurations, MAE rarely reaches significance, whereas RMSE and 푅 2 tend to move together, reaching significance in the same dataset when either does. Table 8 reports the results for the REP regression task. When prior Democratic voting percentage is included as a feature, MAE is the only metric that reaches significance, doing so consistently across all four datasets, while RMSE and 푅 2 do not reach significance in any dataset under this configuration. When the prior Democratic voting percentage S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 27 of 77 Table 6 Wilcoxon test W-statistic and p-value comparison between RuleFit and SR4-Fit over 30 trials across demographic classification datasets. Statistically significant differences (푝 < 0.05) are shown in bold. Dataset AccuracyPrecisionRecallF1 Score W-stat p-value W-stat p-value W-stat p-value W-stat p-value Minimum 74.5 0.0024 33.0 0.0178 19.0 0.0374 73.5 0.0034 Standard 128.0 0.0009 47.0 0.0234 110.0 0.0023 159.0 0.0007 Expanded 341.0 0.0001 192.0 0.0006 192.5 0.0037 367.0 0.0001 Previous 72.5 0.1037 48.0 0.8495 148.5 0.0522 174.0 0.0615 (a) With prior party voting percentages. Dataset AccuracyPrecisionRecallF1 Score W-stat p-value W-stat p-value W-stat p-value W-stat p-value Minimum 418.0 <0.0001 337.0 0.0158 397.0 <0.0001 439.0 <0.0001 Standard 465.0 <0.0001 465.0 <0.0001 464.0 <0.0001 465.0 <0.0001 Expanded 465.0 <0.0001 465.0 <0.0001 416.0 0.0001 465.0 <0.0001 Previous 465.0 <0.0001 465.0 <0.0001 465.0 <0.0001 465.0 <0.0001 (b) Without prior party voting percentages. Table 7 Wilcoxon signed-rank test W-statistic and p-value comparison between RuleFit and SR4-Fit over 30 trials across demographic datasets for Democratic percentage label regression. Statistically significant differences (푝 < 0.05) are shown in bold. Dataset RMSEMAER 2 W-stat p-value W-stat p-value W-stat p-value Minimum 193.0 0.2083 13.0 <0.0001 274.0 0.1967 Standard 52.0 0.0001 375.0 0.9983 415.0 0.0001 Expanded 127.0 0.0150 408.0 0.9998 333.0 0.0194 Previous 425.0 1.0000 221.0 0.4065 37.0 1.0000 (a) With prior Republican voting percentage as feature. Dataset RMSEMAER 2 W-stat p-value W-stat p-value W-stat p-value Minimum 325.0 0.9715 379.0 0.9987 139.0 0.9728 Standard 189.0 0.1855 398.0 0.9997 280.0 0.1643 Expanded 126.0 0.0142 394.0 0.9996 335.0 0.0175 Previous 28.0 <0.0001 325.0 0.9715 439.0 <0.0001 (b) Without prior Republican voting percentage as feature. is removed, the pattern shifts: RMSE and 푅 2 become significant in the Standard, Expanded, and Previous datasets, while MAE does not reach significance in any dataset in this configuration. Across all three tables, no comparison yields a statistically significant result in RuleFit’s favor, consistent with the one-sided test formulation used throughout. Where significant differences are observed, they consistently favor SR4- Fit, reinforcing the pattern described in the main text: SR4-Fit’s predictive advantage over RuleFit is strongest and most consistent in the classification task, particularly under the without-prior-party-voting-percentage configuration, and more selectively significant in the regression tasks, where the specific metric reaching significance (RMSE/푅 2 versus MAE) varies by dataset and feature configuration. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 28 of 77 Table 8 Wilcoxon signed-rank test W-statistic and p-value comparison between RuleFit and SR4-Fit over 30 trials across demographic datasets for Republican percentage label regression. Statistically significant differences (푝 < 0.05) are shown in bold. Dataset RMSEMAER 2 W-stat p-value W-stat p-value W-stat p-value Minimum 252.0 0.6558 145.0 0.0360 213.0 0.6558 Standard 307.0 0.9373 70.0 0.0004 157.0 0.9398 Expanded 185.0 0.1643 89.0 0.0016 281.0 0.1592 Previous 165.0 0.0825 144.0 0.0344 299.0 0.0857 (a) With prior Democratic voting percentage as feature. Dataset RMSEMAER 2 W-stat p-value W-stat p-value W-stat p-value Minimum 242.0 0.5775 247.0 0.6172 224.0 0.5694 Standard 34.0 <0.0001 274.0 0.8033 432.0 <0.0001 Expanded 56.0 0.0001 251.0 0.6482 409.0 0.0001 Previous 137.0 0.0247 212.0 0.3366 327.0 0.0260 (b) Without prior Democratic voting percentage as feature. B. Results of U.S House of Representative Elections Classification Across all configurations of the U.S. House of Representatives election classification task, SR4-Fit consistently emerges as the best combination of a stable, accurate, and compact model, even when compared with all the baselines. The combined evidence from line plots, violin plots, stability tables, and rule-complexity analyses provides a comprehensive view of model behavior both with and without the party-percentage features. In the minimum dataset, the line plots in Figure 10 and Figure 11 show that when prior party voting percentage is included, all models achieve high performance; however, SR4-Fit exhibits the smoothest trajectories across accuracy, precision, recall, and F1 with small variance, outperforming logistic regression, Decision Tree, and XGBoost. The distributional comparisons through violin plots in Figure 18 and Figure 19 confirm that SR4-Fit produces tighter metric distributions than other models, which display broader spreads and heavier fluctuations. When prior party voting percentage is removed, performance declines across models, with XGBoost maintaining the highest predictive scores, while SR4-Fit maintains comparatively smoother trends despite lower accuracy, which is further confirmed through the violin plots in Figure 20 and Figure 21 where the distributions are tighter for SR4-Fit despite lower accuracy. Stability analysis in Table 9 shows that with and without prior party voting percentages, SR4-Fit achieves the highest Dice–Sørensen Index when compared with all rule-based models, showing its structural consistency and robustness. Table 10 shows that in the case of prior party voting percentages as a feature, the Decision Tree has the most compact structure, which is preceded by both SR4-Fit and RuleFit, whereas without including prior party voting percentages, SR4-Fit has a more compact structure with respect to the number of rules. Table 11 shows that Random Forest has lower average rule complexity compared to all the other models. In the standard dataset, the line plots in Figure 12 and Figure 13 show that when prior party voting percentage is included, SR4-Fit exhibits the smoothest trajectories with small variance, outperforming all models. The distributional comparisons through violin plots in Figure 18 and Figure 19 confirm that SR4-Fit produces tighter metric distributions than other models. When the prior party voting percentages are removed, performance declines across models, with SVM maintaining the highest predictive scores. Stability analysis in Table 9 shows that with and without prior party voting percentages, RuleFit achieves the highest Dice–Sørensen Index when compared with all rule-based models, which is closely preceded by SR4-Fit. Table 10 shows that in the case of prior party voting percentages as a feature, the Decision Tree has the most compact structure, whereas without including prior party voting percentages, Random Forest has a more compact structure with respect to the number of rules. Table 11 shows that Random Forest has lower S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 29 of 77 051015202530 Trial 0.984 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 Accuracy Random Forest Mean: 0.9946 ± 0.0029 SVM Mean: 0.9978 ± 0.0021 RuleFit Mean: 0.9982 ± 0.0017 SR4 Mean: 0.9990 ± 0.0012 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Precision Random Forest Mean: 0.9951 ± 0.0055 SVM Mean: 0.9980 ± 0.0029 RuleFit Mean: 0.9983 ± 0.0026 SR4 Mean: 0.9991 ± 0.0018 051015202530 Trial 0.984 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 Recall Random Forest Mean: 0.9945 ± 0.0041 SVM Mean: 0.9978 ± 0.0031 RuleFit Mean: 0.9983 ± 0.0026 SR4 Mean: 0.9989 ± 0.0019 051015202530 Trial 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 F1 Random Forest Mean: 0.9948 ± 0.0028 SVM Mean: 0.9979 ± 0.0020 RuleFit Mean: 0.9983 ± 0.0016 SR4 Mean: 0.9990 ± 0.0011 051015202530 Trial 0.65 0.70 0.75 0.80 0.85 0.90 Accuracy Random Forest Mean: 0.8669 ± 0.0138 SVM Mean: 0.8123 ± 0.0152 RuleFit Mean: 0.7098 ± 0.0217 SR4 Mean: 0.7313 ± 0.0197 051015202530 Trial 0.65 0.70 0.75 0.80 0.85 0.90 Precision Random Forest Mean: 0.8492 ± 0.0186 SVM Mean: 0.7899 ± 0.0221 RuleFit Mean: 0.7015 ± 0.0264 SR4 Mean: 0.7116 ± 0.0271 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 Recall Random Forest Mean: 0.9018 ± 0.0188 SVM Mean: 0.8665 ± 0.0196 RuleFit Mean: 0.7614 ± 0.0310 SR4 Mean: 0.8069 ± 0.0367 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 F1 Random Forest Mean: 0.8745 ± 0.0140 SVM Mean: 0.8261 ± 0.0149 RuleFit Mean: 0.7297 ± 0.0221 SR4 Mean: 0.7555 ± 0.0193 Figure 10: Line plot comparison of model performance metrics for minimum data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. average rule complexity when prior party voting percentages are not included, whereas RuleFit has lower average rule complexity when prior party voting percentages are included. In the expanded dataset, the line plots in Figure 14 and Figure 15 show that when prior party voting percentage is included, SR4-Fit exhibits the smoothest trajectories with small variance, outperforming all models. When the prior party voting percentages are removed, performance declines across models, with SVM maintaining the highest predictive scores. Stability analysis in Table 9 shows that with and without prior party voting percentages, SR4-Fit achieves the highest Dice–Sørensen Index when compared with other rule-based models. Table 10 shows that in the case of prior party voting percentages as a feature, the Decision Tree has the most compact structure, whereas without including prior party voting percentages, Random Forest has a more compact structure with respect to the number of rules. Table 11 shows that Random Forest has lower average rule complexity when prior party voting percentages are not included, whereas both SR4-Fit and RuleFit have lower average rule complexity when prior party voting percentages are included. In the previous dataset, the line plots in Figure 16 and Figure 17 show that when prior party voting percentage is included, SR4-Fit exhibits the smoothest trajectories with small variance, outperforming all models. When the prior party voting percentages are removed, performance declines across models, with Random Forest maintaining the highest predictive scores. Stability analysis in Table 9 shows that with and without prior party voting percentages, SR4-Fit achieves the highest Dice–Sørensen Index when compared with other rule-based models. Table 10 shows that in both cases, the Decision Tree has the most compact structure. Table 11 shows that Random Forest has lower S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 30 of 77 051015202530 Trial 0.984 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 Accuracy Logistic Regression Mean: 0.9961 ± 0.0029 Decision Tree Mean: 0.9937 ± 0.0033 XGBoost Mean: 0.9951 ± 0.0027 SR4 Mean: 0.9990 ± 0.0012 051015202530 Trial 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 Precision Logistic Regression Mean: 0.9968 ± 0.0032 Decision Tree Mean: 0.9943 ± 0.0051 XGBoost Mean: 0.9962 ± 0.0045 SR4 Mean: 0.9991 ± 0.0018 051015202530 Trial 0.9800 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 Recall Logistic Regression Mean: 0.9957 ± 0.0046 Decision Tree Mean: 0.9935 ± 0.0058 XGBoost Mean: 0.9943 ± 0.0056 SR4 Mean: 0.9989 ± 0.0019 051015202530 Trial 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 F1 Logistic Regression Mean: 0.9962 ± 0.0028 Decision Tree Mean: 0.9939 ± 0.0032 XGBoost Mean: 0.9952 ± 0.0027 SR4 Mean: 0.9990 ± 0.0011 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 Accuracy Logistic Regression Mean: 0.7116 ± 0.0185 Decision Tree Mean: 0.7879 ± 0.0179 XGBoost Mean: 0.8852 ± 0.0126 SR4 Mean: 0.7313 ± 0.0197 051015202530 Trial 0.65 0.70 0.75 0.80 0.85 0.90 Precision Logistic Regression Mean: 0.7059 ± 0.0245 Decision Tree Mean: 0.7903 ± 0.0214 XGBoost Mean: 0.8731 ± 0.0162 SR4 Mean: 0.7116 ± 0.0271 051015202530 Trial 0.75 0.80 0.85 0.90 Recall Logistic Regression Mean: 0.7550 ± 0.0264 Decision Tree Mean: 0.8007 ± 0.0339 XGBoost Mean: 0.9093 ± 0.0180 SR4 Mean: 0.8069 ± 0.0367 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 F1 Logistic Regression Mean: 0.7293 ± 0.0194 Decision Tree Mean: 0.7951 ± 0.0212 XGBoost Mean: 0.8907 ± 0.0129 SR4 Mean: 0.7555 ± 0.0193 Figure 11: Line plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics for minimum data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. average rule complexity when prior party voting percentages are not included, whereas SR4-Fit has lower average rule complexity when prior party voting percentages are included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 31 of 77 051015202530 Trial 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 Accuracy Random Forest Mean: 0.9934 ± 0.0033 SVM Mean: 0.9946 ± 0.0028 RuleFit Mean: 0.9967 ± 0.0024 SR4 Mean: 0.9984 ± 0.0016 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 Precision Random Forest Mean: 0.9929 ± 0.0062 SVM Mean: 0.9954 ± 0.0038 RuleFit Mean: 0.9982 ± 0.0028 SR4 Mean: 0.9989 ± 0.0021 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 Recall Random Forest Mean: 0.9944 ± 0.0053 SVM Mean: 0.9940 ± 0.0052 RuleFit Mean: 0.9955 ± 0.0045 SR4 Mean: 0.9980 ± 0.0028 051015202530 Trial 0.988 0.990 0.992 0.994 0.996 0.998 1.000 F1 Random Forest Mean: 0.9936 ± 0.0032 SVM Mean: 0.9947 ± 0.0028 RuleFit Mean: 0.9968 ± 0.0023 SR4 Mean: 0.9985 ± 0.0015 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 Accuracy Random Forest Mean: 0.8817 ± 0.0101 SVM Mean: 0.9013 ± 0.0130 RuleFit Mean: 0.7513 ± 0.0195 SR4 Mean: 0.7967 ± 0.0150 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 Precision Random Forest Mean: 0.8573 ± 0.0166 SVM Mean: 0.8959 ± 0.0144 RuleFit Mean: 0.7330 ± 0.0258 SR4 Mean: 0.7737 ± 0.0224 051015202530 Trial 0.775 0.800 0.825 0.850 0.875 0.900 0.925 0.950 Recall Random Forest Mean: 0.9246 ± 0.0159 SVM Mean: 0.9151 ± 0.0234 RuleFit Mean: 0.8145 ± 0.0174 SR4 Mean: 0.8566 ± 0.0141 051015202530 Trial 0.75 0.80 0.85 0.90 F1 Random Forest Mean: 0.8895 ± 0.0098 SVM Mean: 0.9052 ± 0.0124 RuleFit Mean: 0.7714 ± 0.0180 SR4 Mean: 0.8128 ± 0.0131 Figure 12: Line plot comparison of model performance metrics for standard data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. Table 9 Average Dice–Sørensen Index ± standard deviation across 30 trials for election classification datasets. Bold indicates the best-performing method per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0487±0.0107 0.4776±0.0162 0.4776±0.0162 0.0453±0.0337 Standard 0.0308±0.0087 0.7814±0.0111 0.7370±0.0110 0.0415±0.0293 Expanded 0.0378±0.0111 0.8540±0.0080 0.8540±0.0080 0.0411±0.0288 Previous 0.0587±0.0098 0.7437±0.0108 0.7873±0.0108 0.0471±0.0331 (a) With prior party voting percentage as a feature. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0210±0.0073 0.0400±0.0021 0.2047±0.0065 0.0027±0.0017 Standard 0.0347±0.0145 0.3425±0.0006 0.2066±0.0004 0.0005±0.0005 Expanded 0.0094±0.0058 0.1815±0.0011 0.3056±0.0002 0.0009±0.0008 Previous 0.0618±0.0149 0.6459±0.0098 0.6459±0.0098 0.0314±0.0210 (b) Without prior party voting percentage as feature. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 32 of 77 051015202530 Trial 0.984 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 Accuracy Logistic Regression Mean: 0.9985 ± 0.0016 Decision Tree Mean: 0.9925 ± 0.0031 XGBoost Mean: 0.9953 ± 0.0025 SR4 Mean: 0.9984 ± 0.0016 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 Precision Logistic Regression Mean: 0.9991 ± 0.0018 Decision Tree Mean: 0.9937 ± 0.0054 XGBoost Mean: 0.9963 ± 0.0044 SR4 Mean: 0.9989 ± 0.0021 051015202530 Trial 0.9800 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 Recall Logistic Regression Mean: 0.9980 ± 0.0023 Decision Tree Mean: 0.9918 ± 0.0058 XGBoost Mean: 0.9945 ± 0.0050 SR4 Mean: 0.9980 ± 0.0028 051015202530 Trial 0.986 0.988 0.990 0.992 0.994 0.996 0.998 1.000 F1 Logistic Regression Mean: 0.9986 ± 0.0016 Decision Tree Mean: 0.9927 ± 0.0031 XGBoost Mean: 0.9954 ± 0.0024 SR4 Mean: 0.9985 ± 0.0015 051015202530 Trial 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 Accuracy Logistic Regression Mean: 0.8133 ± 0.0126 Decision Tree Mean: 0.7890 ± 0.0185 XGBoost Mean: 0.8978 ± 0.0111 SR4 Mean: 0.7967 ± 0.0150 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 Precision Logistic Regression Mean: 0.7943 ± 0.0210 Decision Tree Mean: 0.7987 ± 0.0258 XGBoost Mean: 0.8837 ± 0.0143 SR4 Mean: 0.7737 ± 0.0224 051015202530 Trial 0.75 0.80 0.85 0.90 0.95 Recall Logistic Regression Mean: 0.8607 ± 0.0145 Decision Tree Mean: 0.7908 ± 0.0315 XGBoost Mean: 0.9232 ± 0.0157 SR4 Mean: 0.8566 ± 0.0141 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 F1 Logistic Regression Mean: 0.8260 ± 0.0125 Decision Tree Mean: 0.7941 ± 0.0190 XGBoost Mean: 0.9029 ± 0.0107 SR4 Mean: 0.8128 ± 0.0131 Figure 13: Line plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics for standard data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. Table 10 Average number of rules ± standard deviation across 30 trials for election classification datasets. Bold indicates the most compact model. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 86.43±8.70 19.33±1.06 19.33±1.06 9.97±0.76 Standard 68.00±5.19 36.13±0.94 38.30±0.99 10.03±0.85 Expanded 82.07±8.99 54.13±0.94 54.13±0.94 10.03±0.93 Previous 153.93±11.25 39.30±0.99 37.13±0.94 8.77±1.01 (a) With prior party voting percentage as a feature. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 89.70±5.16 177.80±13.47 34.80±1.54 253.87±15.9 Standard 32.50±7.23 76.00±0.00 126.00±0.00 104.00±3.301 Expanded 23.27±6.69 242.73±2.98 144.00±0.00 90.56±5.67 Previous30.03±6.7142.87±0.35 42.87±0.35 22.03±1.62 (b) Without prior party voting percentage as a feature. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 33 of 77 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Accuracy Random Forest Mean: 0.9915 ± 0.0048 SVM Mean: 0.9872 ± 0.0042 RuleFit Mean: 0.9953 ± 0.0033 SR4 Mean: 0.9983 ± 0.0020 051015202530 Trial 0.960 0.965 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Precision Random Forest Mean: 0.9903 ± 0.0072 SVM Mean: 0.9877 ± 0.0069 RuleFit Mean: 0.9959 ± 0.0047 SR4 Mean: 0.9987 ± 0.0024 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Recall Random Forest Mean: 0.9933 ± 0.0054 SVM Mean: 0.9874 ± 0.0057 RuleFit Mean: 0.9952 ± 0.0049 SR4 Mean: 0.9980 ± 0.0032 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 F1 Random Forest Mean: 0.9918 ± 0.0045 SVM Mean: 0.9875 ± 0.0040 RuleFit Mean: 0.9955 ± 0.0031 SR4 Mean: 0.9983 ± 0.0019 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 Accuracy Random Forest Mean: 0.8835 ± 0.0140 SVM Mean: 0.9073 ± 0.0133 RuleFit Mean: 0.8086 ± 0.0125 SR4 Mean: 0.8319 ± 0.0131 051015202530 Trial 0.750 0.775 0.800 0.825 0.850 0.875 0.900 0.925 Precision Random Forest Mean: 0.8573 ± 0.0185 SVM Mean: 0.9087 ± 0.0167 RuleFit Mean: 0.7843 ± 0.0155 SR4 Mean: 0.8085 ± 0.0169 051015202530 Trial 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 Recall Random Forest Mean: 0.9289 ± 0.0155 SVM Mean: 0.9121 ± 0.0208 RuleFit Mean: 0.8673 ± 0.0206 SR4 Mean: 0.8833 ± 0.0157 051015202530 Trial 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 F1 Random Forest Mean: 0.8915 ± 0.0125 SVM Mean: 0.9102 ± 0.0128 RuleFit Mean: 0.8235 ± 0.0119 SR4 Mean: 0.8441 ± 0.0108 Figure 14: Line plot comparison of model performance metrics for expanded data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. Table 11 Average rule complexity (number of conditions per rule) ± standard deviation for election classification datasets across 30 trials. Bold indicates less complex rules. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 1.90±0.03 3.00±0.15 3.00±0.15 4.41±0.20 Standard1.91±0.03 1.84±0.11 2.21±0.11 4.42±0.23 Expanded 1.90±0.02 1.60±0.09 1.60±0.09 4.44±0.30 Previous1.92±0.02 2.19±0.11 1.83±0.11 4.03±0.20 (a) With prior party voting percentage as feature. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 1.99±0.01 8.11±0.14 4.43±0.07 10.95±0.19 Standard 1.97±0.03 5.61±0.32 6.79±0.10 8.36±0.31 Expanded 1.98±0.03 8.05±0.21 7.41±0.32 7.55±0.14 Previous1.97±0.03 2.62±0.03 2.62±0.03 4.64±0.08 (b) Without prior party voting percentage as feature. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 34 of 77 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 Accuracy Logistic Regression Mean: 0.9985 ± 0.0016 Decision Tree Mean: 0.9916 ± 0.0047 XGBoost Mean: 0.9951 ± 0.0026 SR4 Mean: 0.9983 ± 0.0020 051015202530 Trial 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Precision Logistic Regression Mean: 0.9991 ± 0.0018 Decision Tree Mean: 0.9927 ± 0.0071 XGBoost Mean: 0.9961 ± 0.0048 SR4 Mean: 0.9987 ± 0.0024 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Recall Logistic Regression Mean: 0.9980 ± 0.0023 Decision Tree Mean: 0.9911 ± 0.0061 XGBoost Mean: 0.9944 ± 0.0051 SR4 Mean: 0.9980 ± 0.0032 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 F1 Logistic Regression Mean: 0.9986 ± 0.0016 Decision Tree Mean: 0.9919 ± 0.0045 XGBoost Mean: 0.9952 ± 0.0026 SR4 Mean: 0.9983 ± 0.0019 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 Accuracy Logistic Regression Mean: 0.8399 ± 0.0125 Decision Tree Mean: 0.8042 ± 0.0172 XGBoost Mean: 0.8939 ± 0.0117 SR4 Mean: 0.8319 ± 0.0131 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 Precision Logistic Regression Mean: 0.8235 ± 0.0170 Decision Tree Mean: 0.8140 ± 0.0231 XGBoost Mean: 0.8824 ± 0.0168 SR4 Mean: 0.8085 ± 0.0169 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 0.95 Recall Logistic Regression Mean: 0.8775 ± 0.0170 Decision Tree Mean: 0.8048 ± 0.0330 XGBoost Mean: 0.9166 ± 0.0157 SR4 Mean: 0.8833 ± 0.0157 051015202530 Trial 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 F1 Logistic Regression Mean: 0.8495 ± 0.0123 Decision Tree Mean: 0.8087 ± 0.0183 XGBoost Mean: 0.8990 ± 0.0104 SR4 Mean: 0.8441 ± 0.0108 Figure 15: Line plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics for expanded data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 35 of 77 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Accuracy Random Forest Mean: 0.9921 ± 0.0043 SVM Mean: 0.9946 ± 0.0034 RuleFit Mean: 0.9971 ± 0.0024 SR4 Mean: 0.9977 ± 0.0022 051015202530 Trial 0.96 0.97 0.98 0.99 1.00 Precision Random Forest Mean: 0.9912 ± 0.0080 SVM Mean: 0.9952 ± 0.0043 RuleFit Mean: 0.9982 ± 0.0024 SR4 Mean: 0.9979 ± 0.0029 051015202530 Trial 0.980 0.985 0.990 0.995 1.000 Recall Random Forest Mean: 0.9936 ± 0.0049 SVM Mean: 0.9943 ± 0.0051 RuleFit Mean: 0.9963 ± 0.0046 SR4 Mean: 0.9977 ± 0.0032 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 F1 Random Forest Mean: 0.9923 ± 0.0040 SVM Mean: 0.9947 ± 0.0033 RuleFit Mean: 0.9972 ± 0.0023 SR4 Mean: 0.9978 ± 0.0021 051015202530 Trial 0.75 0.80 0.85 0.90 0.95 Accuracy Random Forest Mean: 0.9237 ± 0.0127 SVM Mean: 0.9223 ± 0.0137 RuleFit Mean: 0.7662 ± 0.0209 SR4 Mean: 0.9089 ± 0.0132 051015202530 Trial 0.70 0.75 0.80 0.85 0.90 0.95 Precision Random Forest Mean: 0.9207 ± 0.0158 SVM Mean: 0.9244 ± 0.0163 RuleFit Mean: 0.7479 ± 0.0248 SR4 Mean: 0.9000 ± 0.0191 051015202530 Trial 0.80 0.85 0.90 0.95 Recall Random Forest Mean: 0.9324 ± 0.0143 SVM Mean: 0.9250 ± 0.0162 RuleFit Mean: 0.8239 ± 0.0230 SR4 Mean: 0.9266 ± 0.0131 051015202530 Trial 0.75 0.80 0.85 0.90 0.95 F1 Random Forest Mean: 0.9264 ± 0.0119 SVM Mean: 0.9246 ± 0.0131 RuleFit Mean: 0.7838 ± 0.0208 SR4 Mean: 0.9130 ± 0.0120 Figure 16: Line plot comparison of model performance metrics for previous data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 36 of 77 051015202530 Trial 0.9800 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 Accuracy Logistic Regression Mean: 0.9971 ± 0.0027 Decision Tree Mean: 0.9917 ± 0.0038 XGBoost Mean: 0.9943 ± 0.0026 SR4 Mean: 0.9977 ± 0.0022 051015202530 Trial 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Precision Logistic Regression Mean: 0.9972 ± 0.0036 Decision Tree Mean: 0.9934 ± 0.0067 XGBoost Mean: 0.9949 ± 0.0046 SR4 Mean: 0.9979 ± 0.0029 051015202530 Trial 0.975 0.980 0.985 0.990 0.995 1.000 Recall Logistic Regression Mean: 0.9973 ± 0.0042 Decision Tree Mean: 0.9906 ± 0.0061 XGBoost Mean: 0.9940 ± 0.0047 SR4 Mean: 0.9977 ± 0.0032 051015202530 Trial 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 F1 Logistic Regression Mean: 0.9972 ± 0.0027 Decision Tree Mean: 0.9919 ± 0.0037 XGBoost Mean: 0.9945 ± 0.0026 SR4 Mean: 0.9978 ± 0.0021 051015202530 Trial 0.88 0.89 0.90 0.91 0.92 0.93 0.94 Accuracy Logistic Regression Mean: 0.9199 ± 0.0133 Decision Tree Mean: 0.9195 ± 0.0142 XGBoost Mean: 0.9264 ± 0.0115 SR4 Mean: 0.9089 ± 0.0132 051015202530 Trial 0.86 0.88 0.90 0.92 0.94 Precision Logistic Regression Mean: 0.9178 ± 0.0167 Decision Tree Mean: 0.9126 ± 0.0188 XGBoost Mean: 0.9172 ± 0.0142 SR4 Mean: 0.9000 ± 0.0191 051015202530 Trial 0.90 0.92 0.94 0.96 Recall Logistic Regression Mean: 0.9277 ± 0.0142 Decision Tree Mean: 0.9335 ± 0.0190 XGBoost Mean: 0.9426 ± 0.0145 SR4 Mean: 0.9266 ± 0.0131 051015202530 Trial 0.88 0.89 0.90 0.91 0.92 0.93 0.94 F1 Logistic Regression Mean: 0.9226 ± 0.0126 Decision Tree Mean: 0.9227 ± 0.0136 XGBoost Mean: 0.9296 ± 0.0106 SR4 Mean: 0.9130 ± 0.0120 Figure 17: Line plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics for previous data across 30 trials. The left panel reports results obtained using feature sets that include prior party voting percentage information, whereas the right panel shows results with prior party voting percentages removed. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 37 of 77 AccuracyPrecisionRecallF1 0.975 0.980 0.985 0.990 0.995 1.000 Score Random Forest SVM RuleFit SR4 (a) Minimum AccuracyPrecisionRecallF1 0.980 0.985 0.990 0.995 1.000 Score Random Forest SVM RuleFit SR4 (b) Standard AccuracyPrecisionRecallF1 0.960 0.965 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Score Random Forest SVM RuleFit SR4 (c) Expanded AccuracyPrecisionRecallF1 0.96 0.97 0.98 0.99 1.00 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 18: Violin plot comparison of model performance metrics across multiple datasets for the election classification task that includes prior party voting percentage information. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 38 of 77 AccuracyPrecisionRecallF1 0.9800 0.9825 0.9850 0.9875 0.9900 0.9925 0.9950 0.9975 1.0000 Score Logistic Regression Decision Tree XGBoost SR4 (a) Minimum AccuracyPrecisionRecallF1 0.980 0.985 0.990 0.995 1.000 Score Logistic Regression Decision Tree XGBoost SR4 (b) Standard AccuracyPrecisionRecallF1 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Score Logistic Regression Decision Tree XGBoost SR4 (c) Expanded AccuracyPrecisionRecallF1 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Score Logistic Regression Decision Tree XGBoost SR4 (d) Previous Figure 19: Violin plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics across multiple datasets for the election classification task that includes prior party voting percentage information. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 39 of 77 AccuracyPrecisionRecallF1 0.65 0.70 0.75 0.80 0.85 0.90 Score Random Forest SVM RuleFit SR4 (a) Minimum AccuracyPrecisionRecallF1 0.70 0.75 0.80 0.85 0.90 0.95 Score Random Forest SVM RuleFit SR4 (b) Standard AccuracyPrecisionRecallF1 0.75 0.80 0.85 0.90 0.95 Score Random Forest SVM RuleFit SR4 (c) Expanded AccuracyPrecisionRecallF1 0.70 0.75 0.80 0.85 0.90 0.95 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 20: Violin plot comparison of model performance metrics across multiple datasets for the election classification task that does not include prior party voting percentage information. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 40 of 77 AccuracyPrecisionRecallF1 0.65 0.70 0.75 0.80 0.85 0.90 0.95 Score Logistic Regression Decision Tree XGBoost SR4 (a) Minimum AccuracyPrecisionRecallF1 0.75 0.80 0.85 0.90 0.95 Score Logistic Regression Decision Tree XGBoost SR4 (b) Standard AccuracyPrecisionRecallF1 0.70 0.75 0.80 0.85 0.90 0.95 Score Logistic Regression Decision Tree XGBoost SR4 (c) Expanded AccuracyPrecisionRecallF1 0.86 0.88 0.90 0.92 0.94 0.96 Score Logistic Regression Decision Tree XGBoost SR4 (d) Previous Figure 21: Violin plot comparison of model (logistic regression, Decision Tree, XG boost) performance metrics across multiple datasets for the election classification task that does not include prior party voting percentage information. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 41 of 77 C. Results of U.S House of Representative Elections Regression (DEM) In the minimum dataset, the line plots in Figure 22 and Figure 23 show that when REP percentage is included as a feature, all models achieve relatively low error values, whereas Random Forest attains slightly lower average error values compared to other models. The distributional comparisons through violin plots in Figure 30 and Figure 31 reinforce these findings. When REP percentage is removed, performance declines across all models, with XGBoost attaining slightly lower average error values compared to other models. In contrast, SR4-Fit maintains comparatively smoother trends despite a modest increase in error, which is further confirmed in Figure 32 and Figure 33, where SR4-Fit retains more concentrated distributions. Stability analysis in Table 12 shows that with both cases, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models, showing its structural consistency and robustness across different experimental scenarios. Table 13 shows that in both cases, the Decision Tree has the most compact structure, but the structure is too compact, which may lower robustness and may lower the ability to identify interactions properly. Table 14 shows that in the case of including REP percentage as a feature, SR4-Fit has the lower average rule complexity, whereas in the case of removing REP percentage as a feature, the Decision Tree has the overall lower average rule complexity. In the standard dataset, the line plots in Figure 24 and Figure 25 show that when REP percentage is included as a feature, all models achieve lower error values, whereas XGBoost attains slightly lower average error values compared to other models. When the REP percentage is removed, performance declines across all models, with XGBoost still attaining lower average error values compared to other models. Stability analysis in Table 12 shows that in the case of including REP percentage as a feature, RuleFit achieves the highest Dice–Sørensen Index among rule-based models, whereas when REP percentage is removed, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models. Table 13 shows that in both cases, the Decision Tree has the most compact structure, trailed closely by SR4-Fit. Table 14 shows that in the case of including REP percentage as a feature, RuleFit has the lower average rule complexity, whereas in the case of removing REP percentage as a feature, the SR4-Fit has the overall lower average rule complexity. In the expanded dataset, the line plots in Figure 26 and Figure 27 show that when REP percentage is included as a feature, all models achieve lower error values, whereas XGBoost again attains slightly lower average error values compared to other models. When the REP percentage is removed, performance declines across all models, with XGBoost still attaining lower average error values compared to other models. Stability analysis in Table 12 shows that in the case of including REP percentage as a feature, SR4-Fit achieves the highest Dice–Sørensen Index among rule- based models, whereas when REP percentage is removed, RuleFit achieves the highest Dice–Sørensen Index among rule-based models. Table 13 shows that in both cases, the Decision Tree has the most compact structure. Table 14 shows that in the case of including REP percentage as a feature, SR4-Fit has the lower average rule complexity, whereas in the case of removing REP percentage as a feature, the Decision Tree has the overall lower average rule complexity. In the previous dataset, the line plots in Figure 28 and Figure 29 show that when REP percentage is included as a feature, all models achieve lower error values, whereas XGBoost attains slightly lower average error values compared to other models. When the REP percentage is removed, performance declines across all models, with XGBoost slightly achieving lower average error values compared to other models. Stability analysis in Table 12 shows that with both cases, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models. Table 13 shows that in both cases, the Decision Tree has the most compact structure. Table 14 shows that in the case of including REP percentage as a feature, RuleFit has the lower average rule complexity, whereas in the case of removing REP percentage as a feature, the SR4-Fit has the overall lower average rule complexity. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 42 of 77 051015202530 Trial 7 8 9 10 11 12 RMSE Random Forest Mean: 8.3312 ± 0.8466 SVM Mean: 9.4249 ± 0.9411 RuleFit Mean: 9.8178 ± 0.8773 SR4 Mean: 9.7908 ± 0.8729 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Random Forest Mean: 0.8575 ± 0.0264 SVM Mean: 0.8175 ± 0.0342 RuleFit Mean: 0.8023 ± 0.0333 SR4 Mean: 0.8035 ± 0.0326 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Random Forest Mean: 4.4404 ± 0.2424 SVM Mean: 4.6123 ± 0.2967 RuleFit Mean: 5.6297 ± 0.2636 SR4 Mean: 5.4874 ± 0.2809 051015202530 Trial 14 15 16 17 18 RMSE Random Forest Mean: 14.4292 ± 0.5229 SVM Mean: 16.6142 ± 0.5709 RuleFit Mean: 17.6710 ± 0.5651 SR4 Mean: 17.7100 ± 0.5814 051015202530 Trial 0.30 0.35 0.40 0.45 0.50 0.55 0.60 R² Random Forest Mean: 0.5755 ± 0.0232 SVM Mean: 0.4373 ± 0.0282 RuleFit Mean: 0.3633 ± 0.0324 SR4 Mean: 0.3606 ± 0.0320 051015202530 Trial 10 11 12 13 14 MAE Random Forest Mean: 10.3683 ± 0.4106 SVM Mean: 11.8984 ± 0.4644 RuleFit Mean: 13.2812 ± 0.4524 SR4 Mean: 13.3498 ± 0.4709 Figure 22: Line plot comparison of model performance metrics for minimum data across 30 trials with DEM percentage as label. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. Table 12 Average Dice–Sørensen Index ± standard deviation across 30 trials for the election regression dataset where DEM percentage is the label. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0001±0.0000 0.3333±0.0000 0.3851±0.0116 0.0027±0.0043 Standard 0.0001±0.0000 0.7714±0.0000 0.7519±0.0101 0.0020±0.0030 Expanded 0.0001±0.0000 0.7377±0.0000 0.8337±0.0102 0.0030±0.0043 Previous 0.0001±0.0000 0.7778±0.0000 0.6863±0.0146 0.0058±0.0088 (a) DEM as a label, with REP percentage Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0003±0.0001 0.1032±0.0014 0.2690±0.0122 0.0001±0.0000 Standard 0.0002±0.0001 0.3034±0.0031 0.5404±0.0209 0.0029±0.0045 Expanded 0.0001±0.0001 0.7342±0.0015 0.6638±0.0118 0.3540±0.2354 Previous 0.0020±0.0005 0.7714±0.0000 0.8051±0.0077 0.0989±0.1001 (b) DEM as a label, without REP percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 43 of 77 051015202530 Trial 6 7 8 9 10 11 12 RMSE Lasso Regression Mean: 10.2011 ± 0.8692 Decision Tree Mean: 9.1715 ± 0.6849 XGBoost Mean: 7.5402 ± 0.8154 SR4 Mean: 9.7908 ± 0.8729 051015202530 Trial 0.75 0.80 0.85 0.90 R² Lasso Regression Mean: 0.7867 ± 0.0344 Decision Tree Mean: 0.8279 ± 0.0234 XGBoost Mean: 0.8832 ± 0.0231 SR4 Mean: 0.8035 ± 0.0326 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.1680 ± 0.2470 Decision Tree Mean: 5.0958 ± 0.2206 XGBoost Mean: 4.0285 ± 0.2696 SR4 Mean: 5.4874 ± 0.2809 051015202530 Trial 13 14 15 16 17 18 19 RMSE Lasso Regression Mean: 18.4633 ± 0.5963 Decision Tree Mean: 17.7532 ± 0.6028 XGBoost Mean: 13.4536 ± 0.4572 SR4 Mean: 17.7100 ± 0.5814 051015202530 Trial 0.3 0.4 0.5 0.6 R² Lasso Regression Mean: 0.3050 ± 0.0343 Decision Tree Mean: 0.3574 ± 0.0340 XGBoost Mean: 0.6308 ± 0.0226 SR4 Mean: 0.3606 ± 0.0320 051015202530 Trial 9 10 11 12 13 14 15 MAE Lasso Regression Mean: 13.9348 ± 0.4578 Decision Tree Mean: 13.3121 ± 0.5309 XGBoost Mean: 9.5471 ± 0.3021 SR4 Mean: 13.3498 ± 0.4709 Figure 23: Line plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics for minimum data across 30 trials. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. 051015202530 Trial 7 8 9 10 11 RMSE Random Forest Mean: 8.3613 ± 0.8337 SVM Mean: 9.4026 ± 0.9719 RuleFit Mean: 9.6431 ± 0.8910 SR4 Mean: 9.5461 ± 0.8489 051015202530 Trial 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 R² Random Forest Mean: 0.8564 ± 0.0269 SVM Mean: 0.8182 ± 0.0358 RuleFit Mean: 0.8092 ± 0.0336 SR4 Mean: 0.8131 ± 0.0315 051015202530 Trial 4.5 5.0 5.5 6.0 MAE Random Forest Mean: 4.5192 ± 0.2135 SVM Mean: 4.6868 ± 0.3077 RuleFit Mean: 5.6043 ± 0.2571 SR4 Mean: 5.6699 ± 0.2611 051015202530 Trial 13 14 15 16 17 RMSE Random Forest Mean: 13.6704 ± 0.5072 SVM Mean: 13.6791 ± 0.5564 RuleFit Mean: 15.9318 ± 0.5693 SR4 Mean: 15.9172 ± 0.5248 051015202530 Trial 0.45 0.50 0.55 0.60 0.65 R² Random Forest Mean: 0.6189 ± 0.0235 SVM Mean: 0.6185 ± 0.0238 RuleFit Mean: 0.4824 ± 0.0298 SR4 Mean: 0.4835 ± 0.0263 051015202530 Trial 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 12.5 MAE Random Forest Mean: 9.5664 ± 0.3209 SVM Mean: 9.2337 ± 0.3691 RuleFit Mean: 11.6428 ± 0.4235 SR4 Mean: 11.7396 ± 0.4008 Figure 24: Line plot comparison of model performance metrics for standard data across 30 trials with DEM percentage as label. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 44 of 77 051015202530 Trial 6 7 8 9 10 11 12 RMSE Lasso Regression Mean: 10.1223 ± 0.8445 Decision Tree Mean: 9.3826 ± 0.7603 XGBoost Mean: 7.7027 ± 0.7232 SR4 Mean: 9.5461 ± 0.8489 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 0.925 R² Lasso Regression Mean: 0.7900 ± 0.0332 Decision Tree Mean: 0.8197 ± 0.0275 XGBoost Mean: 0.8783 ± 0.0214 SR4 Mean: 0.8131 ± 0.0315 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.2045 ± 0.2559 Decision Tree Mean: 5.1549 ± 0.2673 XGBoost Mean: 4.2252 ± 0.2281 SR4 Mean: 5.6699 ± 0.2611 051015202530 Trial 12 13 14 15 16 17 RMSE Lasso Regression Mean: 16.4317 ± 0.5442 Decision Tree Mean: 17.0012 ± 0.4858 XGBoost Mean: 12.9099 ± 0.5985 SR4 Mean: 15.9172 ± 0.5248 051015202530 Trial 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 R² Lasso Regression Mean: 0.4495 ± 0.0286 Decision Tree Mean: 0.4106 ± 0.0291 XGBoost Mean: 0.6599 ± 0.0277 SR4 Mean: 0.4835 ± 0.0263 051015202530 Trial 8 9 10 11 12 13 MAE Lasso Regression Mean: 11.9977 ± 0.4001 Decision Tree Mean: 12.5190 ± 0.3985 XGBoost Mean: 8.9721 ± 0.3528 SR4 Mean: 11.7396 ± 0.4008 Figure 25: Line plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics for standard data across 30 trials. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. 051015202530 Trial 7 8 9 10 11 RMSE Random Forest Mean: 8.3080 ± 0.9234 SVM Mean: 9.4252 ± 0.9481 RuleFit Mean: 9.5736 ± 0.9124 SR4 Mean: 9.5018 ± 0.8677 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Random Forest Mean: 0.8580 ± 0.0296 SVM Mean: 0.8175 ± 0.0346 RuleFit Mean: 0.8118 ± 0.0344 SR4 Mean: 0.8148 ± 0.0322 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 MAE Random Forest Mean: 4.4834 ± 0.2596 SVM Mean: 4.8201 ± 0.2978 RuleFit Mean: 5.5623 ± 0.2940 SR4 Mean: 5.7197 ± 0.2476 051015202530 Trial 12 13 14 15 16 RMSE Random Forest Mean: 13.4583 ± 0.5797 SVM Mean: 13.7039 ± 0.5761 RuleFit Mean: 15.1408 ± 0.5358 SR4 Mean: 15.0664 ± 0.5460 051015202530 Trial 0.50 0.55 0.60 0.65 R² Random Forest Mean: 0.6306 ± 0.0257 SVM Mean: 0.6170 ± 0.0261 RuleFit Mean: 0.5325 ± 0.0272 SR4 Mean: 0.5371 ± 0.0271 051015202530 Trial 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 MAE Random Forest Mean: 9.3578 ± 0.3967 SVM Mean: 9.1287 ± 0.3693 RuleFit Mean: 10.7545 ± 0.3676 SR4 Mean: 10.8844 ± 0.4028 Figure 26: Line plot comparison of model performance metrics for expanded data across 30 trials with DEM percentage as label. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 45 of 77 051015202530 Trial 7 8 9 10 11 12 RMSE Lasso Regression Mean: 10.0512 ± 0.8591 Decision Tree Mean: 9.3441 ± 0.8328 XGBoost Mean: 7.8183 ± 0.7098 SR4 Mean: 9.5018 ± 0.8677 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Lasso Regression Mean: 0.7929 ± 0.0337 Decision Tree Mean: 0.8210 ± 0.0301 XGBoost Mean: 0.8746 ± 0.0211 SR4 Mean: 0.8148 ± 0.0322 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.1749 ± 0.2526 Decision Tree Mean: 5.1985 ± 0.2553 XGBoost Mean: 4.1868 ± 0.2487 SR4 Mean: 5.7197 ± 0.2476 051015202530 Trial 12 13 14 15 16 17 18 RMSE Lasso Regression Mean: 15.3528 ± 0.5741 Decision Tree Mean: 16.7925 ± 0.5564 XGBoost Mean: 12.6774 ± 0.5901 SR4 Mean: 15.0664 ± 0.5460 051015202530 Trial 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 R² Lasso Regression Mean: 0.5193 ± 0.0297 Decision Tree Mean: 0.4251 ± 0.0303 XGBoost Mean: 0.6720 ± 0.0274 SR4 Mean: 0.5371 ± 0.0271 051015202530 Trial 8 9 10 11 12 13 MAE Lasso Regression Mean: 10.9097 ± 0.3827 Decision Tree Mean: 12.3420 ± 0.4520 XGBoost Mean: 8.7850 ± 0.3802 SR4 Mean: 10.8844 ± 0.4028 Figure 27: Line plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics for expanded data across 30 trials. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. 051015202530 Trial 7 8 9 10 11 RMSE Random Forest Mean: 8.0384 ± 0.8145 SVM Mean: 9.3657 ± 0.9602 RuleFit Mean: 9.4330 ± 0.8480 SR4 Mean: 9.5722 ± 0.8530 051015202530 Trial 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 R² Random Forest Mean: 0.8672 ± 0.0258 SVM Mean: 0.8197 ± 0.0353 RuleFit Mean: 0.8175 ± 0.0312 SR4 Mean: 0.8121 ± 0.0315 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 MAE Random Forest Mean: 4.3844 ± 0.2179 SVM Mean: 4.7138 ± 0.3063 RuleFit Mean: 5.4944 ± 0.2815 SR4 Mean: 5.4869 ± 0.3006 051015202530 Trial 11 12 13 14 15 16 17 RMSE Random Forest Mean: 11.7369 ± 0.5069 SVM Mean: 12.4142 ± 0.5382 RuleFit Mean: 13.0489 ± 0.8749 SR4 Mean: 12.9741 ± 0.8578 051015202530 Trial 0.45 0.50 0.55 0.60 0.65 0.70 0.75 R² Random Forest Mean: 0.7187 ± 0.0237 SVM Mean: 0.6856 ± 0.0229 RuleFit Mean: 0.6518 ± 0.0472 SR4 Mean: 0.6558 ± 0.0458 051015202530 Trial 7.5 8.0 8.5 9.0 9.5 10.0 10.5 MAE Random Forest Mean: 8.0819 ± 0.3052 SVM Mean: 8.1145 ± 0.3548 RuleFit Mean: 9.1450 ± 0.4411 SR4 Mean: 9.1700 ± 0.4388 Figure 28: Line plot comparison of model performance metrics for previous data across 30 trials with DEM percentage as label. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 46 of 77 051015202530 Trial 6 7 8 9 10 11 12 RMSE Lasso Regression Mean: 9.9834 ± 0.8262 Decision Tree Mean: 9.3854 ± 0.6617 XGBoost Mean: 7.5047 ± 0.7073 SR4 Mean: 9.5722 ± 0.8530 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 0.925 R² Lasso Regression Mean: 0.7958 ± 0.0321 Decision Tree Mean: 0.8198 ± 0.0236 XGBoost Mean: 0.8843 ± 0.0210 SR4 Mean: 0.8121 ± 0.0315 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.1326 ± 0.2512 Decision Tree Mean: 5.1882 ± 0.2976 XGBoost Mean: 4.1533 ± 0.2233 SR4 Mean: 5.4869 ± 0.3006 051015202530 Trial 10 11 12 13 14 15 16 17 RMSE Lasso Regression Mean: 14.4174 ± 0.5705 Decision Tree Mean: 13.3686 ± 0.4301 XGBoost Mean: 11.4409 ± 0.5160 SR4 Mean: 12.9741 ± 0.8578 051015202530 Trial 0.45 0.50 0.55 0.60 0.65 0.70 0.75 R² Lasso Regression Mean: 0.5759 ± 0.0302 Decision Tree Mean: 0.6355 ± 0.0214 XGBoost Mean: 0.7327 ± 0.0234 SR4 Mean: 0.6558 ± 0.0458 051015202530 Trial 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 MAE Lasso Regression Mean: 9.9915 ± 0.3685 Decision Tree Mean: 9.6190 ± 0.3202 XGBoost Mean: 7.8572 ± 0.3211 SR4 Mean: 9.1700 ± 0.4388 Figure 29: Line plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics for previous data across 30 trials. The left panel reports results obtained with party percentage (REP) included, whereas the right panel shows results with party percentage (REP) not included. RMSER²MAE 4 5 6 7 8 9 10 11 12 Score Random Forest SVM RuleFit SR4 (a) Minimum RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (b) Standard RMSER²MAE 4 5 6 7 8 9 10 11 12 Score Random Forest SVM RuleFit SR4 (c) Expanded RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 30: Violin plot comparison of model performance metrics across multiple datasets for the election regression task that has DEM percentage as a label and includes party percentage (REP). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 47 of 77 RMSER²MAE 4 6 8 10 12 Score Lasso Regression Decision Tree XGBoost SR4 (a) Minimum RMSER²MAE 4 6 8 10 12 Score Lasso Regression Decision Tree XGBoost SR4 (b) Standard RMSER²MAE 4 6 8 10 12 Score Lasso Regression Decision Tree XGBoost SR4 (c) Expanded RMSER²MAE 4 5 6 7 8 9 10 11 12 Score Lasso Regression Decision Tree XGBoost SR4 (d) Previous Figure 31: Violin plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics across multiple datasets for the election regression task that has DEM percentage as a label and includes party percentage (REP). Table 13 Average number of rules ± standard deviation across 30 trials for the election regression dataset, where DEM percentage is the label. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 8086.66±199.12 24.00±0.00 20.63±1.43 9.03±1.45 Standard 8162.07±233.69 35.00±0.00 34.13±1.03 12.33±1.03 Expanded 7972.60±224.35 61.00±0.00 52.10±1.09 11.56±1.69 Previous 7856.70±222.97 36.00±0.00 38.83±1.32 11.10±1.21 (a) DEM as a label, with REP percentage. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 8869.63±179.67 67.87±1.82 26.13±2.46 1.00±0.00 Standard 9024.40±202.97 85.70±1.84 46.13±3.35 8.22±2.15 Expanded 8654.03±138.38 59.93±0.25 61.70±2.31 1.00±0.00 Previous 7946.13±164.93 35.00±0.00 32.30±0.65 1.53±0.50 (b) DEM as a label, without REP percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 48 of 77 RMSER²MAE 4 6 8 10 12 14 16 18 Score Random Forest SVM RuleFit SR4 (a) Minimum RMSER²MAE 4 6 8 10 12 14 16 Score Random Forest SVM RuleFit SR4 (b) Standard RMSER²MAE 6 8 10 12 14 16 Score Random Forest SVM RuleFit SR4 (c) Expanded RMSER²MAE 4 6 8 10 12 14 16 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 32: Violin plot comparison of model performance metrics across multiple datasets for the election regression task that has DEM percentage as a label and does not include party percentage (REP). Table 14 Average rule complexity ± standard deviation across 30 trials for the election regression dataset where DEM percentage is the label. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 6.25±0.03 3.00±0.00 2.84±0.01 4.81±0.14 Standard6.17±0.03 1.46±0.00 1.46±0.01 4.80±0.08 Expanded 6.09±0.03 1.78±0.00 1.30±0.01 4.75±0.10 Previous6.11±0.03 1.44±0.00 1.89±0.05 4.73±0.01 (a) DEM as a label, with REP percentage. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 6.36±0.02 5.44±0.04 4.64±0.13 1.01±0.00 Standard6.39±0.02 4.44±0.05 3.28±0.21 4.73±0.16 Expanded 6.34±0.014 1.79±0.01 2.68±0.12 1.2±0.61 Previous6.25±0.03 1.46±0.00 1.39±0.03 2.80±0.61 (b) DEM as a label, without REP percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 49 of 77 RMSER²MAE 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Score Lasso Regression Decision Tree XGBoost SR4 (a) Minimum RMSER²MAE 4 6 8 10 12 14 16 18 Score Lasso Regression Decision Tree XGBoost SR4 (b) Standard RMSER²MAE 4 6 8 10 12 14 16 18 Score Lasso Regression Decision Tree XGBoost SR4 (c) Expanded RMSER²MAE 4 6 8 10 12 14 16 Score Lasso Regression Decision Tree XGBoost SR4 (d) Previous Figure 33: Violin plot comparison of model (LASSO regression, Decision Tree, XG boost) performance metrics across multiple datasets for the election regression task that has DEM percentage as a label and does not include party percentage (REP). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 50 of 77 D. Results of U.S House of Representative Elections Regression (REP) In the minimum dataset, the line plots in Figure 34 and Figure 35 show that when DEM percentage is included as a feature, all models achieve relatively low error values, whereas XGBoost attains slightly lower average error values compared to other models. The distributional comparisons through violin plots in Figure 42 and Figure 43 reinforce these findings. When the DEM percentage is removed, performance declines across all models, with XGBoost still achieving lower average error values compared to other models, which is further confirmed in Figure 44 and Figure 45. Stability analysis in Table 15 shows that in the case of including DEM percentage as a feature, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models, whereas when DEM percentage is removed, RuleFit achieves the highest Dice–Sørensen Index among rule-based models. Table 16 shows that in both cases, the decision tree has the most compact structure, but the structure is too compact, which may lower robustness and may lower the ability to identify interactions properly. Table 17 shows that in the case of including DEM percentage as a feature, the decision tree has the lower average rule complexity, whereas in the case of removing DEM percentage as a feature, the RuleFit has the overall lower average rule complexity. In the standard dataset, the line plots in Figure 36 and Figure 37 show that when DEM percentage is included as a feature, all models achieve lower error values, whereas XGBoost attains slightly lower average error values compared to other models. When the DEM percentage is removed, performance declines across all models, with XGBoost still attaining lower average error values compared to other models. Stability analysis in Table 15 shows that in the case of including DEM percentage as a feature, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models, whereas when DEM percentage is removed, RuleFit achieves the highest Dice–Sørensen Index among rule-based models. Table 16 shows that in both cases, the decision tree has the most compact structure. Table 17 shows that in the case of including DEM percentage as a feature, the decision tree has the lower average rule complexity, whereas in the case of removing DEM percentage as a feature, the RuleFit has the overall lower average rule complexity. In the expanded dataset, the line plots in Figure 38 and Figure 39 show that when DEM percentage is included as a feature, all models achieve lower error values, whereas XGBoost again attains slightly lower average error values compared to other models. When the DEM percentage is removed, performance declines across all models, with XGBoost still attaining lower average error values compared to other models. Stability analysis in Table 15 shows that with both cases, SR4-Fit achieves the highest Dice–Sørensen Index among rule-based models. Table 16 shows that in both cases, the decision tree has the most compact structure. Table 14 shows that in both cases, the decision tree has lower average rule complexity compared to all models. In the previous dataset, the line plots in Figure 40 and Figure 41 show that when DEM percentage is included as a feature, all models achieve lower error values, whereas XGBoost attains lower average error values compared to other models. When the DEM percentage is removed, performance declines across all models, with random forest slightly achieving lower average error values compared to other models. Stability analysis in Table 15 shows that with both cases, RuleFit achieves the highest Dice–Sørensen Index among rule-based models. Table 16 shows that in both cases, the decision tree has the most compact structure. Table 17 shows that in the case of including DEM percentage as a feature, the decision tree has the lower average rule complexity, whereas in the case of removing DEM percentage as a feature, the RuleFit has the overall lower average rule complexity. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 51 of 77 051015202530 Trial 8 9 10 11 RMSE Random Forest Mean: 8.5898 ± 0.7239 SVM Mean: 9.2827 ± 0.7159 RuleFit Mean: 9.9005 ± 0.7747 SR4 Mean: 9.9116 ± 0.7464 051015202530 Trial 0.76 0.78 0.80 0.82 0.84 0.86 0.88 R² Random Forest Mean: 0.8571 ± 0.0223 SVM Mean: 0.8331 ± 0.0249 RuleFit Mean: 0.8102 ± 0.0281 SR4 Mean: 0.8098 ± 0.0272 051015202530 Trial 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 6.25 MAE Random Forest Mean: 4.7623 ± 0.2395 SVM Mean: 4.8655 ± 0.2597 RuleFit Mean: 5.9492 ± 0.2560 SR4 Mean: 5.9229 ± 0.2357 051015202530 Trial 13 14 15 16 17 18 19 RMSE Random Forest Mean: 14.1145 ± 0.5209 SVM Mean: 16.4391 ± 0.5051 RuleFit Mean: 18.0222 ± 0.5030 SR4 Mean: 18.0404 ± 0.4769 051015202530 Trial 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 R² Random Forest Mean: 0.6154 ± 0.0287 SVM Mean: 0.4784 ± 0.0347 RuleFit Mean: 0.3733 ± 0.0368 SR4 Mean: 0.3722 ± 0.0336 051015202530 Trial 10 11 12 13 14 MAE Random Forest Mean: 10.1228 ± 0.3619 SVM Mean: 11.9482 ± 0.3857 RuleFit Mean: 13.8387 ± 0.3745 SR4 Mean: 13.8467 ± 0.3258 Figure 34: Line plot comparison of model performance metrics for minimum data across 30 trials with REP percentage as label. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. Table 15 Average Dice–Sørensen Index ± standard deviation across 30 trials for the election regression dataset where REP percentage is the label. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0001±0.0000 0.3333±0.0000 0.4874±0.0155 0.0092±0.0155 Standard 0.0001±0.0000 0.6279±0.0000 0.6873±0.0078 0.0092±0.0155 Expanded 0.0001±0.0000 0.7377±0.0000 0.7806±0.0075 0.0069±0.0140 Previous 0.0001±0.0000 0.7778±0.0000 0.7132±0.0068 0.0092±0.0155 (a) REP as a label, with DEM percentage. Dataset Random ForestRuleFitSR4-FitDecision Tree Minimum 0.0004±0.0001 0.3048±0.0012 0.2666±0.0121 0.0074±0.0084 Standard 0.0001±0.0000 0.7647±0.0000 0.7584±0.0021 0.0100±0.0110 Expanded 0.0001±0.0000 0.7333±0.0000 0.7452±0.0077 0.4184±0.2733 Previous 0.0015±0.0005 0.7714±0.0000 0.6814±0.0078 0.1356±0.1127 (b) REP as a label, without DEM percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 52 of 77 051015202530 Trial 7 8 9 10 11 12 RMSE Lasso Regression Mean: 10.4081 ± 0.8032 Decision Tree Mean: 10.2168 ± 0.7210 XGBoost Mean: 7.8917 ± 0.6825 SR4 Mean: 9.9116 ± 0.7464 051015202530 Trial 0.725 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Lasso Regression Mean: 0.7901 ± 0.0313 Decision Tree Mean: 0.7979 ± 0.0279 XGBoost Mean: 0.8794 ± 0.0189 SR4 Mean: 0.8098 ± 0.0272 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 7.0 MAE Lasso Regression Mean: 6.5571 ± 0.2233 Decision Tree Mean: 6.2419 ± 0.2767 XGBoost Mean: 4.1754 ± 0.2048 SR4 Mean: 5.9229 ± 0.2357 051015202530 Trial 12 13 14 15 16 17 18 19 20 RMSE Lasso Regression Mean: 18.8152 ± 0.5153 Decision Tree Mean: 17.7415 ± 0.5515 XGBoost Mean: 13.0335 ± 0.4958 SR4 Mean: 18.0404 ± 0.4769 051015202530 Trial 0.2 0.3 0.4 0.5 0.6 0.7 R² Lasso Regression Mean: 0.3170 ± 0.0391 Decision Tree Mean: 0.3926 ± 0.0393 XGBoost Mean: 0.6720 ± 0.0254 SR4 Mean: 0.3722 ± 0.0336 051015202530 Trial 9 10 11 12 13 14 15 MAE Lasso Regression Mean: 14.4313 ± 0.3777 Decision Tree Mean: 13.4765 ± 0.3786 XGBoost Mean: 9.2549 ± 0.3301 SR4 Mean: 13.8467 ± 0.3258 Figure 35: Line plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics for minimum data across 30 trials. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. 051015202530 Trial 7 8 9 10 11 RMSE Random Forest Mean: 8.4458 ± 0.7582 SVM Mean: 8.8359 ± 0.7085 RuleFit Mean: 9.5526 ± 0.7107 SR4 Mean: 9.6165 ± 0.6922 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 R² Random Forest Mean: 0.8617 ± 0.0233 SVM Mean: 0.8486 ± 0.0240 RuleFit Mean: 0.8234 ± 0.0249 SR4 Mean: 0.8210 ± 0.0243 051015202530 Trial 4.5 5.0 5.5 6.0 MAE Random Forest Mean: 4.7334 ± 0.2374 SVM Mean: 4.8081 ± 0.2711 RuleFit Mean: 5.7967 ± 0.2440 SR4 Mean: 5.7351 ± 0.2393 051015202530 Trial 13 14 15 16 RMSE Random Forest Mean: 13.4809 ± 0.5138 SVM Mean: 13.5194 ± 0.5138 RuleFit Mean: 15.6761 ± 0.4845 SR4 Mean: 15.5834 ± 0.4826 051015202530 Trial 0.50 0.55 0.60 0.65 0.70 R² Random Forest Mean: 0.6491 ± 0.0271 SVM Mean: 0.6472 ± 0.0258 RuleFit Mean: 0.5258 ± 0.0307 SR4 Mean: 0.5313 ± 0.0308 051015202530 Trial 9.0 9.5 10.0 10.5 11.0 11.5 12.0 MAE Random Forest Mean: 9.4548 ± 0.3322 SVM Mean: 9.3271 ± 0.3023 RuleFit Mean: 11.6635 ± 0.3133 SR4 Mean: 11.6722 ± 0.3080 Figure 36: Line plot comparison of model performance metrics for standard data across 30 trials with REP percentage as label. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 53 of 77 051015202530 Trial 7 8 9 10 11 RMSE Lasso Regression Mean: 10.0555 ± 0.7654 Decision Tree Mean: 10.3269 ± 0.7080 XGBoost Mean: 7.7687 ± 0.6924 SR4 Mean: 9.6165 ± 0.6922 051015202530 Trial 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Lasso Regression Mean: 0.8041 ± 0.0288 Decision Tree Mean: 0.7937 ± 0.0264 XGBoost Mean: 0.8831 ± 0.0191 SR4 Mean: 0.8210 ± 0.0243 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 7.0 MAE Lasso Regression Mean: 6.3981 ± 0.2342 Decision Tree Mean: 6.5083 ± 0.2257 XGBoost Mean: 4.2088 ± 0.1976 SR4 Mean: 5.7351 ± 0.2393 051015202530 Trial 11 12 13 14 15 16 17 18 RMSE Lasso Regression Mean: 16.2891 ± 0.4961 Decision Tree Mean: 16.8056 ± 0.5262 XGBoost Mean: 12.5381 ± 0.5465 SR4 Mean: 15.5834 ± 0.4826 051015202530 Trial 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 R² Lasso Regression Mean: 0.4881 ± 0.0310 Decision Tree Mean: 0.4547 ± 0.0388 XGBoost Mean: 0.6965 ± 0.0247 SR4 Mean: 0.5313 ± 0.0308 051015202530 Trial 8 9 10 11 12 13 MAE Lasso Regression Mean: 12.1191 ± 0.3500 Decision Tree Mean: 12.4485 ± 0.3536 XGBoost Mean: 8.7036 ± 0.3178 SR4 Mean: 11.6722 ± 0.3080 Figure 37: Line plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics for standard data across 30 trials. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. 051015202530 Trial 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 RMSE Random Forest Mean: 8.3067 ± 0.5816 SVM Mean: 8.7953 ± 0.7084 RuleFit Mean: 9.2576 ± 0.7257 SR4 Mean: 9.2185 ± 0.6978 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 R² Random Forest Mean: 0.8666 ± 0.0171 SVM Mean: 0.8499 ± 0.0240 RuleFit Mean: 0.8340 ± 0.0251 SR4 Mean: 0.8355 ± 0.0237 051015202530 Trial 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 6.25 MAE Random Forest Mean: 4.7268 ± 0.2278 SVM Mean: 4.8451 ± 0.2881 RuleFit Mean: 5.6384 ± 0.2522 SR4 Mean: 5.5496 ± 0.2450 051015202530 Trial 12.0 12.5 13.0 13.5 14.0 14.5 15.0 15.5 RMSE Random Forest Mean: 13.2721 ± 0.5580 SVM Mean: 13.0996 ± 0.5403 RuleFit Mean: 14.4319 ± 0.5689 SR4 Mean: 14.3301 ± 0.5681 051015202530 Trial 0.55 0.60 0.65 0.70 R² Random Forest Mean: 0.6599 ± 0.0272 SVM Mean: 0.6687 ± 0.0267 RuleFit Mean: 0.5978 ± 0.0325 SR4 Mean: 0.6035 ± 0.0317 051015202530 Trial 8.5 9.0 9.5 10.0 10.5 11.0 MAE Random Forest Mean: 9.3160 ± 0.3378 SVM Mean: 8.8793 ± 0.3147 RuleFit Mean: 10.2667 ± 0.3503 SR4 Mean: 10.2648 ± 0.3507 Figure 38: Line plot comparison of model performance metrics for expanded data across 30 trials with REP percentage as label. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 54 of 77 051015202530 Trial 7 8 9 10 11 RMSE Lasso Regression Mean: 9.5504 ± 0.7292 Decision Tree Mean: 9.9734 ± 0.5736 XGBoost Mean: 7.8035 ± 0.6049 SR4 Mean: 9.2185 ± 0.6978 051015202530 Trial 0.750 0.775 0.800 0.825 0.850 0.875 0.900 R² Lasso Regression Mean: 0.8233 ± 0.0260 Decision Tree Mean: 0.8078 ± 0.0210 XGBoost Mean: 0.8821 ± 0.0172 SR4 Mean: 0.8355 ± 0.0237 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.0966 ± 0.2512 Decision Tree Mean: 6.1297 ± 0.2158 XGBoost Mean: 4.3701 ± 0.2616 SR4 Mean: 5.5496 ± 0.2450 051015202530 Trial 11 12 13 14 15 16 17 RMSE Lasso Regression Mean: 14.5614 ± 0.5422 Decision Tree Mean: 16.4664 ± 0.4989 XGBoost Mean: 12.6282 ± 0.5542 SR4 Mean: 14.3301 ± 0.5681 051015202530 Trial 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 R² Lasso Regression Mean: 0.5907 ± 0.0308 Decision Tree Mean: 0.4766 ± 0.0358 XGBoost Mean: 0.6921 ± 0.0253 SR4 Mean: 0.6035 ± 0.0317 051015202530 Trial 8 9 10 11 12 13 MAE Lasso Regression Mean: 10.3801 ± 0.3434 Decision Tree Mean: 12.2317 ± 0.3620 XGBoost Mean: 8.7962 ± 0.3627 SR4 Mean: 10.2648 ± 0.3507 Figure 39: Line plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics for expanded data across 30 trials. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. 051015202530 Trial 7 8 9 10 11 RMSE Random Forest Mean: 7.9491 ± 0.6227 SVM Mean: 8.7498 ± 0.6521 RuleFit Mean: 9.5490 ± 0.6797 SR4 Mean: 9.5061 ± 0.6576 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 R² Random Forest Mean: 0.8777 ± 0.0179 SVM Mean: 0.8517 ± 0.0217 RuleFit Mean: 0.8236 ± 0.0237 SR4 Mean: 0.8252 ± 0.0225 051015202530 Trial 4.5 5.0 5.5 6.0 MAE Random Forest Mean: 4.6023 ± 0.2296 SVM Mean: 4.7979 ± 0.2593 RuleFit Mean: 5.9372 ± 0.2478 SR4 Mean: 5.8877 ± 0.2600 051015202530 Trial 10 11 12 13 14 15 16 17 18 RMSE Random Forest Mean: 11.2336 ± 0.4595 SVM Mean: 11.8757 ± 0.5228 RuleFit Mean: 12.6140 ± 0.9617 SR4 Mean: 12.5449 ± 1.0605 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 R² Random Forest Mean: 0.7563 ± 0.0204 SVM Mean: 0.7277 ± 0.0229 RuleFit Mean: 0.6914 ± 0.0531 SR4 Mean: 0.6944 ± 0.0603 051015202530 Trial 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 MAE Random Forest Mean: 7.7306 ± 0.2928 SVM Mean: 7.8896 ± 0.3168 RuleFit Mean: 8.9164 ± 0.4272 SR4 Mean: 8.9041 ± 0.4411 Figure 40: Line plot comparison of model performance metrics for previous data across 30 trials with REP percentage as label. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 55 of 77 051015202530 Trial 7 8 9 10 11 12 RMSE Lasso Regression Mean: 9.8362 ± 0.6856 Decision Tree Mean: 9.8943 ± 0.7661 XGBoost Mean: 7.2301 ± 0.5179 SR4 Mean: 9.5061 ± 0.6576 051015202530 Trial 0.750 0.775 0.800 0.825 0.850 0.875 0.900 0.925 R² Lasso Regression Mean: 0.8128 ± 0.0250 Decision Tree Mean: 0.8105 ± 0.0275 XGBoost Mean: 0.8989 ± 0.0131 SR4 Mean: 0.8252 ± 0.0225 051015202530 Trial 4.0 4.5 5.0 5.5 6.0 6.5 MAE Lasso Regression Mean: 6.3381 ± 0.2277 Decision Tree Mean: 6.1408 ± 0.2829 XGBoost Mean: 4.0609 ± 0.1872 SR4 Mean: 5.8877 ± 0.2600 051015202530 Trial 10 11 12 13 14 15 16 17 18 RMSE Lasso Regression Mean: 14.1574 ± 0.6363 Decision Tree Mean: 12.9120 ± 0.4563 XGBoost Mean: 11.2905 ± 0.4525 SR4 Mean: 12.5449 ± 1.0605 051015202530 Trial 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 R² Lasso Regression Mean: 0.6129 ± 0.0340 Decision Tree Mean: 0.6782 ± 0.0235 XGBoost Mean: 0.7539 ± 0.0194 SR4 Mean: 0.6944 ± 0.0603 051015202530 Trial 7.5 8.0 8.5 9.0 9.5 10.0 10.5 MAE Lasso Regression Mean: 9.9938 ± 0.3653 Decision Tree Mean: 9.3827 ± 0.3155 XGBoost Mean: 7.9159 ± 0.2565 SR4 Mean: 8.9041 ± 0.4411 Figure 41: Line plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics for previous data across 30 trials. The left panel reports results obtained with party percentage (DEM) included, whereas the right panel shows results with party percentage (DEM) not included. RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (a) Minimum RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (b) Standard RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (c) Expanded RMSER²MAE 4 5 6 7 8 9 10 11 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 42: Violin plot comparison of model performance metrics across multiple datasets for the election regression task that has REP percentage as a label and includes party percentage (DEM). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 56 of 77 RMSER²MAE 4 6 8 10 12 Score Lasso Regression Decision Tree XGBoost SR4 (a) Minimum RMSER²MAE 4 5 6 7 8 9 10 11 12 Score Lasso Regression Decision Tree XGBoost SR4 (b) Standard RMSER²MAE 4 5 6 7 8 9 10 11 Score Lasso Regression Decision Tree XGBoost SR4 (c) Expanded RMSER²MAE 4 6 8 10 12 Score Lasso Regression Decision Tree XGBoost SR4 (d) Previous Figure 43: Violin plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics across multiple datasets for the election regression task that has REP percentage as a label and includes party percentage (DEM). Table 16 Average number of rules ± standard deviation across 30 trials for the election regression dataset, where REP percentage is the label. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 8161.43±200.06 24.00±0.00 15.86±0.34 1.00±0.00 Standard 8225.93±225.83 43.00±0.00 39.00±0.52 1.00±0.00 Expanded 8457.63±232.08 61.00±0.00 56.96±1.12 1.00±0.00 Previous 8205.06±199.47 36.00±0.00 39.26±0.78 1.00±0.00 (a) REP as a label, with DEM percentage. Dataset Random Forest RuleFitSR4-Fit Decision Tree Minimum 8797.16±179.66 22.96±0.18 26.36±2.41 10.50±2.28 Standard 9513.73±133.31 34.00±0.00 32.96±0.18 10.13±1.45 Expanded 9256.36±162.12 60.00±0.00 55.03±1.15 1.00±0.00 Previous 8044.26±152.79 35.00±0.00 38.16±0.91 1.60±0.49 (b) REP as a label, without DEM percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 57 of 77 RMSER²MAE 4 6 8 10 12 14 16 18 Score Random Forest SVM RuleFit SR4 (a) Minimum RMSER²MAE 6 8 10 12 14 16 Score Random Forest SVM RuleFit SR4 (b) Standard RMSER²MAE 6 8 10 12 14 16 Score Random Forest SVM RuleFit SR4 (c) Expanded RMSER²MAE 4 6 8 10 12 14 16 18 Score Random Forest SVM RuleFit SR4 (d) Previous Figure 44: Violin plot comparison of model performance metrics across multiple datasets for the election regression task that has REP percentage as a label and does not include party percentage (DEM). Table 17 Average rule complexity ± standard deviation across 30 trials for the election regression dataset, where REP percentage is the label. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 6.23±0.03 3.00±0.00 2.01±0.02 1.00±0.00 Standard6.17±0.03 2.11±0.00 1.93±0.03 1.00±0.00 Expanded 6.18±0.03 1.78±0.00 1.64±0.03 1.20±0.61 Previous6.16±0.03 1.44±0.00 1.86±0.04 1.00±0.00 (a) REP as a label, with DEM percentage. Dataset Random Forest RuleFit SR4-Fit Decision Tree Minimum 6.32±0.02 3.08±0.02 4.66±0.13 4.59±0.17 Standard6.42±0.01 1.47±0.00 1.48±0.01 4.73±0.13 Expanded 6.39±0.03 1.80±0.00 1.76±0.05 1.20±0.61 Previous6.30±0.02 1.45±0.00 1.95±0.05 2.86±0.50 (b) REP as a label, without DEM percentage. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 58 of 77 RMSER²MAE 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Score Lasso Regression Decision Tree XGBoost SR4 (a) Minimum RMSER²MAE 4 6 8 10 12 14 16 18 Score Lasso Regression Decision Tree XGBoost SR4 (b) Standard RMSER²MAE 4 6 8 10 12 14 16 Score Lasso Regression Decision Tree XGBoost SR4 (c) Expanded RMSER²MAE 4 6 8 10 12 14 16 18 Score Lasso Regression Decision Tree XGBoost SR4 (d) Previous Figure 45: Violin plot comparison of model (LASSO regression, decision tree, XG boost) performance metrics across multiple datasets for the election regression task that has REP percentage as a label and does not include party percentage (DEM). S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 59 of 77 E. Results on Standard Public Datasets Classification Across all six classification datasets and 30 trials, SR4-Fit demonstrates a strong balance of predictive performance, rule stability, and low variance, making it the most robust and interpretable method overall. Although individual models occasionally outperform it on accuracy, SR4-Fit repeatedly delivers the most consistent and stable behavior across trials, as reflected in the line plots and violin plots. In the Breast Cancer dataset, XGBoost has the highest performance with respect to prediction metrics, but SR4- Fit trails very closely behind XGBoost, as seen in Figure 46. The violin plots for breast cancer in Figure 52 and Figure 53 reinforce these findings. Despite this small difference in accuracy, SR4-Fit significantly outperforms in stability, achieving the highest Dice–Sørensen Index, along with RuleFit, as seen in Table 18. Table 19 shows that the decision tree has a compact structure, whereas with respect to average rule complexity in Table 20 random forest has lower average rule complexity compared to all rule-based models. In the E. coli dataset, SVM has the highest performance with respect to prediction metrics, as seen in Figure 47. The violin plots for ecoli in Figure 52 and Figure 53 reinforce these findings. SR4-Fit and RuleFit significantly outperform in stability, achieving the highest Dice–Sørensen Index, as seen in Table 18. Table 19 shows that the decision tree has a compact structure, whereas with respect to average rule complexity in Table 20 random forest has lower average rule complexity compared to all rule-based models. On the Page blocks dataset, random forest has the highest performance with respect to prediction metrics, but SR4- Fit trails very closely behind random forest, as seen in Figure 48. The violin plots for page blocks in Figure 52 and Figure 53 show the distribution of the prediction metrics across trials. SR4-Fit and RuleFit continue to significantly outperform in stability, achieving the highest Dice–Sørensen Index, as seen in Table 18. Table 19 shows that the decision tree has a compact structure, whereas with respect to average rule complexity in Table 20 random forest has lower average rule complexity compared to all rule-based models. On the Pima Indians dataset, random forest continues to have the highest performance with respect to prediction metrics, but SR4-Fit trails very closely behind random forest, as seen in Figure 49. The violin plots for Pima Indians in Figure 52 and Figure 53 show the distribution of the prediction metrics across trials. SR4-Fit and RuleFit continue to significantly outperform in stability, achieving the highest Dice–Sørensen Index, as seen in Table 18. Table 19 shows that the SR4-Fit and RuleFit have a compact structure, whereas, with respect to average rule complexity in Table 20 random forests continue to have lower average rule complexity compared to all rule-based models. In the vehicle dataset, SVM has the highest performance with respect to prediction metrics, but SR4-Fit trails very closely behind SVM, as seen in Figure 50. The violin plots for the vehicle in Figure 52 and Figure 53 show the distribution of the prediction metrics across trials. SR4-Fit and RuleFit continue to significantly outperform in stability, achieving the highest Dice–Sørensen Index, as seen in Table 18. Table 19 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 20 random forests continue to have lower average rule complexity compared to all rule-based models. Finally, in the vehicle dataset, random forest has the highest performance with respect to prediction metrics, as seen in Figure 51. The violin plots for the yeast in Figure 52 and Figure 53 show the distribution of the prediction metrics across trials. SR4-Fit and RuleFit continue to significantly outperform in stability, achieving the highest Dice–Sørensen Index, as seen in Table 18. Table 19 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 20 random forests continue to have lower average rule complexity compared to all rule-based models. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 60 of 77 051015202530 Trial 0.950 0.955 0.960 0.965 0.970 Accuracy Random Forest Mean: 0.9659 ± 0.0034 SVM Mean: 0.9584 ± 0.0033 RuleFit Mean: 0.9618 ± 0.0062 SR4 Mean: 0.9630 ± 0.0062 051015202530 Trial 0.919 0.920 0.921 0.922 0.923 Precision Random Forest Mean: 0.9221 ± 0.0007 SVM Mean: 0.9205 ± 0.0007 RuleFit Mean: 0.9212 ± 0.0013 SR4 Mean: 0.9214 ± 0.0013 051015202530 Trial 0.94 0.95 0.96 0.97 0.98 0.99 1.00 Recall Random Forest Mean: 0.9861 ± 0.0098 SVM Mean: 0.9646 ± 0.0095 RuleFit Mean: 0.9743 ± 0.0176 SR4 Mean: 0.9778 ± 0.0178 051015202530 Trial 0.930 0.935 0.940 0.945 0.950 0.955 0.960 F1 Random Forest Mean: 0.9530 ± 0.0050 SVM Mean: 0.9420 ± 0.0049 RuleFit Mean: 0.9469 ± 0.0090 SR4 Mean: 0.9487 ± 0.0091 051015202530 Trial 0.91 0.92 0.93 0.94 0.95 0.96 0.97 Accuracy Logistic Regression Mean: 0.9603 ± 0.0049 Decision Tree Mean: 0.9426 ± 0.0157 XGBoost Mean: 0.9664 ± 0.0036 SR4 Mean: 0.9630 ± 0.0062 051015202530 Trial 0.88 0.90 0.92 0.94 Precision Logistic Regression Mean: 0.9209 ± 0.0011 Decision Tree Mean: 0.9139 ± 0.0181 XGBoost Mean: 0.9222 ± 0.0007 SR4 Mean: 0.9214 ± 0.0013 051015202530 Trial 0.825 0.850 0.875 0.900 0.925 0.950 0.975 1.000 Recall Logistic Regression Mean: 0.9701 ± 0.0139 Decision Tree Mean: 0.9236 ± 0.0470 XGBoost Mean: 0.9875 ± 0.0102 SR4 Mean: 0.9778 ± 0.0178 051015202530 Trial 0.86 0.88 0.90 0.92 0.94 0.96 F1 Logistic Regression Mean: 0.9448 ± 0.0072 Decision Tree Mean: 0.9180 ± 0.0244 XGBoost Mean: 0.9537 ± 0.0051 SR4 Mean: 0.9487 ± 0.0091 Figure 46: Line plot comparison of model performance metrics for breast cancer data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. Table 18 Average Dice–Sørensen Index ± standard deviation across 30 trials for public classification datasets. Bold indicates the best-performing method per dataset. DatasetRandom ForestRuleFitSR4-FitDecision Tree Breast Cancer 0.3307±0.0186 0.4449±0.0199 0.4449±0.0199 0.0558±0.0540 E. coli0.1160±0.0150 0.1762±0.0140 0.1762±0.0140 0.0414±0.0347 Page Blocks 0.1090±0.0155 0.3232±0.0294 0.3232±0.0294 0.0135±0.0090 Pima Indians 0.0454±0.0078 0.5153±0.0115 0.5153±0.0115 0.0021±0.0032 Vehicle0.1993±0.0146 0.5231±0.0272 0.5231±0.0272 0.0593±0.0360 Yeast0.1182±0.0146 0.1807±0.0182 0.1807±0.0182 0.0121±0.0098 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 61 of 77 051015202530 Trial 0.82 0.84 0.86 0.88 0.90 Accuracy Random Forest Mean: 0.8765 ± 0.0188 SVM Mean: 0.9025 ± 0.0089 RuleFit Mean: 0.8623 ± 0.0184 SR4 Mean: 0.8510 ± 0.0196 051015202530 Trial 0.84 0.86 0.88 0.90 0.92 Precision Random Forest Mean: 0.9019 ± 0.0189 SVM Mean: 0.9227 ± 0.0097 RuleFit Mean: 0.8759 ± 0.0197 SR4 Mean: 0.8638 ± 0.0217 051015202530 Trial 0.82 0.84 0.86 0.88 0.90 Recall Random Forest Mean: 0.8765 ± 0.0188 SVM Mean: 0.9025 ± 0.0089 RuleFit Mean: 0.8623 ± 0.0184 SR4 Mean: 0.8510 ± 0.0196 051015202530 Trial 0.82 0.84 0.86 0.88 0.90 0.92 F1 Random Forest Mean: 0.8789 ± 0.0194 SVM Mean: 0.9047 ± 0.0099 RuleFit Mean: 0.8585 ± 0.0182 SR4 Mean: 0.8472 ± 0.0188 051015202530 Trial 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 Accuracy Logistic Regression Mean: 0.9000 ± 0.0059 Decision Tree Mean: 0.7838 ± 0.0138 XGBoost Mean: 0.8623 ± 0.0216 SR4 Mean: 0.8510 ± 0.0196 051015202530 Trial 0.775 0.800 0.825 0.850 0.875 0.900 0.925 0.950 Precision Logistic Regression Mean: 0.9246 ± 0.0093 Decision Tree Mean: 0.8395 ± 0.0241 XGBoost Mean: 0.8874 ± 0.0217 SR4 Mean: 0.8638 ± 0.0217 051015202530 Trial 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 Recall Logistic Regression Mean: 0.9000 ± 0.0059 Decision Tree Mean: 0.7838 ± 0.0138 XGBoost Mean: 0.8623 ± 0.0216 SR4 Mean: 0.8510 ± 0.0196 051015202530 Trial 0.750 0.775 0.800 0.825 0.850 0.875 0.900 F1 Logistic Regression Mean: 0.9019 ± 0.0067 Decision Tree Mean: 0.7881 ± 0.0204 XGBoost Mean: 0.8651 ± 0.0218 SR4 Mean: 0.8472 ± 0.0188 Figure 47: Line plot comparison of model performance metrics for E. coli data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. Table 19 Average number of rules ± standard deviation across 30 trials for public classification datasets. Lower values indicate more compact models. DatasetRandom Forest RuleFitSR4-Fit Decision Tree Breast Cancer 104.43±11.58 21.13±1.11 21.13±1.11 9.10±0.66 E. coli117.17±13.48 57.00±0.00 57.00±0.00 11.03±0.67 Page Blocks50.27±8.77 43.10±2.15 43.10±2.15 18.23±0.89 Pima Indians 37.47±8.19 15.67±0.48 15.67±0.48 20.77±1.96 Vehicle147.37±11.09 40.07±0.78 40.07±0.78 18.57±1.33 Yeast57.00±8.87 57.00±0.00 57.00±0.00 24.07±1.41 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 62 of 77 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 Accuracy Random Forest Mean: 0.9731 ± 0.0017 SVM Mean: 0.9633 ± 0.0027 RuleFit Mean: 0.9675 ± 0.0026 SR4 Mean: 0.9696 ± 0.0030 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 Precision Random Forest Mean: 0.9727 ± 0.0019 SVM Mean: 0.9623 ± 0.0029 RuleFit Mean: 0.9675 ± 0.0029 SR4 Mean: 0.9699 ± 0.0033 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 Recall Random Forest Mean: 0.9731 ± 0.0017 SVM Mean: 0.9633 ± 0.0027 RuleFit Mean: 0.9675 ± 0.0026 SR4 Mean: 0.9696 ± 0.0030 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 F1 Random Forest Mean: 0.9723 ± 0.0018 SVM Mean: 0.9619 ± 0.0028 RuleFit Mean: 0.9670 ± 0.0028 SR4 Mean: 0.9693 ± 0.0033 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 0.9775 Accuracy Logistic Regression Mean: 0.9644 ± 0.0017 Decision Tree Mean: 0.9681 ± 0.0037 XGBoost Mean: 0.9728 ± 0.0020 SR4 Mean: 0.9696 ± 0.0030 051015202530 Trial 0.960 0.965 0.970 0.975 Precision Logistic Regression Mean: 0.9633 ± 0.0017 Decision Tree Mean: 0.9675 ± 0.0039 XGBoost Mean: 0.9724 ± 0.0021 SR4 Mean: 0.9699 ± 0.0033 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 0.9775 Recall Logistic Regression Mean: 0.9644 ± 0.0017 Decision Tree Mean: 0.9681 ± 0.0037 XGBoost Mean: 0.9728 ± 0.0020 SR4 Mean: 0.9696 ± 0.0030 051015202530 Trial 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 0.9775 F1 Logistic Regression Mean: 0.9636 ± 0.0016 Decision Tree Mean: 0.9669 ± 0.0038 XGBoost Mean: 0.9724 ± 0.0021 SR4 Mean: 0.9693 ± 0.0033 Figure 48: Line plot comparison of model performance metrics for page blocks data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. Table 20 Average rule complexity (number of conditions per rule) ± standard deviation across 30 trials for public classification datasets. Bold indicates less complex rules. DatasetRandom Forest RuleFit SR4-Fit Decision Tree Breast Cancer 1.99±0.01 2.97±0.12 2.97±0.12 3.42±0.15 E. coli1.97±0.01 3.55±0.01 3.55±0.01 3.60±0.08 Page Blocks1.97±0.01 2.47±0.04 2.47±0.04 4.41±0.05 Pima Indians1.96±0.01 2.28±0.07 2.28±0.07 4.61±0.08 Vehicle1.99±0.01 2.05±0.04 2.05±0.04 4.52±0.09 Yeast1.96±0.02 2.66±0.03 2.66±0.03 4.69±0.06 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 63 of 77 051015202530 Trial 0.70 0.71 0.72 0.73 0.74 0.75 0.76 0.77 Accuracy Random Forest Mean: 0.7506 ± 0.0147 SVM Mean: 0.7182 ± 0.0100 RuleFit Mean: 0.7338 ± 0.0110 SR4 Mean: 0.7416 ± 0.0159 051015202530 Trial 0.58 0.60 0.62 0.64 0.66 0.68 0.70 Precision Random Forest Mean: 0.6626 ± 0.0245 SVM Mean: 0.6195 ± 0.0201 RuleFit Mean: 0.6427 ± 0.0196 SR4 Mean: 0.6480 ± 0.0289 051015202530 Trial 0.45 0.50 0.55 0.60 0.65 0.70 Recall Random Forest Mean: 0.5895 ± 0.0325 SVM Mean: 0.5105 ± 0.0149 RuleFit Mean: 0.5438 ± 0.0366 SR4 Mean: 0.5802 ± 0.0467 051015202530 Trial 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 F1 Random Forest Mean: 0.6235 ± 0.0250 SVM Mean: 0.5595 ± 0.0135 RuleFit Mean: 0.5884 ± 0.0232 SR4 Mean: 0.6109 ± 0.0276 051015202530 Trial 0.68 0.70 0.72 0.74 0.76 0.78 0.80 Accuracy Logistic Regression Mean: 0.7240 ± 0.0091 Decision Tree Mean: 0.7281 ± 0.0314 XGBoost Mean: 0.7403 ± 0.0099 SR4 Mean: 0.7416 ± 0.0159 051015202530 Trial 0.55 0.60 0.65 0.70 0.75 Precision Logistic Regression Mean: 0.6246 ± 0.0165 Decision Tree Mean: 0.6387 ± 0.0537 XGBoost Mean: 0.6734 ± 0.0250 SR4 Mean: 0.6480 ± 0.0289 051015202530 Trial 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Recall Logistic Regression Mean: 0.5346 ± 0.0201 Decision Tree Mean: 0.5198 ± 0.0894 XGBoost Mean: 0.5068 ± 0.0393 SR4 Mean: 0.5802 ± 0.0467 051015202530 Trial 0.40 0.45 0.50 0.55 0.60 0.65 0.70 F1 Logistic Regression Mean: 0.5759 ± 0.0152 Decision Tree Mean: 0.5694 ± 0.0667 XGBoost Mean: 0.5770 ± 0.0243 SR4 Mean: 0.6109 ± 0.0276 Figure 49: Line plot comparison of model performance metrics for Pima Indians data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 64 of 77 051015202530 Trial 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 Accuracy Random Forest Mean: 0.9551 ± 0.0060 SVM Mean: 0.9786 ± 0.0051 RuleFit Mean: 0.9386 ± 0.0113 SR4 Mean: 0.9520 ± 0.0106 051015202530 Trial 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 Precision Random Forest Mean: 0.9560 ± 0.0058 SVM Mean: 0.9789 ± 0.0050 RuleFit Mean: 0.9398 ± 0.0112 SR4 Mean: 0.9525 ± 0.0105 051015202530 Trial 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 Recall Random Forest Mean: 0.9551 ± 0.0060 SVM Mean: 0.9786 ± 0.0051 RuleFit Mean: 0.9386 ± 0.0113 SR4 Mean: 0.9520 ± 0.0106 051015202530 Trial 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 F1 Random Forest Mean: 0.9551 ± 0.0060 SVM Mean: 0.9786 ± 0.0052 RuleFit Mean: 0.9386 ± 0.0113 SR4 Mean: 0.9520 ± 0.0106 051015202530 Trial 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 Accuracy Logistic Regression Mean: 0.9447 ± 0.0066 Decision Tree Mean: 0.8655 ± 0.0242 XGBoost Mean: 0.9633 ± 0.0045 SR4 Mean: 0.9520 ± 0.0106 051015202530 Trial 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 Precision Logistic Regression Mean: 0.9475 ± 0.0060 Decision Tree Mean: 0.8707 ± 0.0223 XGBoost Mean: 0.9644 ± 0.0043 SR4 Mean: 0.9525 ± 0.0105 051015202530 Trial 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 Recall Logistic Regression Mean: 0.9447 ± 0.0066 Decision Tree Mean: 0.8655 ± 0.0242 XGBoost Mean: 0.9633 ± 0.0045 SR4 Mean: 0.9520 ± 0.0106 051015202530 Trial 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 F1 Logistic Regression Mean: 0.9449 ± 0.0065 Decision Tree Mean: 0.8656 ± 0.0242 XGBoost Mean: 0.9633 ± 0.0045 SR4 Mean: 0.9520 ± 0.0106 Figure 50: Line plot comparison of model performance metrics for vehicle data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 65 of 77 051015202530 Trial 0.56 0.57 0.58 0.59 0.60 0.61 0.62 0.63 0.64 Accuracy Random Forest Mean: 0.6199 ± 0.0119 SVM Mean: 0.5911 ± 0.0084 RuleFit Mean: 0.5865 ± 0.0135 SR4 Mean: 0.5856 ± 0.0151 051015202530 Trial 0.52 0.54 0.56 0.58 0.60 0.62 Precision Random Forest Mean: 0.6001 ± 0.0119 SVM Mean: 0.5845 ± 0.0091 RuleFit Mean: 0.5673 ± 0.0144 SR4 Mean: 0.5661 ± 0.0166 051015202530 Trial 0.56 0.57 0.58 0.59 0.60 0.61 0.62 0.63 0.64 Recall Random Forest Mean: 0.6199 ± 0.0119 SVM Mean: 0.5911 ± 0.0084 RuleFit Mean: 0.5865 ± 0.0135 SR4 Mean: 0.5856 ± 0.0151 051015202530 Trial 0.54 0.56 0.58 0.60 0.62 F1 Random Forest Mean: 0.6069 ± 0.0118 SVM Mean: 0.5829 ± 0.0084 RuleFit Mean: 0.5717 ± 0.0129 SR4 Mean: 0.5702 ± 0.0149 051015202530 Trial 0.54 0.56 0.58 0.60 0.62 Accuracy Logistic Regression Mean: 0.5981 ± 0.0091 Decision Tree Mean: 0.5590 ± 0.0171 XGBoost Mean: 0.5998 ± 0.0108 SR4 Mean: 0.5856 ± 0.0151 051015202530 Trial 0.52 0.54 0.56 0.58 0.60 Precision Logistic Regression Mean: 0.5865 ± 0.0099 Decision Tree Mean: 0.5471 ± 0.0188 XGBoost Mean: 0.5843 ± 0.0112 SR4 Mean: 0.5661 ± 0.0166 051015202530 Trial 0.54 0.56 0.58 0.60 0.62 Recall Logistic Regression Mean: 0.5981 ± 0.0091 Decision Tree Mean: 0.5590 ± 0.0171 XGBoost Mean: 0.5998 ± 0.0108 SR4 Mean: 0.5856 ± 0.0151 051015202530 Trial 0.50 0.52 0.54 0.56 0.58 0.60 F1 Logistic Regression Mean: 0.5846 ± 0.0090 Decision Tree Mean: 0.5442 ± 0.0186 XGBoost Mean: 0.5883 ± 0.0113 SR4 Mean: 0.5702 ± 0.0149 Figure 51: Line plot comparison of model performance metrics for yeast data across 30 trials. The left panel reports results obtained for models—random forest, SVM, rulefit, and SR4-fit , whereas the right panel shows results with models logistic regression, decision tree, and XGboost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 66 of 77 AccuracyPrecisionRecallF1 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1.00 Score Random Forest SVM RuleFit SR4 (a) Breast Cancer AccuracyPrecisionRecallF1 0.82 0.84 0.86 0.88 0.90 0.92 Score Random Forest SVM RuleFit SR4 (b) E. Coli AccuracyPrecisionRecallF1 0.9575 0.9600 0.9625 0.9650 0.9675 0.9700 0.9725 0.9750 Score Random Forest SVM RuleFit SR4 (c) Page blocks AccuracyPrecisionRecallF1 0.45 0.50 0.55 0.60 0.65 0.70 0.75 Score Random Forest SVM RuleFit SR4 (d) Pima Indians AccuracyPrecisionRecallF1 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 Score Random Forest SVM RuleFit SR4 (e) Vehicle AccuracyPrecisionRecallF1 0.52 0.54 0.56 0.58 0.60 0.62 0.64 Score Random Forest SVM RuleFit SR4 (f) Yeast Figure 52: Violin plot comparison of model (random forest, SVM, RuleFit, and SR4-fit) performance across multiple standard benchmark classification datasets for different prediction metrics. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 67 of 77 AccuracyPrecisionRecallF1 0.825 0.850 0.875 0.900 0.925 0.950 0.975 1.000 Score Logistic Regression Decision Tree XGBoost SR4 (a) Breast Cancer AccuracyPrecisionRecallF1 0.75 0.80 0.85 0.90 0.95 Score Logistic Regression Decision Tree XGBoost SR4 (b) E. coli AccuracyPrecisionRecallF1 0.960 0.965 0.970 0.975 Score Logistic Regression Decision Tree XGBoost SR4 (c) Page blocks AccuracyPrecisionRecallF1 0.3 0.4 0.5 0.6 0.7 0.8 Score Logistic Regression Decision Tree XGBoost SR4 (d) Pima Indians AccuracyPrecisionRecallF1 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 Score Logistic Regression Decision Tree XGBoost SR4 (e) Vehicle AccuracyPrecisionRecallF1 0.50 0.52 0.54 0.56 0.58 0.60 0.62 Score Logistic Regression Decision Tree XGBoost SR4 (f) Yeast Figure 53: Violin plot comparison of model (logistic regression, decision tree, and XGBoost) performance across multiple standard benchmark classification datasets for different prediction metrics. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 68 of 77 F. Results on Standard Public Datasets Regression Across all eight regression datasets and 30 trials, SR4-Fit demonstrates a strong balance of predictive performance, rule stability, and low variance, making it the most robust and interpretable method overall. Although individual models occasionally outperform it on prediction metrics, SR4-Fit repeatedly delivers the most consistent and stable behavior across trials. In the abalone dataset, SVM has the lowest error values, but SR4-Fit trails very closely behind SVM, as seen in Figure 54. The violin plots for abalone in Figure 62 and Figure 63 reinforce these findings. SR4-Fit significantly outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 decision tree has lower average rule complexity compared to all rule-based models. In the bone dataset, random forest has the lowest error values, but SR4-Fit trails very closely behind random forest, as seen in Figure 55. The violin plots for bone in Figure 62 and Figure 63 reinforce these findings. SR4-Fit significantly outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 decision tree has lower average rule complexity compared to all rule-based models. In the diabetes dataset, linear regression has the lowest error values, but SR4-Fit and RuleFit trail very closely behind, as seen in Figure 56. The violin plots for diabetes in Figure 62 and Figure 63 reinforce these findings. SR4-Fit significantly outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 decision tree has lower average rule complexity compared to all rule-based models. In the housing dataset, XGBoost has the lowest error values, but SR4-Fit trails very closely behind XGBoost, as seen in Figure 57. The violin plots for housing in Figure 62 and Figure 63 reinforce these findings. RuleFit outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 RuleFit has lower average rule complexity compared to all rule-based models. In the machine dataset, XGBoost has the lowest error values, but SR4-Fit trails very closely behind XGBoost, as seen in Figure 58. The violin plots for the machine in Figure 62 and Figure 63 reinforce these findings. SR4-Fit outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 SR4-Fit has lower average rule complexity compared to all rule-based models. In the MPG dataset, SVM has the lowest error values, but SR4-Fit and RuleFit trail very closely behind, as seen in Figure 59. The violin plots for MPG in Figure 62 and Figure 63 reinforce these findings. The decision tree outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 decision tree has lower average rule complexity compared to all rule-based models. In the ozone dataset, SVM has the lowest error values, but SR4-Fit trails very closely behind, as seen in Figure 60. The violin plots for the ozone in Figure 62 and Figure 63 reinforce these findings. SR4-Fit outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 SR4-Fit has lower average rule complexity compared to all rule-based models. In the prostate dataset, SVM has the lowest error values, but SR4-Fit trails very closely behind, as seen in Figure 61. The violin plots for the ozone in Figure 62 and Figure 63 reinforce these findings. SR4-Fit outperforms in stability, achieving the highest Dice–Sorensen Index, as seen in Table 21. Table 22 shows that the decision tree has a compact structure, whereas, with respect to average rule complexity in Table 23 SR4-Fit has lower average rule complexity compared to all rule-based models. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 69 of 77 051015202530 Trial 1.9 2.0 2.1 2.2 2.3 2.4 RMSE Random Forest Mean: 2.0469 ± 0.0683 SVM Mean: 2.0235 ± 0.0756 RuleFit Mean: 2.1853 ± 0.0759 SR4 Mean: 2.0696 ± 0.0651 051015202530 Trial 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 R² Random Forest Mean: 0.5677 ± 0.0157 SVM Mean: 0.5776 ± 0.0163 RuleFit Mean: 0.5074 ± 0.0153 SR4 Mean: 0.5581 ± 0.0146 051015202530 Trial 1.3 1.4 1.5 1.6 1.7 MAE Random Forest Mean: 1.4711 ± 0.0454 SVM Mean: 1.4205 ± 0.0509 RuleFit Mean: 1.6093 ± 0.0501 SR4 Mean: 1.5205 ± 0.0440 051015202530 Trial 1.9 2.0 2.1 2.2 2.3 2.4 2.5 RMSE Lasso Regression Mean: 2.0913 ± 0.0671 Decision Tree Mean: 2.2666 ± 0.0803 XGBoost Mean: 2.0473 ± 0.0697 SR4 Mean: 2.0696 ± 0.0651 051015202530 Trial 0.425 0.450 0.475 0.500 0.525 0.550 0.575 0.600 R² Lasso Regression Mean: 0.5487 ± 0.0182 Decision Tree Mean: 0.4700 ± 0.0192 XGBoost Mean: 0.5676 ± 0.0160 SR4 Mean: 0.5581 ± 0.0146 051015202530 Trial 1.4 1.5 1.6 1.7 1.8 MAE Lasso Regression Mean: 1.5395 ± 0.0462 Decision Tree Mean: 1.6467 ± 0.0515 XGBoost Mean: 1.4718 ± 0.0449 SR4 Mean: 1.5205 ± 0.0440 Figure 54: Line plot comparison of model performance metrics for abalone data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. Table 21 Average Dice–Sørensen Index ± standard deviation across 30 trials for public regression datasets. Bold indicates the best-performing method per dataset. Dataset Random ForestRuleFitSR4-FitDecision Tree Abalone 0.0001±0.0000 0.5000±0.0000 0.5011±0.0028 0.2506±0.1684 Bone0.0012±0.0003 0.2727±0.0000 0.3710±0.0326 0.0230±0.0228 Diabetes 0.0001±0.0001 0.5556±0.0000 0.5848±0.0156 0.1425±0.0970 Housing 0.0005±0.0002 0.6190±0.0000 0.5880±0.0143 0.0428±0.0668 Machine 0.0090±0.0020 0.4138±0.0141 0.6146±0.0167 0.0169±0.0205 MPG0.0005±0.0001 0.5000±0.0000 0.3743±0.0095 1.0000±0.0000 Ozone 0.0008±0.0002 0.4286±0.0000 0.4649±0.0138 0.0511±0.0506 Prostate 0.0034±0.0009 0.5032±0.0048 0.5574±0.0427 0.0084±0.0130 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 70 of 77 051015202530 Trial 0.035 0.040 0.045 0.050 0.055 RMSE Random Forest Mean: 0.0399 ± 0.0025 SVM Mean: 0.0491 ± 0.0027 RuleFit Mean: 0.0494 ± 0.0040 SR4 Mean: 0.0418 ± 0.0027 051015202530 Trial 0.4 0.2 0.0 0.2 0.4 R² Random Forest Mean: 0.3242 ± 0.1096 SVM Mean: -0.0234 ± 0.1809 RuleFit Mean: -0.0185 ± 0.0205 SR4 Mean: 0.2667 ± 0.0677 051015202530 Trial 0.025 0.030 0.035 0.040 0.045 MAE Random Forest Mean: 0.0302 ± 0.0020 SVM Mean: 0.0415 ± 0.0028 RuleFit Mean: 0.0394 ± 0.0031 SR4 Mean: 0.0321 ± 0.0022 051015202530 Trial 0.034 0.036 0.038 0.040 0.042 0.044 0.046 0.048 RMSE Lasso Regression Mean: 0.0424 ± 0.0029 Decision Tree Mean: 0.0406 ± 0.0029 XGBoost Mean: 0.0407 ± 0.0023 SR4 Mean: 0.0418 ± 0.0027 051015202530 Trial 0.1 0.2 0.3 0.4 0.5 R² Lasso Regression Mean: 0.2441 ± 0.0671 Decision Tree Mean: 0.3042 ± 0.1054 XGBoost Mean: 0.2966 ± 0.1094 SR4 Mean: 0.2667 ± 0.0677 051015202530 Trial 0.026 0.028 0.030 0.032 0.034 0.036 MAE Lasso Regression Mean: 0.0328 ± 0.0022 Decision Tree Mean: 0.0307 ± 0.0026 XGBoost Mean: 0.0307 ± 0.0019 SR4 Mean: 0.0321 ± 0.0022 Figure 55: Line plot comparison of model performance metrics for bone data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit, whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. 051015202530 Trial 47.5 50.0 52.5 55.0 57.5 60.0 62.5 65.0 RMSE Random Forest Mean: 56.4488 ± 4.4391 SVM Mean: 54.1514 ± 3.3116 RuleFit Mean: 53.9356 ± 3.7982 SR4 Mean: 54.8552 ± 3.9184 051015202530 Trial 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 R² Random Forest Mean: 0.4201 ± 0.0708 SVM Mean: 0.4667 ± 0.0493 RuleFit Mean: 0.4704 ± 0.0610 SR4 Mean: 0.4517 ± 0.0680 051015202530 Trial 37.5 40.0 42.5 45.0 47.5 50.0 52.5 MAE Random Forest Mean: 46.0333 ± 3.9045 SVM Mean: 43.8972 ± 3.0519 RuleFit Mean: 44.0223 ± 3.1952 SR4 Mean: 44.3589 ± 3.1643 051015202530 Trial 45 50 55 60 65 70 RMSE Lasso Regression Mean: 53.2604 ± 3.8749 Decision Tree Mean: 60.5215 ± 4.0650 XGBoost Mean: 57.0343 ± 3.8601 SR4 Mean: 54.8552 ± 3.9184 051015202530 Trial 0.1 0.2 0.3 0.4 0.5 0.6 R² Lasso Regression Mean: 0.4836 ± 0.0604 Decision Tree Mean: 0.3325 ± 0.0799 XGBoost Mean: 0.4072 ± 0.0702 SR4 Mean: 0.4517 ± 0.0680 051015202530 Trial 35 40 45 50 55 MAE Lasso Regression Mean: 43.4326 ± 3.1771 Decision Tree Mean: 48.7924 ± 3.3122 XGBoost Mean: 45.8768 ± 3.3871 SR4 Mean: 44.3589 ± 3.1643 Figure 56: Line plot comparison of model performance metrics for diabetes data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 71 of 77 051015202530 Trial 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 RMSE Random Forest Mean: 3.4319 ± 0.5920 SVM Mean: 3.8288 ± 0.6395 RuleFit Mean: 4.2036 ± 0.5963 SR4 Mean: 4.2193 ± 0.6169 051015202530 Trial 0.65 0.70 0.75 0.80 0.85 0.90 R² Random Forest Mean: 0.8582 ± 0.0418 SVM Mean: 0.8246 ± 0.0460 RuleFit Mean: 0.7888 ± 0.0503 SR4 Mean: 0.7869 ± 0.0517 051015202530 Trial 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 MAE Random Forest Mean: 2.3032 ± 0.2653 SVM Mean: 2.2955 ± 0.2561 RuleFit Mean: 2.8290 ± 0.2866 SR4 Mean: 2.7340 ± 0.3019 051015202530 Trial 3 4 5 6 RMSE Lasso Regression Mean: 4.8381 ± 0.5664 Decision Tree Mean: 4.7440 ± 0.5488 XGBoost Mean: 3.0599 ± 0.4668 SR4 Mean: 4.2193 ± 0.6169 051015202530 Trial 0.60 0.65 0.70 0.75 0.80 0.85 0.90 R² Lasso Regression Mean: 0.7209 ± 0.0526 Decision Tree Mean: 0.7297 ± 0.0579 XGBoost Mean: 0.8881 ± 0.0274 SR4 Mean: 0.7869 ± 0.0517 051015202530 Trial 2.0 2.5 3.0 3.5 4.0 MAE Lasso Regression Mean: 3.3550 ± 0.2814 Decision Tree Mean: 3.1922 ± 0.2526 XGBoost Mean: 2.1069 ± 0.2037 SR4 Mean: 2.7340 ± 0.3019 Figure 57: Line plot comparison of model performance metrics for housing data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. 051015202530 Trial 25 50 75 100 125 150 175 RMSE Random Forest Mean: 34.2804 ± 12.4845 SVM Mean: 99.4272 ± 47.6923 RuleFit Mean: 30.5149 ± 8.6959 SR4 Mean: 29.8233 ± 8.3421 051015202530 Trial 0.2 0.4 0.6 0.8 1.0 R² Random Forest Mean: 0.9191 ± 0.0428 SVM Mean: 0.4185 ± 0.2425 RuleFit Mean: 0.9348 ± 0.0221 SR4 Mean: 0.9340 ± 0.0275 051015202530 Trial 10 20 30 40 50 MAE Random Forest Mean: 14.3279 ± 5.0831 SVM Mean: 33.0458 ± 14.2425 RuleFit Mean: 18.2022 ± 4.1950 SR4 Mean: 18.7728 ± 3.4803 051015202530 Trial 10 20 30 40 50 60 70 RMSE Lasso Regression Mean: 31.3053 ± 5.6181 Decision Tree Mean: 42.4632 ± 15.3619 XGBoost Mean: 25.5142 ± 9.1702 SR4 Mean: 29.8233 ± 8.3421 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 0.9 1.0 R² Lasso Regression Mean: 0.9180 ± 0.0555 Decision Tree Mean: 0.8579 ± 0.1201 XGBoost Mean: 0.9434 ± 0.0499 SR4 Mean: 0.9340 ± 0.0275 051015202530 Trial 5 10 15 20 25 30 MAE Lasso Regression Mean: 20.5150 ± 2.9250 Decision Tree Mean: 21.3430 ± 5.7549 XGBoost Mean: 10.2432 ± 3.4572 SR4 Mean: 18.7728 ± 3.4803 Figure 58: Line plot comparison of model performance metrics for machine data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 72 of 77 051015202530 Trial 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 RMSE Random Forest Mean: 2.7934 ± 0.3639 SVM Mean: 2.6488 ± 0.3103 RuleFit Mean: 2.9411 ± 0.2474 SR4 Mean: 2.9195 ± 0.3017 051015202530 Trial 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 R² Random Forest Mean: 0.8658 ± 0.0333 SVM Mean: 0.8795 ± 0.0271 RuleFit Mean: 0.8524 ± 0.0237 SR4 Mean: 0.8544 ± 0.0273 051015202530 Trial 1.6 1.8 2.0 2.2 2.4 MAE Random Forest Mean: 1.9474 ± 0.1834 SVM Mean: 1.9067 ± 0.1505 RuleFit Mean: 2.1613 ± 0.1592 SR4 Mean: 2.1288 ± 0.1716 051015202530 Trial 0 10 20 30 40 50 RMSE Lasso Regression Mean: 38.2687 ± 4.1432 Decision Tree Mean: 40.9110 ± 4.2186 XGBoost Mean: 39.9642 ± 4.2489 SR4 Mean: 2.9195 ± 0.3017 051015202530 Trial 0.60 0.65 0.70 0.75 0.80 0.85 0.90 R² Lasso Regression Mean: 0.8094 ± 0.0465 Decision Tree Mean: 0.7818 ± 0.0557 XGBoost Mean: 0.7925 ± 0.0484 SR4 Mean: 0.8544 ± 0.0273 051015202530 Trial 0 5 10 15 20 25 30 35 MAE Lasso Regression Mean: 26.6181 ± 2.2264 Decision Tree Mean: 29.1461 ± 2.6130 XGBoost Mean: 27.7720 ± 2.5021 SR4 Mean: 2.1288 ± 0.1716 Figure 59: Line plot comparison of model performance metrics for MPG data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. 051015202530 Trial 3.5 4.0 4.5 5.0 5.5 RMSE Random Forest Mean: 4.2317 ± 0.4180 SVM Mean: 3.9319 ± 0.4173 RuleFit Mean: 4.4040 ± 0.4106 SR4 Mean: 4.3631 ± 0.4470 051015202530 Trial 0.50 0.55 0.60 0.65 0.70 0.75 0.80 R² Random Forest Mean: 0.7014 ± 0.0548 SVM Mean: 0.7422 ± 0.0493 RuleFit Mean: 0.6766 ± 0.0584 SR4 Mean: 0.6818 ± 0.0636 051015202530 Trial 2.5 3.0 3.5 4.0 MAE Random Forest Mean: 3.1676 ± 0.3265 SVM Mean: 2.8981 ± 0.3190 RuleFit Mean: 3.4225 ± 0.3224 SR4 Mean: 3.3658 ± 0.3717 051015202530 Trial 3.5 4.0 4.5 5.0 5.5 6.0 6.5 RMSE Lasso Regression Mean: 4.5327 ± 0.3694 Decision Tree Mean: 4.8989 ± 0.5258 XGBoost Mean: 4.0922 ± 0.4018 SR4 Mean: 4.3631 ± 0.4470 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 R² Lasso Regression Mean: 0.6580 ± 0.0551 Decision Tree Mean: 0.5971 ± 0.0912 XGBoost Mean: 0.7213 ± 0.0490 SR4 Mean: 0.6818 ± 0.0636 051015202530 Trial 2.5 3.0 3.5 4.0 4.5 MAE Lasso Regression Mean: 3.5743 ± 0.2867 Decision Tree Mean: 3.6866 ± 0.3627 XGBoost Mean: 3.0227 ± 0.2867 SR4 Mean: 3.3658 ± 0.3717 Figure 60: Line plot comparison of model performance metrics for ozone data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 73 of 77 051015202530 Trial 0.5 0.6 0.7 0.8 0.9 1.0 1.1 RMSE Random Forest Mean: 0.8064 ± 0.1253 SVM Mean: 0.7812 ± 0.1369 RuleFit Mean: 0.7624 ± 0.1273 SR4 Mean: 0.7503 ± 0.1231 051015202530 Trial 0.2 0.3 0.4 0.5 0.6 0.7 R² Random Forest Mean: 0.4964 ± 0.1388 SVM Mean: 0.5365 ± 0.1030 RuleFit Mean: 0.5560 ± 0.1064 SR4 Mean: 0.5614 ± 0.1336 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 0.9 MAE Random Forest Mean: 0.6536 ± 0.1189 SVM Mean: 0.6018 ± 0.1168 RuleFit Mean: 0.6011 ± 0.1090 SR4 Mean: 0.5962 ± 0.1025 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 RMSE Lasso Regression Mean: 0.7303 ± 0.1393 Decision Tree Mean: 0.9329 ± 0.1188 XGBoost Mean: 0.8759 ± 0.1223 SR4 Mean: 0.7503 ± 0.1231 051015202530 Trial 0.2 0.0 0.2 0.4 0.6 0.8 R² Lasso Regression Mean: 0.5815 ± 0.1494 Decision Tree Mean: 0.3115 ± 0.2137 XGBoost Mean: 0.3964 ± 0.1930 SR4 Mean: 0.5614 ± 0.1336 051015202530 Trial 0.4 0.5 0.6 0.7 0.8 0.9 MAE Lasso Regression Mean: 0.5607 ± 0.1148 Decision Tree Mean: 0.7597 ± 0.1069 XGBoost Mean: 0.6848 ± 0.1087 SR4 Mean: 0.5962 ± 0.1025 Figure 61: Line plot comparison of model performance metrics for prostate data across 30 trials. The left panel reports results obtained for models—random forest, SVM, RuleFit, and SR4-fit , whereas the right panel shows results with models LASSO regression, decision tree, and XGBoost. Table 22 Average number of rules ± standard deviation across 30 trials for public regression datasets. Lower values indicate more compact models. Dataset Random ForestRuleFitSR4-Fit Decision Tree Abalone 8666.53±101.33 16.00±0.00 15.96±0.18 1.00±0.00 Bone8151.33±298.36 11.00±0.00 2.73±0.44 1.03±0.18 Diabetes 7823.36±209.19 18.00±0.00 17.00±0.69 1.00±0.00 Housing 7244.06±142.93 21.00±0.00 21.90±0.95 3.06±1.08 Machine 4166.13±207.00 21.80±1.5177 14.66±0.84 2.66±0.80 MPG9090.80±205.65 16.00±0.00 21.40±1.13 1.00±0.00 Ozone 7968.26±144.58 28.00±0.00 25.70±1.23 2.20±0.48 Prostate 3220.96±75.75 15.90±0.30 12.33±1.24 3.33±0.88 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 74 of 77 RMSER² (×4)MAE 1.4 1.6 1.8 2.0 2.2 2.4 Score Random Forest SVM RuleFit SR4 (a) Abalone RMSER² (×0.3)MAE 0.10 0.05 0.00 0.05 0.10 0.15 Score Random Forest SVM RuleFit SR4 (b) Bone RMSER² (×10)MAE 0 10 20 30 40 50 60 Score Random Forest SVM RuleFit SR4 (c) Diabetes RMSER² (×4)MAE 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Score Random Forest SVM RuleFit SR4 (d) Housing RMSER² (×38.8)MAE 0 25 50 75 100 125 150 175 Score Random Forest SVM RuleFit SR4 (e) Machine RMSER² (×4)MAE 1.5 2.0 2.5 3.0 3.5 Score Random Forest SVM RuleFit SR4 (f) MPG RMSER² (×5)MAE 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Score Random Forest SVM RuleFit SR4 (g) Ozone RMSER² (×4)MAE 0.5 1.0 1.5 2.0 2.5 3.0 Score Random Forest SVM RuleFit SR4 (h) Prostate Figure 62: Violin plot comparison of models (random forest, SVM, RuleFit, and SR4-fit) performance across multiple public regression datasets for different prediction metrics. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 75 of 77 RMSER² (×4)MAE 1.4 1.6 1.8 2.0 2.2 2.4 Score Lasso Regression Decision Tree XGBoost SR4 (a) Abalone RMSER² (×0.2)MAE 0.02 0.04 0.06 0.08 0.10 Score Lasso Regression Decision Tree XGBoost SR4 (b) Bone RMSER² (×10)MAE 0 10 20 30 40 50 60 70 Score Lasso Regression Decision Tree XGBoost SR4 (c) Diabetes RMSER² (×4)MAE 2 3 4 5 6 Score Lasso Regression Decision Tree XGBoost SR4 (d) Housing RMSER² (×33.4)MAE 10 20 30 40 50 60 70 Score Lasso Regression Decision Tree XGBoost SR4 (e) Machine RMSER² (×10)MAE 0 10 20 30 40 50 Score Lasso Regression Decision Tree XGBoost SR4 (f) MPG RMSER² (×5)MAE 2 3 4 5 6 Score Lasso Regression Decision Tree XGBoost SR4 (g) Ozone RMSER² (×4)MAE 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Score Lasso Regression Decision Tree XGBoost SR4 (h) Prostate Figure 63: Violin plot comparison of models (LASSO regression, decision tree, and XGBoost) performance across multiple public regression datasets for different prediction metrics. S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 76 of 77 Table 23 Average rule complexity ± standard deviation across 30 trials for public regression datasets. Bold indicates less complex rules Dataset Random Forest RuleFit SR4-Fit Decision Tree Abalone6.54±0.02 2.00±0.00 1.99±0.01 1.00±0.00 Bone5.95±0.03 2.45±0.00 2.24±0.15 1.73±0.98 Diabetes 5.88±0.02 1.88±0.00 1.82±0.05 1.00±0.00 Housing5.89±0.02 1.76±0.00 2.22±0.07 3.00±0.00 Machine 5.62±0.05 2.61±0.14 1.75±0.07 2.72±0.58 MPG5.98±0.02 2.00±0.00 2.87±0.06 1.00±0.00 Ozone5.86±0.01 2.71±0.00 2.59±0.07 2.98±0.09 Prostate 4.75±0.05 1.98±0.04 1.79±0.14 2.39±0.31 S.S. Murali Krishnan and D.F. Hougen: Preprint submitted to ElsevierPage 77 of 77

Instant research discovery