Paper deep dive
A Leakage-Free Stacked Ensemble Method for Multiclass Classification
S. P. Sharmila, Aruna Tiwari
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/2/2026, 12:58:14 PM
Summary
The paper proposes LFS-FRAME, a leakage-free stacked ensemble framework for multiclass classification that integrates Kolmogorov-Arnold Networks (KAN) for functional learning and XGBoost for rule-based learning. By employing a strict out-of-fold stacking strategy, the method prevents data leakage between base learners and the meta-classifier. Experimental results on malware datasets demonstrate that LFS-FRAME achieves 89.85% accuracy in identifying major families and 81.74% in identifying sub-families, outperforming single-model baselines and traditional stacking methods.
Entities (6)
Relation Signals (6)
LFS-FRAME → employstechnique → out-of-fold stacking
confidence 95% · constructs unbiased meta-features by employing a strict out-of-fold stacking strategy
out-of-fold stacking → prevents → Data Leakage
confidence 95% · ensure complete isolation between training and validation data hence preventing performance leakage
LFS-FRAME → uses → Kolmogorov-Arnold Networks
confidence 95% · integrates functional learning using Kolmogorov-Arnold Networks (KAN)
LFS-FRAME → uses → XGBoost
confidence 95% · rule-based learning via XGBoost
LFS-FRAME → achievesaccuracyon → CIC-MalMem-2022
confidence 90% · overall accuracy is 89.85% in identifying major families... on multi-class datasets
LFS-FRAME → achievesaccuracyon → EnhancedCIC
confidence 90% · 81.74% in identifying sub-families
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multiclass classification is a fundamental problem across a wide range of domains. It is still challenging due to possession of high inter-class similarity, class imbalance datasets, and variability in data distributions. Rule-based classifiers such as XGBoost often achieve stronger performance on structured features, but they are limited in capturing smooth functional relationships among variables. Similarly, neural network models can represent complex nonlinear interactions but frequently suffer from overfitting and generalization issues. To address these limitations, we propose LFS-FRAME, a Leakage-Free Stacked ensemble framework that integrates functional learning using Kolmogorov-Arnold Networks (KAN) and rule-based learning via XGBoost for robust multiclass classification. The proposed framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data hence preventing performance leakage. By learning over probabilistic outputs from heterogeneous base learners, the meta-classifier effectively exploits both global functional patterns and sharp decision boundaries present in the complex data. Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics, and overall accuracy is 89.85% in identifying major families and 81.74% in identifying sub-families relative to strong single-model baselines. These results highlight the effectiveness of leakage-free functional and rule-based stacking for reliable and generalizable multiclass classification.
Tags
Links
- Source: https://arxiv.org/abs/2607.22081v1
- Canonical: https://arxiv.org/abs/2607.22081v1
Trouble viewing inline? Open PDF directly →
Full Text
34,737 characters extracted from source content.
Expand or collapse full text
A Leakage-Free Stacked Ensemble Method for Multiclass Classification S P Sharmila1,2,∗, and Aruna Tiwari1 1 Indian Institute of Technology Indore, Madhya Pradesh, India. 2Siddaganga Institute of Technology, Tumakuru, Karnataka, India. E-mail: phd2201101012, artiwari@iiti.ac.in, sharmila@sit.ac.in Abstract Multiclass classification is a fundamental problem across a wide range of domains. It is still challenging due to possession of high inter-class similarity, class imbalance datasets, and variability in data distributions. Rule-based classifiers such as XGBoost often achieve stronger performance on structured features, but they are limited in capturing smooth functional relationships among variables. Similarly, neural network models can represent complex nonlinear interactions but frequently suffer from overfitting and generalization issues. To address these limitations, we propose LFS-FRAME, a leakage-free stacked ensemble framework that integrates functional learning using Kolmogorov-Arnold Networks (KAN) and rule-based learning via XGBoost for robust multiclass classification. The proposed framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data hence preventing performance leakage. By learning over probabilistic outputs from heterogeneous base learners, the meta-classifier effectively exploits both global functional patterns and sharp decision boundaries present in the complex data. Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics, and overall accuracy is 89.85% in identifying major families and 81.74% in identifying sub-families relative to strong single-model baselines. These results highlight the effectiveness of leakage-free functional and rule-based stacking for reliable and generalizable multiclass classification. I Introduction Ensemble learning has emerged as a cornerstone of modern machine learning, enabling the combination of multiple models to achieve superior predictive performance over individual learners[1]. Initially popularized through bagging (e.g., Random Forests) and boosting (e.g., AdaBoost, Gradient Boosting) in the late 1990s and early 2000s, ensemble methods gained prominence by exploiting the bias-variance tradeoff, reducing overfitting, and improving generalization on complex datasets. Their rise accelerated with the availability of computational resources and large-scale data, leading to widespread adoption across domains like computer vision (e.g., deep ensemble CNNs)[2], finance (risk modeling)[3, 4, 5], healthcare (diagnostic classifiers)[6], and cybersecurity (malware detection)[7, 8, 9]. Among advanced ensemble techniques, stacking stands out for its meta-learning approach, where a second-level model learns to optimally blend predictions from diverse base classifiers. Wolpert[16] formalized stacking in 1992, which renewed interest not only in multiclass settings, but also intrusion detection and multi-label malware classification[17], where it leverages class probability vectors from bases like RF, SVM, and XGBoost to boost accuracy by 5-15% over single models. Despite these advantages, stacking ensembles, particularly for multiclass classification, face several critical challenges that can undermine their effectiveness if unaddressed. Key issues in Multi-class stacking ensembles: Stacking introduces complexities amplified by the high-dimensional probability outputs (B×CB× C features) in multiclass problems. 1. Data leakage: Training the meta-learner on in-sample base predictions leaks label information, inflating performance metrics unrealistically. 2. Meta-learner overfitting: High-dimensional meta-features lead to overfitting unless regularized (e.g., via simple logistic regression). 3. Base model diversity deficiency: Correlated bases provide redundant signals, diminishing stacking gains. 4. Computational overhead: CV-based out-of-fold predictions scale poorly with B, C, and folds. Apart from these the other issues include, class imbalance and interpretability, necessitate careful design, as outlined in prior analyses. To address these issues, in this paper we propose a leakage free multiclass stacked ensemble of functional and rule based models111This paper is presented at IEEE World Congress on Computational Intelligence (WCCI) from 21–26 June 2026 Maastricht, The Netherlands. Unlike conventional stacking ensemble the proposed framework is a structured integration of functional and rule-based learners, designed explicitly for leakage-free, probability-level multiclass classification. Its novelty lies in the principled fusion of complementary inductive biases and its dynamic adaptability to evolving multi-class families. I Background and Related Work Multiclass classification (MCC) is a fundamental problem in machine learning, with wide-ranging applications in pattern recognition, healthcare analytics, bioinformatics, and security-oriented data analysis. As the number of classes increases, classifiers face significant challenges such as inter-class similarity, overlapping decision regions, and severe class imbalance, which often lead to unstable generalization and degraded performance. Traditional single-model approaches frequently struggle in such high-cardinality settings, motivating the adoption of ensemble learning techniques to improve robustness and predictive accuracy [10, 11, 30, 29]. Tree-based ensemble methods, particularly gradient-boosted decision trees, have demonstrated strong empirical performance on structured and tabular datasets. XGBoost, in particular, has become a widely adopted model due to its ability to learn sparse, non-linear decision rules while incorporating regularization to mitigate overfitting [12]. Several studies have reported effectiveness of XGB in multiclass learning scenarios; however, in our proposed work, its limitations concerned with reliance on piecewise-constant approximations is counteracted by adopting functional learning. Neural network–based models provide a complementary perspective by approximating complex functional mappings through continuous transformations. Deep learning approaches have been applied to multiclass classification with varying degrees of success, yet they often suffer from sensitivity to hyperparameter tuning, overfitting under limited data, and reduced interpretability [13]. Recently, Kolmogorov–Arnold Networks (KAN) have been proposed as a theoretically grounded functional learning paradigm based on the Kolmogorov–Arnold representation theorem, enabling multivariate function approximation through compositions of univariate nonlinear functions [14]. KANs have also excelled for being resilient to generative adversarial attacks [15, 26, 27]. While KANs offer improved parameter efficiency and interpretability, their standalone use may be insufficient for modeling sharp decision boundaries required in highly discriminative tasks. Stacking generalization, originally introduced by Wolpert [16], aims to overcome the limitations of individual learners by training a meta-classifier on the predictions of multiple base models. Numerous stacking-based frameworks have been proposed, including homogeneous ensembles, heterogeneous learners, and probability-level fusion strategies [18]. Despite these, many existing stacking approaches inadvertently introduce information leakage, as the meta-classifier is often trained on predictions generated from the same data used to train the base models. Such leakage leads to overly optimistic performance estimates and poor real-world generalization, particularly in multiclass settings where error propagation is amplified [19]. More recent studies have emphasized the importance of out-of-fold (OOF) prediction strategies to mitigate leakage and ensure unbiased meta-feature construction [20]. Separately, hybrid ensembles combining neural and tree-based models have shown potential benefits; however, these approaches often lack a principled integration strategy and theoretical grounding, and few explicitly address leakage prevention in a systematic manner. I Problem Definition Definition: Leakage-Free Multiclass Stacking Framework A Multiclass Stacking Framework built for a dataset D=(xi,yi)i=1ND=(x_i,y_i)_i=1^N, is said to be Leakage-Free if the meta-feature vector for sample xix_i is constructed as: zi=[p^i(1),p^i(2),…,p^i(M)]z_i=[ p^(1)_i, p^(2)_i,…, p^(M)_i] where each p^i(m) p^(m)_i is the out-of-fold class probability vector produced by the mthm^th base learner and no classifier is allowed to observe yiy_i while producing any component of ziz_i. Let the training set be =(xi,yi)i=1nD=\(x_i,y_i)\_i=1^n where xi∈x_i is a feature vector and yi∈1,…,Cy_i∈\1,…,C\ is the true multiclass label (C classes). Suppose we have B base classifiers fbb=1B\f_b\_b=1^B, each mapping fb:→[0,1]Cf_b:X→[0,1]^C so fb(xi)f_b(x_i) is a probability vector over the C classes. The meta‑learner g is a classifier that takes as input the concatenated base‑model outputs: zi=[f1(xi)⊤,f2(xi)⊤,…,fB(xi)⊤]⊤∈[0,1]B×Cz_i= [f_1(x_i) ,f_2(x_i) ,…,f_B(x_i) ] ∈[0,1]^B× C and predicts the final class y^i=g(zi). y_i=g(z_i). In the leaky version, each base model fbf_b is trained on the full training set D: fb=b()f_b=A_b(D) where bA_b is the learning algorithm for base model b. Then the meta‑level training set is built as =(zi,yi)i=1nwherezi=[f1(xi)⊤,…,fB(xi)⊤]⊤.Z=\(z_i,y_i)\_i=1^n z_i= [f_1(x_i) ,…,f_B(x_i) ] . The meta‑learner is trained as g=g().g=A_g(Z). Here the leakage is that fb(xi)f_b(x_i) is computed on a model that has already seen (xi,yi)(x_i,y_i) during training. That is, the label yiy_i has already influenced the learned parameters of fbf_b, so the probability vector fb(xi)f_b(x_i) is correlated with yiy_i in a way that will not hold on new, unseen data. This means, Cov(yi,fb(xi)) is inflated on Cov(y_i,f_b(x_i)) is inflated on D, but this covariance will shrink on test data testD_test, causing the meta‑learner to overfit to training‑specific patterns and leading to optimistic in‑sample performance. IV Proposed Methodology To remove leakage, we have employed out‑of‑fold (OOF) predictions described as follows: By partitioning D into K folds kk=1K\D_k\_k=1^K. For each fold k we perform the following: - Train each base model fb(k)f_b^(k) on ∖kD _k: fb(k)=b(∖k).f_b^(k)=A_b(D _k). - Compute predictions only on kD_k zi(k)=[f1(k)(xi)⊤,…,fB(k)(xi)⊤]⊤,(xi,yi)∈k.z_i^(k)= [f_1^(k)(x_i) ,…,f_B^(k)(x_i) ] , (x_i,y_i) _k. - Concatenate all OOF predictions into the meta‑level training set: oof=(zi(k),yi)(xi,yi)∈k,k=1K.Z_oof=\(z_i^(k),y_i)\_(x_i,y_i) _k,\,k=1^K. - Train the meta‑learner as g=g(oof).g=A_g(Z_oof). Further, for every sample xix_i, the vector zi(k)z_i^(k) is generated by base models fb(k)f_b^(k) that never saw (xi,yi)(x_i,y_i) during training, so fb(k)(xi)⟂yi∣∖kf_b^(k)(x_i) y_i _k in the sense that the label information is not directly leaked into the meta‑level features. At the test time, for a new and unseen sample x∗x^* we perform the following: - Retrain each base model on the full D: fbfull=b().f_b^full=A_b(D). - Compute the meta‑features: z∗=[f1full(x∗)⊤,…,fBfull(x∗)⊤]⊤.z^*= [f_1^full(x^*) ,…,f_B^full(x^*) ] . - Predict: y^∗=g(z∗). y^*=g(z^*). Because g was trained only on OOF predictions that did not leak labels, and the test predictions fbfull(x∗)f_b^full(x^*) are generated on unseen data, there is no label leakage into the meta‑level model in our proposed approach, and the performance estimate generalizes more reliably. In Algorithm 1 we present a leakage-free multiclass stacking framework that integrates KAN and XGBoost as complementary base learners. Reason for choosing this combination is already presented in the section I. The dataset is first partitioned using stratified K-fold cross-validation to preserve class distributions across folds. For each fold, KAN and XGBoost models are trained exclusively on the fold-specific training subset, and class probability predictions are generated only for the corresponding validation subset. These out-of-fold (OOF) probability estimates are stored and later concatenated to form a meta-feature matrix, ensuring that the meta-classifier is trained on predictions obtained from models that have never seen the same samples during training, thereby eliminating information leakage. A multinomial meta-classifier is then trained on these OOF features to learn optimal fusion weights across base models. Finally, KAN and XGBoost are retrained on the full dataset, and their probability outputs for unseen test samples are fused through the trained meta-classifier to produce the final multiclass prediction. This design ensures robust generalization, unbiased performance estimation, and effective utilization of functional (KAN) and rule-based (XGBoost) decision paradigms. Figure 1 further facilitates to enhance the clarity of the proposed framework. The proposed method resolves key gaps in multiclass malware classification by combining leakage-free stacking, probability-level fusion, and complementary functional–rule learners within a scalable and robust ensemble framework. Figure 1: Leakage-free multiclass stacking framework integrating Kolmogorov–Arnold Networks (KAN) and XGBoost using out-of-fold probability fusion. TABLE I: Details of existing works comparison with evaluated dataset Sl. Reference Methodology used MCC(Accuracy %) 1. [21] HyStack Ensemble 4C: 85.04 16C:70.29 2. [22] Hybrid CNN BiLSTM 3C: 84.54 15C:72.6 3. [23] SMOTE DNN 4C: 75.4 16C:68.2 4. [24] RF with Hyp Tuning 4C: 89.07 16C:68.2 5. Ours KAN+XGB 4C: 89.85 16C:81.74 0: D=(xi,yi)i=1ND=\(x_i,y_i)\_i=1^N, number of folds K 0: Trained meta-classifier M Initialisation : 1: Initialize StratifiedKFold with K folds 2: Initialize out-of-fold probability matrices: 3: OOFKAN∈ℝN×COOF_KAN ^N× C, OOFXGB∈ℝN×COOF_XGB ^N× C 4: Forf=1f=1 to K 5: Split D into training set Dtrain(f)D_train^(f) and validation set Dval(f)D_val^(f) 6: Train Kolmogorov–Arnold Network KAN(f)KAN^(f) on Dtrain(f)D_train^(f) 7: Train XGBoost classifier XGB(f)XGB^(f) on Dtrain(f)D_train^(f) 8: Obtain class probability predictions on Dval(f)D_val^(f): 9: PKAN←KAN(f).predict_proba(Dval(f))P_KAN← KAN^(f).predict\_proba(D_val^(f)) 10: PXGB←XGB(f).predict_proba(Dval(f))P_XGB← XGB^(f).predict\_proba(D_val^(f)) 11: Store out-of-fold predictions: 12: OOFKAN[val_idx]←PKANOOF_KAN[val\_idx]← P_KAN 13: OOFXGB[val_idx]←PXGBOOF_XGB[val\_idx]← P_XGB 14: EndFor 15: Construct meta-feature matrix: 16: Z←[OOFKAN||OOFXGB]Z←[OOF_KAN\;||\;OOF_XGB] 17: Train multinomial meta-classifier M on (Z,y)(Z,y) 18: Retrain KANKAN and XGBXGB on the full dataset D 19: For each test sample xtestx_test 20: Obtain probability vectors: 21: pKAN←KAN.predict_proba(xtest)p_KAN← KAN.predict\_proba(x_test) 22: pXGB←XGB.predict_proba(xtest)p_XGB← XGB.predict\_proba(x_test) 23: Concatenate probabilities: 24: ztest←[pKAN||pXGB]z_test←[p_KAN\;||\;p_XGB] 25: Predict final class label: 26: y^←M(ztest) y← M(z_test) 27: EndFor 28: Return meta-classifier M output Algorithm 1 Leakage-Free Multiclass Stacking with KAN and XGBoost TABLE I: Comparison Between Traditional Stacking and the Proposed Leakage-Free Stacked Ensemble Aspect Traditional Stacking Proposed Leakage-Free Stacking (LFS-FRAME) Meta-feature construction Base-model predictions often generated on training data Strictly uses out-of-fold (OOF) predictions for meta-feature construction Information leakage High risk due to reuse of training samples across levels Explicitly eliminated through fold-wise data isolation Generalization reliability Validation and test performance may be optimistically biased Unbiased and stable generalization estimates Base learner diversity Typically homogeneous or weakly heterogeneous models Heterogeneous ensemble: functional (KAN) + rule-based (XGBoost) Error correlation High due to similar inductive biases Reduced via complementary learning mechanisms Multiclass scalability Performance degrades as number of classes increases Maintains macro-level performance in high-class regimes Theoretical grounding Primarily empirical with limited formal justification Grounded in functional approximation and ensemble theory Robustness to class imbalance Often biased toward majority classes Class-weighted learning and probability-level fusion Interpretability Limited or inconsistent Rule-based component retains interpretability Extensibility Pipeline redesign required to add new models Modular and easily extensible stacking architecture TABLE I: KAN and XGB Hyperparameters and values KAN-Hyperparameter Value XGB-Hyperparameter Value hidden_layer_size 512 eta (learning rate) 0.1 regularize_activation 0.0 max_depth 6 regularize_entropy 0.0 subsample 0.8 regularize_ridge 0.2 colsample_bytree 0.8 spline_order 3 seed 42 batch_size 32 - - learning_rate (lr) 0.0025 - - weight_decay 0.0001 - - n_epochs 100 - - TABLE IV: Classification performance accuracy of XGB, KAN and KAN+XGB for both datasets - 16C Dataset KAN-only XGB-only KAN+XGB CIC 69.66 76.21 79.32 EnhancedCIC 71.82 79.32 81.74 V Experimental Details with Results and discussion To evaluate the proposed model we have chosen publicly available dataset having 16 classes and its enhanced version containing more samples with adversarial training. This section provides detailed dataset description followed by the chosen functional and rule-based learning models. We also provide the baseline comparison alongwith the advantages of the proposed framework. The sample source segment is available at our repository https://github.com/sharmilaharsha/Leakfree-WCCI2026. V-A Dataset Description We have employed two datasets in our work: the public volatile-memory dataset CIC-MalMem-2022 (malware memory analysis)[25] and a larger internal dataset called EnhancedVolMem dataset created from CIC-MalMem-2022 via a set of augmentation and synthetic-sample generation techniques (GAN, interpolation, controlled mixing). Further, we summarize contents, provenance and the role each dataset plays in our experiments. CIC-MalMem-2022: Obfuscated malware is a malware that hides to avoid detection and extermination. The obfuscated malware dataset is designed to test obfuscated malware detection methods through memory. The dataset was created to represent as close to a real-world situation as possible using malware that is prevalent in the real world. It is composed of Spyware, Ransomware and Trojan malware, and a balanced dataset that can be used to test obfuscated malware detection systems. This dataset uses debug mode for the memory dump process to avoid the dumping process to show up in the memory dumps. This works to represent a more accurate example of what an average user would have running at the time of a malware attack. EnhancedCIC dataset: This is a larger dataset produced by augmenting CICMalMem-2022 using a combination of controlled synthetic-sample generation and interpolation techniques. Methods used include generative adversarial network (GAN) based synthesis, feature-space interpolation (SMOTE-like and linear interpolation between pairs of feature vectors), and conservative class-balanced oversampling. The goal is to increase per-class sample counts so that multi-class attribution experiments (e.g., 15/16-class tasks) have sufficient data. The train, test and validation split chosen with these datasets is 70:15:15. Sample dataset is available in our GitHub repository. Figure 2: Confusion Matrix derived from KAN+XGB with OOF for 16C. Figure 3: KAN-XGB-16C. V-B Learning Models In this section we describe the functional learner(KAN), rule-based learners(XGB) and the combination with OOF. Kolmogorov–Arnold Network (KAN): The Kolmogorov–Arnold Network (KAN) serves as a functional learner that models smooth multivariate mappings f:ℝd→ℝCf:R^d ^C using compositions of univariate nonlinear basis functions, as motivated by the Kolmogorov–Arnold representation theorem. Given an input feature vector ∈ℝdx ^d, KAN approximates class logits =f()z=f(x), which are transformed into class probabilities via a softmax function. This formulation enables KAN to capture global, continuous feature interactions efficiently; however, its reliance on smooth approximations can limit its ability to model sharp class boundaries when used in isolation. Extreme Gradient Boosting (XGBoost): XGBoost is employed as a rule-based learner that models class posterior probabilities through an additive ensemble of decision trees, expressed as y^=∑m=1Mfm() y= _m=1^Mf_m(x), where each fmf_m represents a regression tree optimized via gradient boosting. In the multiclass setting, a softmax objective is used to estimate P(y=c∣)P(y=c ) for each class c. XGBoost excels at capturing localized, non-linear decision regions and sparse feature interactions. KAN+XGB Out-of-Fold Stacked Ensemble: The proposed ensemble integrates KAN and XGBoost using a leakage-free out-of-fold (OOF) stacking strategy. For each fold k, base learners are trained on ∗train(k)D*train^(k) and generate unbiased probability estimates (k)∗KANp^(k)*KAN and (k)∗XGBp^(k)*XGB for the held-out subset ∗val(k)D*val^(k). These predictions are concatenated to form meta-features =[∗KAN|∗XGB]z=[p*KAN|p*XGB], which are used to train a multinomial meta-classifier. This formulation exploits complementary functional and rule-based representations while preventing information leakage, leading to improved multiclass generalization. Hyperparameters tuning details are tabulated in Table I. V-C Baseline Comparison The comparative results are summarized in Table I for CIC dataset. It highlights clear differences between existing approaches and our approach. Earlier ensemble-based methods such as HyStack[21] achieved reasonable performance for the 4C showing 85.04% accuracy, but there is a substantial drop when extended to 16C classes (showing 70.29%). This indicates limited robustness of this model for fine-grained classes. Similarly, deep learning–based approaches, including the Hybrid CNN–BiLSTM [22] and SMOTE-DNN [23], showed competitive results in 4C (with 84.54% and 75.4% respectively). Yet they suffer from degraded performance under higher number of class cases, with accuracies falling to the lower 60% to 70% for higher number of classes. This suggests that, representation learning and data balancing help in simpler scenarios, but these methods struggle to preserve discriminative capability as number of classes increases. Although, the Random Forest with hyperparameter tuning [24] improves 4C performance to 89.07%, demonstrating the strength of rule-based learners on structured features. However, there is a sharp decline to 68.2% in the 16C, it reveals the limitations in generalization when decision boundaries become more complex and overlapping. In contrast to this, our proposed KAN+XGB framework with OOF consistently outperforms all compared methods across both settings, achieving 89.85% accuracy for 4C and a markedly higher 81.74% for 16C. This improvement in the high-class regime is particularly significant. Thus it demonstrates the ability of the proposed leakage-free stacked ensemble for class-wise discrimination even under increased multiclass complexity. Fig. 2 depicts the confusion matrix. Finally, Fig.3 portrays the effect of stacking over individual learning methods. Table IV presents classification performance accuracy of XGB, KAN and KAN+XGB for both datasets evaluated for 16 class classification. Overall, the comparison indicates that the proposed approach not only exceeds state-of-the-art performance in simpler classification scenarios but, also delivers substantial gains in challenging multiclass scenarios. The combination of functional learning (KAN) and rule-based modeling (XGBoost), coupled with leakage-free stacking, enables better robust and consistent performance existing methods. V-D Advantages of the LFS-FRAME The proposed leakage-free stacked ensemble offers a robust solution for multiclass classification by integrating complementary functional and rule-based learning paradigms within a rigorously unbiased stacking framework. By combining KAN, which effectively model smooth and compositional functional relationships, with XGB, which excels at learning sharp, rule-based decision boundaries, the approach expands the hypothesis space and improves class discrimination. The use of strict out-of-fold stacking ensures complete isolation between training and validation data, eliminating information leakage and enabling reliable generalization. V-E Computational Trade-off Alongwith the comparison between traditional stacking and leakage free stacking presented in Table I, there exists computational trade-offs. Unlike traditional leaky stacking approaches, while leakage-free stacking introduces a linear computational overhead due to out-of-fold training. This cost is both bounded and necessary to ensure unbiased meta-learning, making the resulting performance improvements statistically valid and practically reliable. In traditional stacking especially for malware family identification, base models trained on the full dataset generate meta-features that inevitably leak training data information to the meta-learner, resulting in weak meta-learning validity, unstable generalization, and low reproducibility despite lower training costs, critical flaws when classifying evasive malware families, where over-optimistic performance (e.g., inflated accuracies from leaked biases) fails against real-world variants and concept drift. Leakage-free stacking, using out-of-fold predictions from k-fold cross-validation, eliminates this information leakage by ensuring the meta-learner trains solely on unbiased hold-out predictions, yielding asymptotically equivalent performance with marginally higher O(k)O(k) training overhead but delivering strong meta-learning validity, robust generalization reliability, and high review defensibility. This approach is particularly compelling for malware family detection that reliably identify families in dynamic, adversarial environments unlike biased traditional methods. VI Conclusion This paper presented LFS-FRAME, a leakage-free stacked ensemble framework. It is designed to address the inherent challenges in multiclass classification. The proposed approach leverages complementary inductive biases that are difficult to capture using a single modeling paradigm by integrating functional learning through Kolmogorov-Arnold Networks (KAN) with rule-based learning using XGBoost. To ensure unbiased meta-feature construction a strict out-of-fold stacking strategy is employed to eliminate information leakage resulting in reliable performance estimation. Based on our experimental evaluation across multiple classes of CIC dataset, LFS-FRAME demonstrated excellence against existing ensemble and deep learning–based approaches, particularly in higher multiclass classification. While several baseline methods exhibited competitive accuracy in low-class scenarios, their performance degraded substantially as the number of classes increased. In contrast to this, LFS-FRAME maintained strong metrics, highlighting its robustness and scalability. These results infer that combining functional and rule-based models within a leakage-free stacking architecture provides a stable decision mechanism for complex multiclass problems. In summary, this study establishes leakage-free functional and rule-based stacking as an effective and generalizable strategy for multiclass classification. As a future work this model can be explored for evaluation of other evolving datasets and various ablation studies with advanced statistical significant testing. References [1] Xue, Yajie, et al. “A survey on ensemble classification methods from the dual perspectives of sampling and learning.” Knowledge and Information Systems 68.1 (2026): 16. [2] Wu, Pan, et al. “A Review on Research and Application of AI-based Image Analysis in the field of Computer Vision.” IEEE Access (2025). [3] Zhang, Xiaoming, Lean Yu, and Hang Yin. “Domain adaptation-based multistage ensemble learning paradigm for credit risk evaluation.” Financial Innovation 11.1 (2025): 27. [4] Aruleba, Idowu, and Yanxia Sun. “An improved Ensemble Method with Data Resampling for Credit Risk Prediction.” IEEE Access (2025). [5] Sharmila, S. P., and Harsha Pandit Moger. “An operative application of distributed ledger technology for banking domain.” International Journal of Computer Science and Mobile Computing 10.7 (2021): 68-75. [6] Imrie, Fergus, et al. “Automated ensemble multimodal machine learning for healthcare.” IEEE Journal of Biomedical and Health Informatics (2025). [7] Moujoud, Loubna, Meryeme Ayache, and Abdelhamid Belmekki. “Enhancing malware detection through ensemble learning techniques.” Cluster Computing 28.7 (2025): 477. [8] Vasan, Danish, et al. “An Advanced Ensemble Framework for defending against obfuscated Windows, Android, and IoT malware.” Applied Soft Computing 173 (2025): 112908. [9] Sharmila, S. P., and Narendra S. Chaudhari. “Conceptual study of prevalent methods for cyber-attack prediction.” Expert Clouds and Applications: Proceedings of ICOECA 2022. Singapore: Springer Nature Singapore, 2022. 631-641. [10] Dietterich, Thomas G. “Ensemble methods in machine learning.” International workshop on multiple classifier systems. Berlin, Heidelberg: Springer Berlin Heidelberg, 2000. [11] Rokach, Lior. “Ensemble-based classifiers.” Artificial intelligence review 33.1 (2010): 1-39. [12] Chen, Tianqi. “XGBoost: A Scalable Tree Boosting System.” Cornell University (2016). 22nd ACM SIGKDD International Conference on Knowledge [13] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning” Nature 521.7553 (2015): 436-444. [14] Liu, Ziming, et al. “KAN: Kolmogorov-arnold networks.” arXiv preprint arXiv:2404.19756 Neural Networks (ICLR 2024). [15] S.P. Sharmila, S Gupta, A Tiwari, N S. Chaudhari. “Unveiling Evasive Portable Documents with Explainable Kolmogorov-Arnold Networks Resilient to Generative Adversarial Attacks”, Applied Soft Computing, Volume 182, (2025) [16] Wolpert, David H. “Stacked generalization.” Neural networks 5.2 (1992): 241-259. [17] S.P. Sharmila, S Gupta, A Tiwari, N S. Chaudhari. “Leveraging Memory Forensic Features for Explainable Obfuscated Malware Detection with Isolated Family Distinction Paradigm”, Computers and Electrical Engineering, Volume 123, Part C, (2025) [18] Polikar, Robi. “Ensemble learning.” Ensemble machine learning. Springer, New York, NY, 2012. 1-34. [19] Cawley, Gavin C., and Nicola LC Talbot. “On over-fitting in model selection and subsequent selection bias in performance evaluation.” The Journal of Machine Learning Research 11 (2010): 2079-2107. [20] Bühlmann, Peter. Bin Yu. “Boosting with the l2 loss.” Journal of the American Statistical Association 98.462 (2003): 324-339. [21] K.S. Roy, T. Ahmed. “Malhystack: A hybrid stacked ensemble learning framework with feature engineering schemes for obfuscated malware analysis.” Intell Syst Appl, 20 (2023) [22] S.S. Shafin, G. Karmakar, I. Mareels. “Obfuscated memory malware detection in resource-constrained IoT devices for smart city applications” Sensors, 23 (11) (2023), p. 5348, [23] D. Cevallos-Salas, F. Grijalva, J. Estrada-Jiménez, D. Benítez, R. Andrade. “Obfuscated privacy malware classifiers based on memory dumping analysis” IEEE Access (2024) [24] S.P. Sharmila, A. Tiwari, N.S. Chaudhari. “Obfuscated malware detection using multi-class classification” 2023 IEEE international conference on cloud computing in emerging markets, CCEM (2023), p. 170-175 [25] Arash Habibi Lashkari, CIC-MalMem-2022, Canadian Institute of Cybersecurity, https://w.unb.ca/cic/datasets/malmem-2022.html [26] Sharmila, S. P., and Aruna Tiwari. “PDFInspect: A Unified Feature Extraction Framework for Malicious Document Detection.” 2026 18th International Conference on COMmunication Systems and NETworks (COMSNETS). IEEE, 2026. [27] Sharmila, S. P., Aruna Tiwari, and Narendra S. Chaudhari. “Enhanced Cyber Threat Intelligence by Network Forensic Analysis for Ransomware as a Service (RaaS) Malwares.” arXiv preprint arXiv:2601.13873 (2026). [28] Sharmila, S. P., et al. “Malware Detection Using Hybrid Vision Transformer with CNN backbone.” 2025 International Conference on Emerging Techniques in Computational Intelligence (ICETCI). IEEE, 2025. [29] Sharmila, S. P., et al. “MaxVote Classifier-Based Diversified Ensemble Model for Identifying Malicious Activity with Few-Shot API Data.” 2024 IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE). IEEE, 2024. [30] Sharmila, S. P., Pratyush Shukla, and Narendra S. Chaudhari. “A distinguished method for network intrusion detection using random initialized viterbi algorithm in hidden Markov model.” 2022 OITS International Conference on Information Technology (OCIT). IEEE, 2022.