Paper deep dive
Width Expansion as a Method for Class Incremental Learning
A. L. S. Conde, Y. Elkhatib, C. M. Ranieri
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Class Incremental Learning (Class-IL) requires models to learn new classes over time while preserving previously acquired knowledge without access to past data or task identity. This setting intensifies the stability-plasticity dilemma and makes catastrophic forgetting a central challenge. Existing approaches include regularization, knowledge distillation, replay, and architectural expansion. However, many expansion methods rely on explicit task identifiers or predefined growth strategies, limiting their applicability when task boundaries are unavailable at inference time. This work proposes a dynamic width expansion method that increases the number of neurons within existing layers according to a normalized loss criterion, without requiring task-specific information. An attention mechanism with persistent key-value memory is also incorporated to stabilize feature representations and reduce interference between previously learned and newly introduced classes. The approach is evaluated on Split MNIST and Split CIFAR-100 under the standard Class-IL protocol. Experiments compare fixed-capacity and dynamically expanding architectures, both with and without attention, combined with established continual learning methods including EWC, LwF, and A-GEM. Results show that progressive width expansion consistently improves performance over fixed architectures, particularly when combined with functional methods and A-GEM. The combination of width expansion and attention provides the most consistent gains. Overall, dynamic width expansion based on representational demand provides an effective and flexible strategy for Class-IL, although uncontrolled growth may increase overfitting and computational cost.
Tags
Links
- Source: https://arxiv.org/abs/2609.37702v1
- Canonical: https://arxiv.org/abs/2609.37702v1
Trouble viewing inline? Open PDF directly β
Full Text
61,117 characters extracted from source content.
Expand or collapse full text
Width Expansion as a Method for Class Incremental Learning AndrΓ© L. S. Conde a,β , Yehia Elkhatib b and Cateano M. Ranieri a a Institute of Geosciences and Exact Sciences, SΓ£o Paulo State University (UNESP), Av. 24-A, 1515, Rio Claro, 13506-692, SP, Brazil b School of Computing Science, University of Glasgow, Glasgow, G12 8Q, Scotland, United Kingdom A R T I C L E I N F O Keywords: Class-Incremental Learning Continual Learning Catastrophic Forgetting Network Expansion Width Expansion A B S T R A C T Class Incremental Learning (Class-IL) is a critical frontier in Continual Learning, which requires that models learn new classes over time while preserving previously acquired knowledge without access to past data or task identity. This setting acutely intensifies the stability-plasticity dilemma, making catastrophic forgetting a central challenge. Existing approaches to mitigate forgetting can be broadly categorized into (i) regularization-based methods, which constrain parameter updates; (i) functional methods, which rely on knowledge distillation; (i) replay-based strategies, which revisit stored samples; and (iv) architectural approaches, which expand model capacity over time. Regarding architectural expansion, most methods rely on an explicit task identifier or predefined growth strategies, which limit their applicability in Class-IL settings where task boundaries are not available during inference. To address this challenge, this work proposes a dynamic width expansion method that increases the number of neurons within existing layers based on a normalized loss criterion, eliminating the need for task-specific information. Additionally, an attention mechanism with persistent key-value memory is incorporated to stabilize feature representations and mitigate interference between previously learned and newly introduced classes. The proposed approach is evaluated on Split MNIST and Split CIFAR-100 benchmarks under the standard Class-IL protocol. Experiments compare fixed-capacity models with dynamically expanding architectures, both in isolation and in combination with the attention mechanism, using established continual learning strategies such as EWC, LwF, and A-GEM. The results suggest that progressively increasing the model capacity leads to consistent improvements over fixed architectures, particularly when combined with functional methods and A-GEM. The combination of width expansion and attention mechanisms yields the most consistent gains. In conclusion, a dynamically expanding network width based on representational demand provides an effective and flexible strategy for Class-IL. However, caution is required, as uncontrolled growth may lead to overfitting and increased computational cost. 1. Introduction Deep neural networks have achieved remarkable per- formance across a wide range of tasks when trained under the traditional assumption of independent and identically distributed (i.i.d) data (Cao, 2022). However, in many real- world applications, data are not available all at once but arrive sequentially over time, a problem with non-stationary data that remains unsolved (Hadsell, Rao, Rusu and Pascanu, 2020). In such scenarios, models are required to continu- ously incorporate new information while retaining previ- ously acquired knowledge. This setting is commonly referred to as continual or incremental learning. A central challenge in this paradigm is catastrophic forgetting, in which learning new information leads to significant performance degrada- tion on previously learned tasks or classes (Liu, Zhou, Liu, Zhao, Yao and Shao, 2023). Among the different continual learning scenarios, class- incremental learning (Class-IL) stands out as one of the most challenging. In this setting, the model must learn to discriminate among an ever-growing set of classes using a single shared classifier, without access to task identity during inference. As new classes are introduced incrementally, the β Corresponding author andre.conde@unesp.br (A.L.S. Conde); yehia.elkhatib@glasgow.ac.uk (Y. Elkhatib); cm.ranieri@unesp.br (C.M. Ranieri) ORCID(s): 0000-0002-5227-5271 (A.L.S. Conde); 0000-0003-4639-436X (Y. Elkhatib); 0000-0001-5680-9085 (C.M. Ranieri) model must balance preserving prior knowledge with acquir- ing new information, thereby intensifying the well-known stability-plasticity dilemma (Kim and Han, 2023). A broad range of methods has been proposed to address catastrophic forgetting in Class-IL (Van de Ven, Tuyte- laars and Tolias, 2022). Regularization-based approaches constrain parameter updates to protect knowledge deemed important for previously learned classes. Functional meth- ods, by contrast, aim to preserve the modelβs input-output mapping via knowledge distillation. Replay-based strategies mitigate forgetting by revisiting past data, either through stored samples or generative models. More recently, ar- chitectural approaches based on network expansion have been explored, increasing model capacity as new tasks or classes are introduced. Template-Based methods, in turn, retain compact representations of past classes in the form of prototypes or exemplars to guide future predictions. Finally, model rectification approaches aim to correct the bias intro- duced during incremental updates, typically by re-balancing decision boundaries or calibrating classifier outputs (Zhou, Wang, Qi, Ye, Zhan and Liu, 2024b). While these approaches have shown promising results, they also present limitations. Regularization and functional methods may excessively restrict model plasticity, replay- based methods introduce additional memory or computa- tional overhead, and expansion-based approaches often rely on task-specific components or predefined growth strategies. : Preprint submitted to ElsevierPage 1 of 15 arXiv:2609.37702v1 [cs.LG] 29 Sep 2026 Template-based methods depend on the quality and repre- sentativeness of stored prototypes, and model rectification techniques may require additional calibration steps or as- sumptions about data distribution shifts. (Zhou et al., 2024b) Recent work highlights the growing momentum of expansion- based methods in the CIL literature. Approaches such as RNE (Jiang, Bai and Zhou, 2026) and Orth-DER (Dong, Zhang, Tan, Qiu and Xie, 2026) extend this paradigm through recurrent inter-expert connections and orthogonal- ity, respectively, while applications to open-world streaming (Li, Dou, Li, Gao and Zhou, 2026) and real-time surveil- lance (Hussain, Ullah, Khan, Khan, Yar and Baik, 2026) demonstrate its reach across diverse settings. Nevertheless, most of these methods increase model capacity by adding new task-specific modules, which introduces dependence on task identifiers at inference or leads to accelerated parameter growth. In this work, we propose a complementary strategy that expands representational capacity within existing lay- ers, preserving the single shared classifier required by the standard Class-IL protocol. To further reduce interference between previously learned and newly introduced classes, we incorporate a linear at- tention mechanism augmented with a persistent key-value memory. This mechanism provides a stable reference across incremental steps, helping to mitigate catastrophic forgetting and representational drift, while enabling the model to adaptively maintain its representations to new data. The main contributions of this work are: β’ A dynamic width expansion mechanism for Class-IL (Section 3.2): a task-agnostic method that increases the number of neurons within existing layers based on a normalized loss criterion, eliminating the need for explicit task identifiers during inference. β’ A linear attention mechanism with persistent key- value memory (Section 3.3): a complementary mod- ule that stabilizes feature representations across in- cremental steps and mitigates interference between previously learned and newly introduced classes. β’ A systematic empirical evaluation (Sections 4 and 5): experiments on Split MNIST and Split CIFAR-100 benchmarks compare fixed-capacity and dynamically expanding architectures, in isolation and in combina- tion with the attention mechanism, across established continual learning strategies including EWC, LwF, and A-GEM. 2. Related Work Catastrophic forgetting in Class-IL has motivated a broad range of approaches, which can be grouped into six paradigms (Zhou et al., 2024b): regularization-based, functional, replay-based, architectural, template-based, and model rectification. Regularization-based methods such as EWC (Kirkpatrick, Pascanu, Rabinowitz, Veness, Desjardins, Rusu, Milan, Quan, Ramalho, Grabska-Barwinska, Hassabis, Clopath, Kumaran and Hadsell, 2017) and SI (Zenke, Poole and Ganguli, 2017) constrain parameter updates by penalizing deviations from weights deemed important for prior tasks. While effective at reducing interference, these methods tend to over-restrict plasticity in Class-IL, where all classes share a common output space. Functional methods such as LwF (Li and Hoiem, 2017) and LwM (Dhar, Singh, Peng, Wu and Chellappa, 2019) preserve the modelβs input-output behavior via knowledge distillation, rather than constraining individual parameters. LwF encourages consistency with previous model outputs through a temperature-scaled KL divergence, while LwM extends this by additionally penalizing divergence in inter- mediate attention maps. Both approaches offer more flexibil- ity than parameter constraints, but may become insufficient when substantial representational adaptation is required. Replay-based methods such as ER (Rolnick, Ahuja, Schwarz, Lillicrap and Wayne, 2019) and A-GEM (Chaudhry, Ranzato, Rohrbach and Elhoseiny, 2019) mitigate forgetting by revisiting stored samples from prior classes. A-GEM constrains gradient updates so that new learning does not increase loss on buffered samples. These methods often achieve strong performance, but introduce memory overhead and raise scalability concerns as the number of observed classes grows. Architectural expansion methods increase model capac- ity as new classes are introduced. DER (Yan, Xie and He, 2021) appends a new extractor while freezing the previous extractor at each step; DNE (Hu, Li, Lyu, Gao and Vas- concelos, 2023) extends this with dense cross-step connec- tions; EASE (Zhou, Sun, Ye and Zhan, 2024a) employs lightweight adapters on a frozen pre-trained backbone; and KANets (Fu, Wang, Xu, Li and Yang, 2023) separates old and new knowledge into parallel branches before compress- ing them. Recent approaches such as RNE (Jiang et al., 2026) and Orth-DER (Dong et al., 2026) further extend this paradigm via recurrent inter-expert connections and orthogonal constraints, respectively. A common limitation across these methods is reliance on task-specific modules or explicit task identifiers, which are unavailable under the standard Class-IL protocol. Template-based methods such as iCaRL (Rebuffi, Kolesnikov, Sperl and Lampert, 2017), CoPE (De Lange and Tuytelaars, 2021), and SDC (Yu, Twardowski, Liu, Herranz, Wang, Cheng, Jui and Van De Weijer, 2020) retain compact class prototypes to guide classification and representation learn- ing. Their effectiveness is bounded by exemplar quality and the per-class memory budget, both of which degrade as the class space grows. Model rectification methods such as WA (Zhao, Xiao, Gan, Zhang and Xia, 2020) and FACT (Zhou, Wang, Ye, Ma, Pu and Zhan, 2022) address classifier bias introduced during incremental updates, but operate primarily at the output layer and may be insufficient when feature-level drift is substantial. The method proposed in this work addresses a gap in the architectural expansion paradigm: rather than introduc- ing per-task modules, it expands representational capacity : Preprint submitted to ElsevierPage 2 of 15 within existing layers, guided by a normalized loss criterion, making it directly applicable to the standard Class-IL setting without access to task identity at inference. 3. Proposed Approach The proposed method operates under the standard Class- IL setting, where a model is trained over a sequence of incremental steps, each introducing a disjoint set of new classes. At each step, the model must learn to recognize newly introduced classes while preserving knowledge of previously learned ones, using a single shared classifier and without access to explicit task identifiers during inference. The central motivation for the proposed approach stems from the stability-plasticity dilemma inherent in this setting Van de Ven et al. (2022). As new classes are introduced, a fixed-capacity model must reallocate resources, often at the expense of previously acquired knowledge. To address this, we propose incrementally expanding the number of neurons within existing layers, allowing the model to ac- commodate new classes with reduced interference to older ones. However, increasing capacity alone is insufficient to prevent representational drift, as newly added parameters may disrupt the feature space learned for prior classes. To mitigate this, we further incorporate an attention mechanism with persistent key-value memory, which stabilizes feature representations across incremental steps and reduces inter- ference between previously learned and newly introduced classes. In contrast to network expansion approaches such as DER and DNE, the proposed method increases represen- tational capacity within existing layers rather than intro- ducing new task-specific modules. While DER expands the feature space by concatenating independent extractors and DNE promotes feature reuse via cross-task attention, both approaches rely on the progressive addition of new structures tied to specific learning steps. The proposed approach, called Width Expansion (WE) and its Width Dynamic layers (WD), departs from this paradigm by enabling continuous adap- tation of learned representations, thereby allowing a more flexible balance between stability and plasticity: the model can both retain prior knowledge and refine its feature space as capacity expands. 3.1. Preliminaries and Loss Function The proposed method builds upon a multiclass classifi- cation objective based on the cross-entropy loss, defined as: ξΈ νΆνΈ = β νΎ β ν=1 ν¦ ν ννν(ν ν )(1) where ν¦ ν denotes the ground-truth label and ν ν is the pre- dicted probability for class ν, obtained through the softmax function: ν ν = ν ννν‘ννν₯(ν§ ν ) = ν ν§ ν β νΆ ν=1 ν ν§ν (2) with ν§ ν representing the logit associated with class ν. An important reference point in the Class-IL setting is the expected loss of a model that has not yet adapted to the classes introduced at the current incremental step. Under the assumption that such a model produces approximately uniform predictions over all observed classes, the expected cross-entropy loss is given by: ξΈ ννν₯ = νν(|ν νν_νννν νν |)(3) where|νΆ ν νν | denotes the total number of classes observed up to and including the current incremental step. This quan- tity serves as a scale-invariant upper bound on the loss, and is subsequently used to normalize the expansion criterion described in the following section. 3.2. Expansion Mechanism At the beginning of each incremental step, the modelβs capacity to represent newly introduced classes is assessed before any adaptation takes place. Specifically, the average cross-entropy loss is computed over the training samples of the current step using the model trained at the previous step, as described in Algorithm 1. This value reflects the degree to which the current architecture can accommodate the new classes without modification. Algorithm 1 Mean Loss Computation 1: νΏβ 0; νβ 0 2: for mini-batch (ν₯,ν¦) in the dataset do 3: νβ ννννν(ν₯) 4: νβ νννν‘ννννν(ν,ν¦) 5: νΏβ νΏ + ν β |ν₯|; νβ ν +|ν₯| 6: end for 7: νν£ν_ννν β νΏβν 8: return νν£ν_ννν To obtain a scale-invariant measure of representational saturation, this average loss is normalized byξΈ ννν₯ as defined in Equation 3, yielding a normalized lossν bounded between 0 and 1, as described in Algorithm 2. When the average loss remains below a predefined threshold, the model is assumed to have sufficient capacity to adapt to the new classes without architectural modification, and no expansion is performed. When expansion is required, it is carried out on selected lay- Algorithm 2 Normalized Global Loss Require: νν£ν_ννν , νΆ ν νν , ννν _ν‘βννν βννν 1: νΏ max β ln(|νΆ ν νν |) 2: νβ νν£ν_ννν βνΏ max 3: νβ min(ν, 1) 4: if νν£ν_ννν β€ ννν _ν‘βννν βννν then 5: return 6: end if 7: return ν ers using a combination of the global difficulty signal ν and local neuron utilization statistics. For each width-dynamic : Preprint submitted to ElsevierPage 3 of 15 layer (WD) νΏ, a local usage ratio ν’ is computed, reflecting how actively the layerβs neurons have been engaged during previous training steps. The expansion coefficient is then defined as: ν = νν€ ννν + ν’ β ν€ ννννν (4) where ν€ ννν and ν€ ννννν are weighting factors that control the relative influence of the global loss signal and local utilization estimate, respectively. The number of neurons to be added is proportional to the layer size and is constrained within a bounded range to prevent uncontrolled growth, as detailed in Algorithm 3. Algorithm 3 Expansion per Width Dynamic Layer Require: ν, ννννν, ν€ ννννν , ν€ ννν , νννν€ν‘β_νννν‘νν 1: for Dynamic Width Layer νΏ in ννννν do 2: ν’β νΏ.ν ν‘νν‘ν .ν’ν ννν_ννν‘νν() 3: νβ ν€ loss β ν + ν€ local β ν’ 4: νβ νΏ.νν’ν‘_νννν‘ν’ννν β νννν€ν‘β_νννν‘νν 5: νβ ννν’νν(ν β ν) 6: ν min β (0.05 β νΏ.νν’ν‘_νννν‘ν’ννν if ν > 0.5, else 0) 7: ν max β 0.75 β νΏ.νν’ν‘_νννν‘ν’ννν 8: νβ max(ν min , min(ν,ν max )) 9: if ν > 0 then 10:expand νΏ with ν neurons 11:update connections with other layers 12:update optimizer 13: end if 14: end for This procedure ensures that the expansion process is task-agnostic, bounded, and driven by representational de- mand rather than any task identifier, making it directly applicable to the Class-IL setting. 3.3. Attention Mechanism While width expansion increases the modelβs represen- tational capacity, it does not explicitly address interference that can arise between previously learned and newly intro- duced classes. To further stabilize representations during incremental updates, we incorporate a modified linear atten- tion mechanism inspired by Katharopoulos, Vyas, Pappas and Fleuret (2020), augmented with a persistent key-value memory. Let ν, ν and ν£ denote the input query, key, and value sequences. These inputs are first projected into a shared attention space through learnable linear transformations: ν = ν(ν ν ν), νΎ = ν(ν νΎ ν), ν = ν ν ν£(5) where ν ν , ν νΎ , and ν ν are learnable projection matrices and ν(β ) is a positive feature map defined as: ν(ν₯) = νΈνΏν(ν₯) + 1(6) where νΈνΏν(β ) is an exponential linear unit function (Clev- ert, Unterthiner and Hochreiter, 2016), which enables the ef- ficient linear attention formulation by ensuring non-negative kernel values. Projection q Projection k Projection v QKV K Q Ξ¦(Β·) concat with memory (K mem , V mem ) Linear Attention Figure 1: Attention Mechanism To provide a stable reference across incremental learning steps, a persistent memory module is introduced, composed of learnable key-value pairs: ν ννν ββ νΓν , ν£ ννν ββ νΓν where ν denotes the memory size and ν the attention dimensionality. These parameters are initialized using or- thogonal initialization to encourage diversity among stored representations. During the forward pass, the memory vec- tors are concatenated with the projected keys and values: νΎ β² = [νΎ;ν ννν ], ν β² = [ν ;ν£ ννν ] Given the augmented key-value set, the linear attention output is computed as: νΎν = νΎ β²ν ν β² (7) ν = 1 ν(νΎ β²ν ) (8) ν΄ν‘(ν,νΎ β² ,ν β² ) = (ν β νΎν ) β ν(9) This formulation reduces the quadratic complexity of stan- dard self-attention to linear complexity with respect to se- quence length, making it more suitable for the incremental learning setting. Notably, the persistent memory allows the model to attend to stable key-value representations consol- idated during previous training stages, thereby mitigating representational drift and helping to preserve previously acquired knowledge as new classes are introduced. : Preprint submitted to ElsevierPage 4 of 15 4. Experimental Setup The experiments in this section are designed to answer three interconnected questions. First, is dynamic width ex- pansion better than a fixed-capacity model retaining pre- viously acquired knowledge in the Class-IL setting? Sec- ond, does the proposed attention mechanism with persis- tent key-value memory complement width expansion, or does it provide independent benefit? Third, how do these architectural mechanisms interact with established continual learning strategies - regularization, functional, and replay- based - across benchmarks of varying complexity? To address these questions, we evaluate four architec- tural configurations - a fixed-capacity baseline, width ex- pansion alone (WE), attention alone, and their combination (WE + Attention) - across two benchmarks: Split MNIST, a controlled setting using multilayer perceptrons, and Split CIFAR-100, a more demanding visual recognition scenario using a convolutional network. In both cases, the same continual learning strategies (EWC, SI, LwF, LwM, ER, A-GEM) are applied under identical hyperparameter condi- tions, enabling direct comparison. Lower and upper bounds are provided by sequential training without mitigation and joint training on all data simultaneously, respectively. This structure allows us to isolate the contribution of each ar- chitectural component and assess its combined effect across strategy families and task complexities. 4.1. MNIST Setup The split MNIST benchmark (Hsu, Liu, Ramasamy and Kira, 2019; Van de Ven et al., 2022) provides a controlled incremental learning scenario in which the original MNIST dataset is partitioned into five sequential steps, each intro- ducing two new classes. During training, the model receives data exclusively from the current step and must learn to recognize the newly introduced classes while maintaining performance on all previously observed ones. No task iden- tity is provided during inference, following the standard Class-IL protocol. All models are trained for five epochs per incremental step using the Adam optimizer with a learning rate of 0.001 and momentum parameters ν½ 1 = 0.9 and ν½ 2 = 0.999. Train- ing is performed with mini-batches of 128 samples, while evaluation uses batches of 64 samples. The optimization objective is the cross-entropy loss, computed over the set of classes observed up to the current incremental step. For models incorporating the attention mechanism, the attention dimensionality is set to ν = 128, and both the key and value memory matrices ν ννν and ν£ ννν contain 32 vectors each. Two reference training regimes are considered to bound the expected performance range. The Lower Bound corre- sponds to sequential training without any forgetting miti- gation strategy, representing the worst-case scenario. The Upper Bound corresponds to joint training, an idealized regime in which the model is trained simultaneously on data from all steps, representing the performance ceiling that continual learning methods seek to approximate. To provide a comprehensive comparison, several widely used continual learning strategies are evaluated under iden- tical experimental conditions. For EWC and SI, the reg- ularization strength is set to ν = 10 9 ; while no univer- sally standard value exists for this hyperparameter, values determined empirically via log-scale grid search typically fall between 10 3 and 10 1 5 for both datasets in the Class- IL setting (Kruengkrai and Yamagishi, 2022; Van de Ven et al., 2022). For LwF, the distillation loss uses a temperature parameter ν = 2 and a weighting coefficient ν½ = 1, following the common formulation in literature. Replay- based methods maintain an episodic memory buffer of 1000 samples, storing up to 100 samples per class. In the case of A-GEM, the episodic memory also contains 100 samples per class, and reference gradients are computed using mini- batches of 128 samples drawn from the replay buffer. 4.1.1. Model Architectures Four model configurations are evaluated to isolate and analyze the individual and combined effects of width expan- sion and the proposed attention mechanism. Baseline. The baseline architecture is a multilayer percep- tron with two fully connected hidden layers, each contain- ing 400 neurons, followed by ReLU activations. The net- work takes a flattened MNIST image as input and produces intermediate feature representations, which are forwarded to a shared incremental classifier that predicts all classes observed up to the current step. The incremental classi- fier dynamically expands its output dimensionality as new classes are introduced throughout the learning process. The architecture is illustrated in Figure 3. Width Expansion (WE). The second configuration intro- duces the width expansion mechanism described in Section 3.2. The standard fully connected layers are replaced with width-dynamic (WD) layers, which are capable of increas- ing their number of neurons during training based on the normalized loss criterion and neuron utilization statistics. The network is initialized with the same configuration as the baseline model, containing two hidden layers of 400 neurons each. When the expansion criterion is triggered, new neurons are added to the hidden layers while preserving all previously learned weights and connections. The resulting architecture is illustrated in Figure 4. MLP with Attention. The third configuration augments the baseline architecture with the proposed linear attention module. In this architecture, two fully connected layers are first used to compute intermediate feature representations β 1 and β 2 : β 1 = ν ννΏν(ν 1 ν₯) β 2 = ν ννΏν(ν 2 β 1 ) The attention module then receives the second-layer repre- sentation as the query, while the first-layer representation : Preprint submitted to ElsevierPage 5 of 15 Figure 2: Example of Split MNIST and classification along the three types of incremental learning. ReLU Linear (784, 400) ReLU Linear (400,400) Incremental Classifier Figure 3: MLP model for MNIST ReLU WD (784, 400) ReLU WD (400,400) Incremental Classifier Figure 4: WE model for MNIST serves as both key and value: ν = β 2 , νΎ = β 1 , ν = β 1 Attention is computed as described in Section 3.3, pro- ducing an embedding of dimensionality ν = 128, which is subsequently projected back into the feature space via an additional fully connected layer before being passed to the incremental classifier. The architecture is illustrated in Figure 5. In this formulation, the attention module enables deeper representations to selectively focus on earlier ones, integrating information from multiple representation levels before generating the final feature vector for classification. Width Expansion with Attention. The fourth configu- ration combines the width expansion mechanism with the attention module described above. The standard fully con- nected layers used for feature extraction are replaced with width-dynamic layers, allowing the model to increase its representational capacity during incremental learning as de- scribed in Section 3.2. The attention module is applied in the same manner as in the previous configuration, with the second-layer serving as query and the first-layer representa- tion as both key and value. The attention output is mapped back into the feature space through a linear projection before being forwarded to the incremental classifier. The architec- ture is illustrated in Figure 6. This configuration is designed to combine two com- plementary mechanisms for continual learning: dynamic width expansion increases model capacity as new classes are introduced, while the attention module with persistent memory stabilizes representations across incremental steps, reducing interference between previously and newly learned classes. 4.2. CIFAR-100 Setup The Split CIFAR-100 benchmark (Krizhevsky; Rebuffi et al., 2017) is used to evaluate the proposed approach in a more challenging visual recognition scenario. The original CIFAR-100 dataset is partitioned into 10 sequential incre- mental steps, each introducing 10 new classes. As in the Split MNIST protocol, the model receives data exclusively from the current step during training and must maintain perfor- mance on all previously observed classes without access to task identity during inference. All models are trained using the Adam optimizer with a learning rate of 0.001 and momentum parameters ν½ 1 = 0.9 and ν½ 2 = 0.999. Training is performed using mini-batches of 256 samples, while evaluation uses mini-batches of 128 samples. Each incremental step is trained for 30 epochs, ex- cept for the upper-bound joint-training configuration, which : Preprint submitted to ElsevierPage 6 of 15 Feature Extractor and Feature Separation to Attention Mechanism Linear (784, 400) Linear (400,400) KVQ ReLU Attention Mechanism (128,32) Linear (128,400) Incremental Classifier ReLU Figure 5: MLP model with attention mechanism for MNIST Feature Extractor and Feature Separation to Attention Mechanism WD (784, 400) WD (400,400) KVQ ReLU Attention Mechanism (128,32) WD (128,400) Incremental Classifier ReLU Figure 6: WE model with attention mechanism for MNIST : Preprint submitted to ElsevierPage 7 of 15 Figure 7: Sample images from the CIFAR-100 dataset illustrat- ing the diversity of object categories and visual appearances. is trained on the complete dataset for the same number of epochs. The optimization objective is the cross-entropy loss, computed over all classes observed up to the current incremental step. For models that incorporate the attention mechanism, the attention dimensionality is set to ν = 256, and both the spatial and feature attention memory modules contain 64 key-value vectors each. The same continual learning strategies evaluated on split MNIST are considered here under identical hyperparameter conditions: EWC and SI use a regularization coefficient ν = 10 9 , LwF and LwM use distillation temperature ν = 2, and weighting coefficient ν½ = 1. For replay-based methods, an episodic memory with 10,000 samples is maintained with a maximum of 100 samples per class. In the case of A- GEM, the episodic memory also stores 100 samples per class, and reference gradients are computed using batches of 256 samples drawn from the replay buffer. 4.2.1. Model Architectures Four model configurations are evaluated in the convolu- tional setting to assess the impact of width expansion and the attention mechanism under the greater visual complexity of CIFAR-100. Baseline. The baseline architecture comprises a convolu- tional feature extractor, two fully connected layers, and a shared incremental classifier. The convolutional backbone consists of five blocks with 3 Γ 3 kernels, each followed by batch normalization and ReLU activation. Strided convolu- tions progressively reduce the spatial resolution from 32Γ32 to 4Γ4 while increasing the number of feature channels from 3 to 256 with the first layer mapping to 16 channels and each subsequent doubling that count. The resulting feature map is flattened and processed by two fully connected layers with 400 neurons each, producing the final representation used by the incremental classifier. The convolutional backbone is shared across all model configurations evaluated on CIFAR- 100, and the full baseline architecture is illustrated in Figures 8 and 9. Width Expansion (WE). The second configuration intro- duces the width expansion mechanism in the convolutional setting. The fully connected layers responsible for feature extraction are replaced by width-dynamic layers, which can increase their number of neurons during training based on the normalized loss and neuron utilization statistics. The model is initialized with the same configuration as the base- line, and new neurons are added to the hidden layers when the expansion criterion is triggered, while previously learned weights and connections are preserved. The resulting archi- tecture is illustrated in Figure 10. CNN with Attention. The third configuration augments the baseline convolutional architecture with two attention modules. The first operates over the spatial feature maps produced by the final convolutional layer. The feature tensor ν₯ ββ ν΅ΓνΆΓν»Γν is reshaped into a sequence ofν»β ν spatial tokens of dimensionality νΆ, over which linear self-attention is applied, allowing the model to capture interactions be- tween spatial regions of the image. The resulting attention output is reshaped back to the original spatial format and combined with the input feature map through a residual connection, followed by batch normalization. After flattening, two fully connected layers produce in- termediate representations β 1 and β 2 . A second attention module is then applied in the feature space, following the same cross-level formulation used in the MNIST setting: ν = β 2 , νΎ = β 1 , ν = β 1 This mechanism allows the deeper representations to selec- tively attend to early feature representations while leveraging the persistent key-value memory described in Section 3.3. The attention output is projected back into the feature space through an additional linear layer before being forwarded to the incremental classifier. The full architecture is illustrated in Figure 11. Width Expansion with Attention. The final configuration integrates both the width expansion mechanism and the dual attention modules described above. The fully connected layers used for feature extraction are replaced with width- dynamic layers, allowing the model to increase its represen- tational capacity as new classes are introduced, while both the spatial and feature attention modules are retained. This configuration is designed to jointly exploit two complemen- tary strategies: dynamic capacity growth, which provides additional representational resources when demanded, and attention-based stabilization, which mitigates representa- tional drift and interference across incremental steps. The architecture is illustrated in Figure 12. : Preprint submitted to ElsevierPage 8 of 15 Convolution Convolution Convolution ConvolutionConvolution 3@32x32 16@32x32 32@17x17 64@10x10 128@6x6 256@4x4 Figure 8: Common feature extractor for CIFAR-100 models Convolutional and Fully Connected Layers Flatten Feature Extractor Incremental Classifier ReLU Linear (4096, 400) ReLU Linear (400, 400) Figure 9: CNN model for CIFAR-100 5. Results The experimental results reveal clear and consistent pat- terns across benchmarks, learning strategies, and architec- tural configurations. Taken together, they highlight the com- plementary roles of width expansion and the attention mech- anism in addressing catastrophic forgetting within the Class- IL setting. 5.1. Split MNIST As shown in Table 1, the results on Split MNIST ex- pose a sharp divide between the evaluated strategy fami- lies. Replay-based methods, particularly Exemplar Replay, achieve accuracy values close to the upper bound, sub- stantially outperforming all other approaches. In contrast, regularization-based methods - EWC and SI - fail to pro- vide meaningful mitigation of catastrophic forgetting, with results comparable to the lower bound regardless of the architectural configuration employed. This outcome is con- sistent with prior findings suggesting that parameter-based Convolutional and Fully Connected Dynamic Layers Flatten Feature Extractor Incremental Classifier ReLU WD (4096, 400) ReLU WD (400, 400) Figure 10: CNN model for CIFAR-100 with width dynamic layers : Preprint submitted to ElsevierPage 9 of 15 B,H*W,C Feature Extractor Sum Flatten ReLU Attention Mechanism (256,64) Linear (256, 400) Incremental Classifier Feature Extractor and Feature Separation to Attention Mechanism ReLU Linear (4096, 400) Linear (400, 400) K VQ Convolutional and Self-Attention with the proposed memory Batch Normalization Attention Mechanism (256,64) Self Attention Out_Att (B, C, H, W) Reshape Figure 11: CNN model with attention mechanism for CIFAR-100 B,H*W,C Feature Extractor Sum Flatten ReLU Attention Mechanism (256,64) WD (256, 400) Incremental Classifier Feature Extractor and Feature Separation to Attention Mechanism ReLU WD (4096, 400) WD (400, 400) K VQ Convolutional and Self-Attention with the proposed memory Batch Normalization Attention Mechanism (256,64) Self Attention Out_Att (B, C, H, W) Reshape Figure 12: CNNWE model with attention mechanism for CIFAR-100 constraints are poorly suited to the Class-IL setting, where all classes share a common output space and the required plasticity tends to exceed what these methods allow. Among functional regularization methods, LwF demon- strates a considerably more favorable response to the pro- posed architectural modifications. The combination of width expansion and attention achieves the highest accuracy within this group, reaching 44.04%, compared to 29.79% for the fixed MLP baseline. This improvement reflects the con- tribution of both mechanisms: width expansion provides additional representational capacity as new classes are intro- duced, while the attention module with persistent memory stabilizes feature representations across incremental steps, reducing the interference that distillation alone cannot fully prevent. Notably, the attention mechanism isolation intro- duces grater training variability, as evidenced by the higher standard deviation observed for the MLP with Attention con- figuration under LwF. When combined with width expan- sion, however, the model becomes more stable, suggesting that the two mechanisms interact constructively rather than independently. A-GEM also benefits consistently from the proposed modifications. Accuracy increases from 28.34% for the fixed MLP to 41.25% for the width expansion with attention configuration, with width expansion contributing a mean- ingful share of this gain even before the attention module is incorporated. As illustrated in Figure 13, the improve- ments obtained by architectural enhancements are more pronounced for the LwF than for A-GEM. This asymme- try is interpretable: replay-based methods already mitigate forgetting through explicit sample reuse, and the marginal benefit of representational improvements is therefore smaller than for functional methods, which rely entirely on structural mechanisms to preserve prior knowledge. : Preprint submitted to ElsevierPage 10 of 15 StrategyMLPWEMLP + AttentionWE + Attention Joint97.76 Β±0.12--- None19.62 Β±0.07--- EWC19.60 Β±0.0919.41 Β±0.1018.67 Β±3.0619.75 Β±0.17 SI19.63 Β±0.0519.39 Β±0.1119.07 Β±2.5919.68 Β±0.04 LwF29.79 Β±0.8529.28 Β±2.0039.63 Β±6.2544.04 Β±3.93 ER88.80 Β±0.7383.44 Β±0.9356.74 Β±33.1287.74 Β±0.60 A-GEM28.34 Β±8.5433.97 Β±8.5538.38 Β±14.7341.25 Β±4.93 Table 1 Results for Split MNIST Values represent Mean Β± Standard Deviation MLP WE MLP-Attention WE-Attention LwF 203040506070 Accuracy (%) MLP WE MLP-Attention WE-Attention A-GEM Figure 13: Comparison between LwF and A-GEM on Split- Mnist It is worth noting that ER exhibits an unexpected drop in performance under the MLP with attention configuration, with a large associated standard deviation. This instability likely to reflects the sensitivity of the attention module to training dynamics in the absence of additional capacity, and is substantially reduced when attention is combined with width expansion. 5.2. Split CIFAR-100 without Pretraining As shown in Table 2, all methods experience a significant reduction in performance on Split CIFAR-100, reflecting the considerably greater visual complexity of this benchmark relative to Split MNIST. Regularization-based approaches again fail to provide meaningful improvements over the lower bound, and this pattern holds across all architectural configurations, reinforcing the conclusion that parameter- based constraints are insufficient to address the challenges of Class-IL in complex visual settings. The proposed width expansion yields consistent im- provements across all functional and replay-adjacent strate- gies. The effect is particularly pronounced for A-GEM, where accuracy increases from 11.72% to 17.91% with width expansion alone, and reaches 25.17% when the attention mechanism is added - the best result among all methods that not rely on direct sample replay. This progression suggests that the two mechanisms provide complementary benefits: expansion increases the representational resources available for new classes, while the attention module mitigates the drift in previously learned representations that expansion alone cannot prevent. For LwF, width expansion also yields a consistent im- provement, with accuracy increasing from 12.58% to 14.20% under expansion alone, and to 17.91% when attention is incorporated. As illustrated in Figure 14, the attention mech- anism has a stronger impact when combined with width expansion than when applied in isolation, mirroring the pattern observed on Split MNIST and further supporting the interpretation that attention-based stabilization is most effec- tive when sufficient representational capacity is available. Among replay-based methods, ER shows no meaningful benefit from width expansion or attention, with results re- maining stable across all architectural configurations. This is consistent with the earlier observation that methods rely- ing on stored samples are less sensitive to representational improvements, since explicit rehearsal already provides a strong mechanism for preserving prior knowledge. 5.3. Split CIFAR-100 with Pretraining Table 3 reports results for the convolutional setting in which the backbone is initialized with weights pretrained on CIFAR-10 and subsequently frozen during incremental training. Although pretraining might intuitively be expected to provide a stronger representational foundation, the results show that models trained from scratch outperform their pretrained counterparts across nearly all strategies and con- figurations. : Preprint submitted to ElsevierPage 11 of 15 StrategyCNNWECNN + AttentionWE + Attention Joint48.56 Β±0.51--- None8.05 Β±0.17--- EWC6.46 Β±0.347.27 Β±0.116.49 Β±0.307.53 Β±0.11 SI5.82 Β±0.168.43 Β±0.085.89 Β±0.178.30 Β±0.09 LwF12.58 Β±0.3014.20 Β±0.3415.13 Β±0.5617.91 Β±0.90 LwM10.64 Β±0.2612.52 Β±0.3810.39 Β±0.8512.18 Β±0.86 ER33.37 Β±0.9431.21 Β±0.6231.72 Β±0.8832.41 Β±0.63 A-GEM11.72 Β±3.3817.91 Β±3.8414.39 Β±3.5525.17 Β±6.50 Table 2 Results for Split CIFAR-100 without pretraining Values represent Mean Β± Standard Deviation StrategyCNNWECNN + AttentionWE + Attention Joint44.09 Β±0.40--- None8.04 Β±0.06--- EWC7.58 Β±0.247.64 Β±0.177.83 Β±0.147.86 Β±0.15 SI7.35 Β±0.117.38 Β±0.065.61 Β±0.215.65 Β±0.22 LwF14.19 Β±0.3214.43 Β±0.4016.13 Β±0.5916.41 Β±0.69 ER31.72 Β±0.2631.60 Β±0.5633.38 Β±0.5733.28 Β±0.38 A-GEM8.10 Β±0.148.15 Β±0.1013.23 Β±1.6715.37 Β±4.94 Table 3 Results for Split CIFAR-100 with pretraining values represent Mean Β± Standard Deviation This counterintuitive outcome can be attributed to the representational mismatch between CIFAR-10 and CIFAR- 100. Because the convolutional layers are frozen after pre- training, the model is constrained to a fixed feature space that was optimized for a simpler and categorically different distri- bution. The resulting representations lack the expressiveness required for the finer-grained distinctions among CIFAR- 100 classes, forcing the classifier and the proposed expan- sion mechanisms to compensate for suboptimal features - a task that fundamentally limits achievable performance. Pretraining does, however, reduce training variability: pretrained models exhibit smaller standard deviations across runs, acting as a strong implicit regularizer. This stability comes at a clear cost to plasticity, as frozen convolutional layers cannot adapt their representations to the new data distribution encountered during incremental learning. The proposed width expansion, which operates exclusively on the fully connected layers in this configuration, can only partially compensate for the rigidity of the fixed feature extractor. As illustrated in Figures 15 and 16, pretrained models consistently underperform their non-pretrained counterparts across both LwF and A-GEM, with the gap particularly evident in the A-GEM setting. This pattern reinforces a central conclusion of the study: in the Class-IL setting, the ability to adapt feature representations incrementally is more important than the quality of the initial representation space. A well-initialized but fixed feature extractor introduces con- straints that ultimately hinder the kind of flexible adaptation that continual learning demands. 6. Threats to Validity A critical assessment of this studyβs methodology and results reveals several potential threats to the validity of its conclusions. Acknowledging these limitations is essential for contextualizing the findings and guiding future research. 6.1. Internal Validity Internal validity concerns whether the observed im- provements can be attributed to the proposed mechanisms rather than confounding factors. Several design choices sup- port this claim. Hyperparameters for the baseline strategies (ν = 10 9 for EWC and SI, ν = 2 and ν½ = 1 for LwF and LwM, buffer sizes of 100 samples per class) were set fol- lowing established practice in the Class-IL literature (Van de Ven et al., 2022; Kruengkrai and Yamagishi, 2022) and held constant across all architectural configurations, ensuring that strategy-level differences do not inflate observed gains. All results are reported as means and standard deviations across multiple independent runs with different random seeds, pro- viding a measure of result stability. However, the expansion- specific hyperparameters - including loss threshold, growth factor, and weighting coefficients ν€ ννν and ν€ ννννν - were not subjected to systematic held-out grid search, and their interaction with specific strategy-architecture combinations cannot be fully ruled out as a source of variance. The high standard deviation observed for A-GEM under some configurations (e.g., 14.73 on Split MNIST under MLP + Attention) suggests that certain combinations are sensitive to initialization or training dynamics in ways not fully captured by the reported statistics. : Preprint submitted to ElsevierPage 12 of 15 CNN WE CNN-Attention WE-Attention LwF 101520253035 Accuracy (%) CNN WE CNN-Attention WE-Attention A-GEM Figure 14: Comparison between LwF and A-GEM on Split CIFAR-100 6.2. External Validity External validity concerns the generalization of findings beyond the evaluated conditions. Two benchmarks of sub- stantially different complexity were used: Split MNIST, a controlled setting, and Split CIFAR-100, a more demand- ing visual recognition task. This range provides meaning- ful coverage of the Class-IL landscape. Nevertheless, both benchmarks use balanced class splits of equal and fixed size, a condition that may not hold in real-world deployments, where class arrival frequency, distributional shift, and task duration may vary unpredictably. The convolutional back- bone used for CIFAR-100 is relatively shallow compared to architectures used in recent state-of-the-art work; perfor- mance on large-scale benchmarks such as Split ImageNet, or in settings based on pre-trained transformer backbones, cannot be directly inferred from the present results. Addi- tionally, all evaluations assume a fixed incremental schedule with clearly delineated steps; behavior of the expansion mechanism under irregular, fine-grained, or blurred task boundaries remains an open question. CNN WE CNN-Attention WE-Attention LwF (with Pretraining) 12141618 Accuracy (%) CNN WE CNN-Attention WE-Attention LwF (without Pretraining) Figure 15: LwF comparison between with and without pre- training. 6.3. Construct Validity Construct validity concerns whether the evaluation met- rics adequately capture the properties of interest. Average classification across all observed classes at the end of train- ing is used as the primary metric, consistent with the stan- dard Class-IL evaluation protocol (Van de Ven et al., 2022). This metric, however, does not distinguish between forget- ting of early classes and failure to learn later ones - two failure modes with distinct implications for the stability- plasticity trade-off. Furthermore, it does not capture the computational cost incurred by dynamic expansion, which is a practically relevant consideration in resource-constrained deployment scenarios. Future evaluations should comple- ment accuracy with backward transfer, forward transfer, and parameter count over time to provide a more complete characterization of each mechanismβs contribution. 7. Conclusion This work investigated catastrophic forgetting in Class- IL through the lens of architectural adaptability, proposing a method based on dynamic width expansion within existing : Preprint submitted to ElsevierPage 13 of 15 CNN WE CNN-Attention WE-Attention Agem (with Pretraining) 101520253035 Accuracy (%) CNN WE CNN-Attention WE-Attention Agem (without Pretraining) Figure 16: A-GEM comparison between with and without pretraining. network layers, complemented by a linear attention mech- anism with a persistent key-value memory. Unlike network expansion approaches such as DER and DNE, which grow model capacity through the addition of task-specific mod- ules and therefore depend on explicit task identifiers, the proposed method expands representational capacity within existing layers based on a normalized cross-entropy criterion that reflects representational demand. This design makes the approach directly applicable to the Class-IL setting, where task boundaries are not available during inference. Experimental results across Split MNIST and Split CIFAR-100 demonstrate that the combination of width expansion and the attention mechanism provides consistent improvements in knowledge retention. On Split MNIST, the width expansion with the attention module configuration achieves the highest accuracy among functional regulariza- tion methods, reaching 44.04%, while on Split CIFAR-100, the same configuration yields 25.17% - the best result among methods that do not rely on direct sample replay. Across both benchmarks, the attention mechanism with persistent key-value memory plays a central role in stabilizing feature representations and mitigating representational drift during incremental updates. The results also reveal that the two mechanisms interact constructively: attention-based stabi- lization is most effective when combined with sufficient representational capacity, and width expansion alone yields more variable results in the absence of the representational anchoring provided by the persistent memory. A recurring and significant finding across all experi- ments is that adaptability in feature extraction layers proves more consequential than the quality of the initial repre- sentation space. Models trained from scratch consistently outperformed those with pretrained and frozen convolu- tional backbones, as the latter introduce representational constraints that limit the plasticity required to accommodate new data distributions over time. This result suggests that, in complex continual learning scenarios, the capacity to revise learned representations incrementally should be prioritized over initialization-based stability. Several directions emerge from the present work. First, the expansion mechanism currently operates on fully con- nected layers; extending it to convolutional layers could allow the model to adapt its feature extraction capacity di- rectly, rather than relying solely on downstream dense layers to compensate for representational bottlenecks. Second, se- lective post-expansion pruning could recover computational efficiency without sacrificing accuracy, addressing the risk of unbounded parameter growth observed in the current results. Third, the normalized global loss criterion used to trigger expansion could be replaced by layer-wise or neuron- wise importance signals, enabling finer-grained allocation of representational resources across the network. Finally, eval- uation on larger-scale and class-imbalanced benchmarks, as well as in settings with irregular overlapping task bound- aries, would provide a more complete picture of the ap- proachβs applicability to real-world continual learning sce- narios. CRediT authorship contribution statement AndrΓ© L. S. Conde: Writing β original draft, Visualiza- tion, Methodology, Investigation, Software, Formal analy- sis, Conceptualization. Yehia Elkhatib: review & editing, Investigation, Validation. Cateano M. Ranieri: review & editing, Investigation, Project administration, Supervision, Funding acquisition. Acknowledgements This work was partially supported by the SΓ£o Paulo Research Foundation (FAPESP), grant 2025/13241-3. The Article Processing Charge (APC) was funded by the Brazil- ian Federal Agency for Support and Evaluation of Graduate Education β CAPES (ROR identifier: 00x0ma614). For the purposes of open access, the authors have applied a Creative Commons C BY license to any accepted version of the article. : Preprint submitted to ElsevierPage 14 of 15 References Cao, L., 2022. Beyond iid: Non-iid thinking, informatics, and learning. IEEE Intelligent Systems 37, 5β17. Chaudhry, A., Ranzato, M., Rohrbach, M., Elhoseiny, M., 2019. Efficient lifelong learning with a-GEM, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id=Hkf2_sC5FX. Clevert, D.A., Unterthiner, T., Hochreiter, S., 2016. Fast and Accu- rate Deep Network Learning by Exponential Linear Units (ELUs). URL: http://arxiv.org/abs/1511.07289, doi:10.48550/arXiv.1511.07289. arXiv:1511.07289 [cs]. De Lange, M., Tuytelaars, T., 2021. Continual Prototype Evolution: Learn- ing Online from Non-Stationary Data Streams, in: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, Montreal, QC, Canada. p. 8230β8239. URL: https://ieeexplore.ieee.org/ document/9711397/, doi:10.1109/ICCV48922.2021.00814. Dhar, P., Singh, R.V., Peng, K.C., Wu, Z., Chellappa, R., 2019. Learning without memorizing, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5138β5146. Dong, M., Zhang, Z., Tan, X., Qiu, J., Xie, Y., 2026. Dynamic expan- sion orthogonal network for class-incremental learning. Knowledge- Based Systems 341, 115868. URL: https://linkinghub.elsevier.com/ retrieve/pii/S0950705126005940, doi:10.1016/j.knosys.2026.115868. Fu, Z., Wang, Z., Xu, X., Li, D., Yang, H., 2023. Knowledge aggre- gation networks for class incremental learning. Pattern Recognition 137, 109310. URL: https://linkinghub.elsevier.com/retrieve/pii/ S0031320323000110, doi:10.1016/j.patcog.2023.109310. Hadsell, R., Rao, D., Rusu, A.A., Pascanu, R., 2020. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences 24, 1028β1040. Hsu, Y.C., Liu, Y.C., Ramasamy, A., Kira, Z., 2019. Re-evaluating Contin- ual Learning Scenarios: A Categorization and Case for Strong Baselines. URL: http://arxiv.org/abs/1810.12488, doi:10.48550/arXiv.1810.12488. arXiv:1810.12488 [cs]. Hu, Z., Li, Y., Lyu, J., Gao, D., Vasconcelos, N., 2023. Dense network ex- pansion for class incremental learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 11858β11867. Hussain, A., Ullah, W., Khan, N., Khan, Z.A., Yar, H., Baik, S.W., 2026. Class-incremental learning network for real-time anomaly recognition in surveillance environments.Pattern Recognition 170, 112064. URL: https://linkinghub.elsevier.com/retrieve/pii/ S0031320325007241, doi:10.1016/j.patcog.2025.112064. Jiang, K., Bai, X., Zhou, F., 2026. Recurrent Network Expansion for Class Incremental Learning. IEEE Transactions on Neural Networks and Learning Systems 37, 122β135. URL: https://ieeexplore.ieee.org/ document/11142762/, doi:10.1109/TNNLS.2025.3601373. Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F., 2020. Transform- ers are RNNs: Fast autoregressive transformers with linear attention, in: I, H.D., Singh, A. (Eds.), Proceedings of the 37th International Conference on Machine Learning, PMLR. p. 5156β5165. URL: https: //proceedings.mlr.press/v119/katharopoulos20a.html. Kim, D., Han, B., 2023. On the stability-plasticity dilemma of class- incremental learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 20196β20204. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., Hadsell, R., 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114, 3521β3526. URL: https://w.pnas. org/doi/abs/10.1073/pnas.1611835114, doi:10.1073/pnas.1611835114, arXiv:https://w.pnas.org/doi/pdf/10.1073/pnas.1611835114. Krizhevsky, A., . Learning Multiple Layers of Features from Tiny Images . Kruengkrai, C., Yamagishi, J., 2022. Mitigating the Diminishing Effect of Elastic Weight Consolidation, in: Calzolari, N., Huang, C.R., Kim, H., Pustejovsky, J., Wanner, L., Choi, K.S., Ryu, P.M., Chen, H.H., Donatelli, L., Ji, H., Kurohashi, S., Paggio, P., Xue, N., Kim, S., Hahm, Y., He, Z., Lee, T.K., Santus, E., Bond, F., Na, S.H. (Eds.), Proceedings of the 29th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Gyeongju, Re- public of Korea. p. 4568β4574. URL: https://aclanthology.org/2022. coling-1.403/. Li, Y., Dou, H., Li, G., Gao, G., Zhou, H., 2026. INSERTION: From traditional incremental learning to open-world stream learning. Pattern Recognition 176, 113163. URL: https://linkinghub.elsevier.com/ retrieve/pii/S0031320326001287, doi:10.1016/j.patcog.2026.113163. Li, Z., Hoiem, D., 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40, 2935β2947. Liu, H., Zhou, Y., Liu, B., Zhao, J., Yao, R., Shao, Z., 2023. Incremental learning with neural networks for computer vision: a survey. Artificial Intelligence Review 56, 4557β4589. URL: https://doi.org/10.1007/ s10462-022-10294-2, doi:10.1007/s10462-022-10294-2. Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H., 2017. icarl: Incremental classifier and representation learning, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, p. 2001β2010. Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., Wayne, G., 2019. Ex- perience replay for continual learning, in: Wallach, H., Larochelle, H., Beygelzimer, A., d'AlchΓ©-Buc, F., Fox, E., Garnett, R. (Eds.), Ad- vances in Neural Information Processing Systems, Curran Associates, Inc. URL: https://proceedings.neurips.c/paper_files/paper/2019/ file/fa7cdfad1a5aaf8370ebeda47a1f1c3-Paper.pdf. Van de Ven, G.M., Tuytelaars, T., Tolias, A.S., 2022. Three types of incremental learning. Nature Machine Intelligence 4, 1185β1197. Yan, S., Xie, J., He, X., 2021. Der: Dynamically expandable represen- tation for class incremental learning, in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3013β3022. doi:10.1109/CVPR46437.2021.00303. Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., Van De Weijer, J., 2020. Semantic Drift Compensation for Class- Incremental Learning, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Seattle, WA, USA. p. 6980β6989. URL: https://ieeexplore.ieee.org/document/9156964/, doi:10.1109/CVPR42600.2020.00701. Zenke, F., Poole, B., Ganguli, S., 2017. Continual learning through synaptic intelligence. Proc Mach Learn Res 70, 3987β3995. Zhao, B., Xiao, X., Gan, G., Zhang, B., Xia, S.T., 2020. Maintain- ing Discrimination and Fairness in Class Incremental Learning, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), p. 13205β13214. URL: https://ieeexplore.ieee. org/document/9156766/, doi:10.1109/CVPR42600.2020.01322. iSSN: 2575- 7075. Zhou, D.W., Sun, H.L., Ye, H.J., Zhan, D.C., 2024a. Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning, in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Seattle, WA, USA. p. 23554β 23564. URL: https://ieeexplore.ieee.org/document/10656913/, doi:10. 1109/CVPR52733.2024.02223. Zhou, D.W., Wang, F.Y., Ye, H.J., Ma, L., Pu, S., Zhan, D.C., 2022. Forward Compatible Few-Shot Class-Incremental Learning, in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, New Orleans, LA, USA. p. 9036β9046. URL: https: //ieeexplore.ieee.org/document/9878986/, doi:10.1109/CVPR52688.2022. 00884. Zhou, D.W., Wang, Q.W., Qi, Z.H., Ye, H.J., Zhan, D.C., Liu, Z., 2024b. Class-incremental learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 9851β9873. doi:10.1109/TPAMI. 2024.3429383. : Preprint submitted to ElsevierPage 15 of 15