Paper deep dive
In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models
Benjamin Smith, Levin Kuhlmann, Kaushik Roy, Gideon Kowadlo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/21/2026, 3:14:16 AM
Summary
The paper introduces 4MAS (4 Module Awake/Sleep), a novel macroarchitecture for continual learning that mimics biological memory consolidation. It utilizes two asymmetric hemispheres (Left for stability/routine, Right for plasticity/exploration), each with dedicated Long-Term Memory (generative VAEs) and Short-Term Memory buffers. The system alternates between an 'awake' phase for learning new tasks and a 'sleep' phase for cross-hemispheric consolidation to prevent catastrophic forgetting and representational drift. 4MAS achieves competitive results on Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR-100 datasets.
Entities (15)
Relation Signals (12)
4MAS → achievesaccuracyon → Split-MNIST
confidence 95% · 4MAS achieves 98.3% accuracy on Split-MNIST
4MAS → achievesaccuracyon → Split-Fashion-MNIST
confidence 95% · 4MAS... achieves 84.9%... on Split-Fashion-MNIST
4MAS → achievesaccuracyon → Split CIFAR-100
confidence 95% · 4MAS... improves on the unilateral B-IR baseline... 29.29%... on Split-CIFAR-100
Benjamin Smith → authored → 4MAS
confidence 95% · Benjamin Smith... propose 4MAS
4MAS → solves → Catastrophic Forgetting
confidence 95% · 4MAS demonstrates strong resilience to catastrophic forgetting... preventing the drift that typically destabilises single-generator networks.
4MAS → consistsof → Long-Term Memory (LTM)
confidence 90% · Each hemisphere contains: 1. A generative Long-Term Memory (LTM) representing the neocortex
4MAS → consistsof → Short-Term Memory (STM)
confidence 90% · 2. A Short-Term Memory (STM) buffer representing the hippocampus
4MAS → uses → Left Hemisphere
confidence 88% · 4MAS employs two generative long-term memories as an asymmetrical ensemble... Left Hemisphere... configured for stability
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves competitive results on the Split-MNIST, Split-Fashion-MNIST and Split-CIFAR-100 datasets, with 98.3%, 84.9%, and 29.29% accuracy respectively.
Tags
Links
- Source: https://arxiv.org/abs/2608.19514v1
- Canonical: https://arxiv.org/abs/2608.19514v1
Trouble viewing inline? Open PDF directly →
Full Text
59,337 characters extracted from source content.
Expand or collapse full text
Graphical Abstract In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models Benjamin Smith, Levin Kuhlmann, Kaushik Roy, Gideon Kowadlo Modelling biological features of memory consolidation: hippocampal replay, REM sleep, and bilateral redundancy Effective protection against forgetting Split-MNIST: 96.2% Split-Fashion-MNIST: 80.2% Latent spaces with synthetic sample confidence measures The problem: catastrophic forgetting Auto-regressive asymmetric models representing each hemisphere Training on new data while awake; self- training consolidation during sleep In Two Minds about Lifelong Learning Exploring Hemispheric Redundancy and Specialisation in Neural Models Training time Performance Task A Task B Emergent behaviour of hemispheric specialisation (Novelty vs Routinisation) arXiv:2608.19514v1 [cs.LG] 20 Aug 2026 In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models Benjamin Smith a,∗ , Levin Kuhlmann a , Kaushik Roy c , Gideon Kowadlo 1,b a Data Science and AI, Monash University, Wellington Road, Clayton, 3800, Victoria, Australia b Cerenaut, Victoria, Australia c CSIRO Robotics, CSIRO, Australia Abstract Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typ- ically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavail- able due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without ex- periencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolida- tion. We focus on three concepts: experience replay, REM sleep, and bilater- ality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architec- ture achieves competitive results on the Split-MNIST, Split-Fashion-MNIST and Split-CIFAR-100 datasets, with 98.3%, 84.9%, and 29.29% accuracy re- ∗ Corresponding author spectively. Keywords: Continual Learning, Lifelong Learning, Experience Replay, Bilateral Deep Learning 1. Introduction Contemporary artificial neural networks suffer from catastrophic forget- ting (or catastrophic interference) [1, 2], where learning new tasks rapidly erodes previously acquired skills. While standard training assumes indepen- dent, identically distributed (i.i.d.) data, real-world applications present se- quentially correlated distributions where class presentation frequencies vary arbitrarily. This forgetting stems from the stability-plasticity dilemma: adapting weights to new data modifies parameters that are critical for earlier tasks [3, 4]. Joint training across all data streams is fundamentally impossible in continual learning settings where future data are unavailable and past data are restricted by storage or privacy constraints. Naive sequential fine-tuning, while feasible, suffers from severe catastrophic forgetting. Continual learning algorithms seek to resolve this dilemma without full dataset retraining. In contrast, biological brains mitigate forgetting through specialised neu- ral architectures and consolidation states. The mammalian hippocampus uses experience replay to weave together recent and distant memories, trans- ferring knowledge to the neocortex during sleep [5, 6, 7]. Furthermore, sleep phases like rapid-eye-movement (REM) sleep exhibit highly coordinated bi- lateral (inter-hemispheric) activity, which contrasts with the unihemispheric states of slow-wave sleep [8, 9]. Finally, there is evidence that biological brains leverage two lateralised cortical hemispheres that acquire distinct rep- resentations, with the right hemisphere handling novel information and the left hemisphere optimised for stable routine tasks [10, 11]. Existing generative replay methods such as Generative Replay struggle to scale because a single generator must capture all historical distributions, leading to compounding representational drift [12]. To address these limita- tions, we propose 4MAS (4 Module Awake/Sleep), a novel continual learn- ing macroarchitecture that implements two asymmetric hemispheres—each with dedicated short- and long-term memory modules—and an explicit of- fline sleep phase for cross-hemispheric consolidation; representing hippocam- pus and neocortex respectively. By distributing replay across two specialised 3 generators and anchoring the latent space during sleep, 4MAS minimises representational drift. Our main contributions are: • We propose a dual-hemisphere continual learning architecture (4MAS) that splits generative replay across specialised exploratory and conser- vative models, mimicking biological lateralisation. • We introduce an explicit wake-sleep training cycle that uses cross- hemispheric consolidation (mutual fine-tuning during a simulated sleep phase) to stabilise latent spaces and prevent representational drift. • We demonstrate that 4MAS achieves competitive Class-IL accuracies on Split-MNIST (98.3%), Split-Fashion-MNIST (84.9%), and Split- CIFAR-100 (29.29%); while displaying extremely low representational drift across tasks. 1.1. Biological motivation Inspired by this neurobiological template, 4MAS employs two generative long-term memories as an asymmetrical ensemble: one retains plasticity for novel task acquisition, while the other stabilises historical patterns. Gen- erative memories are paired with short-term memory buffers, allowing for rehearsal akin to hippocampal replay. Training follows an ultradian-inspired cycle, alternating between a unihemispheric awake phase for task learning and a bilateral sleep phase for cross-hemispheric memory harmonisation. 2. Background and related work Continual learning is typically evaluated under three scenarios [13] of increasing difficulty: • Task-IL (Task Incremental Learning): The model learns a se- quence of tasks with distinct data distributions and separate output spaces. The task identity is explicitly provided during both training and inference, allowing the model to select task-specific parameters or heads. • Domain-IL (Domain Incremental Learning): The model is ex- posed to a sequence of tasks with varying input distributions that share a common output space. The task identity is never provided, requiring the model to adapt to changing domains without contextual routing. 4 • Class-IL (Class Incremental Learning): The most challenging paradigm, where both input distributions and output spaces vary across tasks. No task identifiers are available at training or inference. The model must classify inputs across all classes seen so far, making it highly susceptible to catastrophic forgetting. When evaluating these models, performance is typically benchmarked against fine-tuning (sequential training without forgetting mitigation; lower bound) and joint-training (simultaneous training on all data; upper bound). To bridge the gap to joint-training under Class-IL constraints, algorithms are commonly grouped into regularisation, parameter isolation, and rehearsal. 2.1. Regularisation Regularisation methods restrict gradient updates to protect parameters that are critical for prior tasks. Early approaches froze lower layers of a net- work after training on a task [14]. Modern algorithms estimate parameter importance by calculating contribution to loss reduction (e.g., Synaptic In- telligence [15]), approximating Bayesian inference (e.g., Elastic Weight Con- solidation [4]), or evaluating output function sensitivity (e.g., Memory Aware Synapses [16]). Recent methods perform geometric analysis to locate stable flat minima in the distribution manifold [17, 18] or approximate prior losses using the Hessian matrix eigenvalues [19]. While regularisation is highly effective in Task-IL, it struggles in Class-IL [20]. The strict constraints designed to protect existing knowledge prevent the model from adapting to novel classes, leading to representational paralysis as constraints accumulate over sequential tasks [18]. 2.2. Parameter isolation Parameter isolation limits training to a subset of parameters or dynam- ically expands the network structure. A common design adds task-specific output heads to a pretrained feature-extractor backbone [21, 22]. Under network expansion, the network freezes historical weights and adds new neu- rons or columns for each task [23, 24]. Another branch of research leverages Adaptive Resonance Theory to dynamically create category nodes as new distributions appear [25]. However, these approaches struggle in Class-IL settings because they re- quire task identity during inference to route data to the appropriate sub- network. Without explicit task identifiers, the model lacks an intrinsic mech- 5 anism for choosing the correct task pathway. Although sparse neural acti- vation in large networks can mitigate routing issues [26], parameter isola- tion remains difficult to scale without introducing capacity exhaustion or inference-routing failures [27, 28]. 2.3. Rehearsal: replay and generative replay Rehearsal techniques interleave historical data with new task inputs. Ex- perience replay stores a memory buffer of real samples from earlier tasks, and Gradient Episodic Memory (GEM) uses those stored examples as inequality constraints on each update so the loss on previous tasks does not increase, which reduces forgetting while still allowing positive backward transfer [29]. Maximally Interfered Retrieval (MIR) instead keeps a finite replay memory, estimates the parameter update from the current batch, and then replays the buffered samples whose losses would increase the most under that up- date, making it distinct from GEM because it prioritises most-interfered sam- ples rather than enforcing explicit gradient constraints [30]. AdaER further adapts replay by using Contextually-Cued Memory Recall to select memories based on both data-conflicting and task-conflicting cues, and it also updates the buffer with Entropy-Balanced Reservoir Sampling to keep a more bal- anced, informative memory, distinguishing it from MIR’s interference-only retrieval and GEM’s constraint-based updates [31]. While buffer-based replay is effective, it scales poorly because represent- ing complex, high-dimensional distributions requires a prohibitive number of stored samples [32]. Generative replay avoids storing real data by training a generative model to synthesise historical inputs [33]. To scale beyond simple datasets, models reconstruct latent feature representations rather than raw training samples. For example, Generative Feature Replay (GFR) uses a feature extractor to train a generator on latent distributions [34], mimicking biological systems where memory replay occurs at representational rather than raw sensory levels [35]. Brain-Inspired Replay (B-IR) combines generative replay with parameter isolation to achieve state-of-the-art results [12]. Nonetheless, gen- erative models suffer from representational drift as the feature extractor up- dates over time, requiring distillation constraints or frozen features to anchor the latent space [36]. 6 (a) Training Latent space (b) Generation Latent space (c) Inference CONFCONF < EncoderEncoderEncoder ● ● ● ... ● ● ● ● ● ● ... ● ● ● ● ● ● ... ● ● ● ● ● ● ... ● ● ● Decoder DecoderDecoder Decoder 4 3 1 24 3 1 24 3 1 2 Figure 1: Long-term memory inputs and outputs: (a) LTM training on image-label pairs. (b) Synthetic sample generation with temperature τ. (c) Inference confidence-based selec- tion. 3. Methods We present 4MAS (4 Module Awake/Sleep), a macroarchitecture for con- tinual learning designed to model biological memory consolidation processes. As illustrated in Figure 1, the system consists of two lateralised hemispheres. Each hemisphere contains: 1. A generative Long-Term Memory (LTM) representing the neocortex, which learns task distributions and classifies incoming data. 2. A Short-Term Memory (STM) buffer representing the hippocampus, which stores a small set of episodic exemplars. The architecture restricts data flow to a biologically inspired model where task acquisition is unihemispheric and offline consolidation is bilateral, facil- itating knowledge transfer and specialisation. 3.1. Benchmarks and experimental design To evaluate 4MAS under the Class-IL constraint, we employ Split-MNIST [37, 38] and Split-Fashion-MNIST [39, 40], shown in Figure 2. The standard 10-class datasets are split into 5 sequential tasks of 2 classes each, presented 7 (a) Split-MNIST Task 1: -Class 1 -Class 2 Task 2: -Class 3 -Class 4 Task 3: -Class 5 -Class 6 Task 4: -Class 7 -Class 8 Task 5: -Class 9 -Class 10 Task 1: -Class 1 -Class 2 Task 2: -Class 3 -Class 4 Task 3: -Class 5 -Class 6 Task 4: -Class 7 -Class 8 Task 5: -Class 9 -Class 10 (b) Split-Fashion-MNIST Figure 2: Class-IL datasets: (a) Split-MNIST, (b) Split-Fashion-MNIST. without task identifiers at both training and inference. Hyperparameter tun- ing and initial method development were conducted primarily on these two MNIST variants. To test scalability and out-of-the-box generalizability, we then expanded evaluation to the more challenging Split-CIFAR-100 dataset [38]. Unless otherwise stated, each configuration is run for 10 random seeds, and we report the mean and standard error of the mean of each metric across seeds. The model trains for 10 epochs per class, with a synthetic epoch length of 10,000 samples. Ablation studies disentangle the contributions of the STM buffers, the dual-hemisphere ensemble, and the sleep phase. 3.2. Sleep/wake phases The training cycle alternates between a unihemispheric “awake” phase and a bilateral “sleep” phase for each task, Figure 3. During the awake phase, Algorithm 1, each LTM hemisphere is trained on three randomly interleaved sources: new task data, exemplars stored in the ipsilateral STM from prior tasks, and self-generated synthetic samples. After training, the LTM evaluates the training data and updates its STM buffer with samples that meet the selection criteria (Section 3.4). During the sleep phase, Algorithm 2, LTMs undergo fine-tuning on con- tralateral representations (generations from the opposite LTM and exemplars 8 (a) Awake training ☀ new task (b) Sleep training no new tasks Left LTM training pathwaysRight LTM training pathwaysSTM storage pathways left LTMright LTM right STM left STM left STMright STM LTM training on a new task is stabilised by interleaving ipsilateral STM & LTM samples from prior tasks weakly-learned samples from new task are added to STM left LTMright LTM Figure 3: Awake/sleep training phases: (a) Awake training: LTMs learn new task data interleaved with STM samples. (b) Sleep training: LTMs are fine-tuned on contralateral representations. from the opposite STM) during a period where no new training data are available. To allow subtle adjustments to internal representations without destroying learned task-specific parameters, the sleep learning rate is scaled down by a multiplier λ = 0.1, mimicking the lower firing rates observed across brain regions during REM sleep [41]. 9 Algorithm 1 Incremental task – awake phase 1: Input: Task t ∈ [1,∞), learning rate α, model parameters θ left , θ right , memories M left , M right , classes seen C, memory capacity M. 2: for h∈left,right do 3: w ←w i C i=1 where w i = 1/C 4: for e = 1...E awake do 5:Draw batch β t from D t . 6:if t > 1 then generate replayG h ∼ p θ h (x|z) and set β ← β t ∪G h ∪ M h else β ← β t 7:β ← class_weighted_rebalance(β,w) 8:θ h ← θ h − α∇ θ L(θ h ;β) 9:g ← bincount arg max c ˆy(G h ) 10:w ←1/(g i + ε) C i=1 11: end for 12: for i = 1...C do 13: P h,i ← Oversample(D t,i ∪M h,i ) ▷ Expand candidate pool for class i 14: M h,i ← TopK ⌊M/C⌋ P h,i , CONF (ˆy) ▷ Select top-k highest confidence 15: end for 16: M h ← S C i=1 M h,i 17: end for Algorithm 2 Incremental task – sleep phase 1: Input: Task t ∈ [1,∞), learning rate α, models θ left , θ right , memories M left , M right , sleep learning multiplier λ = 0.1. 2: for h in left,right do 3: w ←w i C i=1 where w i = 1/C 4: for e = 1 : E sleep do 5:β ← θ h (z)∪M h 6:τ h ← θ h − λα∇ τ L(θ h ;β). 7: end for 8: end for 10 3.3. Long-term memories Each LTM is both a generator and a classifier within a single variational auto-encoder (VAE). The classification vector is appended to the input image, and the model is trained to reconstruct the joint vector x ⌢ y. This joint parameter space ensures that synthetic images and labels are tightly coupled. Let ˆy denote the reconstructed label channel, normalised over the C classes seen so far. We define prediction confidence as the negative cross-entropy of ˆy against its own arg-max class: CONF (ˆy) =−L CE ˆy, onehot(arg max c ˆy c ) = log max c ˆy c (1) CONF ∈ (−∞, 0], with larger values indicating higher confidence. Since L CE is the same term the VAE minimises on the label channel, Equation 1 doubles as a measure of reconstruction quality and requires no separate dis- criminative head, providing stability when learning new tasks [42]. Confi- dence is used in three places: rejecting low-confidence generations during replay, ranking candidates for STM storage (Section 3.4), and arbitrating between hemispheres at inference. Standard VAE latent sampling z ∼N (0,I) can suffer from posterior col- lapse, producing low-variance synthetic samples that degrade replay quality over sequential tasks. To resolve this, we apply a temperature gain τ to the latent coordinates during sampling: z = τX,X ∼ N (0,I) [43]. Setting τ > 1 expands the explored region of the latent space, generating sharper and more diverse samples, Figure 4. During replay, low-confidence generations near class boundaries are rejected to avoid interpolating between classes. At inference, inputs are processed by both LTMs, and the class prediction from the hemisphere with higher confidence is selected. To maintain class balance during replay, generation frequencies are weighted inversely to generation counts Algorithm 1, line 10, biased linearly by task age. 3.4. Short-term memories and lateralisation Each STM acts as a memory buffer storing 50 exemplars. We evaluate two buffer storage selection mechanisms: 1. Comparative Confidence Selection (CCS): Samples from the task and prior STM are passed through the post-training LTM and ranked by confidence. In symmetric configurations, moderate-confidence quan- tile range samples (∼ 50%) are selected in both hemispheres to rep- resent distribution boundaries while retaining discernibility. Under 11 4 32 + + 2+ 3 + 4 4 3 2 + + 2 + 3 + 4 Split-MNIST latent space 4 32 + + 2+ 3 + 4 4 3 2 + + 2 + 3 + 4 Split-Fashion-MNIST latent space Figure 4: 2-dimensional mapping of the LTM’s latent space after training. τ scales the sampling radius in units of the prior standard deviation. High temperature τ > 1 increases the diversity of output representations (16 latent dimensions used in final configurations). asymmetric configurations, STM storage selection is also lateralised: the Left Hemisphere selects high-confidence anchor samples to rein- force representations against drift, while the Right Hemisphere selects moderate-confidence samples to explore decision boundaries. Specif- ically, we filter replay samples using the raw probability max c ˆy c = exp(CONF (ˆy)) ∈ [0, 1], retaining candidates within the moderate- confidence quantile range L,R =0.1, 0.5. This design helps avoid VAE posterior collapse and representational drift. 2. Latent Space Cluster Centroids (LSCC): We apply K-Means clus- tering in the VAE latent space across task and STM samples, retaining exemplars closest to the centroids. This grounds representations and further prevents representational drift. To model biological hemispheric lateralisation (stability vs. plasticity), we configure hyperparameter asymmetry. The Left Hemisphere (LH) is con- figured for stability and routine processing (lower generator temperature τ and task-age bias favouring older tasks). The Right Hemisphere (RH) is con- figured for plasticity and exploration (higher generator temperature τ and no age bias), enabling rapid adaptation to novel distributions. 12 4. Results and discussion Our results show that 4MAS achieves competitive Class-IL accuracies across all benchmarks: 98.3± 0.0% on Split-MNIST, 84.9± 0.3% on Split- Fashion-MNIST, and 29.29± 0.29% on Split-CIFAR-100 (Table 1). These results are competitive with other recent Class-IL methods, such as B-IR (93.5%, 74.6%, and 27.85% accuracy) [38] and AdaER (89.6% and 74.0% accuracy) [31]. As shown in Table 1, 4MAS demonstrates strong resilience to catastrophic forgetting. On Split-MNIST, 4MAS achieves 98.3% accu- racy, closely matching the Joint training ceiling of 98.0%. On Split-Fashion- MNIST, our method achieves 84.9± 0.3%, representing a significant advance- ment over B-IR (74.6± 0.4%) and GR (71.3± 0.3%). This performance boost is directly tied to our low Forgetting scores (1.0% on Split-MNIST, 10.6% on Split-Fashion-MNIST) and exceptionally low representational drift (1.2 and 1.0, respectively, compared to GR’s 215.0 and 48.5). This demonstrates that the dual-hemisphere and sleep-phase architecture stabilises internal la- tent representations, preventing the drift that typically destabilises single- generator networks. On the challenging Split-CIFAR-100 dataset and when scaling each ar- chitecture to 140M trainable parameters, 4MAS improves on the unilateral B-IR baseline (27.85± 0.55%) and significantly outperforms other generative replay methods. The performance gap relative to Joint training (51.9±0.4%) is primarily due to the expressive capacity of the standard flat VAE decoder, which struggles to reconstruct high-frequency details for 100 complex classes. 4.1. Backward and Forward Transfer Analysis To evaluate how sequence learning affects historical and future task per- formance, we analyze Backward Transfer (BWT) and Forward Transfer (FWT), as defined in [29]: BWT = 1 T − 1 T−1 X i=1 (R T,i − R i,i ), FWT = 1 T − 1 T X i=2 (R i−1,i − ̄ b i ) (2) where R T,i represents test accuracy on task i after training on task T, and ̄ b i denotes random baseline performance for task i. 13 Table 1: Class-IL evaluation across Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR- 100 benchmarks. Accuracy metrics report final task performance after learning all tasks. DatasetMethodFinal Acc (%) Forgetting (%) BWT (%) FWT (%) Compute (x) Split-MNIST Joint98.5± 0.01.0−1.00.01.0 4MAS (Our method)98.3± 0.01.2−1.20.01.0 B-IR93.5± 0.26.4± 0.2 −6.4± 0.20.2± 0.00.2± 0.0 GR91.2± 0.48.9± 0.4 −8.9± 0.40.3± 0.042.1± 1.8 LwF24.2± 0.40.10.10.015.2 EWC19.9± 0.079.2−79.20.061.3 oEWC19.9± 0.079.2−79.20.058.1 SI19.9± 0.079.3−79.30.033.5 Fine-Tuning19.7± 0.179.5± 0.1 −79.5± 0.10.3± 0.084.2± 2.8 Split-Fashion-MNIST Joint88.3± 0.27.5± 0.2 −7.5± 0.20.01.0 4MAS (Our method)84.9± 0.310.6−10.60.01.0 B-IR74.6± 0.430.2± 0.3 −30.2± 0.30.5± 0.00.2± 0.0 GR71.3± 0.325.4± 0.3 −25.4± 0.30.6± 0.148.5± 2.3 LwF20.1± 0.20.30.30.012.9 EWC19.9± 0.079.2−79.20.055.5 oEWC19.8± 0.479.2−79.20.050.3 SI19.9± 0.079.3−79.30.030.4 Fine-Tuning20.0± 0.079.4± 0.0 −79.4± 0.00.5± 0.177.3± 2.3 Split-CIFAR-100 Joint51.9± 0.416.2± 0.4 −16.2± 0.40.02.4 4MAS (Our method, 140M)29.29± 0.2925.8± 1.2−25.8± 1.20.00.43± 0.07 GR7.9± 0.166.1± 0.5 −66.1± 0.50.010.3 B-IR (140M)27.85± 0.5552.9−52.90.05.1 LwF10.8± 0.24.6± 0.3 −3.9± 0.30.01.9 EWC8.1± 0.181.6± 0.4 −81.6± 0.40.04.2 SI9.2± 0.279.8± 0.4 −79.8± 0.40.03.1 Fine-Tuning8.1± 0.181.3± 0.4 −81.3± 0.40.03.6 Backward Transfer Dynamics.. Table 1 demonstrates that 4MAS consis- tently minimises negative BWT compared to existing baselines. On Split- MNIST, Split-Fashion-MNIST, and Split-CIFAR-100, 4MAS achieves BWT scores of−1.2%,−10.6%, and−25.8% respectively, significantly outperform- ing competitive memory and replay methods (B-IR and GR). 4MAS tracks closely with the offline Joint Training baseline (e.g., −25.8% vs. −16.2% on CIFAR-100), confirming that our approach effectively freezes and preserves past decision boundaries during new class assimilation. Forward Transfer Limitations in Class-IL.. Across all methods, FWT re- mains near 0.0%. This behaviour is characteristic of Class-IL benchmarks evaluated from scratch: without a shared pre-trained feature extractor, fea- ture representations learned on early tasks do not inherently transfer zero- shot accuracy to orthogonal class boundaries in subsequent tasks. Thus, performance superiority in 4MAS is driven almost entirely by backward sta- bility rather than forward inductive bias. 14 (a) 0%25%50%75%100%125%150%175%200% 68% 70% 72% 74% 76% 78% 80% Sleep phase learning rate system (ensemble) accuracy LH accuracy RH accuracy (b) task 1 awake task 1 sleep task 2 awake task 2 sleep task 3 awake task 3 sleep task 4 awake task 4 sleep task 5 awake task 5 sleep 20% 30% 40% 50% 60% 70% 80% Right Dominance Left Hemispheric dominance Split-Fashion-MNIST - 20 epochs Split-Fashion-MNIST - 10 epochs Split-MNIST - 10 epochs Figure 5: (a) Performance variation with sleep phase learning rates (LR) as a percentage of awake phase LR, Split-Fashion-MNIST. (b) Memory consolidation during sleep phase increased LH dominance, while new task learning during awake phase increased RH dom- inance; produced by lateralisation techniques (class representation bias by age in LH and asymmetric generator temperatures) 4.2. Sleep tuning We evaluated sleep phase learning rates (LR) ranging from 0% to 200% of the awake phase rate (Figure 5a). On Split-Fashion-MNIST a reduced sleep rate of 5–10% (≈ 2× 10 −5 ) performed best (84.9± 0.3%), while remov- ing sleep (LR = 0%) yielded 74.0± 0.59%. Larger learning rates degraded performance, confirming that low sleep learning rates enable fine-tuning on the contralateral representation without overwriting specialised features. Furthermore, average ensemble accuracy exceeded either isolated hemi- sphere. This gain stems from effective confidence-based routing between specialised hemispheres rather than standard ensemble variance reduction; indeed, without sleep-phase consolidation, the ensemble fails to outperform the strongest individual hemisphere (Section 4.6). Hemispheric dominance shifted dynamically: wake training on new tasks increased RH dominance, while sleep consolidation restored LH dominance (Figure 5b). This shift aligns with Goldberg’s Novelty-Routine hypothesis, reflecting a transition from initial RH-driven processing of novel representations to consolidated, routinised LH schemas. 4.3. Lateralisation Figure 6a shows that the LH accuracy decays slowly, maintaining stabil- ity, while the RH learns new tasks quickly but forgets faster, supporting the 15 (a) task 1 awake task 1 sleep task 2 awake task 2 sleep task 3 awake task 3 sleep task 4 awake task 4 sleep task 5 awake task 5 sleep 40% 50% 60% 70% 80% 90% 100% Hemispheric accuracy by task Task 1 (RH) Task 2 (RH) Task 3 (RH) Task 4 (RH) Task 5 (RH) Task 1 (LH) Task 2 (LH) Task 3 (LH) Task 4 (LH) Task 5 (LH) (b) task 1 awake task 1 sleep task 2 awake task 2 sleep task 3 awake task 3 sleep task 4 awake task 4 sleep task 5 awake task 5 sleep 70% 75% 80% 85% 90% 95% 100% -asymmetry study left =1 right =3, with sleep left =1 right =2, with sleep left =1 right =1, with sleep left =1 right =2, no sleep left =1 right =1, no sleep left =2 right =2, no sleep t t t t t t t t t t t t t Figure 6: Fashion-MNIST: (a) Hemispheric accuracy for each task. (b) τ-asymmetry results, with and without sleep training phase. τ - right 12345 τ - left 10.63010.71220.74450.71320.6984 20.70210.68950.66620.63420.6178 30.67680.61160.59070.55950.5504 40.65930.61280.55610.55510.5233 50.64530.5760.54170.5550.5148 Table 2: LH and RH generator temperature vs Split-Fashion-MNIST accuracy. Degraded performance observed for symmetrical temperatures. stability-plasticity lateralisation described in the Novelty-Routine hypothesis [44]. Lateralising the generator temperature (τ) significantly improved per- formance (Table 2, Figure 6b). Symmetrical temperatures (τ = 1 or τ = 2 for both) caused degraded performance or limited sleep benefits, whereas asym- metrical configurations (τ = 1 for LH, τ ∈ [2, 4] for RH) showed consistent performance gains after each sleep phase. This indicates that specialisation, elicited by asymmetrical parameterisation, provides an advantage and better use of total resources. Applying a task-age class representation bias to the LH only (biasing training toward older tasks) improved the retention of earlier tasks by 15–26% (Figure 7a), allowing the ensemble to retain stability in LH while maintaining plasticity in RH. 16 Confidence - RH 90%70%50%30%10%0%K-Means Confidence - LH 90%0.75620.76620.76680.76680.75360.75050.7656 70%0.76180.77240.77530.77250.76070.75890.7796 50%0.76950.77060.77330.76670.76480.75730.7779 30%0.77030.77180.77580.76270.75650.72470.7799 10%0.76730.77040.76750.75090.73510.7270.7757 0%0.75470.77260.76130.74700.72850.72050.7662 K-Means0.77480.77760.77990.77640.76070.75880.776 Table 3: Comparison of mechanisms for selecting memories for STM buffer storage. Storing latent space centroids via K-Means clustering produced the most consistent results on Split-Fashion-MNIST, whereas highly asymmetric confidence selection (L = 0.1/R = 0.5) achieved peak performance on the more complex Split-CIFAR-100 benchmark. 4.4. STM buffer selection Evaluating CCS and LSCC selection mechanisms (Table 3) showed that storing moderate-confidence samples (∼ 50%) under CCS performed best, whereas selecting low-confidence outliers led to VAE posterior collapse. Stor- ing latent space cluster centroids via K-Means (LSCC) yielded the best and most consistent results across MNIST and Fashion-MNIST tasks, indicating that the STM buffer is most effective when its primary role is grounding the latent space against representational drift rather than importing weakly- learned outliers. This aligns with neurobiological evidence showing that of- fline memory reactivation helps to preserve multiday representational stabil- ity [45]. The robust performance of the asymmetric threshold configuration (L = 0.1,R = 0.5) on CIFAR-100 is explained by a functional division of labour. The left hemisphere, operating at a low generator temperature (τ = 1), be- haves as a stable anchor that preserves core, high-confidence representations. By setting a very conservative threshold (L = 0.1), we prevent representa- tional drift during consolidation. Conversely, the right hemisphere, operating at a high generator temperature (τ = 3), acts as a flexible explorer. Stor- ing intermediate-confidence boundary samples (R = 0.5) allows it to explore variations and refine task boundaries, leading to significantly enhanced en- semble consolidation. 4.5. Capacity and architectural scaling To evaluate how model capacity influences continual learning perfor- mance, we systematically compared the parameter scaling behaviour of 4MAS 17 (a) 1.51.00.50.00.51.01.5 Class Age Bias Factor ( Prefer New Tasks | Equal | Prefer Old Tasks ) 40% 50% 60% 70% 80% 90% 100% Class representation bias (Mean ± SEM) task 1 task 2 task 3 task 4 task 5 (b) 02004006008001000 63% 65% 68% 70% 73% 75% 78% 80% STM buffer size (for each hemisphere) accuracy (with sleep phase) accuracy (no sleep phase) Figure 7: (a) Biasing class representation by recency; weighting training heavily toward older classes significantly boosts early-class accuracy with only minor performance degra- dation on recent classes. (b) Effect of STM memory buffer size on accuracy; gains from larger buffers plateau after 200 samples, while also diminishing the relative impact of the sleep phase. against the unilateral B-IR baseline across Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR-100 (Figure 8). At lower parameter ranges (e.g., < 30 M parameters), monolithic gener- ative models demonstrate superior sample and parameter efficiency. This is primarily because 4MAS splits its total parameter budget across two distinct hemispheric models (LH and RH) and requires sleep-phase cross- replay to consolidate knowledge, introducing an architectural capacity over- head. When the overall parameter budget is highly constrained, the split- hemisphere bottleneck limits the representation capacity of the individual generators. However, 4MAS demonstrates superior scalability as model capacity in- creases. Unilateral networks typically suffer from severe representational drift and catastrophic interference when forced to represent a large number of con- flicting class distributions in a single unified latent space [38]. Consequently, performance of B-IR (and other unilateral models) plateaus or degrades at larger scales. In contrast, 4MAS’s bilateral hemispheric partitioning and stability-plasticity division of labour mitigate representational drift, enabling monotonic scaling. At larger parameter scales (≥ 70 M parameters), 4MAS consistently outperforms B-IR on the more complex Fashion-MNIST bench- mark and approaches parity on the CIFAR-100 benchmark. 18 (a) 1M2M5M10M20M50M100M200M Total Model Parameters 20 30 40 50 60 70 80 90 100 Final Accuracy (%) Split-MNIST (5 tasks, class-IL) 4MAS (ours) GR B-IR EWC SI LwF (b) 1M2M5M10M20M50M100M200M Total Model Parameters 60 65 70 75 80 Final Accuracy (%) Split-Fashion-MNIST (5 tasks, class-IL) 4MAS (ours) GR B-IR (c) 10152030405070100150200 Total Parameters (Millions) 5 10 15 20 25 30 35 Final Class-IL Accuracy (% / 10-Task Joint) 18.90% 19.57% 24.45% Rank 1 Peak: 29.29% 21.00% 23.01% 26.27% 27.85% Split-CIFAR-100 Capacity Scaling: 4MAS vs. B-IR 4MAS (Ours, Rank 1 Peak) B-IR (Baseline) Figure 8: Capacity scaling curves comparing our bilateral 4MAS architecture against the unilateral B-IR baseline across (a) Split-MNIST, (b) Split-Fashion-MNIST, and (c) Split- CIFAR-100. The plots show final accuracy as a function of total model parameters (in millions). 4.6. Ablation studies To evaluate the contribution of each component, we performed abla- tions on the STM buffer size, the sleep phase, and the dual-hemisphere structure. Varying the unilateral STM buffer size from 0 to 1,000 samples (Figure 7b) showed diminishing gains beyond 200 samples, improving Split- Fashion-MNIST accuracy from 73.4% (fully ablated STM) to 81.3%. In the absence of a sleep phase, small buffer sizes severely degraded performance (63.2% at size 0), while larger buffers offset this loss, narrowing the sleep phase benefit to < 1%. Ablation comparisons (Figure 9) show that configurations including the sleep phase consistently perform best on both datasets. Without sleep, the ensemble accuracy matches its single best-performing hemisphere, confirming that sleep-based cross-replay is crucial for bilateral knowledge integration. In particular, the comparative confidence levels of each hemisphere are aligned during sleep training, allowing for system accuracy to exceed either hemi- sphere’s individual accuracy. 5. Limitations and future work Generative models in this study were standard non-convolutional VAEs, chosen to evaluate task-agnostic macroarchitectures. Future research should scale generators to convolutional VAEs, GANs, or diffusion models to handle more complex image distributions. Additionally, regularisation mechanisms (e.g., EWC or SI) could be integrated in tandem with our replay structure to further boost stability, as demonstrated in hybrid models like B-IR [12]. 19 task 1 awake task 1 sleep task 2 awake task 2 sleep task 3 awake task 3 sleep task 4 awake task 4 sleep task 5 awake task 5 sleep 86% 88% 90% 92% 94% 96% 98% 100% Ablations - Split-MNIST accuracy with STM, with sleep no STM, with sleep with STM, no sleep no STM, no sleep no STM, RH only with STM, RH only no STM, LH only with STM, LH only task 1 awake task 1 sleep task 2 awake task 2 sleep task 3 awake task 3 sleep task 4 awake task 4 sleep task 5 awake task 5 sleep 60% 65% 70% 75% 80% 85% 90% 95% 100% Ablations - Split-Fashion-MNIST accuracy with STM, with sleep no STM, with sleep with STM, no sleep no STM, no sleep no STM, RH only with STM, RH only no STM, LH only with STM, LH only Figure 9: Ablation performance for Split-MNIST and Split-Fashion-MNIST. Configura- tions including the sleep phase consistently perform best on both datasets, where sleep informed hemispheric confidence levels lead to system accuracy exceeding individual hemi- sphere accuracies. While we evaluated 4MAS up to a 50-task Split-CIFAR-100 benchmark, autonomous agents in the real world require handling longer task horizons. Subsequent work should test 4MAS under unbounded dynamic streams, ex- plore diverse forms of hemispheric asymmetry, and investigate additional bio- logical mechanisms, such as slow-wave sleep modelling, to enhance persistent lifelong learning. 6. Conclusion Lifelong learning in artificial networks remains constrained by catastrophic forgetting. Addressing this challenge, this paper introduced 4MAS, a novel continual learning macroarchitecture derived from biological memory consol- idation. By modelling experience replay, bilateral sleep consolidation, and inter-hemispheric lateralisation, 4MAS employs two asymmetric LTM/STM hemispheres that coordinate via a consolidation sleep phase. Our empirical results on Split-MNIST, Split-Fashion-MNIST, and CIFAR-100 demonstrate competitive performance and robust knowledge retention against representa- tional drift. our analysis reveals that functional specialisation is essential to these gains: while symmetrical configurations degraded performance or re- stricted sleep benefits, asymmetrical parameterisation consistently produced post-sleep performance enhancements, demonstrating that lateralised spe- cialisation improves overall resource utilisation. 20 This work illustrates how system-level neurobiological structures can be abstracted to manage the stability-plasticity trade-off. Moving beyond local weight-level constraints, system-level bilateral consolidation offers a promis- ing path toward persistent and adaptable artificial intelligence. References [1] B. Wickramasinghe, G. Saha, K. Roy, Continual Learning: A Review of Techniques, Challenges, and Future Directions, IEEE Transactions on Artificial Intelligence 5 (6) (2024) 2526–2546. doi:10.1109/TAI.2023. 3339091. URL https://ieeexplore.ieee.org/document/10341211 [2] M. McCloskey, N. J. Cohen, Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem, in: G. H. Bower (Ed.), Psychology of Learning and Motivation, Vol. 24, Academic Press, 1989, p. 109–165. doi:10.1016/S0079-7421(08)60536-8. URL https://w.sciencedirect.com/science/article/pii/ S0079742108605368 [3] R. Kemker, M. McClure, A. Abitino, T. Hayes, C. Kanan, Measur- ing Catastrophic Forgetting in Neural Networks (Nov. 2017). doi: 10.48550/arXiv.1708.02072. URL http://arxiv.org/abs/1708.02072 [4] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hasabis, C. Clopath, D. Kumaran, R. Hadsell, Overcoming catas- trophic forgetting in neural networks (2017). doi:10.1073/pnas. 1611835114. URL https://w.pnas.org/doi/10.1073/pnas.1611835114 [5] S. Corkin, What’s new with the amnesic patient H.M.?, Nature Reviews Neuroscience 3 (2) (2002) 153–160. doi:10.1038/nrn726. URL https://w.nature.com/articles/nrn726 [6] B. Giri, H. Miyawaki, K. Mizuseki, S. Cheng, K. Diba, Hippocam- pal Reactivation Extends for Several Hours Following Novel Ex- perience, The Journal of Neuroscience 39 (5) (2019) 866–875. doi:10.1523/JNEUROSCI.1950-18.2018. 21 URL https://w.jneurosci.org/lookup/doi/10.1523/ JNEUROSCI.1950-18.2018 [7] T. L. Hayes, G. P. Krishnan, M. Bazhenov, H. T. Siegelmann, T. J. Sejnowski, C. Kanan, Replay in Deep Learning: Current Approaches and Missing Biological Elements, Neural Computation (2021) 1– 44doi:10.1162/neco_a_01433. URL https://direct.mit.edu/neco/article/doi/10.1162/neco_ a_01433/107071/Replay-in-Deep-Learning-Current-Approaches-and [8] M. Tamaki, J. Bang, T. Watanabe, Y. Sasaki, Night Watch in One Brain Hemisphere during Sleep Associated with the First- Night Effect in Humans, Current Biology 26 (9) (2016) 1190–1194. doi:10.1016/j.cub.2016.02.063. URL https://linkinghub.elsevier.com/retrieve/pii/ S0960982216301749 [9] M. Ghosh, F.-C. Yang, S. P. Rice, V. Hetrick, A. L. Gonzalez, D. Siu, E. K. Brennan, T. T. John, A. M. Ahrens, O. J. Ahmed, Running speed and REM sleep control two distinct modes of rapid interhemispheric communication, Cell Reports 40 (1) (2022) 111028. doi:10.1016/j.celrep.2022.111028. URL https://linkinghub.elsevier.com/retrieve/pii/ S2211124722008221 [10] L. F. Koziol, The Novelty-Routinization Principle of Brain Organi- zation, in: L. F. Koziol (Ed.), The Myth of Executive Function- ing: Missing Elements in Conceptualization, Evaluation, and Assess- ment, Springer International Publishing, Cham, 2014, p. 27–31. doi: 10.1007/978-3-319-04477-4_8. URL https://doi.org/10.1007/978-3-319-04477-4_8 [11] C. S. Prat, J. Gallée, B. L. Yamasaki, Getting language right: Relating individual differences in right hemisphere contributions to language learning and relearning, Brain and Language 239 (2023) 105242. doi:10.1016/j.bandl.2023.105242. URL https://w.sciencedirect.com/science/article/pii/ S0093934X23000214 22 [12] G. M. van de Ven, H. T. Siegelmann, A. S. Tolias, Brain-inspired replay for continual learning with artificial neural networks, Nature Communi- cations 11 (1) (2020) 4069. doi:10.1038/s41467-020-17866-2. URL https://w.nature.com/articles/s41467-020-17866-2 [13] G. M. v. d. Ven, A. S. Tolias, Three scenarios for continual learning (Apr. 2019). doi:10.48550/arXiv.1904.07734. URL http://arxiv.org/abs/1904.07734 [14] S. Gutstein, O. Fuentes, E. Freudenthal, Knowledge transfer in deep convolutional neural nets, International Journal on Artificial Intelligence Tools 17 (03) (2008) 555–567. doi:10.1142/S0218213008004059. URL https://w.worldscientific.com/doi/abs/10.1142/ S0218213008004059 [15] F. Zenke, B. Poole, S. Ganguli, Continual Learning Through Synaptic Intelligence, Proceedings of machine learning research (2017). [16] R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, T. Tuytelaars, Memory Aware Synapses: Learning what (not) to forget, in: Proceed- ings of the European conference on computer vision (ECCV), 2018, p. 139–154. URL https://openaccess.thecvf.com/content_ECCV_2018/html/ Rahaf_Aljundi_Memory_Aware_Synapses_ECCV_2018_paper.html [17] S. I. Mirzadeh, M. Farajtabar, R. Pascanu, H. Ghasemzadeh, Under- standing the Role of Training Regimes in Continual Learning, Advances in Neural Information Processing Systems 33 (2020) 7308–7320. URL https://proceedings.neurips.c/paper/2020/hash/ 518a38c9a0173d0b2dc088166981cf8-Abstract.html?ref=https: //githubhelp.com [18] G. Shi, J. Chen, W. Zhang, L.-M. Zhan, X.-M. Wu, Overcoming Catas- trophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima, Advances in neural information processing systems 34 (2021). [19] Y. Kong, L. Liu, H. Chen, J. Kacprzyk, D. Tao, Overcoming Catas- trophic Forgetting in Continual Learning by Exploring Eigenvalues of Hessian Matrix, IEEE Transactions on Neural Networks and Learn- ing Systems 35 (11) (2024) 16196–16210. doi:10.1109/TNNLS.2023. 23 3292359. URL https://ieeexplore.ieee.org/document/10190202 [20] M. D. Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, T. Tuytelaars, A continual learning survey: Defying for- getting in classification tasks, IEEE Transactions on Pattern Analy- sis and Machine Intelligence (2021) 1–1ArXiv:1909.08383 [cs]. doi: 10.1109/TPAMI.2021.3057446. URL http://arxiv.org/abs/1909.08383 [21] A. S. Razavian, H. Azizpour, J. Sullivan, S. Carlsson, CNN Features off-the-shelf: an Astounding Baseline for Recognition (May 2014). doi: 10.48550/arXiv.1403.6382. URL http://arxiv.org/abs/1403.6382 [22] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, T. Dar- rell, DeCAF: A Deep Convolutional Activation Feature for Generic Vi- sual Recognition (Oct. 2013). doi:10.48550/arXiv.1310.1531. URL http://arxiv.org/abs/1310.1531 [23] A. V. Terekhov, G. Montone, J. K. O’Regan, Knowledge Transfer in Deep Block-Modular Neural Networks, in: S. P. Wilson, P. F. Verschure, A. Mura, T. J. Prescott (Eds.), Biomimetic and Biohybrid Systems, Springer International Publishing, Cham, 2015, p. 268–279. doi:10. 1007/978-3-319-22979-9_27. [24] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, R. Hadsell, Progressive Neural Networks (Oct. 2022). doi:10.48550/arXiv.1606.04671. URL http://arxiv.org/abs/1606.04671 [25] S. Grossberg, Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world, Neural Networks 37 (2013) 1–47. doi:10.1016/j.neunet.2012.09.017. URL https://w.sciencedirect.com/science/article/pii/ S0893608012002584 [26] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, J. Dean, Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (Jan. 2017). doi:10.48550/arXiv.1701. 24 06538. URL http://arxiv.org/abs/1701.06538 [27] Z. Chen, A. Wuerkaixi, S. Cui, H. Li, D. Li, J. Zhang, B. Han, G. Niu, H. Liu, Y. Yang, S. Yang, C. Zhang, T. Ren, Learning without Isolation: Pathway Protection for Continual Learning, arXiv:2505.18568 [cs] (May 2025). doi:10.48550/arXiv.2505.18568. URL http://arxiv.org/abs/2505.18568 [28] N. Omi, S. Sen, A. Farhadi, Load Balancing Mixture of Experts with Similarity Preserving Routers, arXiv:2506.14038 [cs] (Oct. 2025). doi: 10.48550/arXiv.2506.14038. URL http://arxiv.org/abs/2506.14038 [29] D. Lopez-Paz, M. A. Ranzato, Gradient Episodic Memory for Contin- ual Learning, Advances in Neural Information Processing Systems 30 (2017). URL https://proceedings.neurips.c/paper/2017/hash/ f87522788a2be2d171666752f97ddebb-Abstract.html [30] R. Aljundi, E. Belilovsky, T. Tuytelaars, L. Charlin, M. Caccia, M. Lin, L. Page-Caccia, Online Continual Learning with Maximal Interfered Retrieval, Advances in Neural Information Processing Systems 32 (2019). URL https://proceedings.neurips.c/paper/2019/hash/ 15825aee15eb335c13f9b559f166e8-Abstract.html [31] X. Li, B. Tang, H. Li, AdaER: An adaptive experience replay approach for continual lifelong learning, Neurocomputing 572 (2024) 127204. doi:10.1016/j.neucom.2023.127204. URL https://w.sciencedirect.com/science/article/pii/ S0925231223013279 [32] Y. Balaji, M. Farajtabar, D. Yin, A. Mott, A. Li, The Effectiveness of Memory Replay in Large Scale Continual Learning (Oct. 2020). doi: 10.48550/arXiv.2010.02418. URL http://arxiv.org/abs/2010.02418 [33] H. Shin, J. K. Lee, J. Kim, J. Kim, Continual Learning with Deep Generative Replay (Dec. 2017). doi:10.48550/arXiv.1705.08690. URL http://arxiv.org/abs/1705.08690 25 [34] X. Liu, C. Wu, M. Menta, L. Herranz, B. Raducanu, A. D. Bagdanov, S. Jui, J. v. d. Weijer, Generative Feature Replay For Class-Incremental Learning (Apr. 2020). doi:10.48550/arXiv.2004.09199. URL http://arxiv.org/abs/2004.09199 [35] E. T. Rolls, X. Yan, G. Deco, Y. Zhang, V. Jousmaki, J. Feng, A ven- tromedial visual cortical ‘Where’ stream to the human hippocampus for spatial scenes revealed with magnetoencephalography, Communications Biology 7 (1) (2024) 1–16. doi:10.1038/s42003-024-06719-z. URL https://w.nature.com/articles/s42003-024-06719-z [36] V. Khan, S. Cygert, K. Deja, T. Trzcinski, B. Twardowski, Looking Through the Past: Better Knowledge Retention for Generative Replay in Continual Learning, IEEE Access 12 (2024) 45309–45317. doi:10. 1109/ACCESS.2024.3379148. URL https://ieeexplore.ieee.org/document/10474374 [37] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324. doi:10.1109/5.726791. URL https://ieeexplore.ieee.org/document/726791/ [38] G. M. Van De Ven, T. Tuytelaars, A. S. Tolias, Three types of incre- mental learning, Nature Machine Intelligence 4 (12) (2022) 1185–1197. doi:10.1038/s42256-022-00568-3. URL https://w.nature.com/articles/s42256-022-00568-3 [39] H. Xiao, K. Rasul, R. Vollgraf, Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms (Sep. 2017). doi:10. 48550/arXiv.1708.07747. URL http://arxiv.org/abs/1708.07747 [40] G. Sokar, D. C. Mocanu, M. Pechenizkiy, SpaceNet: Make Free Space For Continual Learning, Neurocomputing 439 (2021) 1–11, arXiv:2007.07617 [cs]. doi:10.1016/j.neucom.2021.01.078. URL http://arxiv.org/abs/2007.07617 [41] N. Niethard, M. Hasegawa, T. Itokazu, C. N. Oyanedel, J. Born, T. R. Sato, Sleep-Stage-Specific Regulation of Cortical Excita- tion and Inhibition, Current Biology 26 (20) (2016) 2739–2749. 26 doi:10.1016/j.cub.2016.08.035. URL https://w.cell.com/current-biology/abstract/ S0960-9822(16)30979-4 [42] P. Skierś, K. Deja, Joint Diffusion models in Continual Learning (Oct. 2025). doi:10.48550/arXiv.2411.08224. URL http://arxiv.org/abs/2411.08224 [43] F. Zhou, T. Zhao, L. V. Nguyen, Z. Yao, A Parallel Gumbel-Softmax VAE Framework with Performance-Based Tuning, in: ECAI 2024, IOS Press, 2024, p. 1784–1791. doi:10.3233/FAIA240689. URL https://ebooks.iospress.nl/doi/10.3233/FAIA240689 [44] E. Goldberg, A New Look at the Old Riddle : Novelty, Routines and the Evolution of the Bicameral Brain 20 (3) (2018). [45] A. D. Grosmark, F. T. Sparks, M. J. Davis, A. Losonczy, Reac- tivation predicts the consolidation of unbiased long-term cognitive maps, Nature Neuroscience 24 (11) (2021) 1574–1585. doi:10.1038/ s41593-021-00920-7. URL https://w.nature.com/articles/s41593-021-00920-7 Appendix A. Hyperparameter Searches and Optimisation This appendix provides detail on the hyperparameter tuning sweeps con- ducted to optimise the 4MAS architecture across the three benchmarks: Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR-100. These searches cover the grid-search sweeps logged in the main project database (comprising 4,328 unique trials) as well as the sequential Bayesian optimisation sweeps conducted using the Optuna framework. Hyperparameter sweeps reported in this Appendix are single runs. Appendix A.1. Split-MNIST and Split-Fashion-MNIST Grid Searches For both Split-MNIST and Split-Fashion-MNIST, we ran systematic pa- rameter grids to evaluate the interaction of model size, latent representation dimensions, and generator capacity under our bilateral sleep-consolidation model. The range of hyperparameters explored in these grid searches is sum- marised in Table A.4. 27 Table A.4: Hyperparameter search space for Split-MNIST and Split-Fashion-MNIST. HyperparameterExplored Values Model Capacity (Parame- ters) 248K, 477K, 736K, 970K, 4.9M, 9.9M, 11.8M, 16.0M, 26.3M, 39.2M, 54.5M, 72.4M, 92.7M Short-Term Memory Size (N stm ) 50 per hemisphere (10 samples per class) VAE Generator Dimension 140, 250, 360, 450, 1450, 2250, 2500, 3000, 4000, 5000, 6000, 7000, 8000 VAE Latent Space Dimen- sion 16, 32, 64, 128, 256 Awake-PhaseLearning Rate (LR awake ) 2× 10 −4 (Adam) Sleep-Phase Learning Rate (LR sleep ) 2× 10 −5 (Adam) Awake-Phase Gating End- Weight 10.0 Left Generator Tempera- ture (τ L ) 1.0 Right Generator Tempera- ture (τ R ) 3.0 Oversampling Factor (O)10, 50 Large model capacity paired with a larger VAE dimension and low la- tent dimensionality (16) consistently yielded optimal Class-IL accuracy by preserving high-fidelity reconstructions without latent-space drift. Appendix A.2. Split-CIFAR-100 Optuna Optimisation Sweeps Due to the increased complexity of CIFAR-100, we executed 14 dis- tinct optimisation sweeps using the Optuna framework, focusing on architec- tural changes, pre-training steps, and latent alignment strategies. The best- performing trial configuration for each sweep is documented in Table A.5. Appendix A.3. Split-CIFAR-100 Architectural Grid Search To clarify the contribution of supervised contrastive loss, left-hemisphere parameter freezing, and bilateral sleep-consolidation (BSS) active gating, we conducted a 16-configuration grid search on Split-CIFAR-100. The complete experimental results of this grid search are summarised in Table A.6. 28 Appendix A.4. Split-CIFAR-100 Ablation and Asymmetry Trials To systematically investigate the roles of comparative confidence selection (CCS), latent space cluster centroids (LSCC), pre-training tasks (1 vs. 5), and synaptic intelligence (SI) regularisation on the Split-CIFAR-100 bench- mark, we evaluated several design configurations. The results are summarised in Table A.7. Appendix A.5. Split-CIFAR-100 Architecture Capacity and Parameter Scal- ing To investigate the capacity limits and scaling robustness of 4MAS relative to B-IR, we evaluated both architectures across a range of total parameter capacities (from 14M to 140M parameters). The comparative results are summarized in Table A.8. As shown in Table A.8, when properly initialized with pre-trained con- volutional features (using the ‘–pre-convE‘ flag), the B-IR baseline scales ro- bustly and monotonically with parameter capacity, rising from 21.0% (13.3M scale) to 27.85% (143.8M scale). Similarly, 4MAS scales monotonically and robustly across all parameter scales, rising from 13.16% (14M scale) to 18.41% (35M scale), 24.45 ± 0.32% (70M scale), and achieving a peak accuracy of 29.26% at 130.8M capacity (fb=1.0, mb=20.0, bg=0.90) under strict single-task pre-training constraints, outperforming the unilateral B-IR base- line as well as the 147.8M v5 configuration (29.01%) which utilized multi-task pre-training. This breakthrough is achieved by resolving representation an- choring bottlenecks at scale via an expanded rehearsal buffer (stmsize = 4000). Importantly, 4MAS maintains significantly lower forgetting (25.8± 1.2% at 70M scale and 23.4% at 140M scale) than B-IR (52.82± 1.97% and 52.92%), verifying that bilateral sleep consolidation consistently provides su- perior retention of past task knowledge across all capacity scales. Appendix A.6. VAE Hyperparameter Seed Replications and Task Trajectories To verify the statistical consistency and trajectory stability of top-performing VAE hyperparameter configurations and Quality-Gated rehearsal regimes, multiple random seed replications were evaluated across all 10 tasks on Split- CIFAR-100. Figure A.10 illustrates the post-REM joint accuracy trajectory as a function of class scale (10 to 100 classes) with standard deviation error bars. 29 Figure A.10: Continual learning post-REM accuracy trajectories across class scale (10 to 100 classes) for top VAE configurations and Quality-Gated replay regimes, showing mean performance and standard deviation error bars across random seeds. 30 Table A.5: Best configurations and parameters across Split-CIFAR-100 Optuna sweeps. Optuna Study NameTrials Best Class-IL (%) Key Parameters and Configurations fourmas_cifar100_v1890.00Default VAE, latent dim 64, LR 0.0014, SGD fourmas_cifar100_v23428.82Task-IL scenario, latent dim 128, buffer 2000, Adam fourmas_cifar100_v3104.11Class-IL, buffer 5000, VAE temp 1.15 (LH) / 1.54 (RH) fourmas_cifar100_v4267.63Class-IL,buffer5000, SWS/REM epochs 75/50, LR 0.0022 fourmas_cifar100_ multi_arch 43.77Pre-trained tap layer 2 feature extractor, latent dim 64 fourmas_cifar100_ scratch_sota 156.85Scratch training (no ImageNet), buffer 10000, latent dim 128 fourmas_cifar100_ novel_designs 249.54Evaluated classifier freezing poli- cies, distill weight 0.16 fourmas_cifar100_ refined_sota 179.49VAE dim 400, latent dim 64, gat- ing end-weight 2.0 fourmas_cifar100_ final_sota 259.56Optimised lr 0.0023, REM/SWS epochs 150/100 fourmas_cifar100_ fixed_sota 139.99Cross-entropy loss function, lin- ear classifier, LR 0.0018 fourmas_cifar100_ sota_push 3411.06Latent dim 192, gating end- weight 1.5, LR 0.0019 fourmas_cifar100_ sota_breakthrough_v2 259.56VAE dim 800, latent dim 192, gating end-weight 1.0 fourmas_cifar100_ pretrain_multi_task 1612.25Bootloaded multi-task pre- training (1 task), latent dim 192 fourmas_cifar100_ vqvae 6210.35Vector Quantized VAE, gating end-weight 2.0, LR 0.0017 fourmas_cifar100_ vqvae_high_cap 329.47High capacity VQ-VAE, code- book size 2048, latent dim 128 31 Table A.6: Grid search performance over generative model types, supervised contrastive (SupCon) weights, left-hemisphere parameter freezing, and BSS active gating. ID Model Type SupCon Weight LH Freeze BSS Active Class-IL (%) Task-IL (%) 1vae0.0FalseFalse1.5212.04 2vae0.0FalseTrue1.029.61 3vae0.0TrueFalse1.1911.17 4vae0.0TrueTrue1.139.26 5vae0.5FalseFalse2.1317.33 6vae0.5FalseTrue2.0114.90 7vae0.5TrueFalse1.0310.32 8vae0.5TrueTrue1.0610.51 9 vqvae0.0FalseFalse1.4110.31 10 vqvae0.0FalseTrue1.0714.19 11 vqvae0.0TrueFalse1.1110.45 12 vqvae0.0TrueTrue1.089.38 13 vqvae0.5FalseFalse1.5212.41 14 vqvae0.5FalseTrue1.0010.87 15 vqvae0.5TrueFalse1.0410.47 16 vqvae0.5TrueTrue1.019.96 Table A.7: Split-CIFAR-100 performance across selection strategies (CCS, LSCC, Hybrid), pre-training task horizons, and local SI regularisation. Selection Strategy (LH / RH) Left Centroid (L) Right Centroid (R) Pre-training Tasks Synaptic Intelligence (SI) Class-IL Acc (%) CCS (Symmetric Baseline)0.50.51No21.38 CCS (Symmetric)0.50.52No25.34 CCS (Symmetric)0.50.53No29.18 CCS (Symmetric)0.50.54No30.46 CCS (Symmetric)0.50.55No29.01 CCS (Left-Skewed)0.70.31No21.78 CCS (Right-Skewed)0.30.71No23.80 CCS (Asymmetric Champion)0.10.51No25.97 CCS (Asymmetric Low)0.10.11No22.96 CCS (Asymmetric High)0.20.81No22.34 CCS (Sweeter Spot)0.050.551No25.63 CCS (Asymmetric + LH SI)0.10.51Yes16.27 LSCC (Symmetric K-Means)K-MeansK-Means1No8.79 LSCC (Symmetric K-Means)K-MeansK-Means5No10.39 Hybrid (LSCC / CCS Asymmetric)K-Means0.51No13.20 Hybrid (LSCC / CCS Asymmetric)K-Means0.52No17.58 Hybrid (LSCC / CCS Asymmetric)K-Means0.53No20.09 Hybrid (LSCC / CCS Asymmetric)K-Means0.54No18.72 Hybrid (LSCC / CCS Asymmetric)K-Means0.55No22.75 Hybrid (LSCC / CCS, Seed 1001)K-Means0.55No20.14 32 Table A.8: Split-CIFAR-100 Class-IL performance across parameter capacity scales. Model Size ArchitectureTotal Parameters Class-IL Acc (%) Forgetting (%) 14M Scale4MAS (Ours, Scaled-down)14.1M13.1625.8 B-IR (Standard Baseline)*13.3M21.061.1 35M Scale4MAS (Ours, Scaled-down)34.5M18.4125.8 B-IR (Ours, Scaled-down)35.8M23.0158.9 70M Scale4MAS (Ours, Champion)70.0M24.45± 0.3225.8± 1.2 B-IR (Ours, Scaled)62.7M26.27± 1.3752.82± 1.97 140M Scale 4MAS (Ours, Rank 1 Peak)130.8M29.29± 0.2925.8 4MAS (Ours, Peak v5)*147.8M29.0123.4 4MAS (Ours, v10)130.8M27.1873.7 4MAS (Ours, v9)130.8M27.1351.0 B-IR (Ours, Scaled)143.8M27.8552.92 33