Paper deep dive
How to Find and Reuse Policies for Continuous Adaptation in Lifelong Reinforcement Learning
Saptarshi Nath, Inish M. D'Souza, Antonio Carta, Soheil Kolouri, Andrea Soltoggio
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In lifelong reinforcement learning, retaining previously learned policies is not sufficient for effective transfer to a new task. Useful knowledge may be distributed across several prior policies, and its relevance may change as the learner acquires experience. One hypothesis is that task similarity can be effectively used in a continual learning setting to find and combine previously learned policies. To test it, Adaptive Mask Selection and Composition (AMSC) is designed to estimate similarity from online experience via non-parametric Wasserstein task embeddings from state-action-reward samples. The z-score-normalized sparsemax of the similarity scores are used to derive a variable-size support to periodically choose and weight policies to form a prior when learning a new task. On CT-graph and MiniGrid, AMSC achieves higher mean performance and forward transfer than the evaluated modular composition baselines while exhibiting no forgetting. Results on Continual World suggest that identifying relevant prior knowledge and determining its layer-specific composition may require additional layer-specific tuning. Ablations show that selecting relevant sources and determining how strongly to reuse them are central to these gains. Independently measured pairwise transfer is also positively associated with task-embedding similarity. These results indicate that task similarity can be an effective criterion to select and weight specific knowledge for reuse in lifelong reinforcement learning.
Tags
Links
- Source: https://arxiv.org/abs/2610.03119v1
- Canonical: https://arxiv.org/abs/2610.03119v1
Trouble viewing inline? Open PDF directly →
Full Text
125,472 characters extracted from source content.
Expand or collapse full text
Preprint HOW TO FIND AND REUSE POLICIES FOR CONTIN- UOUS ADAPTATION IN LIFELONG REINFORCEMENT LEARNING Saptarshi Nath, Inish D’Souza & Andrea Soltoggio Department of Computer Science Loughborough University s.nath,i.m.d-souza,a.soltoggio@lboro.ac.uk Antonio Carta Department of Computer Science University of Pisa antonio.carta@unipi.it Soheil Kolouri Department of Computer Science soheil.kolouri@vanderbilt.edu ABSTRACT In lifelong reinforcement learning, retaining previously learned policies is not suf- ficient for effective transfer to a new task. Useful knowledge may be distributed across several prior policies, and its relevance may change as the learner acquires experience. One hypothesis is that task similarity can be effectively used in a con- tinual learning setting to find and combine previously learned policies. To test it, Adaptive Mask Selection and Composition (AMSC) is designed to estimate sim- ilarity from online experience via non-parametric Wasserstein task embeddings from state-action-reward samples. The z-score-normalized sparsemax of the sim- ilarity scores are used to derive a variable-size support to periodically choose and weight policies to form a prior when learning a new task. On CT-graph and Min- iGrid, AMSC achieves higher mean performance and forward transfer than the evaluated modular composition baselines while exhibiting no forgetting. Results on Continual World suggest that identifying relevant prior knowledge and deter- mining its layer-specific composition may require additional layer-specific tuning. Ablations show that selecting relevant sources and determining how strongly to reuse them are central to these gains. Independently measured pairwise transfer is also positively associated with task-embedding similarity. These results indi- cate that task similarity can be an effective criterion to select and weight specific knowledge for reuse in lifelong reinforcement learning. 1 1INTRODUCTION Lifelong learning requires both retention and reuse of prior knowledge. Although forgetting remains a central challenge, retention alone does not ensure positive transfer to later tasks. As experience accumulates, a model must identify relevant knowledge without allowing irrelevant or conflicting knowledge to cause interference. Prior work has extensively studied retention and increasingly evaluates forward transfer (Rusu et al., 2016; Kirkpatrick et al., 2017; Zenke et al., 2017; Mallya & Lazebnik, 2018; Rolnick et al., 2019; von Oswald et al., 2020), but commonly reuses shared or aggregated knowledge. Selective reuse remains comparatively underexplored. This distinction is especially consequential in reinforcement learning (RL). Transferred knowledge affects the actions taken and the state-action distribution from which the policy learns. An unsuit- able prior can bias exploration and induce negative transfer (Taylor & Stone, 2009; Ahn et al., 2025; Uchendu et al., 2023). Sparse rewards provide little early evidence that a policy is harmful (Gimel- farb et al., 2021). Moreover, useful structure may be distributed unequally across several priors. 1 https://github.com/Chocological45/amsc 1 arXiv:2610.03119v1 [cs.LG] 2 Oct 2026 Preprint Existing lifelong RL methods transfer through replay, modular routing, parameter isolation, and learned composition (Isele & Cosgun, 2018; Kessler et al., 2022; Powers et al., 2022; Mendez et al., 2022; Ben-Iwhiwhu et al., 2023; Zhang et al., 2024; Malagón et al., 2024; Hu et al., 2025; Huang et al., 2025). Modular approaches can preserve completed policies, but reusing the full library may admit interfering priors. Learning how strongly to reuse each policy from the target objective can identify useful policies but may converge only after their opportunity to improve early learning has passed. Measured task similarity can instead act as a method of estimating the relevance of prior policies without optimization (Carroll & Seppi, 2005; Taylor & Stone, 2009; Li et al., 2019; Gimelfarb et al., 2021; Wu et al., 2025; Nath et al., 2026). In task-incremental lifelong RL, the availability of prior policies is determined by curriculum history, completed policies are frozen, and the number of useful sources is unknown. A suitable selector must therefore adapt to the current library. As experience is collected on a new task, both the selected knowledge and its contribution to learning may also change. We ask whether online task similarity can determine both which policies to reuse and how strongly they should contribute under these constraints. −0.4−0.20.00.20.40.60.8 Plasticity (FWT) 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Performance (AUC) Method Fine-tuning No-reuse SER-s SER-r SDW OWL SANE Mask-BLC Mask-LC CKA-RL CompoNet MOSAIC MOSAIC-noperf AMSC 0.0 0.1 0.2 0.3 0.4 0.5 Stability (FGT) Figure 1: Transfer-retention trade-off on CT28 and MG16. Each marker reports mean FWT and AUC across seven seeds; colour denotes FGT. Er- ror bars are 95% confidence intervals. Adaptive Mask Selection and Composition (AMSC) learns task-specific policy masks over a fixed backbone network. It uses online task similarity to select a variable-size support of prior masks and determine their composition weights.Selected and current mask scores are normalized before composition with an ℓ 2 straight-through estimator, and only the current policy-mask scores are optimized. AMSC se- lects without needing task descriptions, task la- bels, or transfer probes. AMSC is evaluated on CT-graph (CT28), Min- iGrid (MG16), and Continual World (CW10). It obtains AUC/FWT/FGT of 0.94/0.94/0.00 on CT28 and 0.73/0.54/0.00 on MG16 (Fig- ure 1). In CT28 and MG16, similarity-derived weights perform comparably to learned com- position weights. In CW10, similarity-derived weights underperform with respect to learned mask composition. Learning composition over a sparse selection of prior policies may provide a performance benefit. Ablations show that selecting relevant policies from the agent’s library and determining how to reuse each through similarity or learned weights are critical to more effective forward transfer and successful learning in lifelong RL. Results also show that task similarity is pos- itively associated with independently measured pairwise transfer. Lastly, results suggest that masks can be quantized with minimal forgetting and loss in AUC and FWT. 2RELATED WORK Selective reuse in lifelong reinforcement learning. Forward transfer and forgetting are distinct objectives in lifelong RL. Preserving earlier tasks does not imply faster learning on later tasks (Wol- czyk et al., 2021; 2022). Prior work promotes selective reuse through sparse or compositional sub- networks (Sokar et al., 2022; Ostapenko et al., 2021), related and unrelated task sequences (Ke et al., 2020), selective replay (Isele & Cosgun, 2018; Huang et al., 2025), and adaptive structure alloca- tion (Lee et al., 2021; Dekhovich et al., 2023; Savadikar et al., 2026). These methods select data, capacity, or structures, often to determine what knowledge is worth retaining. Modular routing and composition. Modular lifelong RL isolates task-specific knowledge and transfers through routing or composition. Prior methods activate modules, select policy heads, infer contexts, or route adapters (Powers et al., 2022; Kessler et al., 2020; Zhang et al., 2024; Julian et al., 2025); others compose masks, modules, policies, or task vectors (Ben-Iwhiwhu et al., 2023; Kessler et al., 2022; Hu et al., 2025; Malagón et al., 2024). 2 Preprint Task similarity and optimal transport. Task similarity predicts transfer and supports policy se- lection in RL (Carroll & Seppi, 2005; Taylor & Stone, 2009; Li et al., 2019; Gimelfarb et al., 2021), while transferability methods use representations, probe networks, likelihoods, or parameter sub- spaces (Zamir et al., 2018; Achille et al., 2019; Nguyen et al., 2020; Lin et al., 2022; Wu et al., 2025). Optimal transport compares task distributions through OTDD and Wasserstein embeddings (Courty et al., 2018; Alvarez-Melis & Fusi, 2020; Liu et al., 2025), and supports memory, selection, distillation, context detection, and expert diversity (Wang et al., 2022; 2024; Xu et al., 2023; Dick et al., 2024; Hihn & Braun, 2022). 3PRELIMINARIES 3.1PROBLEM SETTING Consider a task-incremental lifelong RL setting with a sequence of K tasksT k K k=1 . Each task is an MDPT k = (S, A,p k ,r k ,γ rl ), where S and A are the state and action spaces, p k is the transition kernel, r k is the reward function, and γ rl is the discount factor. Tasks arrive sequentially with known boundaries, and only the current task environment is available for interaction. The learner trains on each task for a fixed budget before proceeding to the next task. A lifelong learning agent aims to maintain performance on previously learned tasks, or improve it through backward transfer, while continuing to learn new tasks through forward transfer. Forward transfer measures whether knowledge acquired from earlier tasks accelerates learning of a new task (Parisi et al., 2019; Khetarpal et al., 2022; Wolczyk et al., 2021). 3.2MODULAR POLICY MASKS Score-based masking learns task-specific subnetworks over a shared, fixed backbone. Supermasks in Superposition stores multiple tasks as binary masks over the same weights (Wortsman et al., 2020). Modulating Masks adapts this mechanism to lifelong RL and introduces learned linear combinations of the current and completed task masks (Ben-Iwhiwhu et al., 2023). CKA-RL is a related modu- lar composition method that combines and merges dense task-specific residuals rather than binary masks (Hu et al., 2025). LetW ℓ L ℓ=1 be the fixed policy-backbone matrices and S ℓ k the real-valued score matrix for task k at masked layer ℓ. The mask in the forward pass is M ℓ k = g(S ℓ k ), g(x) = 1 x≥0 ,(1) where g is applied elementwise. The effective parameters are W ℓ ⊙ M ℓ k . A straight-through esti- mator propagates gradients through g. Let S ℓ,∗ i be the frozen score matrix stored after task i, and let P(k) ⊆ 1,...,k− 1 contain the priors admitted for task k. A generic convex score composition is S ℓ,comb k = X i∈P(k) α ℓ k,i S ℓ,∗ i + α ℓ k,k S ℓ k , X i∈P(k)∪k α ℓ k,i = 1,(2) with non-negative weights. Ben-Iwhiwhu et al. (2023); Hu et al. (2025) use the complete prior library and learn layer-specific weights. For m > 0, Ben-Iwhiwhu et al. (2023) initializes the coefficients uniformly in Mask-LC or balances prior and current-mask mass in Mask-BLC. For m =|P(k)| > 0, Mask-LC: α ℓ,(0) k,k = α ℓ,(0) k,i = 1 m + 1 , Mask-BLC: α ℓ,(0) k,k = 1 2 , α ℓ,(0) k,i = 1 2m , i∈ P(k). (3) When m = 0, the current mask has an initial weight of 1. Prior scores remain frozen, and the final composition is consolidated, S ℓ,∗ k ← S ℓ,comb k , for later tasks. 4ADAPTIVE MASK SELECTION AND COMPOSITION AMSC consists of three operations: normalized sparsemax retrieval, scale-normalized mask scores, and composition from the retrieval weights. 3 Preprint 4.1MASK LIBRARY AND ONLINE TASK REPRESENTATIONS At task k, AMSC initializes a new current score matrix S ℓ k and the policy library contains the frozen score matrices S ℓ,∗ i L i<k,ℓ=1 following the formulation defined in Section 3.2. Let D k be the current-task experience buffer and X (t) k a sampled batch matrix. AMSC uses an embedding map ψ(X (t) k ) = e (t) k ∈ R d e that places online experience from different tasks in a common vector space. The main experiments use a non-parametric linear Wasserstein embedding (WTE) of state– action–reward samples (Liu et al., 2025) which has been demonstrated in RL tasks (Nath et al., 2026). Discrete actions are one-hot encoded, states are standardized within each sampled batch, and rewards are retained as observed. Given a fixed reference matrix X 0 with R samples and an optimal transport matrix Γ ⋆ , the WTE batch embedding is e (t) k = vec (RΓ ⋆ ) ⊤ X (t) k − X 0 √ R ! .(4) The current representation is smoothed online as ̄ e (t) k = normalize ρ ̄ e (t−1) k + (1− ρ)e (t) k ,(5) where the first update initializes the normalized representation and ρ = 0.5 thereafter. Completed task representations remain fixed. Appendix B.1 formalizes the optimal transport plan and the em- pirical distributions represented by X (t) k and X 0 . 4.2SPARSE RETRIEVAL Cosine similarity has been used to evaluate task relationships through gradient alignment, and task embeddings (Lopez-Paz & Ranzato, 2017; Achille et al., 2019; Yu et al., 2020a; Nath et al., 2026). Following this, AMSC compares the current representation of task k with every completed task using cosine similarity, s k,i = cos( ̄ e k , ̄ e i ), i < k.(6) Let s k = [s k,1 ,...,s k,k−1 ] ⊤ . AMSC retrieves w k = sparsemax(zscore(s k )/τ ),P(k) =i < k : w k,i > 0.(7) The z-score has zero mean and unit variance within the available library, and τ = 1 in all main experiments. Sparsemax projects these relative scores onto the probability simplex and assigns exact zeros, so P i∈P(k) w k,i = 1 (Martins & Astudillo, 2016). 4.3SCALE-NORMALIZED FIXED COMPOSITION Positive rescaling of a task’s scores changes its influence when scores from several tasks are com- bined before thresholding. Large magnitude scores become more difficult to alter. Consequently, raw score magnitude can override the linear combination weights. We therefore normalize each se- lected prior and current score to unit ℓ 2 norm in the forward pass, using a straight-through estimator to preserve the current score’s gradient, b S ℓ i = S ℓ i + sg S ℓ i max(∥S ℓ i ∥ F ,ε m ) − S ℓ i ,(8) where sg denotes stop-gradient. Learning composition weights from the RL objective requires the agent to discover useful sources through task interaction. AMSC instead can use the same online similarity signal for both retrieval and weighting, making source contributions available at the first retrieval event without optimization. Let m = |P(k)|. Instead of learning the composition, AMSC applies weights computed from task similarity which, C ℓ k = b S ℓ k ,m = 0, 1 m + 1 b S ℓ k + m m + 1 X i∈P(k) w k,i b S ℓ i , m > 0, M ℓ k = g(C ℓ k ).(9) 4 Preprint For m > 0, the current mask receives mass 1/(m + 1), while the selected priors share m/(m + 1) according to their retrieval weights. These weights are shared across layers. Each task begins without retrieved priors. Once sufficient experience is available, AMSC updates the current embedding and recomputes equation 7–equation 9 between training iterations to reflect new data. At the task boundary, the final composed scores C ℓ k are consolidated as S ℓ,∗ k ← C ℓ k , frozen, and added to the reusable mask library. Appendix B.2 Algorithm 1 gives the complete training pseudo-code. 5RESULTS 5.1EXPERIMENTAL SETUP Benchmarks. We evaluate AMSC on CT28, a sparse-reward decision-tree curriculum with hierar- chical tasks (Soltoggio et al., 2023); MG16, sparse-reward MiniGrid navigation tasks with shuffled observations (Chevalier-Boisvert et al., 2018); and CW10, Continual World continuous robotic con- trol tasks (Wolczyk et al., 2021). In CT28 and MG16, tasks within each family share a common observation space and increase in complexity. Families differ in their representations while sharing input dimensions and actions. These curricula test reuse when prior knowledge varies in relevance in relation to the next task. Details and pairwise-transfer matrices are reported in Appendix C. Baselines. We compare against replay methods (SER (Isele & Cosgun, 2018) and SDW (Huang et al., 2025)), modular methods (Mask-LC/BLC (Ben-Iwhiwhu et al., 2023), CKA-RL (Hu et al., 2025), CompoNet (Malagón et al., 2024), OWL (Kessler et al., 2022), SANE (Powers et al., 2022), MOSAIC (Nath et al., 2026), and ProgNet (Rusu et al., 2016)), fine-tuning, and learning from scratch. We also evaluate retrieval guided by independently measured pairwise transfer. Tables 1 and 2 report comparisons to baselines on CT28, MG16 and CW10. Appendix E provides details of the hyperparameters for AMSC and baselines. Protocol and Metrics. All empirical results are evaluated with seven shared seeds. CT28 and MG16 use PPO with 51, 200 and 512, 000 steps per task, respectively. CW10 uses SAC with 1M steps per task. We report mean task area under the evaluation curve (AUC), forward transfer (FWT), and forgetting (FGT) from normalized performance between [0, 1], following (Wolczyk et al., 2021). We report these metrics together because zero forgetting can reflect strong retention but not neces- sarily strong policies. Appendix E specifies training and evaluation hyperparameters, Appendix F.3 reports hyperparameter sensitivity, and Appendix G reports the details of how the metrics are com- puted. 5.2COMPARISON TO BASELINES Selective reuse improves learning. AMSC obtains strong AUC and FWT with zero measured forgetting on CT28 and MG16 (Table 1), occupying a favorable region of the transfer-retention trade- off (Figure 1) with respect to baselines. AMSC achieves AUC/FWT of 0.94/0.94 and 0.73/0.54 on CT28 and MG16, respectively. From-scratch learning also avoids forgetting but obtains only 0.22 and 0.23 AUC. Surprise-prioritized SER slightly exceeds AMSC on CT28 (0.96/0.95) but incurs 0.54 FGT, while reservoir SER obtains 0.63/0.20 on MG16 with 0.07 FGT. Comparison with modular reuse. AMSC achieves the highest mean AUC and FWT among mod- ular methods on CT28 and MG16 (Table 1). These methods differ in how they reuse prior knowl- edge: Mask-LC/BLC learn weights over all prior masks, CompoNet learns to combine policy-unit outputs, and MOSAIC selects masks through absolute similarity and performance gates before opti- mizing composition. AMSC instead periodically updates both its selected sources and their weights using library-relative similarity. On CT28, AMSC achieves AUC/FWT of 0.94/0.94, compared with 0.81/0.79 for MOSAIC and 0.69/0.64 for CompoNet. On MG16, both baselines have nega- tive FWT, whereas AMSC achieves 0.54. AMSC’s gains over Mask-LC, CompoNet, and MOSAIC are significant on both metrics and curricula (p ≤ 0.023). Its MG16 advantage over Mask-BLC (0.73/0.54 versus 0.55/0.23, p = 0.016 and p = 0.016, respectively). These results associate 5 Preprint Table 1: Evaluation on CT28 and MG16. Each cell reports the mean AUC/FWT/FGT with an 95% confidence interval (Colas et al., 2018). AVG is computed per seed by averaging CT28 and MG16, then bootstrapping the seven paired averages. Bold and underlining denote the best and second-best non-oracle result for each metric. Significance tests are reported in Appendix H. CT28MG16AVG MethodAUC↑FWT↑FGT↓AUC↑FWT↑FGT↓AUC↑FWT↑FGT↓ Fine-tuning.36 +.02 −.02 .22 +.03 −.03 .38 +.02 −.02 .30 +.05 −.05 .10 +.08 −.09 .34 +.08 −.07 .33 +.02 −.02 .16 +.05 −.06 .36 +.03 −.03 From-scratch .22 +.01 −.01 .00 +.00 −.00 .00 +.00 −.00 .23 +.03 −.02 .00 +.00 −.00 .00 +.00 −.00 .22 +.01 −.01 .00 +.00 −.00 .00 +.00 −.00 SER-s.96 +.02 −.03 .95 +.03 −.03 .54 +.17 −.18 .42 +.06 −.07 -.13 +.17 −.17 .51 +.10 −.09 .69 +.03 −.04 .41 +.08 −.08 .52 +.09 −.11 SER-r.88 +.05 −.05 .87 +.06 −.06 .09 +.08 −.07 .63 +.04 −.03 .20 +.14 −.20 .07 +.11 −.09 .76 +.02 −.02 .53 +.08 −.10 .08 +.09 −.06 SDW.76 +.11 −.10 .73 +.13 −.13 -.01 +.01 −.01 .53 +.05 −.06 .02 +.12 −.18 .15 +.13 −.12 .65 +.06 −.07 .37 +.10 −.14 .07 +.06 −.06 OWL.15 +.01 −.01 -.31 +.17 −.20 .18 +.03 −.04 .21 +.03 −.02 -.29 +.18 −.19 .22 +.03 −.02 .18 +.02 −.02 -.30 +.09 −.09 .20 +.02 −.02 SANE.46 +.02 −.03 .37 +.03 −.03 .00 +.00 −.00 .36 +.09 −.08 .18 +.11 −.09 .00 +.00 −.00 .41 +.04 −.04 .28 +.05 −.04 .00 +.00 −.00 Mask-BLC.27 +.02 −.02 .10 +.06 −.06 .00 +.00 −.00 .55 +.10 −.10 .23 +.21 −.19 .00 +.00 −.00 .41 +.06 −.06 .17 +.13 −.12 .00 +.00 −.00 Mask-LC.48 +.03 −.03 .35 +.06 −.07 .00 +.00 −.00 .40 +.14 −.13 -.24 +.41 −.37 .00 +.00 −.00 .44 +.06 −.06 .06 +.21 −.20 .00 +.00 −.00 CKA-RL.36 +.05 −.04 .25 +.05 −.05 .36 +.05 −.05 .34 +.11 −.09 -.21 +.20 −.24 .40 +.14 −.12 .35 +.07 −.06 .02 +.11 −.13 .38 +.08 −.06 CompoNet.69 +.08 −.10 .64 +.09 −.12 .00 +.00 −.00 .31 +.07 −.08 -.85 +.24 −.25 .00 +.00 −.00 .50 +.05 −.06 -.10 +.15 −.15 .00 +.00 −.00 MOSAIC.81 +.03 −.03 .79 +.03 −.03 .00 +.00 −.00 .29 +.11 −.11 -.50 +.40 −.45 .00 +.00 −.00 .55 +.05 −.06 .14 +.20 −.22 .00 +.00 −.00 MOSAIC¬I perf .81 +.03 −.03 .78 +.03 −.03 .00 +.00 −.00 .29 +.13 −.11 -.60 +.47 −.50 .00 +.00 −.00 .55 +.06 −.05 .09 +.22 −.24 .00 +.00 −.00 Transfer oracle .82 +.00 −.01 .72 +.07 −.13 .00 +.00 −.00 .66 +.08 −.09 .54 +.10 −.10 .00 +.00 −.00 .74 +.04 −.04 .63 +.08 −.09 .00 +.00 −.00 AMSC.94 +.02 −.03 .94 +.02 −.03 .00 +.00 −.00 .73 +.06 −.06 .54 +.12 −.14 .00 +.00 −.00 .84 +.03 −.03 .74 +.06 −.07 .00 +.00 −.00 explicit, relative selection and composition with stronger reuse than dense composition or fixed absolute selection. Table 2: CW10 performance across seven seeds. Entries report means with 95% confidence intervals. MethodAUCFWTFGT Fine-tuning .34 +.17 −.18 -7.12 +7.20 −13.38 .39 +.19 −.21 From-scratch .41 +.04 −.04 .00 +.00 −.00 .00 +.00 −.00 PNN.34 +.02 −.03 -.31 +.14 −.14 .00 +.00 −.00 Mask-LC .47 +.06 −.06 -.22 +.34 −.40 .00 +.00 −.00 Mask-BLC .53 +.04 −.04 .12 +.19 −.20 .00 +.00 −.00 CKA-RL .39 +.10 −.14 -2.12 +1.76 −2.24 .00 +.00 −.00 MOSAIC .26 +.04 −.03 -4.76 +3.74 −6.41 .00 +.00 −.00 AMSC.48 +.05 −.05 -1.21 +.79 −.98 .00 +.00 −.00 Early learning across reuse mechanisms. Over the first 20% of CT28 tasks 1–27, AMSC obtains AUC 20 = 0.86, compared with 0.35 for MOSAIC (p = 0.016). The first pos- itive training reward appears after 2.34k versus 5.83k steps (p = 0.016). SER-surprise reaches 0.91 early AUC and SER- reservoir 0.86; neither differs significantly from AMSC on this metric. Thus AMSC’s early advantage is clear relative to MOSAIC, while replay can match or exceed its early learn- ing. Appendix Tables 8 and 9 report aggregate and per-task results. 5.3LIMITS OF SIMILARITY WEIGHTING Does fixed similarity weighting extend to continuous control? On CW10, AMSC obtains 0.48 AUC with zero measured forgetting, but its FWT is−1.21; Mask-BLC obtains 0.53 AUC and 0.12 FWT (Table 2). Their composition weights overlap on some tasks, yet AMSC often allocates more mass to prior masks later in the curriculum. At task 9, mean actor prior mass is 0.78 for AMSC versus 0.46 for Mask-BLC (Appendix F.2). AMSC applies the same similarity-derived proportions across masked layers, whereas Mask-BLC learns layer-specific weights. This difference suggests that identifying relevant prior policies and determining how to compose their knowledge are distinct challenges, even when learned composition addresses both through a single set of weights. 5.4TASK REPRESENTATION SENSITIVITY One question is whether the performance of AMSC is dependent on the specific representation ap- proach, or whether alternatives to WTE can be used with similar performance. Table 3 compares WTE (Liu et al., 2025), sliced-Wasserstein embeddings (Naderializadeh et al., 2021), an empirical- Fisher adaptation of Task2Vec (Achille et al., 2019), recurrent encoders motivated by PEARL and VariBAD (Rakelly et al., 2019; Zintgraf et al., 2021), and FID-inspired Fréchet moments (Heusel et al., 2017). All comparisons receive complete (s,a,r,s ′ ,d) as Task2Vec-style and learned con- 6 Preprint 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 Causal transfer ( AUC) 0.0 0.2 0.4 0.6 0.8 1.0 Selection frequency D2D3D4D5D6D7D8 Cross D2 D3 D4 D5 D6 D7 CT28 0.75 1.000.330.120.030.01<0.01 0.670.300.130.050.01 0.580.310.210.07 0.530.300.20 0.440.32 0.39 (A) Retrieval and causal transfer D2D3D4D5D6D7D8 0.0 0.2 0.4 0.6 0.8 1.0 Mean composition mass 0.62 0.50 0.33 0.27 0.25 0.23 0.22 0.50 0.22 0.09 0.45 0.22 0.10 0.42 0.23 0.16 0.06 0.39 0.23 0.16 0.33 0.25 0.30 0.38 (B) Similarity-weighted composition Component Current D2 D3 D4 D5 D6 D7 Cross 0.40.60.81.0 0.00 0.25 0.50 0.75 1.00 Pairwise transfer ( AUC) Pearson r=0.67 Spearman =0.52 n=350 (C) Similarity and transfer ( AUC) R2R3R4R5 Target depth Cross R2 R3 R4 MG16 0.750.090.01<0.01 0.910.270.06 0.720.28 0.65 R2R3R4R5 Target depth 0.0 0.2 0.4 0.6 0.8 1.0 Mean composition mass 0.61 0.42 0.34 0.30 0.52 0.18 0.04 0.47 0.20 0.45 0.39 0.06 Component Current R2 R3 R4 Cross 0.40.60.81.0 Online task similarity 0.5 0.0 0.5 1.0 Pairwise transfer ( AUC) Pearson r=0.48 Spearman =0.42 n=120 Figure 2: AMSC on CT28 (top) and MG16 (bottom). Same-family source policies share the target task’s observation space. (A) Mean pairwise ∆AUC by source category. Annotations give the mean total sparsemax retrieval mass assigned to that category. (B) Mean composition mass after applying Equation 9 to selected sources. (C) Online similarity versus pairwise transfer; color denotes selection frequency and the line shows bin means with target-cluster 95% confidence intervals. CT28 includes depths D4–D8, whereas MG16 includes all causal non-self pairs. Retrieval uses seven seeds; pairwise transfer uses one source-policy seed. text encoder variants require the reward and next state as part of the objective. The main AMSC experiments use only (s,a,r) (also reported in Table 3 for reference). Table 3: Task-representation sensitivity of AMSC. Reports AUC/FWT with 95% confi- dence intervals across seven seeds. T denotes complete transitions; R and L denote random and online-learned. Bold and underlining in- dicate the best and second-best results. MethodCT28 AUC↑/FWT↑ MG16 AUC↑/FWT↑ WTE (T) .93 +.04 −.04 /.92 +.04 −.04 .61 +.07 −.08 /.38 +.12 −.14 SWE (T) .92 +.04 −.03 /.91 +.05 −.05 .67 +.04 −.04 /.39 +.15 −.16 Fisher (T2V) .87 +.06 −.07 /.85 +.08 −.10 .45 +.12 −.10 /.21 +.17 −.18 Context (R) .92 +.03 −.03 /.91 +.03 −.04 .64 +.06 −.07 /.41 +.10 −.12 Context (L) .90 +.03 −.03 /.87 +.06 −.06 .44 +.09 −.10 /.14 +.21 −.28 Fréchet.90 +.03 −.03 /.87 +.06 −.07 .70 +.07 −.07 /.47 +.13 −.16 WTE (SAR) .94 +.02 −.03 /.94 +.02 −.03 .73 +.06 −.06 /.54 +.12 −.14 Representation choice is task-dependent but AMSC remains agnostic. Among the variants, WTE–transition has the highest mean AUC/FWT on CT28 (0.93/0.92), while Fréchet moments lead on MG16 (0.70/0.47). SWE and random recurrent fea- tures also yield positive transfer on both curricula. AMSC therefore accommodates several representa- tions, although performance depends on the specific distributions being measured and specific configura- tions used and no single approach is universally best. 5.5RETRIEVAL, COMPOSITION, AND CAUSAL TRANSFER We test whether similarity identifies useful prior knowledge using independently measured pairwise transfer. For each source–target pair, we mea- sure ∆AUC (target task AUC with a source minus AUC without it). Does task similarity rank useful priors? Figure 2C presents the correlation between pairwise transfer and similarity. Online similarity correlates with pairwise transfer on CT28 depths 4–8 (Pear- son r = 0.67, 95% CI [0.57, 0.77] and MG16 (r = 0.48, [0.31, 0.62]) with Spearman ρ = 0.52 and 0.42. These correlations suggest that similarity is informative of which prior masks are likely to transfer. Does retrieval exploit this ranking? We define a pair as regularly selected if it is retrieved in at least half of its available selection events. On CT28, regularly selected pairs have mean ∆AUC = 0.66, versus 0.02 for other pairs. These pairs cover 81% of positive-transfer pairs and include no negative-transfer pairs. On MG16, corresponding means are 0.49 and 0.15, but retrieval covers only 19 of 47 positive-transfer pairs as a result of the sparse selection. 7 Preprint How is reuse allocated within priors? On CT28, support grows from one prior at depth 3 to ap- proximately four at depth 8, reducing current-mask mass from 0.50 to 0.22. At depth 8, predecessors at depths 7 and 6 receive 0.30 and 0.25 of the mass. MG16 shows a similar pattern: at depth-5 (R5), the current mask and depth-4 (R4) and depth-3 (R3) masks receive 0.30, 0.45, and 0.20 and below 0.01 to selected cross-family masks. Reuse therefore concentrates on related hierarchical sub-task masks obtained earlier in learning, while retaining a current-task component (Figure 2B). 5.6ABLATION STUDIES Table 4:Ablation of AMSC’s selection and composition mechanism.All runs use WTE embeddings of state–action–reward tu- ples. Values are means with 95% confidence intervals. All configurations obtain zero FGT. Configuration CT28 AUC↑/FWT↑ MG16 AUC↑/FWT↑ No Sim.-α .90 +.03 −.03 /.88 +.04 −.04 .61 +.10 −.10 /.41 +.15 −.15 No L2-STE .89 +.03 −.03 /.86 +.03 −.03 .59 +.09 −.10 /.21 +.26 −.29 No Periodicity .87 +.04 −.05 /.85 +.05 −.06 .63 +.11 −.13 /.42 +.14 −.15 Top-1.88 +.04 −.04 /.86 +.05 −.04 .63 +.09 −.08 /.39 +.16 −.16 Learned-α .92 +.03 −.03 /.91 +.04 −.04 .60 +.11 −.11 /.33 +.15 −.14 Random.35 +.02 −.02 /.13 +.07 −.06 .36 +.05 −.06 /.08 +.13 −.13 Transf. Oracle .82 +.00 −.01 /.72 +.07 −.13 .66 +.08 −.09 /.54 +.10 −.10 AMSC.94 +.02 −.03 /.94 +.02 −.03 .73 +.06 −.06 /.54 +.12 −.14 Controls. Table 4 tests whether the performance gains of AMSC are attributed to the combination of components. Random permutes selected pri- ors while preserving how many are selected and their specific similarity-derived weights. Learned- α optimizes the retrieved weights, whereas No Sim.-α tests a uniform composition over selected prior masks. No Periodicity retains the first re- trieval for the entire learning duration. Top-1 re- tains the highest-similarity prior after sparsemax selection. Transf. Oracle uses independent pair- wise ∆AUC measurements to guide retrieval of multiple masks but is not an upper bound on joint composition. Does source relevance matter beyond sparsity? Randomizing which prior masks are selected significantly reduces AUC/FWT to 0.35/0.13 on CT28 and 0.36/0.08 on MG16 (p = 0.016). Be- cause support size and weights are preserved, these reductions show that sparse composition alone is insufficient and selecting relevant priors is central to the gains. Do retrieval updates and multiple sources improve transfer? Freezing the first retrieval lowers AUC/FWT to 0.87/0.85 on CT28 and 0.63/0.42 on MG16. These decreases suggest that revis- ing the support and weights as experience accumulates contributes beyond the initial selection. In AMSC, selection remains stable throughout learning. Consecutive-selection Jaccard similarity is 0.94–1.00 on CT28 and 0.97–1.00 on MG16. AMSC successfully identifies relevant prior masks early on in learning then gradually refines its selection. Restricting retrieval to one prior mask re- sults in 0.88/0.86 and 0.63/0.39, respectively. Since total prior mass is unchanged, this comparison supports composing multiple sources rather than assigning the entire reuse allocation to the highest- ranked prior. Transfer-informed retrieval. Transfer oracle also obtains positive FWT, obtaining AUC/FWT of 0.82/0.72 on CT28 and 0.66/0.54 on MG16. It nevertheless falls below AMSC on CT28 and matches only its FWT on MG16. Selection is guided by pair-wise transfer, therefore it does not provide an upper bound on joint composition. Do composition weights need to be learned? Learning weights within the retrieved support yields AUC/FWT of 0.92/0.91 on CT28 and 0.60/0.33 on MG16, compared with 0.94/0.94 and 0.73/0.54 for AMSC. Optimization thus provides no improvement in mean performance in these experiments. Replacing similarity-derived source proportions with uniform weights also lowers performance, to 0.90/0.88 and 0.61/0.41. These comparisons suggest that similarity provides use- ful information about how strongly to reuse selected masks, beyond determining which masks enter the composition; however, task similarity in AMSC cannot determine how knowledge should be uniquely composed at every layer of the network as indicated by the earlier CW10 results. Does mask-score normalization matter? Composition weights determine source contributions only relative to the magnitudes of the scores being combined. Removing L2-STE lowers AUC/FWT to 0.89/0.86 on CT28 and 0.59/0.21 on MG16. These results support normalizing scores before composition so that differences in score magnitude do not override the intended similarity-derived allocation. 8 Preprint 5.7ADDRESSING PER-TASK MASK MEMORY COSTS Continuous-value masks are memory intensive. We test whether masks can be quantized with minimal performance loss. Ternary quantization maps scores to −1, 0, 1, preserving their sign while suppressing near-zero scores. On CT28, ternary quantization achieves AUC/FWT/FGT of 0.94/0.91/0.04 compared to 0.941/0.935/0.000 for Float32 masks. Ternary scores also reduce per-task storage by 16× relative to Float32, while retaining better AUC/FWT (Appendix A). 6DISCUSSION AND FUTURE WORK The results suggest that task similarity can guide effective policy reuse without directly optimizing retrieval or composition weights when relevant prior knowledge is available. On CT28 and MG16, AMSC achieves competitive AUC and FWT relative to replay methods while avoiding forgetting through per-task masks. The random-selection ablation indicates that sparsity alone is insufficient. Periodic retrieval further improves reuse as more experience is collected and task representations are updated. The differing performance of transition embeddings and Fréchet moments also suggests that repre- sentation choice depends on the distributions. Future work could evaluate combinations of priors beyond pairwise transfer to determine how reliably embedding similarity predicts joint transfer. The CW10 results highlight a distinction between identifying relevant knowledge and determining how to reuse it. Similarity-derived weights shared across layers may be too restrictive for effective com- position. Future work could test whether task embeddings can guide layer-specific reuse or whether similarity-based selection could be combined with learned composition, including an adaptive bal- ance between prior and current knowledge. Another direction is determining when an agent should abstain from reuse when no relevant prior knowledge exists. This work examines how sparsity and selectivity can enable effective forward transfer as the policy library grows, while managing that growth remains a complementary challenge. Since available knowledge influences what can be effectively learned, consolidation and compression should be evaluated for their effects on future transfer as well as retained performance. Finally, extending AMSC beyond known task boundaries requires detecting distribution changes and deciding when to retrieve, adapt, or create a new mask. 7CONCLUSION This work introduced Adaptive Mask Selection and Composition (AMSC) to address how lifelong agents can selectively reuse prior knowledge when learning new tasks. AMSC uses task similarity to periodically select and compose a sparse set of policy masks, without directly optimizing selec- tion or composition. On CT28 and MG16, it outperforms modular baselines in mean AUC and forward transfer and remains competitive with replay methods while retaining prior policies without forgetting. These findings show that task similarity can provide an effective alternative to learning compositions, with ablations supporting the importance of relevant selection and updating reuse dur- ing learning. The advantage is not universal, however, as the CW10 results suggest that identifying relevant prior knowledge and adapting its composition across layers may require different mecha- nisms. The effectiveness of similarity-guided reuse depends on how prior knowledge is composed per-layer. Together, the results position task similarity as a practical approach for selective reuse. ACKNOWLEDGMENTS This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under contract No. HR001121901 (Shared Experience Lifelong Learning). REFERENCES Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), p. 6429–6438, 2019. doi: 10.1109/ICCV.2019.00653. 9 Preprint Hongjoon Ahn, Jinu Hyeon, Youngmin Oh, Bosun Hwang, and Taesup Moon. Prevalence of negative transfer in continual reinforcement learning: Analyses and a simple baseline.In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=KAIqwkB3dT. David Alvarez-Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, p. 21428–21439. Curran Associates, Inc., 2020.URL https://proceedings.neurips.c/paper_files/paper/2020/ file/f52a7b2610fb4d3f74b4106fb80b233d-Paper.pdf. Eseoghene Ben-Iwhiwhu, Saptarshi Nath, Praveen Kumar Pilly, Soheil Kolouri, and Andrea Soltog- gio. Lifelong Reinforcement Learning with Modulating Masks. Transactions on Machine Learn- ing Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id= V7tahqGrOq. J.L. Carroll and K. Seppi. Task similarity measures for transfer in reinforcement learning task libraries. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, p. 803–808 vol. 2, 2005. doi: 10.1109/IJCNN.2005.1555955. Maxime Chevalier-Boisvert, Lucas Willems, and Suman Pal. Minimalistic Gridworld Environment for OpenAI Gym. https://github.com/maximecb/gym-minigrid, 2018. Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments. arXiv preprint arXiv:1806.08295, 2018. Nicolas Courty, Rémi Flamary, and Mélanie Ducoffe. Learning wasserstein embeddings. In Interna- tional Conference on Learning Representations, 2018. URL https://openreview.net/ forum?id=SJyEH91A-. Aleksandr Dekhovich, David M. J. Tax, Marcel H. F Sluiter, and Miguel A. Bessa. Continual prune- and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 53 (14):17849–17864, 2023. doi: 10.1007/s10489-022-04441-z. URL https://doi.org/10. 1007/s10489-022-04441-z. Jeffery Dick, Saptarshi Nath, Christos Peridis, Eseoghene Benjamin, Soheil Kolouri, and Andrea Soltoggio. Statistical Context Detection for Deep Lifelong Reinforcement Learning, 2024. Michael Gimelfarb, Scott Sanner, and Chi-Guhn Lee. Contextual policy transfer in reinforcement learning domains via deep mixtures-of-experts. In Cassio de Campos and Marloes H. Maathuis (eds.), Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, p. 1787–1797. PMLR, 27–30 Jul 2021. URL https://proceedings.mlr.press/v161/gimelfarb21a.html. Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. Deep Reinforcement Learning That Matters. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Apr. 2018. doi: 10.1609/aaai.v32i1.11694. URL https://ojs.aaai. org/index.php/AAAI/article/view/11694. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter.Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.c/paper_files/ paper/2017/file/8a1d694707eb0fefe65871369074926d-Paper.pdf. Heinke Hihn and Daniel Alexander Braun. Mixture-of-variational-experts for continual learning. In ICLR Workshop on Agent Learning in Open-Endedness, 2022. URL https://openreview. net/forum?id=HhzzNQPZLb9. 10 Preprint Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum? id=nZeVKeeFYf9. Jinwu Hu, ZiHao Lian, Zhiquan Wen, ChenghaoLi, Guohao Chen, Xutao Wen, Bin Xiao, and Mingkui Tan.Continual knowledge adaptation for reinforcement learning.In The Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025. URL https: //openreview.net/forum?id=QRlVickNdN. Zhiyi Huang, Xiaohan Shan, and Jianmin Li. Lifelong reinforcement learning with similarity-driven weighting by large models, 2025. URL https://arxiv.org/abs/2503.12923. David Isele and Akansel Cosgun. Selective Experience Replay for Lifelong Learning. In Proceed- ings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, AAAI’18/IAAI’18/EAAI’18. AAAI Press, 2018. ISBN 978- 1-57735-800-8. Jack Julian, Yun Sing Koh, and Albert Bifet. Building adaptive knowledge bases for evolving contin- ual learning models. npj Artificial Intelligence, 1(1):26, 2025. doi: 10.1038/s44387-025-00028-4. URL https://doi.org/10.1038/s44387-025-00028-4. Zixuan Ke, Bing Liu, and Xingchang Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. In Proceedings of the 34th International Conference on Neural Informa- tion Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546. Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, and Stephen J Roberts. Unclear: A straightforward method for continual reinforcement learning. In Proceedings of the 37th Interna- tional Conference on Machine Learning, 2020. Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, and Stephen J. Roberts. Same state, different task: Continual reinforcement learning without interference, 2022. Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. Towards continual reinforce- ment learning: A review and perspectives. Journal of Artificial Intelligence Research, 75:1401– 1476, 12 2022. doi: 10.1613/jair.1.13673. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017. Nils Koster, Oliver Grothe, and Achim Rettinger. Signing the Supermask: Keep, Hide, Invert. In International Conference on Learning Representations, 2022. URL https://openreview. net/forum?id=e0jtGTfPihs. Seungwon Lee, Sima Behpour, and Eric Eaton. Sharing less is more: Lifelong learning in deep networks with selective layer transfer. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, p. 6065–6075. PMLR, 18–24 Jul 2021. URL https://proceedings. mlr.press/v139/lee21a.html. Siyuan Li, Fangda Gu, Guangxiang Zhu, and Chongjie Zhang. Context-aware policy reuse. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Sys- tems, AAMAS ’19, p. 989–997, Richland, SC, 2019. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450363099. Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. TRGP: Trust region gradient projection for continual learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=iEvAf8i6JjO. 11 Preprint Xinran Liu, Yikun Bai, Yuzhe Lu, Andrea Soltoggio, and Soheil Kolouri. Wasserstein Task Em- bedding for Measuring Task Similarities. Neural Networks, 181:106796, 2025. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2024.106796. URL https://w.sciencedirect. com/science/article/pii/S0893608024007202. David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017. Mikel Malagón, Josu Ceberio, and Jose A. Lozano. Self-composing policies for scalable continual reinforcement learning. In Proceedings of the 41st International Conference on Machine Learn- ing, ICML’24. JMLR.org, 2024. Arun Mallya and Svetlana Lazebnik. Packnet: Adding Multiple Tasks to a Single Network by Itera- tive Pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, p. 7765–7773, 2018. Andre Martins and Ramon Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Ma- chine Learning Research, p. 1614–1623, New York, New York, USA, 20–22 Jun 2016. PMLR. URL https://proceedings.mlr.press/v48/martins16.html. Jorge A Mendez, Harm van Seijen, and Eric Eaton. Modular Lifelong Reinforcement Learning via Neural Composition. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=5XmLzdslFNN. Navid Naderializadeh, Joseph F Comer, Reed Andrews, Heiko Hoffmann, and Soheil Kolouri.Pooling by sliced-wasserstein embedding.In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, p. 3389–3400. Curran Associates, Inc., 2021.URL https://proceedings.neurips.c/paper_files/paper/2021/ file/1bc2029a8851ad344a8d503930dfd7f7-Paper.pdf. Saptarshi Nath, Christos Peridis, Eseoghene Benjamin, Xinran Liu, Soheil Kolouri, Peter Kinnell, Zexin Li, Cong Liu, Shirin Dora, and Andrea Soltoggio. Policy search, retrieval, and composition via task similarity in collaborative agentic systems. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29):24504–24512, Mar. 2026. doi: 10.1609/aaai.v40i29.39633. URL https://ojs.aaai.org/index.php/AAAI/article/view/39633. Cuong Nguyen, Tal Hassner, Matthias Seeger, and Cedric Archambeau. LEEP: A new mea- sure to evaluate transferability of learned representations. In Hal Daumé I and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, p. 7294–7305. PMLR, 13–18 Jul 2020. URL https://proceedings.mlr.press/v119/nguyen20b.html. Oleksiy Ostapenko, Pau Rodríguez, Massimo Caccia, and Laurent Charlin. Continual Learning via Local Module Composition. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS ’21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393. German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Con- tinual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2019.01.012. URL https://w. sciencedirect.com/science/article/pii/S0893608019300231. Sam Powers, Eliot Xing, and Abhinav Gupta. Self-activating neural ensembles for continual rein- forcement learning. In Sarath Chandar, Razvan Pascanu, and Doina Precup (eds.), Proceedings of The 1st Conference on Lifelong Learning Agents, volume 199 of Proceedings of Machine Learn- ing Research, p. 683–704. PMLR, 22–24 Aug 2022. URL https://proceedings.mlr. press/v199/powers22a.html. 12 Preprint Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off-policy meta-reinforcement learning via probabilistic context variables. In Kamalika Chaudhuri and Rus- lan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learn- ing, volume 97 of Proceedings of Machine Learning Research, p. 5331–5340. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/rakelly19a.html. David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019. Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive Neural Networks. arXiv preprint arXiv:1606.04671, 2016. Chinmay Savadikar, Michelle Dai, and Tianfu Wu. Cheem: Continual learning by reuse, new, adapt and skip - a hierarchical exploration-exploitation approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 25066–25076, June 2026. Ghada Sokar, Decebal Constantin Mocanu, and Mykola Pechenizkiy. Avoiding forgetting and allowing forward transfer in continual learning via sparse networks.In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Greno- ble, France, September 19–23, 2022, Proceedings, Part I, p. 85–101, Berlin, Heidelberg, 2022. Springer-Verlag. ISBN 978-3-031-26408-5. doi: 10.1007/978-3-031-26409-2_6. URL https://doi.org/10.1007/978-3-031-26409-2_6. Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Christos Peridis, Pawel Ladosz, Jeffery Dick, Praveen K. Pilly, and Soheil Kolouri. The configurable tree graph (ct-graph): measurable prob- lems in partially observable and distal reward environments for lifelong reinforcement learning, 2023. URL https://arxiv.org/abs/2302.10887. Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(56):1633–1685, 2009. URL http://jmlr.org/ papers/v10/taylor09a.html. Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, Sergey Levine, and Karol Hausman. Jump- Start Reinforcement Learning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, p. 34556–34583. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/ v202/uchendu23a.html. Johannes von Oswald, Christian Henning, Benjamin F. Grewe, and João Sacramento. Continual learning with hypernetworks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=SJgwNerKvB. Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions, 2024. URL https://arxiv.org/abs/2403.11435. Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tiehang Duan, and Mingchen Gao.Improv- ing task-free continual learning by distributionally robust memory evolution.In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Pro- ceedings of Machine Learning Research, p. 22985–22998. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/wang22v.html. Anderson M. Winkler, Gerard R. Ridgway, Matthew A. Webster, Stephen M. Smith, and Thomas E. Nichols. Permutation inference for the general linear model. NeuroImage, 92:381–397, 2014. ISSN 1053-8119. doi: https://doi.org/10.1016/j.neuroimage.2014.01.060. URL https:// w.sciencedirect.com/science/article/pii/S1053811914000913. 13 Preprint Maciej Wolczyk, Michał Zaj ̨ac, Razvan Pascanu, Łukasz Kuci ́ nski, and Piotr Miło ́ s. Continual world: A robotic benchmark for continual reinforcement learning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=5qsptDcsdEj. Maciej Wolczyk, Michał Zaj ̨ac, Razvan Pascanu, Łukasz Kuci ́ nski, and Piotr Miło ́ s.Disen- tangling transfer in continual reinforcement learning. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Process- ing Systems, volume 35, p. 6304–6317. Curran Associates, Inc., 2022.doi: 10.52202/ 068431-0456.URL https://proceedings.neurips.c/paper_files/paper/ 2022/file/2938ad0434a6506b125d8adaff084a4a-Paper-Conference.pdf. Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in Superposition. Advances in Neural Information Processing Systems, 33:15173–15184, 2020. Yanru Wu, Jianning Wang, Xiangyu Chen, Enming Zhang, Yang Tan, Hanbing Liu, and Yang Li. Exploiting task relationships in continual learning via transferability-aware task embeddings. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=V8FnYzDX35. Zihao Xu, Xuan Tang, Yufei Shi, Jianfeng Zhang, Jian Yang, Mingsong Chen, and Xian Wei. Con- tinual learning via manifold expansion replay. In 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), p. 733–738, 2023. doi: 10.1109/SMC53992.2023.10393967. Tianhe Yu,Saurabh Kumar,Abhishek Gupta,Sergey Levine,Karol Hausman,and Chelsea Finn.Gradient surgery for multi-task learning.In H. Larochelle, M. Ran- zato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Pro- cessing Systems, volume 33, p. 5824–5836. Curran Associates, Inc., 2020a.URL https://proceedings.neurips.c/paper_files/paper/2020/file/ 3fe78a8acf5fda99de95303940a2420c-Paper.pdf. Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, p. 1094–1100. PMLR, 2020b. Amir Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning, 2018. URL https://arxiv.org/abs/ 1804.08328. Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, p. 3987–3995. PMLR, 2017. Tiantian Zhang, Zichuan Lin, Yuxing Wang, Deheng Ye, Qiang Fu, Wei Yang, Xueqian Wang, Bin Liang, Bo Yuan, and Xiu Li. Dynamics-adaptive continual reinforcement learning via progressive contextualization. IEEE Transactions on Neural Networks and Learning Systems, 35(10):14588– 14602, 2024. doi: 10.1109/TNNLS.2023.3280085. Luisa Zintgraf, Sebastian Schulze, Cong Lu, Leo Feng, Maximilian Igl, Kyriacos Shiarlis, Yarin Gal, Katja Hofmann, and Shimon Whiteson. VariBAD: Variational Bayes-Adaptive Deep RL via Meta-Learning. J. Mach. Learn. Res., 22(1), January 2021. ISSN 1532-4435. 14 Preprint AMASK-SCORE SENSITIVITY AND QUANTIZED STORAGE Motivation and relation to binary supermasks. AMSC represents task i in layer ℓ by a real- valued score tensor S ℓ i over a frozen weight tensor W ℓ . The forward pass uses a binary gate M ℓ i = I[S ℓ i ≥ 0], h ℓ+1 = σ (W ℓ ⊙ M ℓ i )h ℓ ,(10) and propagates gradients to S ℓ i with a straight-through estimator (STE). This follows Wortsman et al. (2020); Ben-Iwhiwhu et al. (2023), which optimize a subnetwork over a frozen backbone. Related signed-supermask work allows connections to be kept, hidden, or inverted (Koster et al., 2022). AMSC instead keeps the deployed gate in Equation 10 binary. It retains continuous scores for the linear combination before the gate is applied in the forward pass on the resulting mask. Score magnitude can therefore affect composition even though only the score sign determines an individual task’s subnetwork. Wortsman et al. (2020) also superimposes learned binary masks, but optimizes their mixture to infer a task at evaluation time. AMSC composes completed policies during target-task learning, then consolidates the composition as the next reusable policy. The following experiments test whether continuous score information is required for this transfer mechanism and whether completed masks can instead be stored in quantized form. Magnitude-controlled perturbations. We first test whether score magnitude identifies stable gates. For a score tensor with D ℓ entries, define its root-mean-square (RMS) scale as r ℓ i = v u u t 1 D ℓ D ℓ X j=1 (S ℓ i,j ) 2 .(11) For a selected index setI, each score receives independent zero-mean Gaussian noise, e S ℓ i,j = S ℓ i,j + ε ℓ i,j , ε ℓ i,j ∼N 0, (αr ℓ i ) 2 , j ∈I.(12) Zero-mean noise preserves each score in expectation. RMS scaling makes α dimensionless and comparable across layers with different score scales. Thus, α = 0.5 sets the noise standard deviation to one half of the corresponding layer RMS. For a nonzero score s, the probability that Equation 10 changes state is Pr[I[s + ε > 0]̸= I[s > 0]| s] = Φ − |s| αr ℓ i ,(13) where Φ is the standard-normal cumulative distribution function. Equation 13 predicts frequent gate changes near zero and rapidly fewer changes as normalized magnitude increases. The structural sweep perturbs the lowest-magnitude, random, or highest-magnitude frac- tion of each tensor.We evaluate fractions f ∈ 0.10, 0.25, 0.50 and scales α ∈ 0.01, 0.05, 0.10, 0.25, 0.50, 1.00 using final checkpoints from the seven CT28 and MG16 seeds. Matched-state policy analysis perturbs actor-relevant trunk and head scores and measures deterministic-action disagreement. Fresh-rollout return estimates use three episodes per task and serve as a diagnostic rather than an equivalence test. Perturbation results. Across actor heads and shared trunks, 3.7–4.8% of scores lie below 0.05 RMS and 7.1–9.4% lie below 0.10 RMS. At α = 0.5, perturbing the lowest-magnitude quartile changes 37.3% of the selected CT28 gates and 39.3% of the selected MG16 gates. The corre- sponding rates are 14.2% and 16.4% for random scores, but only 0.2% and 0.5% for the highest- magnitude quartile. Low-magnitude perturbations also change the selected action on 6.5% of CT28 states and 6.2% of MG16 states, compared with 0.8% for high-magnitude perturbations on both benchmarks. One-sided Wilcoxon signed-rank tests support the paired low-versus-high differences in CT28 (p = 0.0078) and MG16 (p = 0.017). Mean task-return changes at α = 0.5 are−0.005 on CT28 and−0.007 on MG16 for low-magnitude perturbations. The low-versus-high return differences are not significant (p = 0.60 and p = 0.61, respectively). Score magnitude therefore predicts gate and action stability, but the diagnostic rollout data do not establish that low-magnitude scores can be removed without a performance cost. The difference between action changes and mean-return changes also indicates behavioral redundancy in the learned subnetworks. 15 Preprint 10 6 10 4 10 2 10 0 Normalized magnitude |S|/RMS(S) 0% 20% 40% 60% 80% 100% Scores with magnitude x (\%) CT28 10 5 10 3 10 1 10 1 Normalized magnitude |S|/RMS(S) MG16 shared trunk actor head critic head (a) Score-magnitude distributions. 0.00.20.40.60.81.0 Noise scale (layer RMS) 0.0% 10.0% 20.0% 30.0% 40.0% Perturbed scores changing gate (%) CT28 0.00.20.40.60.81.0 Noise scale (layer RMS) MG16 Magnitude region low random high (b) Gate changes under perturbation. 0.20.40.60.81.0 Noise scale (layer RMS) 0.0% 10.0% 20.0% 30.0% Action disagreement (%) CT28 0.20.40.60.81.0 Noise scale (layer RMS) MG16 Magnitude region low random high (c) Action disagreement on matched states. lowrandomhigh Magnitude region ( = 0.5 × RMS) 0.02 0.00 0.02 Mean task-return change CT28 lowrandomhigh Magnitude region ( = 0.5 × RMS) 0.04 0.02 0.00 0.02 0.04 MG16 (d) Diagnostic return change at α = 0.5. Figure 3: Mechanistic analysis of continuous mask scores. Panel (a) gives the empirical cumulative distribution of up to 2,048 sampled scores per task–layer tensor, normalized by layer RMS. Panels (b)–(d) perturb 25% of scores from each magnitude region. Lines report seed means; bands in (b) and (c) span seed minima and maxima. Return estimates in (d) use three episodes per task. Quantized mask libraries. The second experiment tests whether continuous completed-task scores are required for subsequent composition. Binary storage retains only score sign, Q bin (s) = +1, s > 0, −1, s≤ 0, (14) whereas ternary storage introduces an RMS-scaled dead zone, Q δ (s;r) = +1, s > δr, 0, |s|≤ δr, −1, s <−δr. (15) Increasing δ widens the dead zone and maps a larger fraction of the library to zero. The value 0 removes a near-boundary connection. The value −1 remains inactive when a stored task is evalu- ated alone. These values differ during composition because AMSC combines score tensors before thresholding. For a new task k, the composed score is b S ℓ k = a k,k N (S ℓ k ) + X i∈P(k) a k,i N (Q(S ℓ i )), M ℓ k = I[ b S ℓ k > 0],(16) where N is the L2-STE transformation and Q is either Equation 14 or Equation 15. After task k, its consolidated score is quantized before entering the reusable library. The experiment therefore evaluates online learning with quantized sources rather than post-hoc compression of an already completed run. Quantization results. Float32 AMSC obtains AUC/FWT/FGT of 0.941/0.935/0.000. Binary storage reduces AUC/FWT to 0.750/0.709 despite preserving each stored score’s sign. Score sign alone therefore does not preserve the information required by later composition. Ternary storage reaches its best trade-off at δ = 0.50, obtaining 0.935/0.912/0.041 while setting 29.9% of stored scores to zero. Paired tests against Float32 do not reject zero AUC (p = 0.43) or FWT difference (p = 0.22), although this does not establish equivalence. FGT increases by 0.041 (p = 0.031). At δ = 0.625, AUC/FWT/FGT is 0.915/0.908/0.056 with 41.4% zeros; the AUC reduction is significant (p = 0.023) while the FWT reduction is not (p = 0.086). Increasing δ further mono- tonically increases sparsity but reduces acquisition and retention, reaching 0.624/0.568/0.265 at δ = 1. Thus, the dead zone can remove near-boundary connections, but aggressive thresholding deletes gates required by completed policies. 16 Preprint 00.10.250.50.6250.750.8751.0 Ternary threshold (RMS); 0 is binary 0.5 0.6 0.7 0.8 0.9 1.0 Performance Ternary threshold sweep AUCFWT 00.10.250.50.6250.750.8751.0 Ternary threshold (RMS); 0 is binary 0.0 0.1 0.2 0.3 FGT Retention cost Float32Ternary full L2 Ternary prior L2 Ternary no L2 0.0 0.2 0.4 0.6 0.8 1.0 Performance Normalization at = 0.5 AUCFWT Float32 mask Ternary mask LoRA-4 local LoRA-8 local LoRA-4 composed 0.50 0.25 0.00 0.25 0.50 0.75 1.00 Performance Matched fixed-backbone LoRA controls AUCFWT (a) Quantization, normalization, and matched LoRA controls. Mask float32 Mask 1-bit Mask 2-bit LoRA-4 float32 LoRA-8 float32 LoRA-8 float16 10 2 Theoretical storage (KiB/task) 428.1 13.4 26.8 24.2 48.4 24.2 Fixed-width storage 0.1 0.25 0.5 0.625 0.75 0.875 1 Ternary threshold (RMS) 0 20 40 60 Stored zero scores (%) Empirical sparsity (b) Theoretical storage and ternary sparsity. Figure 4: Quantized storage of completed task masks on CT28. In (a), dashed lines in the threshold sweep give Float32-mask performance and error bars denote asymmetric 95% bootstrap confidence intervals across seven seeds. The LoRA controls use the same frozen signed-constant backbone. Storage in (b) counts packed values only and excludes container and index overhead. Why normalization is necessary. For a ternary tensor,∥Q δ (S ℓ i )∥ F = p n ℓ i , where n ℓ i is its num- ber of nonzero entries. Without normalization, the effective influence of source i is therefore pro- portional to a k,i p n ℓ i , rather than the intended similarity coefficient a k,i . The trainable current score introduces an additional arbitrary continuous scale. L2-STE gives every component unit for- ward norm, making the composition coefficients comparable while retaining the identity gradient for the current score. This calibration is particularly important after ternary quantization because all retained entries have magnitude one, leaving density as the only source-dependent tensor scale. At δ = 0.50, removing normalization yields AUC/FWT/FGT of 0.283/0.070/0.291. Normalizing only the priors also fails (0.269/0.049/0.005) because unit-norm sources remain incommensurate with the raw current score. Joint normalization is therefore required for sparsemax weights to control actual mask contributions. Storage comparison. CT28 contains D m = 109,600 mask scores per task. Float32 scores require 428.1 KiB per task. Packed binary and ternary scores require 13.4 and 26.8 KiB, respectively. For comparison, a rank-r LoRA adapter (Hu et al., 2022) over the same layers stores D LoRA (r) = r X ℓ (d in ℓ + d out ℓ ) = 1,548r(17) 17 Preprint values per task. Rank-4 and rank-8 Float32 adapters therefore require 24.2 and 48.4 KiB; rank-8 Float16 also requires 24.2 KiB. Ternary masks are similar in theoretical size to rank-4 LoRA and smaller than rank-8 Float32 LoRA, whereas binary masks are smaller than both. A task-local rank-4 adapter obtains AUC/FWT/FGT of 0.198/−0.231/0.000, and rank 8 obtains 0.201/−0.150/0.000. Both fall below the 0.22 AUC of full from-scratch learning. They combine a low-rank capacity constraint with no mechanism for reusing earlier solutions, which is especially restrictive in the hi- erarchical CT28 curriculum. In contrast, normalized-sparsemax retrieval and composition of rank-4 deltas raises performance to 0.885/0.870/0.000. At 24.2 KiB per task, this adapter is slightly smaller than a packed ternary mask, but remains below the latter by 0.050 AUC (paired 95% CI [0.020, 0.087]) and 0.042 FWT ([−0.016, 0.101]). Composed LoRA must express and recompress the transferred update within a rank-4 subspace, whereas mask composition can independently com- bine all frozen backbone connections. The LoRA controls retain zero measured forgetting, while ternary quantization incurs a small boundary loss of 0.041 ([0.020, 0.061]) when a completed con- tinuous score is replaced by its quantized representation. These results suggest that quantized masks can approach low-rank-adapter storage while preserving stronger composition in this setting. 18 Preprint BIMPLEMENTATION DETAILS B.1WASSERSTEIN TASK EMBEDDING CONSTRUCTION AMSC constructs WTEs from experience generated by the current policy (Liu et al., 2025). The current-task buffer D k stores transitions (s,a,r, s ′ ,d). At an embedding update, AMSC samples N (s,a,r) entries uniformly with replacement from D k . The stored state has already received the benchmark-specific policy-input scaling. Let d s denote the flattened state dimension and let d a denote the action-feature dimension. Discrete actions are one-hot encoded; continuous actions retain their vector representation: φ(a n ) = onehot(a n )∈ R d a , discrete actions, a n ∈ R d a ,continuous actions. (18) The state block is standardized independently for each sampled batch. The implementation com- putes one scalar mean and standard deviation over all Nd s state entries, rather than separate statistics for each state coordinate. For batch update t, μ (t) s = mean n,j s n,j , σ (t) s = std n,j s n,j , b s n = s n − μ (t) s max(σ (t) s ,ε x ) .(19) The implementation uses ε x = 10 −8 . The input row and embedding batch are then x n = [ b s ⊤ n ,φ(a n ) ⊤ ,r n ] ⊤ ,X (t) k = [x 1 ,..., x N ] ⊤ ∈ R N×d x ,(20) where d x = d s + d a + 1. The reward is used at the scale stored by the RL replay buffer; WTE applies no additional reward normalization. At initialization, AMSC samples a reference matrix X 0 = [x 0 1 ,..., x 0 R ] ⊤ ∈ R R×d x , with each coordinate sampled independently from U [0, 1]. A seeded random generator draws X 0 once, and the same ordered reference is retained for all tasks in a run. The reference is not estimated from a task and is not updated during training. Reusing X 0 fixes the coordinates and dimension of every task embedding. The empirical batch and reference distributions have uniform masses a = 1 N 1 N ,b = 1 R 1 R .(21) Let C n,r = ∥x n − x 0 r ∥ 2 2 be entry (n,r) of the cost matrix C. AMSC uses the unregularized Earth Mover’s Distance solver in the Python Optimal Transport library to compute Γ ⋆ = argmin Γ∈Π(a,b) ⟨Γ, C⟩ F ,Π(a, b) = Γ∈ R N×R + : Γ1 R = a, Γ ⊤ 1 N = b .(22) Since each reference point receives mass 1/R, its barycentric projection into the current empirical distribution is y r = R N X n=1 Γ ⋆ n,r x n .(23) The WTE records the displacement of every ordered reference point under this projection: e (t) k = 1 √ R vec [y 1 − x 0 1 ,..., y R − x 0 R ] ⊤ ∈ R Rd x .(24) Equation equation 24 is the expanded form of Equation equation 4. The construction contains no learned encoder or embedding loss. The first update initializes ̄ e (1) k = normalize(e (1) k ). Subsequent updates combine the stored normal- ized representation with the raw batch embedding, then normalize the result, ̄ e (t) k = normalize ρ ̄ e (t−1) k + (1− ρ)e (t) k .(25) 19 Preprint Here, normalize(e) = e/ max(∥e∥ 2 ,ε e ), with ε e = 10 −8 and ρ = 0.5. This is the update in Equation equation 5 used for the reported CT28 and MG16 experiments. Completed embeddings are frozen at task boundaries, and the current-task buffer is cleared before the next task. Every retrieval event therefore compares an evolving current-task estimate with fixed representations of the completed tasks. The reported CT28 and MG16 experiments use R = 50. CT28 samples N = 128 transitions and updates both the embedding and retrieval support every PPO iteration. MG16 samples N = 1024, updates the embedding every iteration, and updates the retrieval support every five iterations. The CW10 implementation uses R = 50, N = 1000, and an update period of 1000 environment steps. B.2PSEUDO-CODE Algorithm 1 gives the AMSC training procedure. Task boundaries determine when a completed mask and embedding are frozen and added to their respective libraries. Embedding and retrieval updates use periods f emb and f sel , respectively. Retrieval weights are not optimized by the RL loss; gradients update only the current task’s mask scores. Algorithm 1 AMSC training with adaptive similarity-guided selection Require: Task sequenceT k K k=1 , per-task training budget B, rollout length z Require: WTE batch size N , reference size R, EMA factor ρ, sparsemax temperature τ Require: Update periods f emb and f sel 1: Initialize frozen backboneW ℓ L ℓ=1 , mask library M, and embedding table E 2: Sample and fix reference matrix X 0 ∈ R R×d x 3: for k = 1 to K do 4:Initialize current mask-score matricesS ℓ k L ℓ=1 5:Set D k ← ∅, P(k)← ∅, steps← 0, and t← 0 6:while steps < B do 7:FormC ℓ k L ℓ=1 and the binary policy masks using Equation equation 9 8:Collect a rollout of z transitions using the resulting policy 9:Append transitions to D k ; update steps← steps + z and t← t + 1 10:Compute the RL loss and update onlyS ℓ k L ℓ=1 through Equation equation 8 11:if|D k |≥ N and t mod f emb = 0 then 12:Sample N tuples from D k and construct X (t) k 13:Compute e (t) k using Equations equation 22–equation 24 14:if this is the first embedding update for task k then 15:Set ̄ e k ← normalize(e (t) k ) 16:else 17:Update ̄ e k using Equation equation 5 18:end if 19:end if 20:if k > 1, ̄ e k exists, and t mod f sel = 0 then 21:Compute s k , w k , and P(k) using Equations equation 6–equation 7 22:end if 23:end while 24:RecomputeC ℓ k L ℓ=1 using Equation equation 9 25:Store S ℓ,∗ k ← C ℓ k for every ℓ in M, and store ̄ e k in E 26: end for 20 Preprint CBENCHMARKS For CT28 and MG16, we use family to denote tasks that share an observation encoding and a nested task construction across depths. All tasks within each benchmark have the same formal observation dimensions and action interface. Cross-family tasks differ in their observation dataset or symbolic remapping, rather than in tensor shape, and can still share behavioral structure that supports policy transfer. CT-Graph CT28. CT-Graph is a configurable decision-tree environment for continual RL (Soltoggio et al., 2023). At each decision state, the agent selects a branch and receives a sparse reward of 1 only after reaching the target terminal node. Each state is represented by an image gen- erated from a family-specific image dataset. Datasets do not overlap across families, whereas tasks within one family share observations for their common graph prefix. CT28 is based on the evaluation setup used in (Nath et al., 2026), and contains four families with depths 2 through 8, giving 28 tasks. Within each family, the solution at depth d forms a prefix of the solution at depth d + 1. A depth-8 target is encountered after 24–27 previous tasks. Six of the available policies are lower-depth members of the target family, and the remaining policies belong to other families. CT28 therefore tests retrieval from a library in which relevant predecessors form a minority that decreases as the curriculum progresses. MiniGrid MG16. MiniGrid provides partially observable grid-world navigation tasks with sym- bolic object, color, and state observations (Chevalier-Boisvert et al., 2018). MG16 extends Mul- tiRoom into four room-chain families. Each family contains tasks R2 through R5. Under a fixed layout seed, Rd + 1 preserves the complete Rd layout and adds one room and connecting transi- tion. The agent receives sparse reward for reaching the goal in the final room. The action space and physical task remain fixed across families. MG16 assigns each family a distinct symbolic remapping of wall, door, and floor cells. The remap- ping changes the observed object type and colour but preserves collision, traversal, and door-toggle affordances. Each remapping remains fixed across depth, so lower-depth policies share the target family’s observation code. Figure 5 shows the four profiles. Tasks are ordered by room depth, giving four R2 tasks, followed by four R3, R4, and R5 tasks. No family identifier is included in the policy observation. B B B B W W W W W W W W W W W W W W W W W W W D D D D G ▲ F F F K K K K K K K K K K K K K K K K K K K K K K K K K K G B B B B ▲ K K K K K L L L L L L L L L L L L L L L L L L L L L L L L L L O O O G ▲ B B B B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O L L LLG ▲ Figure 5: Depth-5 layouts for the four MG16 task families. Symbols denote the observed represen- tation of each functional cell; the underlying wall, door, and floor affordances remain unchanged. Identity uses grey walls (W), blue ball floors (B), and yellow doors (D). KeyBoxSwap uses yellow key walls (K), grey floors (F), and green box doors (B). LavaBall uses green lava walls (L), yellow key floors (K), and blue ball doors (O). BallLava uses purple ball walls (O), purple box floors (B), and red lava doors (L). Continual World CW10. Continual World evaluates continual RL on sequences of Meta- World robotic manipulation tasks (Wolczyk et al., 2021). We use one pass through the ten- task CW10 curriculum: hammer, push-wall, faucet-close, push-back, stick-pull, handle-press-side, push, shelf-place, window-close, and peg-unplug-side. Each task is trained for 1M environment steps with SAC, giving 10M steps in total. CW10 pro- 21 Preprint vides shaped rewards and four-dimensional continuous actions. It therefore tests AMSC beyond the sparse-reward, discrete-action conditions of CT28 and MG16. Our experiments use the current MetaWorld-v2 goal-observable task definitions (Yu et al., 2020b). Each observation is a 39-dimensional state vector, and no one-hot task label is appended. Initial object and goal configurations remain randomized across episodes. Evaluation uses binary task success rather than shaped return. An episode is successful if the envi- ronment reports success at any step before termination. Every 50,000 training steps, the determinis- tic policy is evaluated for ten episodes on every task encountered so far. Each evaluation uses a fresh environment and a task-specific seed, and does not alter the training environment or replay buffer. The resulting success-rate matrix supplies the CW10 AUC, FWT, and FGT measurements. Figure 6: The ten MetaWorld-v2 tasks in the CW10 curriculum, shown in execution order from left to right and top to bottom Wolczyk et al. (2021). 22 Preprint DBASELINES Comparison protocol. All methods use seven seeds, identical task orders, and the same interac- tion budget. CT28 and MG16 use the common PPO architecture and optimizer settings reported in Section 5. Task labels are used at task switches and evaluation but are not included in policy obser- vations or policy selection. CW10 uses the SAC schedule and evaluation protocol in Appendix C. Architectures are matched where the algorithms permit direct substitution. CompoNet and ProgNet retain their released two-layer, 256-unit ReLU models and a matching no-reuse run is used to calcu- late their FWT. Other CW10 methods use the four-layer, 256-unit CW architecture with first-layer LayerNorm. Learning references and regularization. Fine-tuning updates one policy throughout the curricu- lum without a retention mechanism. No-reuse trains an independent policy on each task and provides the zero-transfer learning curves used to calculate FWT. Replay. SER (Isele & Cosgun, 2018) maintains short-term FIFO memory and bounded episodic memory. We report surprise-prioritized and reservoir-sampled SER variants. SDW (Huang et al., 2025) estimates task similarity and uses it to control replay and policy/value consistency weights. These methods select or weight stored experience rather than stored policy modules. Parameter isolation and routing. ProgNet (Rusu et al., 2016) adds a frozen policy column for each task and learns lateral connections from earlier columns. OWL (Kessler et al., 2022) combines task-specific heads with oEWC on shared features and selects a head using an exponential-weights bandit. SANE (Powers et al., 2022) maintains an ensemble of actor–critic modules and activates a module using optimistic value uncertainty. Our task-incremental SANE configuration clones the selected module at each task boundary and freezes earlier modules. Policy composition. Mask-LC and Mask-BLC (Ben-Iwhiwhu et al., 2023) learn softmax coeffi- cients over the current mask and every earlier mask. Mask-LC initializes the active coefficients uni- formly. Mask-BLC initially allocates half of the mass to the current mask and divides the remainder among the earlier masks. CKA-RL (Hu et al., 2025) learns task-specific parameter residuals around a shared policy and fuses residuals into a bounded knowledge pool. CompoNet (Malagón et al., 2024) adds one policy unit per task and uses learned attention to combine the outputs of frozen earlier units with the current unit. These baselines learn routing or composition without first con- structing a library-relative sparse support. Similarity-guided composition. MOSAIC (Nath et al., 2026) was introduced for knowledge ex- change among concurrently learning agents. We adapt MOSAIC as a baseline, limiting candidates to frozen masks from earlier tasks but retaining MOSAIC’s absolute cosine threshold, performance gate, learned composition, and acquisition protocol. A successful acquisition consolidates the active composition and reinitializes its coefficients from source and target performance. We also evaluate MOSAIC without the performance gate, denoted MOSAIC¬I perf . 23 Preprint EHYPERPARAMETERS Table 5 reports the principal AMSC configuration and shared optimization settings. Table 6 reports method-specific settings for the CT28 and MG16 baselines. Table 7 reports the corresponding CW10 settings. Methods share the optimization and evaluation protocol for each benchmark unless stated otherwise. Table 5: Hyperparameters, seeds, and training budgets for the principal AMSC experiments. One CT28 or MG16 PPO iteration collects 4×128 = 512 environment samples. The CW10 configuration uses the 39-dimensional MetaWorld-v2 observations. ComponentHyperparameterCT28MG16CW10 OptimizationLearnerPPOPPOSAC Network width3× 2003× 2004× 256 ActivationReLUReLUTanh(first), LeakyReLU (remaining) OptimizerRMSpropRMSpropAdam Actor/critic learning rate1.5× 10 −4 1.5× 10 −4 10 −3 /10 −3 Number of workers441 Discount factor0.990.990.99 GAE parameter0.990.99— Entropy coefficient0.10.01Automatic Rollout length per worker128128— Optimization epochs88— Number of mini-batches6464— PPO ratio clip0.10.1— Gradient clipping55— Replay-buffer size—10 6 Replay batch size—128 Learning starts—1,000 steps Initial random actions—10,000 steps Update interval—50 steps Gradient steps per update—50 Target smoothing coefficient—0.005 AMSCTask representationWTE–SARWTE–SARWTE–SAR Reference size R505050 Embedding sample count N12810241,000 Embedding-update frequency1 iteration1 iteration1,000 steps Selection frequency1 iteration5 iterations1,000 steps EMA coefficient ρ0.50.50.5 Similarity normalizationz-scorez-scorez-score Sparsemax temperature τ1.01.01.0 Mask-score normalizationℓ 2 -STEℓ 2 -STEℓ 2 -STE Mask type and scopeDiscrete actor– critic Discrete actor– critic Discrete full ac- tor CriticMaskedMaskedDense,task- local LayerNormNoneNoneTask-specific affine TrainingBudget per task51,200 steps512,000 steps1,000,000 steps Total budget1,433,600 steps8,192,000 steps10,000,000 steps Evaluation interval10 iterations20 iterations50,000 steps Evaluation episodes101010 per seen task SeedsExperiment seeds86, 87, 88, 89, 90, 91, 92 CT-Graph configuration seed3— CT-Graph image seeds1, 2, 3, 4— 24 Preprint Table 6: Method-specific CT28 and MG16 hyperparameters for parameter isolation, routing, and policy-composition methods. AlgorithmHyperparameterValue (CT28/MG16) SERStorage strategySurprise/reservoir Episodic-memory size20,000 FIFO-memory size2,000 Batch size256 Replay epochs1 FIFO ratio0.5 Coverage distance1.0 SDWBuffer size10 5 Batch size256 Probe batch size128 Base replay ratio0.8 Similarity smoothing2 Maximum policy/value weights0.01/0.005 Minimum consistency weight−10 −4 Similarity-feature weights0.4, 0.4, 0.2 OWLoEWC coefficient500 Bandit learning rate0.88 Bandit decay0.9 Exploration ε0 SANEMaximum modulesCT28: 28; MG16: 16 Uncertainty scale1.0 Task-boundary cloningEnabled Mask-LCMask typeDiscrete Prior supportFull library Coefficient initializationUniform Mask-BLCMask type and supportSame as Mask-LC Initial composition massCurrent: 0.5; all priors: 0.5 CKA-RLKnowledge-pool capacity5 α scale10 −2 α learning rate2.5× 10 −4 α initializationRandom normal Major mass0.6 Fixed/global/scaled αFixed: no; global: yes; scaled: yes Fused componentsActor: yes; encoder: no ∆Θ modeT Reset criticEnabled CompoNetAttention dimension200 Composed outputPolicy logits MOSAIC-LLTask representationWTE–SAR Reference size R50 Embedding sample count N128 Cosine-similarity threshold0.5 Selection capNone Selection frequencyCT28: 10; MG16: 25 iterations Performance gateSource return > current target return Acquisition updateConsolidate composition and reset coeffi- cients Coefficient initializationReward-guided MOSAIC¬I perf Performance gateDisabled; otherwise as MOSAIC-L 25 Preprint Table 7: Method-specific CW10 baseline hyperparameters. Compatible methods use the shared SAC configuration in Table 5. CompoNet and ProgNet retain their released architectures and use the matched native no-reuse run as their FWT reference. AlgorithmHyperparameterCW10 value No-reuseNetwork4×256, first-layer LayerNorm No-reuse (for CompoNet and ProgNet)Network2×256, ReLU, no LayerNorm Mask-LCMask typeContinuous Mask scopeActor and critics, trunks and heads LayerNormTask-specific affine Prior supportFull library CompositionLearned Coefficient initializationUniform Mask-BLCMask type, scope, and supportSame as Mask-LC Initial composition massCurrent: 0.5; all priors: 0.5 CKA-RLKnowledge-pool capacity8 α initializationRandom normal α scale10 −3 Major mass0.6 Fixed/global/scaled αFixed: no; global: yes; scaled: yes Fused componentsHeads: yes; shared trunk: no ∆Θ modeT CompoNetNetwork2×256, ReLU, no LayerNorm CompositionLearned attention ProgNetNetwork2×256, ReLU, no LayerNorm Prior columnsFrozen MOSAIC-LLMask type and scopeContinuous full network LayerNormTask-specific affine Task representationWTE–SAR Reference size R50 Embedding sample count N1,000 Selection frequency1,000 steps Cosine-similarity threshold0.5 Performance estimateMean success over 10 recent online episodes Performance gateSource success > current tar- get success Acquisition updateConsolidate composition and reset coefficients Coefficient initializationSuccess-guided 26 Preprint FADDITIONAL RESULTS F.1EARLY ACQUISITION We compare early acquisition against MOSAIC, which also retrieves masks for composition, and the surprise- and reservoir-based SER replay methods. AUC 20 averages the first two of ten target-task evaluations, covering the first 20% of each CT28 task budget. Time to first reward is the end of the first 512-step training iteration with positive mean reward; time to first positive evaluation return is measured at evaluation checkpoints. Both times are right-censored at the 51,200-step task budget. Aggregate statistics exclude task 0 because no source policy is then available. Across tasks 1–27, AMSC obtains AUC 20 = 0.86, compared with 0.35 for MOSAIC (two-sided paired p = 0.016). Its first positive training reward appears after 2.34k rather than 5.83k steps (p = 0.016). SER-surprise achieves AUC 20 = 0.91 and reaches positive reward after 1.59k steps; SER-reservoir obtains 0.86 and 3.27k. Neither SER variant differs significantly from AMSC on early AUC or time to first reward. The MOSAIC comparison supports an early advantage for AMSC within mask-based composition; it does not imply an early advantage over replay. Tables 8 and 9 provide the aggregate and per-task results. Table 8: CT28 early acquisition over tasks 1–27 across seven matched seeds. Entries are means with conservative 95% bootstrap CI. T + and T eval+ are time to first positive training reward and evaluation return, in thousands of environment steps, right-censored at 51.20k. Exact two-sided paired sign-flip p-values compare AMSC with each method; no multiple-comparison adjustment is applied. MethodAUC 20 T + (k)T eval+ (k)p AUC 20 p T + p T eval+ AMSC0.86± 0.02 2.34± 0.91 8.05± 1.19— MOSAIC 0.35± 0.03 5.83± 0.84 14.44± 1.350.0160.0160.016 SER-s0.91± 0.04 1.59± 0.72 7.12± 1.380.1250.3120.422 SER-r0.86± 0.05 3.27± 1.98 10.10± 2.65 1.0000.5160.281 F.2SIMILARITY-DERIVED COMPOSITION VS. LEARNED COMPOSITION IN CW10 We compare the composition weights at the end of training of AMSC and Mask-BLC. Figure 7 shows the coefficient assigned to the current-task mask at each layer, alongside within-task evalu- ation success. AMSC uses one similarity-derived allocation across its masked actor layers. Mask- BLC learns separate weights for each actor and critic layer. Averaged over tasks 1–9, Mask-BLC assigns approximately 0.87 to the current mask in its middle actor layers, but only 0.01 in the action-mean head. Thus, a single task-level similarity-derived weighting cannot recover the layer- dependent composition learned by Mask-BLC in CW10. The largest mean Mask-BLC–AMSC gaps occur on task 9 (peg-unplug-side, 0.92 versus 0.63 mean evaluation success) and task 3 (push-back, 0.41 versus 0.23). The task-9 gap occurs in all seven paired seeds, whereas the task-3 gap is less consistent. AMSC also exceeds Mask-BLC on task 6 (0.51 versus 0.22), so the difference is not uniform across the curriculum. Task 3 remains above its from-scratch reference; it explains part of the between-method performance gap, but not AMSC’s negative forward transfer. Normalized FWT can additionally amplify small deficits on tasks whose from-scratch performance is close to one. Figure 8 identifies the prior tasks receiving actor coefficient weight. AMSC’s sparse support ex- cludes some sources entirely, while Mask-BLC distributes learned coefficients over the available library. F.3HYPERPARAMETER SENSITIVITY Each sweep changes one AMSC hyperparameter while retaining a benchmark-specific sweep ref- erence for all others. Table 10 reports every value considered. Frequencies are measured in PPO iterations, and bold values denote the sweep-reference settings. Each setting contains seven seeds. The reported performance intervals are the minimum and maximum of the seven-seed means across the swept values, rather than confidence intervals. 27 Preprint Table 9: Per-task CT28 early acquisition across seven matched seeds. The first four numeric columns are AUC 20 ; the remaining four give T + in thousands of environment steps. Task A–D2 (†) has no previous source and is excluded from the aggregate comparison. AUC 20 T + (k steps) TaskAMSC MOSAIC SER-s SER-rAMSC MOSAIC SER-s SER-r A–D2 † 1.001.001.000.930.950.950.660.73 B–D20.791.000.640.790.660.732.561.76 C–D20.931.001.001.00 0.800.660.511.02 D–D20.790.860.710.861.101.461.681.17 A–D31.000.641.001.000.802.560.510.51 B–D31.000.291.001.000.951.390.510.51 C–D31.000.500.930.930.663.000.590.51 D–D30.931.001.001.00 0.881.460.510.51 A–D41.000.431.001.000.953.510.510.51 B–D41.000.501.001.000.804.170.510.51 C–D40.710.071.001.000.953.070.510.51 D–D41.000.360.861.001.104.900.510.51 A–D50.930.001.001.00 1.102.190.510.51 B–D51.000.211.001.00 1.026.880.510.51 C–D51.000.141.001.00 1.024.750.510.51 D–D51.000.140.931.000.805.191.240.51 A–D61.000.071.001.000.887.680.510.59 B–D61.000.141.001.00 1.027.170.510.51 C–D60.140.001.001.006.076.800.800.66 D–D61.000.430.570.93 0.886.445.920.51 A–D71.000.641.001.001.105.190.660.51 B–D70.930.000.790.291.028.270.808.70 C–D70.500.000.860.29 11.3410.832.125.63 D–D71.000.430.790.861.0210.027.978.05 A–D81.000.501.000.640.887.830.510.66 B–D80.710.000.930.43 1.2410.900.6615.07 C–D80.500.000.930.4317.8517.040.7322.24 D–D80.430.070.640.79 6.2913.3910.0215.07 Table 10: Sensitivity ranges for AMSC. Each AUC/FWT interval is the range of mean performance across the values in the adjacent sweep. All settings obtain zero measured FGT. N is the number of task samples, R is the reference-distribution size, and u is the reference seed. MG16 uses N = 512 outside the sample-count sweep. Bold marks sweep references. CT28MG16 ParameterSwept valuesAUCFWTSwept valuesAUCFWT Temperature τ0.25, 0.5, 1, 2, 40.89–0.96 0.88–0.95 0.25, 0.5, 1, 2, 40.50–0.67 0.16–0.50 Samples N32, 64, 128, 256, 5120.94–0.95 0.93–0.95 128, 256, 512, 1024, 2048 0.54–0.73 0.23–0.54 Reference size R10, 25, 50, 100, 2000.89–0.94 0.88–0.94 10, 25, 50, 100, 2000.55–0.70 0.35–0.52 Embedding period f emb 1, 2, 5, 100.84–0.94 0.83–0.94 1, 2, 5, 100.57–0.62 0.34–0.43 Selection period f sel 1, 2, 5, 100.83–0.94 0.81–0.94 1, 2, 5, 100.57–0.65 0.35–0.43 EMA coefficient ρ0, 0.25, 0.5, 0.75, 0.90.93–0.95 0.92–0.95 0, 0.25, 0.5, 0.75, 0.90.62–0.68 0.43–0.51 Reference seed u11, 37, 71, 98, 1310.92–0.96 0.91–0.96 11, 37, 71, 98, 1310.62–0.68 0.39–0.50 Tables 11– 17 report the complete setting-level results, including bootstrap confidence intervals. Representation construction and retrieval. The CT28 reference records AUC/FWT of 0.94/0.94 and remains stable, with both metrics remaining between 0.93 and 0.95. Moderate tem- perature changes are also tolerated. τ = 0.5 records 0.96/0.95, whereas flattening the retrieval distribution with τ = 4 reduces the result to 0.89/0.88. Reference sizes of 10, 25, and 50 remain near 0.94/0.94, while sizes of 100 and 200 obtain 0.89/0.88 and 0.91/0.90, respectively. MG16 is more sensitive to representation quality. The independently repeated reference config- uration records 0.62/0.43; its difference from the earlier N = 512 reference run (0.62/0.37) is not significant (two-sided paired p = 0.92/0.41). Lowering τ to 0.5 raises the sweep mean to 28 Preprint H1H2H3H4 mean logstd Push-wall Faucet-close Push-back Stick-pull Handle-press Push Shelf-place Window-close Peg-unplug 0.500.500.500.500.500.50 0.500.500.500.500.500.50 0.500.500.500.500.500.50 0.360.360.360.360.360.36 0.450.450.450.450.450.45 0.420.420.420.420.420.42 0.290.290.290.290.290.29 0.330.330.330.330.330.33 0.220.220.220.220.220.22 A AMSC actor H1H2H3H4 mean logstd 0.520.931.000.560.000.57 0.200.890.750.230.000.52 0.070.810.790.800.000.28 0.120.770.950.780.000.33 0.160.680.700.420.000.26 0.160.960.960.780.040.34 0.201.001.000.920.000.34 0.120.800.730.280.000.32 0.181.001.000.710.020.35 B Mask-BLC actor H1H2H3H4 out 0.490.990.900.900.88 0.640.830.570.170.18 0.400.840.810.620.48 0.200.640.700.590.40 0.590.850.590.130.12 0.521.000.900.560.11 0.210.860.970.510.08 0.520.860.620.210.13 0.621.000.890.160.00 C Mask-BLC critics 0.0 0.2 0.4 0.6 0.8 1.0 Current-mask coefficient 123456789 Task index 0.0 0.2 0.4 0.6 0.8 1.0 Mean within-task success D Task outcomes AMSC Mask-BLC 0.20.40.60.81.0 Steps on task (million) 0.0 0.2 0.4 0.6 0.8 1.0 E Task 9: Peg-unplug 0.20.40.60.81.0 Steps on task (million) 0.0 0.2 0.4 0.6 0.8 1.0 F Task 3: Push-back Figure 7: CW10 task-end composition and acquisition across seven seeds. Top: mean current-task mask coefficients by task and layer. Bottom: mean evaluation success by task and learning curves for the two tasks with the largest Mask-BLC–AMSC success gaps. Curves use 20 evaluation points per task. Coefficients describe the saved training-time allocation, not the effective contribution of each mask after task-boundary consolidation. 012345678 Prior source task 1 2 3 4 5 6 7 8 9 Target task 0.50 0.430.07 0.50 0.290.35 0.52 0.390.090.050.05 0.180.310.21 0.240.42 0.110.030.350.070.050.15 AMSC 012345678 Prior source task 1 2 3 4 5 6 7 8 9 Target task 0.40 0.270.30 0.130.330.09 0.060.100.070.28 0.050.070.230.090.19 0.040.110.140.100.05 0.040.050.060.070.16 0.030.030.080.050.060.230.060.08 0.030.070.070.110.060.06 Mask-BLC 0.0 0.2 0.4 0.6 0.8 1.0 Mean actor coefficient across layers and seeds Figure 8: Mean actor coefficient assigned to each prior source task, averaged over layers and seven seeds. Rows are target tasks and columns are earlier source tasks; gray cells denote sources not yet available. Only coefficients of at least 0.03 are annotated. 0.67/0.50, while τ = 4 obtains 0.50/0.16. Increasing N from 128 to 1024 changes AUC/FWT from 0.54/0.23 to 0.73/0.54, but 2048 samples reduce FWT to 0.30. Reference-size results are 29 Preprint Table 11: Sparsemax-temperature sensitivity. Values are means with asymmetric 95% bootstrap confidence intervals across seven seeds. Bold entries denote the sweep-reference setting; All runs obtain zero measured FGT. CT28MG16 τAUCFWTFGTτAUCFWTFGT 0.250.93 +0.04 −0.04 0.92 +0.05 −0.05 0.000.250.63 +0.07 −0.08 0.45 +0.09 −0.08 0.00 0.50.96 +0.03 −0.03 0.95 +0.04 −0.04 0.000.50.67 +0.10 −0.09 0.50 +0.12 −0.11 0.00 10.94 +0.02 −0.02 0.94 +0.02 −0.03 0.00 10.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 20.93 +0.03 −0.03 0.92 +0.03 −0.04 0.00 20.54 +0.10 −0.12 0.31 +0.12 −0.14 0.00 40.89 +0.03 −0.02 0.88 +0.04 −0.03 0.0040.50 +0.07 −0.07 0.16 +0.18 −0.22 0.00 Table 12: Task-sample-count sensitivity. Values are means with asymmetric 95% bootstrap con- fidence intervals across seven seeds. Bold entries denote the principal sample counts (CT28: N = 128; MG16: N = 1024). All runs obtain zero measured FGT. CT28MG16 NAUCFWTFGTNAUCFWTFGT 320.94 +0.03 −0.04 0.93 +0.04 −0.04 0.001280.54 +0.11 −0.12 0.23 +0.22 −0.28 0.00 640.95 +0.02 −0.03 0.95 +0.03 −0.03 0.00 2560.57 +0.12 −0.13 0.33 +0.17 −0.18 0.00 128 0.94 +0.02 −0.02 0.94 +0.02 −0.03 0.00 5120.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 2560.95 +0.03 −0.03 0.95 +0.03 −0.04 0.001024 0.73 +0.06 −0.06 0.54 +0.12 −0.14 0.00 5120.95 +0.03 −0.04 0.95 +0.03 −0.03 0.00 20480.63 +0.10 −0.13 0.30 +0.29 −0.44 0.00 likewise non-monotonic: R = 10 obtains 0.70/0.52, while R = 25 obtains 0.55/0.35. These point estimates show that more samples do not monotonically improve acquisition. Update schedule and smoothing. Frequent updates are most important on CT28. Increasing either f emb or f sel from 1 to 10 reduces AUC/FWT from 0.94/0.94 to 0.84/0.83 and 0.83/0.81, respectively. On MG16, increasing f emb from 1 to 10 changes 0.62/0.43 to 0.57/0.34. Selection frequency is less ordered: the sweep-reference period of 5 has the highest FWT (0.43), whereas a period of 10 has the highest AUC (0.65). EMA smoothing is not required for the best point estimate in either sweep. CT28 peaks at ρ = 0.25 with 0.95/0.95, and MG16 peaks without EMA at 0.68/0.51. Changing the random reference seed produces AUC ranges of 0.92–0.96 and 0.62–0.68, and FWT ranges of 0.91–0.96 and 0.39–0.50, on CT28 and MG16, respectively. These sweeps describe one-factor sensitivity around the stated references; they do not establish robustness to joint hyperparameter changes or document how the principal configuration was selected. Retention is unaffected across all settings. The observed sensitivity concerns acquisition. F.4PER-TASK ACQUISITION AND RETENTION Tables 18 and 19 report task-level AUC, FWT, and FGT for AMSC and full-library learned com- position. Table 20 additionally reports CW10 comparisons with Mask-LC and Mask-BLC. These results expose variation hidden by curriculum averages. The supplied MG16 per-task table summa- rizes an earlier result set: averaging its displayed task AUCs gives approximately 0.62, rather than the principal 0.73. It is retained as a separately identified analysis and must not be used as a de- composition of the current main-text aggregate. CT28 gains are broad but diminish on several late, high-depth tasks. On MG16, both methods exhibit negative normalized FWT on some R2 cold-start tasks, while AMSC improves acquisition across the later R3–R5 tasks. Both parameter-isolation methods retain every completed task, yielding zero measured FGT throughout. 30 Preprint Table 13: Reference-distribution-size sensitivity. Values are means with asymmetric 95% bootstrap confidence intervals across seven seeds. Bold entries denote the sweep-reference setting. All runs obtain zero measured FGT. CT28MG16 RAUCFWTFGTRAUCFWTFGT 100.94 +0.03 −0.04 0.94 +0.04 −0.05 0.00100.70 +0.08 −0.09 0.52 +0.14 −0.14 0.00 250.94 +0.03 −0.03 0.93 +0.03 −0.04 0.00 250.55 +0.09 −0.09 0.35 +0.16 −0.16 0.00 50 0.94 +0.02 −0.02 0.94 +0.02 −0.03 0.0050 0.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 1000.89 +0.03 −0.02 0.88 +0.04 −0.03 0.00 1000.69 +0.10 −0.10 0.50 +0.14 −0.12 0.00 2000.91 +0.03 −0.03 0.90 +0.03 −0.03 0.002000.66 +0.10 −0.11 0.48 +0.14 −0.14 0.00 Table 14: Embedding-update-period sensitivity. Values are means with asymmetric 95% bootstrap confidence intervals across seven seeds. Bold entries denote the sweep-reference setting. All runs obtain zero measured FGT. CT28MG16 f emb AUCFWTFGTf emb AUCFWTFGT 10.94 +0.02 −0.02 0.94 +0.02 −0.03 0.0010.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 20.91 +0.03 −0.03 0.91 +0.04 −0.03 0.00 20.59 +0.14 −0.15 0.41 +0.18 −0.19 0.00 50.90 +0.03 −0.03 0.90 +0.04 −0.05 0.00 50.58 +0.13 −0.15 0.36 +0.21 −0.25 0.00 100.84 +0.03 −0.03 0.83 +0.05 −0.05 0.00 100.57 +0.13 −0.13 0.34 +0.14 −0.13 0.00 Table 15: Selection-update-period sensitivity. Values are means with asymmetric 95% bootstrap confidence intervals across seven seeds. Bold entries denote the sweep-reference setting; MG16 uses N = 512 outside the sample-count sweep. All runs obtain zero measured FGT. CT28MG16 f sel AUCFWTFGTf sel AUCFWTFGT 10.94 +0.02 −0.02 0.94 +0.02 −0.03 0.0010.64 +0.08 −0.08 0.40 +0.18 −0.20 0.00 20.91 +0.03 −0.02 0.90 +0.03 −0.03 0.0020.57 +0.14 −0.15 0.35 +0.14 −0.16 0.00 50.86 +0.02 −0.02 0.85 +0.02 −0.02 0.00 50.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 100.83 +0.03 −0.02 0.81 +0.04 −0.04 0.00 100.65 +0.08 −0.09 0.42 +0.12 −0.13 0.00 Table 16: Embedding-EMA sensitivity. Values are means with asymmetric 95% bootstrap confi- dence intervals across seven seeds. Bold entries denote the sweep-reference setting; MG16 uses N = 512 outside the sample-count sweep. All runs obtain zero measured FGT. CT28MG16 ρAUCFWTFGTρAUCFWTFGT 00.95 +0.03 −0.03 0.94 +0.03 −0.04 0.00 00.68 +0.10 −0.12 0.51 +0.14 −0.14 0.00 0.250.95 +0.02 −0.03 0.95 +0.02 −0.03 0.00 0.250.66 +0.10 −0.10 0.48 +0.14 −0.14 0.00 0.5 0.94 +0.02 −0.02 0.94 +0.02 −0.03 0.000.5 0.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 0.750.93 +0.03 −0.03 0.92 +0.03 −0.03 0.000.750.67 +0.09 −0.08 0.48 +0.12 −0.10 0.00 0.90.93 +0.03 −0.03 0.92 +0.04 −0.04 0.000.90.66 +0.12 −0.13 0.47 +0.17 −0.17 0.00 31 Preprint Table 17: Random-reference-seed sensitivity. Values are means with asymmetric 95% bootstrap confidence intervals across seven seeds. Bold entries denote the sweep-reference setting; MG16 uses N = 512 outside the sample-count sweep. All runs obtain zero measured FGT. CT28MG16 uAUCFWTFGTuAUCFWTFGT 110.96 +0.02 −0.03 0.96 +0.02 −0.03 0.00110.65 +0.08 −0.08 0.47 +0.12 −0.13 0.00 370.93 +0.03 −0.03 0.92 +0.04 −0.04 0.00 370.65 +0.09 −0.09 0.46 +0.14 −0.15 0.00 710.92 +0.03 −0.03 0.91 +0.04 −0.03 0.00710.62 +0.08 −0.09 0.39 +0.14 −0.14 0.00 98 0.94 +0.02 −0.02 0.94 +0.02 −0.03 0.00 98 0.62 +0.08 −0.09 0.43 +0.13 −0.12 0.00 1310.92 +0.03 −0.03 0.91 +0.04 −0.04 0.00 1310.68 +0.10 −0.12 0.50 +0.16 −0.17 0.00 Table 18: Per-task CT28 acquisition and retention. Values are means across seven seeds. FWT is undefined when the single-task expert saturates the normalization denominator and can fall below −1 when the expert’s residual headroom is small. AMSCMask-LC TaskAUCFWTFGTAUCFWTFGT A–D21.00—0.001.00—0.00 B–D20.96—0.00 0.97—0.00 C–D20.99—0.001.00—0.00 D–D20.960.000.000.890.000.00 A–D31.001.000.00 1.001.000.00 B–D31.001.000.000.800.630.00 C–D31.001.000.00 0.990.950.00 D–D30.990.980.000.59-0.280.00 A–D41.001.000.000.970.980.00 B–D41.001.000.00 0.660.670.00 C–D40.940.970.000.390.400.00 D–D41.001.000.00 0.370.370.00 A–D50.990.990.000.760.790.00 B–D51.001.000.000.340.340.00 C–D51.001.000.000.410.420.00 D–D51.001.000.000.000.000.00 A–D61.001.000.000.840.880.00 B–D61.001.000.000.000.000.00 C–D60.600.630.000.000.000.00 D–D61.001.000.000.000.000.00 A–D71.001.000.000.690.710.00 B–D70.990.990.00 0.000.000.00 C–D70.560.560.000.000.000.00 D–D71.001.000.000.000.000.00 A–D81.001.000.000.660.680.00 B–D80.940.970.000.000.000.00 C–D80.670.680.000.000.000.00 D–D80.770.810.00 0.000.000.00 32 Preprint Table 19: Per-task MG16 acquisition and retention for the earlier result set (mean displayed AUC approximately 0.62). This table does not decompose the principal 0.73 AUC result. Values are means across seven seeds. FWT is undefined when the single-task expert saturates the normalization denominator and can fall below−1 when the expert’s residual headroom is small. AMSCMask-LC TaskAUCFWTFGTAUCFWTFGT BallLava–R20.830.240.000.810.230.00 LavaBall–R20.81-1.270.00 0.78-2.410.00 KeyBoxSwap–R20.910.210.000.81-2.140.00 Identity–R20.790.210.000.40-2.570.00 BallLava–R30.720.720.000.630.630.00 LavaBall–R30.560.560.000.440.440.00 KeyBoxSwap–R30.740.720.00 0.470.470.00 Identity–R30.500.460.00 0.13-0.420.00 BallLava–R40.620.620.000.360.360.00 LavaBall–R40.690.690.000.330.330.00 KeyBoxSwap–R40.450.450.00 0.420.420.00 Identity–R40.240.240.000.180.180.00 BallLava–R50.610.610.00 0.370.380.00 LavaBall–R50.570.570.000.130.130.00 KeyBoxSwap–R50.490.490.000.130.130.00 Identity–R50.380.390.00 0.060.060.00 Table 20: Per-task CW10 acquisition and retention across seven shared seeds. AUC averages the 20 active-task evaluations; FWT compares trapezoidal areas with the seed-matched from-scratch run. FWT is undefined when the reference headroom is below 10 −3 and can be strongly negative when that headroom is small. FGT compares task-end and final success. Window-close FWT uses only one non-saturated seed; its AUC and FGT and all other rows use seven seeds. AMSCMask-LCMask-BLC TaskAUC FWT FGTAUC FWT FGTAUC FWT FGT 0: Hammer0.24-1.620.000.36-0.670.000.36-0.67 0.00 1: Push-wall0.31-0.05 -0.010.370.080.000.370.080.00 2: Faucet-close0.95-3.270.000.990.220.000.990.130.00 3: Push-back0.230.230.000.440.44-0.010.410.420.00 4: Stick-pull0.01-0.070.000.00-0.090.000.00-0.09 0.00 5: Handle-press-side0.95-4.670.000.990.930.000.990.430.00 6: Push0.510.430.000.09-0.11 -0.010.220.060.00 7: Shelf-place0.010.01-0.010.000.000.000.020.020.00 8: Window-close0.991.000.000.980.800.000.980.500.00 9: Peg-unplug-side0.62-2.090.000.53-2.920.000.920.600.00 33 Preprint GEVALUATION METRICS Consider a curriculum of N tasks with an equal interaction budget ∆ per task and total budget T = N ∆. Task i is trained during [t i ,e i ], where t i = (i− 1)∆ and e i = i∆. Let p i (t) ∈ [0, 1] be the normalized evaluation performance on task i at checkpoint t. Each checkpoint value averages the evaluation episodes conducted without training. CT28 returns and CW10 success rates already lie in [0, 1]. For MG16, we divide each return by the corresponding task’s theoretical shortest-path return and clip the result to [0, 1]. The same task- specific normalization is applied to lifelong and single-task runs. Evaluations made before a task is first encountered are excluded from acquisition metrics. Acquisition AUC. Suppose task i’s active training block contains M i regularly spaced evaluation checkpoints t i,1 ,...,t i,M i . Following the discrete aggregation used in the Continual World result code (Wolczyk et al., 2021), the task-local acquisition score is AUC i = 1 M i M i X j=1 p i (t i,j ).(26) This is the mean height of the normalized learning curve while task i is being trained. The reported curriculum score is the task macro-average AUC = 1 N N X i=1 AUC i .(27) Thus, AUC measures acquisition performance rather than retention after a task block and lies in [0, 1]. Forward Transfer. Forward transfer compares each lifelong learning curve with a seed-matched single-task reference trained from scratch (Wolczyk et al., 2021). In the FWT calculation, task-local areas are estimated by the normalized trapezoidal rule. For a sequence q 1 ,...,q M , define A(q 1:M ) = 1 2 q 1 + P M−1 j=2 q j + 1 2 q M M − 1 .(28) Let A CL i and A ST i denote this area for the lifelong and single-task learning curves, respectively. Then FWT i = A CL i − A ST i 1− A ST i ,FWT = 1 |I FWT | X i∈I FWT FWT i .(29) The set I FWT contains every task for which 1 − A ST i ≥ 10 −3 . This exclusion avoids unstable ratios when the single-task reference is effectively saturated. No task is otherwise removed from the reported FWT; in particular, CT28’s four initial depth-2 tasks are retained. They are excluded only from selection analyses that require an earlier hierarchy-relevant source. Zero FWT matches single-task acquisition, positive values indicate faster acquisition, and negative values indicate slower acquisition. The normalized score is at most one but can be substantially negative when a strong single-task reference appears in the denominator. AUC and FWT therefore need not rank methods identically: AUC measures absolute acquisition, whereas FWT first normal- izes each task’s improvement by the remaining headroom over its single-task reference and then macro-averages the task scores. The reported AUC and the trapezoidal area used by FWT also differ slightly in endpoint weighting. Forgetting. Continual World forgetting compares each task’s performance at the end of its training block with its performance at the final curriculum checkpoint (Wolczyk et al., 2021): FGT i = p i (e i )− p i (T ),FGT = 1 N N X i=1 FGT i .(30) Positive values indicate forgetting and negative values indicate subsequent improvement. A value of zero indicates no net average change; complete measured retention additionally requires every per-task difference to be zero. The final task contributes zero because e N = T . 34 Preprint HSIGNIFICANCE TESTING Statistical protocol. Deep reinforcement learning results vary across random seeds and environ- ment stochasticity (Henderson et al., 2018; Colas et al., 2018). Each comparison uses the seven shared seeds 86–92. We compute one metric value per seed, then test the mean paired difference ∆ = AMSC− comparison using an exact two-sided sign-flip test over all 2 7 = 128 sign assign- ments (Winkler et al., 2014). The tables also report 95% percentile-bootstrap confidence intervals for the mean paired difference using 10,000 paired resamples with seed zero. We report unadjusted p-values with 95% confidence intervals. Green cells denote p < 0.05; red cells denote p ≥ 0.05. Color indicates evidence for a difference, not which method is better. Positive ∆ favors AMSC for AUC and FWT, whereas negative ∆ favors AMSC for FGT. Baseline comparisons. Tables 21, 22, and 23 use the principal configurations reported in the main text. CT28 uses N = 128; MG16 uses the seven completed N = 1024,R = 50,ρ = 0.5 runs from the sample-count sweep. With seven pairs, the smallest attainable two-sided raw p-value is 2/128 = 0.015625. Table 21: CT28 baseline comparisons across seven matched seeds. Each metric reports the exact two-sided raw p and a 95% paired percentile-bootstrap CI for ∆ = AMSC− comparison. Green denotes p < 0.05; red denotes p ≥ 0.05. Tests are not adjusted for multiple comparisons. Color indicates significance, not which method is better. AUCFWTFGT Methodp95% CIp95% CIp95% CI Fine-tuning0.016[0.56, 0.60]0.016[0.70, 0.74]0.016[−0.40,−0.36] From-scratch 0.016[0.69, 0.75]0.016[0.91, 0.96]1.000[0.00, 0.00] SER-s0.422[−0.06, 0.02]0.578[−0.06, 0.02]0.016[−0.70,−0.36] SER-r 0.188[0.00, 0.12]0.172[0.00, 0.14]0.125[−0.18,−0.02] SDW0.047[0.06, 0.28]0.047[0.07, 0.34]0.500[0.00, 0.02] OWL 0.016[0.76, 0.82]0.016[1.09, 1.43]0.016[−0.21,−0.14] SANE0.016[0.44, 0.53]0.016[0.52, 0.61]1.000[0.00, 0.00] Mask-BLC0.016[0.63, 0.69]0.016[0.78, 0.89]1.000[0.00, 0.00] Mask-LC 0.016[0.44, 0.49]0.016[0.53, 0.65]1.000[0.00, 0.00] Mask-LC (L2)0.016[0.37, 0.47]0.016[0.44, 0.58]1.000[0.00, 0.00] CKA-RL0.016[0.51, 0.64]0.016[0.60, 0.75]0.016[−0.41,−0.31] CompoNet 0.016[0.15, 0.37]0.016[0.18, 0.43]1.000[0.00, 0.00] MOSAIC0.016[0.08, 0.17]0.016[0.11, 0.20]1.000[0.00, 0.00] MOSAIC¬I perf 0.016[0.09, 0.17]0.016[0.11, 0.20]1.000[0.00, 0.00] Table 22: MG16 baseline comparisons across seven matched seeds. Each metric reports the exact two-sided raw p and a 95% paired percentile-bootstrap CI for ∆ = AMSC− comparison. Green denotes p < 0.05; red denotes p ≥ 0.05. Tests are not adjusted for multiple comparisons. Color indicates significance, not which method is better. AUCFWTFGT Methodp95% CI p95% CI p95% CI Fine-tuning0.016[0.38, 0.50]0.016[0.33, 0.56]0.016[−0.41,−0.27] From-scratch0.016[0.45, 0.57]0.016[0.40, 0.66]1.000[0.00, 0.00] SER-s0.016[0.25, 0.36]0.016[0.50, 0.85]0.016[−0.60,−0.41] SER-r0.031[0.05, 0.16]0.016[0.23, 0.47]0.375[−0.18, 0.02] SDW0.016[0.14, 0.26]0.016[0.43, 0.62]0.094[−0.28,−0.03] OWL0.016[0.46, 0.59]0.016[0.67, 1.00]0.016[−0.24,−0.20] SANE0.016[0.31, 0.46]0.016[0.28, 0.43]1.000[0.00, 0.00] Mask-BLC0.016[0.09, 0.27]0.016[0.17, 0.45]1.000[0.00, 0.00] Mask-LC0.016[0.20, 0.45]0.016[0.44, 1.10]1.000[0.00, 0.00] Mask-LC (L2)0.016[0.21, 0.35]0.016[0.16, 0.35]1.000[0.00, 0.00] CKA-RL0.016[0.34, 0.47]0.016[0.64, 0.87]0.016[−0.54,−0.28] CompoNet 0.016[0.34, 0.51]0.016[1.18, 1.63]1.000[0.00, 0.00] MOSAIC0.016[0.35, 0.53]0.016[0.72, 1.38]1.000[0.00, 0.00] MOSAIC¬I perf 0.016[0.34, 0.56]0.016[0.74, 1.54]1.000[0.00, 0.00] CW10 comparisons. CW10 uses seed-matched online success evaluations. AMSC’s FWT is lower than Mask-BLC (p = 0.031) and from-scratch learning (p = 0.016) at the nominal 0.05 35 Preprint threshold. The AUC difference from Mask-BLC is not significant (p = 0.328). FWT excludes task– seed reference headroom below 10 −3 ; near-saturated reference curves can still amplify negative values. Table 23: CW10 baseline comparisons across seven matched seeds. ProgNet uses its native two- layer scratch FWT reference; other methods use the four-layer reference. Each metric reports the exact two-sided raw p and a 95% paired percentile-bootstrap CI for ∆ = AMSC− comparison. Green denotes p < 0.05; red denotes p ≥ 0.05. Tests are not adjusted for multiple comparisons. Color indicates significance, not which method is better. AUCFWTFGT Methodp95% CIp95% CIp95% CI Fine-tuning0.234[−0.04, 0.35]0.703[−1.15, 18.58]0.062[−0.58,−0.18] From-scratch 0.062[0.02, 0.12]0.016[−2.19,−0.43]0.500[−0.01, 0.00] ProgNet0.016[0.08, 0.21]0.141[−1.90,−0.06]0.500[−0.01, 0.00] Mask-LC0.781[−0.08, 0.10]0.062[−2.01,−0.22]1.000[−0.01, 0.01] Mask-BLC0.328[−0.12, 0.03]0.031[−2.21,−0.58]0.500[−0.01, 0.00] CKA-RL0.141[0.00, 0.20]0.453[−0.59, 2.77]0.500[−0.01, 0.00] MOSAIC 0.016[0.16, 0.29]0.016[0.49, 9.03]0.500[−0.01, 0.00] Composition and retrieval controls. Table 24 compares each control with configuration-matched AMSC runs across seven shared seeds. CT28 uses N = 128, while all MG16 comparisons use N = 1024, R = 50, and ρ = 0.5. Each control changes only the specified retrieval or composition component. FGT is omitted because all controls and their matched AMSC references have zero measured forgetting. Table 24: CT28 and MG16 paired ablation comparisons across seven matched seeds. Each metric reports the exact two-sided unadjusted p-value and a 95% paired bootstrap CI for ∆ = AMSC− control. Green denotes p < 0.05; red denotes p ≥ 0.05. Positive differences favor AMSC. CT28 uses N = 128. MG16 uses N = 1024, with R = 50 and ρ = 0.5. CT28MG16 AUCFWTAUCFWT Configurationp95% CIp95% CIp95% CIp95% CI Learned β0.484[−0.02, 0.06]0.453[−0.02, 0.08]0.031[0.04, 0.23]0.047[0.06, 0.38] Uniform0.172[−0.01, 0.09]0.172[−0.01, 0.11]0.047[0.04, 0.20]0.016[0.05, 0.21] No ℓ 2 -STE0.047[0.01, 0.08]0.016[0.04, 0.11]0.734[−0.10, 0.16]0.172[−0.05, 0.36] Frozen retrieval 0.047[0.03, 0.12]0.062[0.03, 0.14]0.031[0.02, 0.22]0.016[0.05, 0.22] Top-one0.078[0.02, 0.11]0.078[0.02, 0.12]0.844[−0.12, 0.09]0.844[−0.16, 0.10] Matched random 0.016[0.56, 0.62]0.016[0.74, 0.85]0.016[0.17, 0.34]0.016[0.18, 0.39] Transfer-informed0.016[0.10, 0.15]0.016[0.15, 0.33]0.531[−0.14, 0.05]0.031[−0.27,−0.08] 36 Preprint Table 25: Alignment of initial and final AMSC retrieval with aggregate pairwise transfer. Weighted ∆AUC normalizes weights over retrieved sources and therefore excludes current-mask mass. Paren- theses give support-size-matched random expectations. Statistics aggregate seven lifelong runs; the transfer value assigned to each pair comes from one source-policy training seed. BenchmarkPhaseSupportMean ∆AUCWeighted ∆AUCPositive precisionPositive-mass recallBest source CT28Initial3.110.52 (0.10)0.490.61 (0.19)0.87 (0.23)0.98 (0.47) Final2.570.57 (0.10)0.620.67 (0.19)0.86 (0.20)0.96 (0.43) MG16Initial2.420.47 (0.20)0.510.74 (0.46)0.63 (0.33)0.84 (0.41) Final1.940.53 (0.20)0.550.80 (0.46)0.58 (0.27)0.84 (0.36) H.1PAIRWISE TRANSFER MATRICES Figures 9 and 10 report the unaggregated pairwise transfer measurements used in the retrieval anal- ysis. Each cell is ∆AUC, the target-task AUC obtained with the row source minus the AUC of the corresponding no-source run. Positive values indicate improved learning over the target budget and negative values indicate interference. Rows are source policies and columns are target tasks; tasks are grouped by family and ordered by increasing depth or radius. The complete matrices include self-pairs and sources that occur after the target in the lifelong curriculum. The curriculum-restricted analysis in Section 5.5 retains only sources available when the target is encountered. H.2PER-TASK POLICY-MASK UTILIZATION Figures 11 and 12 report how AMSC distributes composition mass across the current and retrieved policy masks. Rows are target tasks and columns are policy-mask components. Diagonal cells therefore correspond to the current mask, while colored cells below the diagonal correspond to prior masks available at that point in the curriculum. Values are the mean fixed composition coefficients across retrieval events and seven seeds. Grey cells identify available priors that are never selected; blank cells correspond to future masks that are not yet available. These coefficients quantify mask reuse, rather than the fraction of backbone parameters activated by a mask. H.3INITIAL RETRIEVAL AND TEMPORAL REFINEMENT We compare AMSC’s first eligible retrieval and its final retrieval with the pairwise-transfer measure- ments above. The first event occurs after the current-task experience buffer contains enough samples to construct an embedding. We exclude task runs with fewer than two measured candidate sources because they contain no meaningful source-selection decision. Table 25 reports alignment with measured transfer. Positive-transfer precision is the fraction of selected sources with ∆AUC > 0. Positive-mass recall is given by P i∈P(k) max(∆AUC i→k , 0) P i<k max(∆AUC i→k , 0) ,(31) which weights recovered sources by transfer magnitude rather than counting all positive pairs equally. Best-source inclusion indicates whether the support contains a source attaining the largest available ∆AUC. Parenthesized values are analytic expectations under uniform sampling without replacement of a support with the same size. The initial support is already strongly transfer-aligned. Its mean ∆AUC exceeds the random expec- tation by 0.42 on CT28 and 0.27 on MG16; seed-bootstrap 95% confidence intervals are [0.40, 0.43] and [0.25, 0.29]. Initial retrieval also contains the best available source in 98% and 84% of task runs. Refinement trades some positive-transfer mass recall for greater precision, consistent with pruning an initially over-inclusive support. Although only 10% of consecutive CT28 retrieval events and 7% of MG16 events change sup- port, these changes accumulate over training. Across seeds, CT28 support changes add 23 and remove 122 source occurrences; their mean ∆AUC values are 0.39 and 0.16, respectively. MG16 changes add 8 and remove 55 occurrences, with corresponding means of 0.28 and 0.22. The source- 37 Preprint A2A3A4A5A6A7A8B2B3B4B5B6B7B8 C2C3C4C5C6C7C8 D2D3D4D5D6D7D8 Target task A2 A3 A4 A5 A6 A7 A8 B2 B3 B4 B5 B6 B7 B8 C2 C3 C4 C5 C6 C7 C8 D2 D3 D4 D5 D6 D7 D8 Source policy 0.310000-.1-.2000000001000-.1-.4-.10000 0.3111000000000001.9000-.1-.1-.10000 0.311111-.1.2.400000010000-.2-.2-.10000 0.31111.900.40000-.1-.6000000-.3-.10000 0.31.9111-.1.2000000-.200000-.10-.10000 0.31.91110-.4000000-.1.1.3000-.2-.5-.10000 0.3.81110-.2-.4.100000-.1.200000-.2-.10000 -.1-.3000000.5.90000-.2-.300000-.1-.3-.10000 -.10000000.5111000-10000000-.10000 -.1-.1000000.5111100-.300000-.1-.3-.10000 0.1.90000-.1.5111100-.2000000-.5-.10000 0-.3.200000.5111100-.100000-.1-.3-.10000 -.2.1.200000.5111110-.200000-.6-.2-.10000 0-.2000000.511111-.1-.700000-.4-.5-.10000 0.300000-.1.3.100000011000-.1-.1-.10000 0-.2000000.500000001000000-.10000 0.200.1000-.10000000110000-.4-.10000 0.2.80000-.3.10000000110000-.1-.10000 0.2000000.20000000111100-.1-.10000 0.3100000.5.500000011.91100-.10000 0.20000000000000011.911-.1-.5-.10000 0.1000000-.300000-.2-.9000000.1.91110 0.3000000-.200000-.1-1000000.1.9.9000 -.1-.200000-.1.2.10000000.10000.1.91110 0-.2.10000-.2.100000-.1-.3000000.1.91111 000.1.1000.300000-.1-.5000000.1.91110 0.100000-.2-.400000-.2-1000000.1.911.90 0.200000-.2.100000-.1-1000000.1.91111 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 Pairwise transfer, AUC Figure 9: Raw pairwise transfer on CT28. Target training is initialized from the source policy and then fine-tuned. Cell annotations report ∆AUC relative to learning the target without a source; red denotes positive transfer and blue denotes negative transfer. Source policies are taken from the family-isolated seed-92 runs. Family labels A–D and depths 2–8 define the row and column order. 38 Preprint BallLava R2BallLava R3BallLava R4BallLava R5LavaBall R2LavaBall R3LavaBall R4LavaBall R5 KeyBoxSwap R2KeyBoxSwap R3KeyBoxSwap R4KeyBoxSwap R5 Identity R2Identity R3Identity R4Identity R5 Target task BallLava R2 BallLava R3 BallLava R4 BallLava R5 LavaBall R2 LavaBall R3 LavaBall R4 LavaBall R5 KeyBoxSwap R2 KeyBoxSwap R3 KeyBoxSwap R4 KeyBoxSwap R5 Identity R2 Identity R3 Identity R4 Identity R5 Source policy .06.93.98.98-.06-.54.50.42.34.73.480.06.1000 .06.94.600.08.3000.34.71.810-.24-.19.550 .06.391.96.06.3600.34.92.780.18.24.730 .06.94.89.98.10.28.360.36.92.87.88.06.21.580 -.51000.16-.54.800-.58000-.46-.1000 0.8100.14.46.90.82-.58.6500-.39-.7100 -.39000.16.4211.19.3100-.07-.2300 -.02.4600.16.46.98.98.19000-.12-.4600 .06.540.82.08.2600.36.8500.06-.1300 .02.800.80.12.34.180.36.96.97.93.12.2100 0.74.800.12.3400.28.96.97.91.12.15.740 .06.88.18.98.16.16.600.38.94.93.94.10.1900 -.40000.12.4400.34.96.93.82.12.23.910 -.02.4600.14.42.040.34.90.81.43.12.23.970 -.08.8400.12.4000.30.90.91.83.12.23.950 .06.760.58.12.4400.34.92.870.12.21.910 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 Pairwise transfer, AUC Figure 10: Raw pairwise transfer on MG16. Each source mask is composed with a newly initialized target mask using an initially equal mixture, and both the target mask and mixture coefficients are optimized on the target task. Cell annotations report ∆AUC relative to the matched no-source run; red denotes positive transfer and blue denotes negative transfer. Source masks are taken from the family-isolated seed-86 runs. Tasks are grouped by observation-remapping family and ordered by radius 2–5. 39 Preprint A D2A D3A D4A D5A D6A D7A D8 B D2B D3B D4B D5B D6B D7B D8 C D2C D3C D4C D5C D6C D7C D8 D D2D D3D D4D D5D D6D D7D D8 Policy-mask component A D2 A D3 A D4 A D5 A D6 A D7 A D8 B D2 B D3 B D4 B D5 B D6 B D7 B D8 C D2 C D3 C D4 C D5 C D6 C D7 C D8 D D2 D D3 D D4 D D5 D D6 D D7 D D8 Target task 1.00 0.500.50------ 0.270.390.33------------ 0.120.210.410.25------------------ 0.020.030.280.400.27------------------------ 0.010.020.180.230.340.22------------------------------ 0.01<.010.050.150.230.350.21------------------------------------ 0.500.50 ----0.500.50---- ------0.210.460.33-------- --------0.070.210.460.26------------ ----------0.030.160.210.380.22---------------- ------------0.010.030.130.260.350.22-------------------- --------------<.01<.010.070.170.280.270.20------------------------ 0.030.480.50 --------0.500.50-- ------------0.160.500.33---- ----------------0.010.190.470.33------ --------------------0.020.040.240.410.29-------- ------------------------0.020.030.110.240.330.27---------- ----------------------------<.01<.010.050.120.210.340.27------------ --0.50--0.50 ------------0.500.50 ------------------0.230.430.33 ------------------------0.150.280.320.25 ------------------------------0.030.180.190.380.22 ------------------------------------0.010.070.220.210.290.20 ------------------------------------------0.010.020.070.200.280.250.18 Mean current and prior weights: fixed sparsemax composition Available prior, never selected 0.0 0.2 0.4 0.6 0.8 1.0 Mean composition weight Figure 11: Per-task policy-mask utilization on CT28. Cell annotations report the mean composition coefficient assigned to each current or prior mask. The block structure shows that AMSC places most prior mass on masks from the same task family, while retaining limited cross-family reuse where the online representation retrieves it. 40 Preprint BallLava R2BallLava R3BallLava R4BallLava R5LavaBall R2LavaBall R3LavaBall R4LavaBall R5 KeyBoxSwap R2KeyBoxSwap R3KeyBoxSwap R4KeyBoxSwap R5 Identity R2Identity R3Identity R4Identity R5 Policy-mask component BallLava R2 BallLava R3 BallLava R4 BallLava R5 LavaBall R2 LavaBall R3 LavaBall R4 LavaBall R5 KeyBoxSwap R2 KeyBoxSwap R3 KeyBoxSwap R4 KeyBoxSwap R5 Identity R2 Identity R3 Identity R4 Identity R5 Target task 1.00 0.520.47<.01<.01-- 0.200.470.33<.010.01--<.01---- 0.060.180.490.27----<.01--<.01<.01------ 0.500.50 0.040.120.530.31---- <.01<.01<.010.190.490.32-------- 0.010.010.01<.010.050.270.380.27--<.01<.01------ 0.50--0.50 <.01<.01----0.510.49<.01 <.01<.01<.01------0.180.440.37--<.01 ------<.01--------0.020.160.480.35------ 0.03--0.540.43 --------0.020.070.520.39 ------------0.01<.010.010.170.480.33 --------------------<.01<.010.050.200.450.30 Mean current and prior weights: fixed sparsemax composition Available prior, never selected 0.0 0.2 0.4 0.6 0.8 1.0 Mean composition weight Figure 12: Per-task policy-mask utilization on MG16. Cell annotations report the mean composi- tion coefficient assigned to each current or prior mask. In contrast to CT28, AMSC assigns non- negligible mass to several cross-family components, consistent with the less family-separable trans- fer structure of MG16. 41 Preprint Table 26: Temporal retrieval diagnostics. J denotes support Jaccard similarity. Improved and worsened are fractions among task runs whose first and final supports differ, measured by source- normalized, similarity-weighted ∆AUC. BenchmarkConsecutive JFirst–final JChangedSupportWeighted ∆AUCImprovedWorsened CT280.970.820.433.11→ 2.570.49→ 0.620.700.23 MG160.970.820.392.42→ 1.940.51→ 0.550.740.16 normalized weighted improvement from first to final retrieval is 0.13 on CT28 (95% CI [0.11, 0.15]) and 0.03 on MG16 ([0.01, 0.05]). These diagnostics associate online refinement with greater aggregate pairwise transfer; they do not provide time-local transfer labels. A No Periodicity control tests repeated retrieval by retaining the first valid support and weights throughout each task. On CT28, AUC/FWT falls from 0.94/0.94 to 0.87/0.85, with paired p = 0.047 and p = 0.062. On MG16, retaining the first retrieval lowers AUC/FWT from 0.73/0.54 to 0.63/0.42, with paired (p = 0.031) and (p = 0.016), respectively, using matched (N = 1024) configurations. These results support updating retrieval as target-task experience accumulates. The pairwise transfer measure- ments summarize complete target learning curves from one source-policy seed, so their alignment with retrieval should be interpreted as a transfer proxy rather than direct evidence of instantaneous benefit. 42