Paper deep dive
Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution
Jiahong Liu, Ming Shen, Xiaohao Liu, Rex Ying, Menglin Yang, Tat-Seng Chua, Irwin King
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/25/2026, 5:15:19 AM
Summary
The paper introduces Hyperbolic Multimodal Continual Learning (HMCL), a method designed to preserve the Lorentz geometry of hyperbolic multimodal models during sequential task learning. Unlike existing methods that operate in Euclidean space, HMCL ensures that parameter updates correspond to a shared hyperbolic isometry, thereby maintaining within-modality similarity, cross-modal correspondence, and semantic hierarchy. The authors propose two variants: Closest-Admissible (CA), which finds the optimal shared rotation via a Lyapunov equation, and Minimal-Rotation (MR), which fixes rotation to zero. Evaluated on a 16-task stream with three hyperbolic backbones, HMCL significantly reduces geometric drift (81.2–95.5%) and improves backward transfer and final performance compared to baselines.
Entities (8)
Relation Signals (7)
HMCL-CA → isvariantof → HMCL
confidence 95% · We formulate a joint closest-admissible (CA) correction... Both variants correct the displacement realized by AdamW
HMCL-MR → isvariantof → HMCL
confidence 95% · its minimal-rotation (MR) special case fixes this rotation to zero.
HMCL → uses → Lorentz Geometry
confidence 95% · We address this gap with Hyperbolic Multimodal Continual Learning (HMCL)... explicitly preserve the Lorentz geometry
HMCL → reduces → geometric drift
confidence 93% · Representation analyses find 81.2 to 95.5 percent less radial, angular, cross-modal, and paired-distance drift
HMCL → improves → Backward Transfer
confidence 92% · HMCL improves final performance and backward transfer over sequential fine-tuning and four continual-learning baselines
HMCL → corrects → AdamW
confidence 90% · Both variants correct the displacement realized by AdamW
MERU → isbasefor → HMCL
confidence 85% · MERU [7]... show that hyperbolic geometry can support effective multimodal representations... This article generalizes that foundation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Existing continual-learning methods protect parameters, replayed examples, or Euclidean feature subspaces. When applied to hyperbolic multimodal models, they do not explicitly preserve the Lorentz geometry that jointly encodes within-modality similarity, cross-modal correspondence, and semantic hierarchy; sequential updates can therefore retain task scores while still distorting previously learned relations. We address this gap with Hyperbolic Multimodal Continual Learning (HMCL). We show that preserving the old multimodal geometry amounts to restricting all modalities to one shared hyperbolic isometry, which induces a family of admissible first-order parameter changes. We formulate a joint closest-admissible (CA) correction that retains the shared rotation best matching the candidate modal updates; its minimal-rotation (MR) special case fixes this rotation to zero. Both variants correct the displacement realized by AdamW, and task anchoring bounds within-task accumulation while preserving learning freedom. Across a unified 16-task classification-retrieval stream with three hyperbolic backbones, HMCL improves final performance and backward transfer over sequential fine-tuning and four continual-learning baselines; HMCL-CA gives the highest Overall score on every backbone. A modality-extended stream confirms the retrieval gains. Representation analyses find 81.2 to 95.5 percent less radial, angular, cross-modal, and paired-distance drift; ImageNet-WordNet results show better semantic ancestry and radial hierarchy.
Tags
Links
- Source: https://arxiv.org/abs/2609.29329v1
- Canonical: https://arxiv.org/abs/2609.29329v1
Trouble viewing inline? Open PDF directly →
Full Text
176,650 characters extracted from source content.
Expand or collapse full text
_set:Ne tcboxmath _set:Ne Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution Jiahong Liu Affiliation: The Chinese University of Hong Kong Ming Shen Affiliation: The Chinese University of Hong Kong Xiaohao Liu Affiliation: National University of Singapore Rex Ying Affiliation: Yale University Menglin Yang Affiliation: The Hong Kong University of Science and Technology (Guangzhou)Email: jiahong.liu21@gmail.com, king@cse.cuhk.edu.hk Tat-Seng Chua Affiliation: National University of Singapore Irwin King Affiliation: The Chinese University of Hong Kong Abstract Existing continual-learning methods protect parameters, replayed examples, or Euclidean feature subspaces. When applied to hyperbolic multimodal models, they do not explicitly preserve the Lorentz geometry that jointly encodes within-modality similarity, cross-modal correspondence, and semantic hierarchy; sequential updates can therefore retain task scores while still distorting previously learned relations. We address this gap with Hyperbolic Multimodal Continual Learning (HMCL). We show that preserving the old multimodal geometry amounts to restricting all modalities to one shared hyperbolic isometry, which induces a family of admissible first-order parameter changes. We formulate a joint closest-admissible (CA) correction that retains the shared rotation best matching the candidate modal updates; its minimal-rotation (MR) special case fixes this rotation to zero. Both variants correct the displacement realized by AdamW, and task anchoring bounds within-task accumulation while preserving learning freedom. Across a unified 16-task classification–retrieval stream with three hyperbolic backbones, HMCL improves final performance and backward transfer over sequential fine-tuning and four continual-learning baselines; HMCL-CA gives the highest Overall score on every backbone. A modality-extended stream confirms the retrieval gains. Representation analyses find 81.2–95.5% less radial, angular, cross-modal, and paired-distance drift; ImageNet–WordNet results show better semantic ancestry and radial hierarchy. Code and data download links are in the code repository. Keywords: continual learning; hyperbolic geometry; multimodal learning; vision–language representation learning 1 Introduction Figure 1: Continual adaptation of hyperbolic representations. Hyperbolic multimodal embeddings jointly encode task semantics and their hierarchy. Unconstrained sequential optimization may gradually distort this geometry and weaken structures learned from earlier tasks. Geometry-aware updates can reduce this form of forgetting. Multimodal contrastive learning (MCL) has become a standard approach for constructing representations shared across data types [1, 2, 3]. Trained with large paired corpora and contrastive objectives, these models align modality-specific observations in a semantic space that supports effective zero-shot transfer [4, 5, 6]. Most existing systems place this shared space in a high-dimensional Euclidean domain and therefore treat the organization of semantic concepts as flat. Semantic organization across modalities is rarely flat, however. A broad concept generally covers numerous specialized instances, creating asymmetric, partially ordered relations between different levels of abstraction. Uniform Euclidean neighborhoods do not explicitly distinguish such general-to-specific structure, making these relations difficult to represent without distortion. Hyperbolic Multimodal Learning. Negative-curvature geometry offers a useful alternative for hierarchical semantics. MERU [7] and subsequent compositional-entailment learning [8] show that hyperbolic geometry can support effective multimodal representations. Geodesic proximity reflects semantic similarity, whereas radial position and entailment relations encode specificity. Generic concepts tend to lie nearer the origin, and more specific concepts tend to lie farther away. Hyperbolic embeddings can therefore represent alignment and abstraction in the same space. Despite these advances, existing hyperbolic multimodal models have been developed mainly for static training distributions [7, 8, 9, 10]. Real systems instead encounter new tasks, concepts, and domains over time, yet standard continual-learning mechanisms do not explicitly protect the Lorentz relations that organize multimodal similarity and hierarchy. How to adapt such models sequentially without deforming their learned geometry therefore remains open. Continual Learning as Geometric Preservation. For a hyperbolic model, forgetting involves more than reduced accuracy on an earlier task. Parameter changes may deform within-modality neighborhoods, cross-modal correspondences, and the radial ordering that records semantic abstraction. Once these quantities drift, both similarity and hierarchy inherited from old tasks are altered. This observation motivates our central question: which parameter changes preserve the previously learned hyperbolic multimodal geometry while leaving sufficient freedom to acquire a new task? Our Approach. We view hyperbolic multimodal continual learning11 1 A preliminary version of this work appeared at ICML 2026. as optimization over updates that satisfy geometric preservation constraints, which we call admissible updates. Under the assumptions in Section 5, old-task geometry is retained when all modalities change through the same hyperbolic isometry. A first-order form of this condition describes the corresponding parameter updates. We then pose a joint closest-admissible problem over the modal parameter changes and the generator of their shared rotation. Its unique solution is obtained from one Lyapunov equation and defines HMCL-CA; constraining the generator to zero yields the projection used by HMCL-MR. For the AdamW implementation evaluated in this work, HMCL corrects the parameter step produced after moment estimation and coordinate-wise scaling, and task anchoring limits its accumulation within a task. Contributions. To the best of our knowledge, this work is the first to study continual hyperbolic multimodal representations explicitly through their geometry. Our main contributions are: • We relate retention of old multimodal knowledge to a shared hyperbolic isometry and derive the associated set of first-order parameter updates. • We formulate a joint closest-admissible problem whose shared-rotation solution is obtained from one Lyapunov equation. HMCLCAHMCL_CA permits the common rotation selected by this solution, while HMCLMRHMCL_MR restricts the same admissible family to zero rotation. Task anchoring preserves admissibility and bounds within-task accumulation. • Across three backbones, both HMCL variants improve final classification and retrieval scores, overall performance, and BWT over all external baselines, with CA attaining the best Overall score in every backbone block. Representation-space analyses further show that these task-level gains coincide with 81.2–95.5% less radial, angular, cross-modal, and paired-distance drift and with improvements in every reported ImageNet–WordNet hierarchy metric. The placement study identifies post-AdamW correction with task anchoring as the strongest stability–plasticity configuration. Extension beyond the conference version. The ICML paper introduced the geometric preservation conditions and the zero-rotation MR update on a 15-task stream [11]. This article generalizes that foundation with the rotation-permitting CA update, optimizer-consistent analysis, and broader geometric evaluation: (i) Closest-admissible CA solution. We express preservation as a joint constrained least-squares problem over every modal parameter change and one shared skew-symmetric generator. Its Lyapunov-equation solution gives the rotation-permitting HMCLCAHMCL_CA update; fixing = =0 recovers the conference-version HMCLMRHMCL_MR update and places both in one admissible family (Section 6.1). (i) Finite-step preservation and controlled accumulation. Following the backbones’ AdamW training paradigm, we correct the realized parameter displacement after moment estimation and preconditioning. We show that task-anchored CA and MR updates remain in the shared admissible family and that their accumulation is uniformly bounded, and we derive residual-drift and complexity results for practical low-rank protection (Section 6.2 and Appendix F). (i) Hierarchy-aware large-scale evaluation. We add ImageNet-1K to form a unified 16-task continual stream across MERU-L, MERU-B, and HyCoCLIP-B. Its WordNet structure directly verifies that the predictive-retention gains coincide with better preservation of semantic ancestry and abstraction order, rather than only higher aggregate scores (Section 7.4). (iv) Controlled component and trajectory analysis. The component study shows that correcting AdamW’s realized displacement is stronger than correcting its raw gradient and that task anchoring provides a further stability gain; the complete post-AdamW update therefore gives the best stability–plasticity balance (Section 7.6). Representation-drift and hierarchy analyses further verify that the gain accompanies less radial and relational distortion (Sections 7.3–7.5). 2 Related Work 2.1 Hyperbolic Multimodal Learning The volume of hyperbolic space grows exponentially with radius. This property makes hyperbolic space useful for hierarchical or scale-free data and has motivated a broad literature on non-Euclidean representation learning [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], including recent calls to extend non-Euclidean geometry to foundation models [22]. Hyperbolic contrastive learning has been developed for node embeddings and graph-level anomaly detection, together with explicit analyses of dimensional collapse [23, 24, 25]. Universal cone constructions further recover implicit hierarchies without prespecified parent–child relations [26]. In recommendation and retrieval, hyperbolic geometry captures long-tailed interactions and hierarchical neighborhoods [27, 28, 29]. In computer vision, it has also been studied for image representation [30], hierarchy-aware zero-shot recognition [31], and contrastive modeling of scene–object structure [32]; its benefits can depend on the task and radial geometry [33]. The same idea has recently been applied to multimodal learning, where images and language often have clear relations of abstraction and specificity. MERU [7] introduced large-scale hyperbolic vision–language pretraining by mapping CLIP-like representations to a Lorentz hyperboloid. Later studies developed the setting in several directions. Pal et al. [8] modeled compositional entailment within each modality, relating complex scenes to visual primitives and compound expressions to simpler concepts. Ibrahimi et al. [34] analyzed the geometry and uncertainty encoded by hyperbolic vision–language embeddings, whereas Ramasinghe et al. [35] replaced proximity-based cross-modal alignment with an angular objective that accommodates the modality gap. Other work scales hyperbolic learning to multimodal large language models [9], filters underspecified image–text pairs using entailment-cone apertures [10], and applies hierarchical alignment to synthetic-caption detection [36] or open-vocabulary segmentation [37]. Low-rank adaptation has also become a broad foundation-model adaptation family, including multimodal and continual settings [38]; related work performs such adaptation directly on a hyperbolic manifold [39]. HySAC [40] organizes safety concepts through a hyperbolic hierarchy, while hyperbolic multimodal unlearning studies the selective removal of concepts from MERU [41]. These works focus on pretraining, task-specific adaptation, safety, or removal; they do not study preservation of shared Lorentz relations throughout sequential multi-task acquisition. 2.2 Continual Learning Continual learning seeks to acquire tasks in sequence without erasing capabilities learned previously [3]. Existing methods typically rely on replay, regularization, or parameter isolation. Replay approaches retain representative examples and reuse them during later training, as in GEM [42] and DER [43]. Regularization methods such as EWC [44] penalize changes to parameters deemed important for earlier tasks. Parameter-isolation strategies instead reserve different model capacity for different tasks; examples include Progressive Neural Networks [45], PackNet [46], and Piggyback [47]. Continual vision–language learning introduces an additional requirement: retention of cross-modal alignment together with modality-specific features [48]. Unlike CLIP domain adaptation and generalization [49], it must preserve a growing task history. Ni et al. [48] described Spatial Disorder, which combines within-modality rotation and between-modality deviation, as an important failure mode in continual CLIP adaptation. Later methods preserve zero-shot behavior through distillation [50], use mixture-of-experts adapters for task acquisition [51], or apply representation-level contrastive regularization, as in C-FLAT [52]. Personalized LLM research likewise identifies lifelong updating, evolving user memories, and forgetting-resistant adaptation as core challenges [53]. As a concrete parameter-efficient instance, PerFit models personalization through a shared low-rank representation shift and user-specific deviations, then intervenes directly in hidden states [54]. Related geometric work treats long-horizon memory as a nonuniform dynamical space [55] and tailors Lorentz representations to heterogeneous federated clients [56]. Hyperbolic continual learning has also been studied for hierarchical image recognition [57] and multimodal few-shot class-incremental learning [58], but these lines do not enforce preservation of one shared Lorentz isometry across modalities and sequential tasks. Our setting also considers the semantic hierarchy encoded by radial coordinates and entailment cones. This motivates update constraints that account for hyperbolic geometry as well as cross-modal alignment. 2.3 Projected Updates with Adaptive Optimizers Projection-based continual-learning methods often express retention as a constraint on a gradient or a descent direction [42]. With scalar-step stochastic gradient descent, projecting the direction and projecting the resulting parameter displacement are equivalent up to the learning rate. AdamW breaks this equivalence through momentum, element-wise preconditioning, and decoupled decay [59]. Riemannian adaptive optimizers instead study parameters that themselves lie on a manifold and use manifold-wise gradients and updates [60]. Our setting is different. The Lorentz head is Euclidean-parameterized, while the continual-learning constraint requires earlier outputs to remain on one shared Lorentz-isometry orbit. 3 Preliminaries 3.1 Hyperbolic Geometry Hyperbolic space is a non-Euclidean manifold of constant negative curvature. We work with its Lorentz, or hyperboloid, realization because it supports stable numerical optimization [14, 61]. Appendix B reviews the hierarchy motivation, Lorentz model, and Lorentz neural layers. Lorentz model. Let the curvature be −1/K-1/K for K>0K>0. A d-dimensional hyperbolic space can be represented by the upper sheet of a hyperboloid in (d+1)(d+1)-dimensional Minkowski space. With metric tensor =diag(−1,1,…,1)∈ℝ(d+1)×(d+1)G=diag(-1,1,…,1) ^(d+1)×(d+1), the manifold is ℍKd=∈ℝd+1∣⊤=−K,x0>0.H^d_K=\x ^d+1 Gx=-K,\;x_0>0\. (1) A point =[x0;[1:d]]⊤∈ℍKdx=[x_0;x_[1:d]] ^d_K contains one time-like component x0x_0 and d space-like components [1:d]x_[1:d], consistent with the signature of the Lorentzian metric. Lorentzian product and distance. For ,∈ℍKdx,y ^d_K, their Lorentzian inner product is ⟨,⟩ℒ=⊤=−x0y0+∑i=1dxiyi. ,y _L=x Gy=-x_0y_0+ _i=1^dx_iy_i. (2) The associated geodesic distance is dℒK(,)=Karcosh(−⟨,⟩ℒ/K)=Karcosh(−⊤/K).d_L^K(x,y)= Karcosh\! (- ,y _L/K )= Karcosh\! (-x Gy/K ). (3) Lorentz transformation layer. Consider a Lorentz embedding ∈ℝd+1z ^d+1 with positive time-like coordinate z0z_0 and spatial block s∈ℝdz_s ^d. Following [62], a Lorentz transformation layer maps it as f(,)=[K+‖22],f(W;z)= bmatrix K+\|Wz\|_2^2\\ Wz bmatrix, (4) where ∈ℝd×(d+1)W ^d×(d+1). The reconstructed first coordinate ensures that the output remains on ℍKdH^d_K. We partition the parameters as =[0s],0∈ℝd×1,s∈ℝd×d,W=[\,w_0\;\;W_s\,], _0 ^d× 1,\;W_s ^d× d, so that 0w_0 and sW_s act on the time-like and space-like inputs, respectively. 3.2 Hyperbolic Multimodal Learning Let m,m′=(im,im′)i=1N⊂m×m′X^m,m =\(x_i^m,x_i^m )\_i=1^N ^m×X^m be N semantically matched observations from modalities m and m′m . A multimodal model embeds them as matrices m,m′∈ℝN×(d+1)Z^m,Z^m ^N×(d+1), whose iith rows imz_i^m and im′z_i^m lie on ℍKdH^d_K. Contrastive similarity. Pairwise similarity is defined by negative hyperbolic distance: m→m′=−dℒK(m,m′)=−Karcosh(−m(m′)⊤/K).S^m→ m =-d_L^K(Z^m,Z^m )=- Karcosh\! (-Z^mG(Z^m ) /K ). (5) Here arcosharcosh is evaluated element-wise and Sijm→m′=−dℒK(im,jm′)S_ij^m→ m =-d_L^K(z_i^m,z_j^m ). The directional contrastive objective is ℒm→m′=−1N∑i=1Nlogexp(Siim→m′/τ)∑j=1Nexp(Sijm→m′/τ),L_m→ m =- 1N _i=1^N (S_i^m→ m /τ) _j=1^N (S_ij^m→ m /τ), (6) where τ is the temperature. Training uses the symmetric loss ℒcontrast=12(ℒm→m′+ℒm′→m).L_contrast= 12 (L_m→ m +L_m → m ). (7) Entailment cones. Hierarchy is represented by assigning each im∈ℍKdz_i^m _K^d an entailment cone whose aperture is aper(im)=sin−1(2κK∥(im)[1:d]∥2),aper(z_i^m)= ^-1\! ( 2κ K\|(z_i^m)_[1:d]\|_2 ), (8) with κ=0.1κ=0.1 controlling behavior near the origin. Points with a smaller spatial norm represent broader concepts and receive wider cones; larger norms correspond to more specific concepts and narrower cones. The exterior angle from imz_i^m to im′z_i^m is γim,m′ _i^m,m :=⟨im,im′⟩ℒ/K, := _i^m,z_i^m _L/K, (9) ext(im,im′) (z_i^m,z_i^m ) =cos−1((im′)0+(im)0γim,m′∥(im)[1:d]∥2(γim,m′)2−1). = ^-1\! ( (z_i^m )_0+(z_i^m)_0 _i^m,m \|(z_i^m)_[1:d]\|_2 ( _i^m,m )^2-1 ). The entailment objective [15] penalizes a target that falls outside the source cone: ℒentail(im,im′)=max(0,ext(im,im′)−aper(im)).L_entail(z_i^m,z_i^m )= \! (0,ext(z_i^m,z_i^m )-aper(z_i^m) ). (10) This loss imposes the desired cross-modal partial order, for example by locating a general textual description above its visual instance in the hierarchy. 4 Problem Formulation 4.1 Continual Hyperbolic Multimodal Learning We study a sequence of multimodal tasks whose representations share one hyperbolic space. At every stage, the learner absorbs the current task while retaining the within-modality, cross-modal, and hierarchical relations established by earlier tasks. Our setup follows the multimodal contrastive continual-learning protocol of [63]. For each modality m∈1,…,Mm∈\1,…,M\, a frozen pretrained encoder ℰm:m→ℍKdE_m:X^m _K^d is followed at task t by a trainable Lorentz transformation Tm(t):ℍKd→ℍKdT_m^(t):H_K^d _K^d. Applied to data from the previous task, the encoder first produces t−1m,∗=ℰm(Xt−1m)∈ℝN×(d+1),Z_t-1^m,*=E_m(X_t-1^m) ^N×(d+1), (11) where every row lies on ℍKdH_K^d and the superscript ∗* indicates a frozen pretrained representation. The stage-t transformation then maps these representations through the trainable Lorentz head: t−1m,t=Tm(t)(t−1m,∗)=f(m,t,t−1m,∗)∈ℝN×(d+1).Z_t-1^m,t=T_m^(t)(Z_t-1^m,*)=f\! (W^m,t;Z_t-1^m,* ) ^N×(d+1). (12) The same stage processes current-task observations as tm,t=Tm(t)(ℰm(Xtm))∈ℝN×(d+1).Z_t^m,t=T_m^(t)\! (E_m(X_t^m) ) ^N×(d+1). (13) Our notation qm,sZ_q^m,s therefore identifies the data task q, modality m, and processing stage s. For instance, t−1m,∗Z_t-1^m,* contains frozen features from task t−1t-1, whereas t−1m,tZ_t-1^m,t contains their Lorentz embeddings after the stage-t transformation. Appendix A summarizes all symbols. 4.2 Geometric Stability and Plasticity Stability. We describe retention of earlier hyperbolic knowledge through changes in the representation geometry. Because geodesic similarity is a monotone function of the Lorentzian inner product, we use three invariants22 2 Since arcosh(⋅)arcosh(·) in Appendix 21 is strictly monotone, equal similarities are equivalent to equal Lorentzian inner products in Definition 2.. For every pair of distinct modalities m,m′∈1,…,Mm,m ∈\1,…,M\: (P1) Intra-modal preservation: t−1m,t(t−1m,t)⊤=t−1m,t−1(t−1m,t−1)⊤Z_t-1^m,tG(Z_t-1^m,t) =Z_t-1^m,t-1G(Z_t-1^m,t-1) ; (P2) Inter-modal preservation: t−1m,t(t−1m′,t)⊤=t−1m,t−1(t−1m′,t−1)⊤Z_t-1^m,tG(Z_t-1^m ,t) =Z_t-1^m,t-1G(Z_t-1^m ,t-1) ; (P3) Hierarchical preservation: ∥(t−1m,t)[1:d]∥2=∥(t−1m,t−1)[1:d]∥2\|(z_t-1^m,t)_[1:d]\|_2=\|(z_t-1^m,t-1)_[1:d]\|_2. The first condition retains the relations internal to each modality. The second fixes cross-modal correspondences, and the third prevents a point from moving to a different radial level of the entailment hierarchy. Together, they protect semantic similarity, modality alignment, and specificity. Appendix C gives further justification. Figure 2: Geometry of continual hyperbolic multimodal learning. Geodesic proximity encodes semantic similarity, while radial distance from the origin records specificity. Under our preservation conditions, all modalities undergo one common hyperbolic isometry. From the top-down view, first-order preservation of radial hierarchy limits a permitted update to the tangent direction. Plasticity. Stability also leaves enough freedom to learn the current task. At stage t, the embeddings tm,tm=1M\Z_t^m,t\_m=1^M fit the joint multimodal distribution pt(1,2,…,M)p_t^(1,2,…,M). The transformation layers capture new task-specific patterns and associations without using all of their representation capacity. Core difficulty. Conditions (P1)–(P3) describe this balance, but applying them directly during gradient-based training is difficult. Section 5 gives an equivalent geometric view that leads to practical constraints on parameter updates. 5 Geometric Conditions for Admissible Updates We next convert the preservation requirements into a geometric description of the parameter changes that are permitted during continual adaptation. 5.1 Characterizing Preserved Representations The following result identifies the transformation that can relate old-task embeddings before and after a learning stage without changing the three invariants. Theorem 1 (Geometric Characterization of Preservation). Suppose the task-t−1t-1 representations satisfy the full-rank condition in Appendix D.1, and the stage-to-stage correspondence follows the continuous optimizer path from the identity component. Conditions (P1), (P2), and (P3) are preserved from stage t−1t-1 to stage t if and only if one transformation =(1⊤~),~∈SO(d),R= pmatrix1&0 \\ 0& R pmatrix, R (d), acts on every modality, so that t−1m,t=t−1m,t−1⊤,m∈1,…,M.Z_t-1^m,t=Z_t-1^m,t-1R , m∈\1,…,M\. Proof sketch. The intra- and inter-modal conditions preserve every required Lorentzian inner product among the task-t−1t-1 embeddings. Theorem 3 and Corollary 4 in Appendix D.1 then imply a unique Lorentz transformation ∈SO+(1,d)L ^+(1,d) shared by all modalities. The forward sheet fixes the time orientation, while continuity from the identity fixes the determinant sign. Hierarchical preservation additionally fixes every spatial norm and therefore every time-like coordinate. Full rank then gives ⊤0=0L e_0=e_0, leaving only the block-diagonal spatial rotation R shown above. Appendix D contains the complete argument. Role of non-degeneracy. Full rank is the worst-case identifiability condition: it uniquely determines the shared isometry over the entire ambient space. Practical HMCL protects only the retained low-rank subspace and therefore does not require this global uniqueness; deriving the update under the maximally constrained case gives a conservative, robust admissible rule. 5.2 First-Order Preservation of Hierarchy On the Lorentz manifold, the time-like coordinate is determined by the spatial block. We exploit this dependence to obtain a local constraint that prevents a continual update from changing the radial level of an old embedding. Corollary 1 (Updates under (P3)). Let Δ=t−1m,t−t−1m,t−1 =z_t-1^m,t-z_t-1^m,t-1 be the change applied to an old-task embedding t−1m,t−1z_t-1^m,t-1. If its time-like coordinate is invariant to first order, as required by (P3), then its spatial change obeys [1:d]⊤Δ[1:d]=0.z_[1:d] _[1:d]=0. (14) Takeaway. Corollary 1 requires the spatial displacement of an old embedding to be orthogonal to its current radial direction. In other words, a first-order hierarchy-preserving change lies along the tangent direction and does not introduce time-like drift and therefore avoids radial drift. Geometric Interpretation. Figure 2 illustrates two implications of the analysis. Hyperbolic distance represents semantic similarity, while radius represents hierarchical specificity. Theorem 1 shows that both quantities remain unchanged across a stage transition when all modalities share the same hyperbolic isometry. From the top-down view, Corollary 1 further limits the local motion to a direction that keeps the radius fixed. A general continual-learning update is not subject to this condition and may distort the geometry even when its task loss is small. 6 Hyperbolic Multimodal Continual Learning The preceding results describe the admissible family in representation space. We now transfer this family to the parameters of a Lorentz transformation layer and obtain a practical training update. 6.1 Geometrically Admissible HMCL Updates Proposition 1 (Admissible Parameter Changes). Consider the Lorentz layer from Section 3.1, parameterized by m,t=[0m,tsm,t]W^m,t=[\,w^m,t_0\;\;W^m,t_s\,]. Assume that Theorem 1 holds at stage t−1t-1. Define the applied parameter displacement as Δm=m,t−m,t−1 ^m=W^m,t-W^m,t-1. In the infinitesimal regime ‖Δm‖F→0\| ^m\|_F→ 0, the collection of modal displacements is admissible if and only if there is one skew-symmetric matrix such that, simultaneously for every modality m, t−1m,∗(Δm)⊤=t−1m,t−1[:d]⊤,⊤=−.Z^m,*_t-1 ( ^m ) =Z^m,t-1_t-1[1\!:\!d] , =- . (15) The left-hand side of (15) is the first-order change of an old spatial representation. The right-hand side is the tangent motion of the rotation exp() ( ). Crucially, the same appears for every modality. The condition therefore permits a common change of spatial coordinates, but excludes independent modal rotations that would alter cross-modal relations. Equation (15) includes both parameter blocks. Under the subspace regularity condition in Appendix E.2, the same output motion has a representative whose time-input column remains fixed; Appendix E.1 gives the precise condition. For every modality, we use the blockwise form Δ0m _0^m =, =0, (16) t−1m,∗[1:d](Δsm)⊤ _t-1^m,*[1:d]( _s^m) =t−1m,t−1[1:d]⊤. =Z_t-1^m,t-1[1:d] . Corollary 2 (HMCL-CA: Closest-Admissible Update). Given candidate parameter displacements m=[0msm]m=1M\ δ^m=[\, δ_0^m\;\; δ_s^m\,]\_m=1^M, where 0m∈ℝd×1 δ_0^m ^d× 1 is the column acting on the time-like input coordinate and sm∈ℝd×d δ_s^m ^d× d is the block acting on the spatial input coordinates, HMCL-CA jointly selects the admissible displacements and their common rotation by solving minimizeΔm, \ ^m\,\, minimize 12∑m=1M‖Δm−m‖F2 12 _m=1^M\| ^m- δ^m\|_F^2 (17) subject to to ⊤=−,Δ0m=, =- , _0^m=0, t−1m,∗[1:d](Δsm)⊤=t−1m,t−1[1:d]⊤, _t-1^m,*[1:d]( _s^m) =Z_t-1^m,t-1[1:d] , m=1,…,M. m=1,…,M. We denote the solution by CA,mm=1M\ δ^CA,m\_m=1^M and ⋆ . Under the regularity condition in Appendix E.2, the parameter displacements are unique and are obtained from one Lyapunov equation for ⋆ . Thus, HMCL-CA retains the common rotation that best matches all modal candidates, rather than solving a separate rotation for each modality. Corollary 3 (HMCL-MR: Minimal-Rotation Update). Selecting = =0 in (17) gives, for each modality, t−1,sm,∗(sMR,m)⊤=,0MR,m=,Z^m,*_t-1,s ( δ^MR,m_s ) =0, δ^MR,m_0=0, (18) where t−1,sm,∗=t−1m,∗[1:d]Z^m,*_t-1,s=Z^m,*_t-1[1:d]. For an orthogonal projector t−1P_t-1 onto the protected old-feature subspace, the unique closest solution is sMR,m=sm(−t−1). δ^MR,m_s= δ^m_s (I-P_t-1 ). (19) Here, t−1P_t-1 projects onto the protected old-feature subspace. This is the closest candidate displacement that induces no first-order motion on that subspace. HMCL-MR is therefore the = =0 member of the same closest-admissible problem. Appendix E.1 proves Proposition 1; Appendices E.2 and E.3 give the corresponding derivations. In summary, hyperbolic preservation determines one admissible family: HMCL-CA selects its closest shared-rotation update, and HMCL-MR selects the zero-rotation member. 6.2 Stepwise Correction and Task Anchoring We follow the standard training paradigm of the hyperbolic multimodal backbones, which optimizes their Euclidean-parameterized Lorentz heads with AdamW. Because AdamW’s moment estimation and coordinate-wise scaling make its realized parameter displacement different from the raw gradient, HMCL applies post-AdamW projection to that realized displacement. It then uses task anchoring to contract the corrected parameters toward the current task start. During task t, J denotes the total number of optimizer steps and k∈0,…,J−1k∈\0,…,J-1\ indexes them, with 0m,t=m,t−1W^m,t_0=W^m,t-1 and Jm,t=m,tW^m,t_J=W^m,t. A first subscript k denotes the within-task optimizer step; a second subscript after a comma denotes an input block, with 00 for the time-like input column and s for the spatial input block. Thus, for r∈opt,⋆r∈\opt, \, km,t=[k,0m,tk,sm,t],kr,m,t=[k,0r,m,tk,sr,m,t].W_k^m,t=[\,w_k,0^m,t\;\;W_k,s^m,t\,], δ_k^r,m,t=[\, δ_k,0^r,m,t\;\; δ_k,s^r,m,t\,]. In particular, the single subscript in 0m,tW_0^m,t identifies the full parameter matrix at optimizer step 00; it is not a block label. With the AdamW state left implicit, one optimizer step produces ~k+1m,t W^m,t_k+1 =Optkm(km,t,km,t), =Opt_k^m(W^m,t_k,g^m,t_k), kopt,m,t δ^opt,m,t_k =~k+1m,t−km,t. = W^m,t_k+1-W^m,t_k. Here km,tg_k^m,t is the raw gradient, whereas kopt,m,t δ_k^opt,m,t is the realized parameter displacement proposed by AdamW before HMCL correction. It already includes the learning rate, moment estimation, coordinate-wise preconditioning, and decoupled weight decay; it is not a gradient. To state the geometric correction at the level of this proposed step, let km,t=[k,0m,tk,sm,t] ξ_k^m,t=[\, ξ_k,0^m,t\;\; ξ_k,s^m,t\,] denote an arbitrary parameter displacement at within-task step k, with the same time-input/spatial-input partition as km,tW_k^m,t. For any collection km,tm=1M\ ξ_k^m,t\_m=1^M, define its Lorentz defect by ℰℒ,t−1(km,t)=min∑m=1M⊤=−‖t−1m,∗[1:d](k,sm,t)⊤−t−1m,t−1[1:d]⊤‖F2+∑m=1M‖k,0m,t‖22.E_L,t-1(\ ξ_k^m,t\)= _ =- _m=1^M \| aligned &Z_t-1^m,*[1:d]( ξ_k,s^m,t) \\[-2.0pt] &-Z_t-1^m,t-1[1:d] aligned \|_F^2+ _m=1^M\| ξ_k,0^m,t\|_2^2. Proposition 1 gives ℰℒ,t−1=0E_L,t-1=0 exactly for a fixed time-input column and one spatial Lorentz rotation shared across modalities. Let t−1ℒP^L_t-1 be the Frobenius closest-point map onto this zero-defect family, denoted by t−1A_t-1. At optimizer step k, the defect is evaluated at km,t=kopt,m,t ξ_k^m,t= δ_k^opt,m,t, whose two blocks are k,0opt,m,t δ_k,0^opt,m,t and k,sopt,m,t δ_k,s^opt,m,t. We collect the modal displacements as kopt,t=(kopt,m,t)m=1M δ_k^opt,t=( δ_k^opt,m,t)_m=1^M, with analogous notation for k⋆,t δ_k ,t. For 0≤β<10≤β<1, the combined update is k⋆,t δ_k ,t =t−1ℒ(kopt,t), =P^L_t-1( δ_k^opt,t), (20) k+1m,t ^m,t_k+1 =β0m,t+(1−β)(km,t+k⋆,m,t). = ^m,t_0+(1-β) (W^m,t_k+ δ ,m,t_k ). For a modal tuple =(m)m=1MU=(U^m)_m=1^M, write ‖⊕F:=(∑m‖m‖F2)1/2\|U\|_ F:=( _m\|U^m\|_F^2)^1/2. Theorem 2 (Anchored Admissibility and Bounded Accumulation). For the update in (20) on a protected representation system fixed throughout task t, the cumulative displacement Jt=(Jm)m=1MA_J^t=(A_J^m)_m=1^M, where Jm=Jm,t−0m,tA_J^m=W_J^m,t-W_0^m,t, satisfies Jmm=1M∈t−1\A_J^m\_m=1^M _t-1 and ℰℒ,t−1(Jm)=0E_L,t-1(\A_J^m\)=0. Thus, the protected representations undergo one shared first-order Lorentz rotation across all modalities. The projection is non-expansive in the joint Frobenius norm. If ‖kopt,t‖⊕F≤umax\| δ_k^opt,t\|_ F≤ u_ for every k, then task anchoring with 0<β<10<β<1 also gives the uniform bound ‖Jt‖⊕F≤(1−β)umax/β\|A_J^t\|_ F≤(1-β)u_ /β. Takeaway. HMCL separates stability from plasticity at each optimizer step. Projection removes only the part of AdamW’s realized displacement that violates the shared Lorentz-isometry constraint, leaving the admissible component free to learn the current task. Task anchoring then limits cumulative drift without leaving this family. 7 Experiments Our evaluation is organized around four questions concerning continual adaptation of hyperbolic multimodal models: • RQ1: Does HMCL improve the stability–plasticity balance, measured by final performance and backward transfer, across hyperbolic multimodal backbones? • RQ2: Does HMCL reduce radial, angular, cross-modal, and paired-distance drift on previously learned tasks? • RQ3: Does the resulting representation preserve semantic hierarchies in quantitative and qualitative tests? • RQ4: Within the unified HMCL update, what are the separate effects of correction placement and task anchoring? Accordingly, Section 7.2 addresses RQ1, Section 7.3 addresses RQ2, Sections 7.4 and 7.5 address RQ3, and Section 7.6 addresses RQ4. 7.1 Experimental Setup Task Stream and Datasets. The continual stream combines 14 multimodal classification tasks with the COCO [64] and Flickr30K [65] cross-modal retrieval tasks. ImageNet-1K [66] is inserted after Caltech-101, yielding 16 tasks in total. Non-replay methods never revisit earlier-task data, whereas replay baselines access it only through a bounded buffer. Appendices G.1 and G.2 provide the complete dataset statistics, canonical order, and alternative-order protocols. Metrics. For retrieval, the implementation records image-to-text and text-to-image Recall@k for k∈1,5,10k∈\1,5,10\. The main comparison uses symmetric R@5, i.e., the mean of image-to-text and text-to-image R@5, for both retrieval tasks. Classification performance is top-1 accuracy. Overall is the unweighted mean of the 14 final classification accuracies and the two final symmetric retrieval scores. We define signed backward transfer as BWT=Performancefinal−PerformancerightaftertaskBWT=Performance_final-Performance_right\ after\ task; thus, a negative value denotes forgetting. The main table reports this quantity separately for classification, retrieval, and all tasks. Reported entries are means and sample standard deviations over five paired seeds. Baselines. Our comparison covers the principal families of continual-learning strategies under the same hyperbolic objective: ordinary sequential fine-tuning (Vanilla), parameter regularization (EWC), replay-based constrained optimization (GEM), vision–language distillation with flatness-aware optimization (C-FLAT), and dual-sided null-space projection (DNS) [48, 42, 52, 63]. This breadth tests HMCL against methods that preserve parameters, examples, representations, or update subspaces; none of them explicitly preserves a shared Lorentz isometry across modalities. Implementation Details. We follow each backbone’s hyperbolic multimodal training paradigm: the pretrained encoder is frozen, continual adaptation is confined to a bias-free linear Lorentz head, and AdamW [59] optimizes the common objective. Classification and retrieval tasks use 10 and 15 epochs with learning rates 5×10−45× 10^-4 and 5×10−55× 10^-5, respectively; all use batch size 1024, curvature −0.1-0.1, and one accumulated AdamW step per epoch. Main HMCL runs apply post-AdamW correction followed by task anchoring. Every method shares the task order, cached 512-dimensional features, schedule, optimizer-step budget, and five paired seeds. Appendix H.5 gives the complete configurations and baseline memory budgets, while Appendices H.8, H.6, and H.7 report sensitivity and timing details. 7.2 Main Results (RQ1) To answer RQ1, we compare final task performance and signed backward transfer under the matched 16-task protocol across all three hyperbolic multimodal backbones. Table 1: Continual-learning results on the unified 16-task stream (mean ± sample standard deviation over five paired seeds). Classification is top-1 accuracy, retrieval is symmetric R@5, and BWT is final minus immediate performance. Light-blue rows denote the complete post-AdamW, task-pullback HMCL variants; boldface marks the best result in each backbone block. The Δ rows give HMCLCAHMCL_CA’s relative improvement over Vanilla, using the reduction in negative magnitude for BWT. All methods share the task order, cached features, training schedule, and optimizer-step budget. Classification Retrieval Overall Backbone Method Acc ↑ A↑BWT_A R@5 ↑ R5↑BWT_R5 Performance ↑ BWT ↑ MERU-L Vanilla 40.251±0.20240.251± 0.202 −5.461±0.180-5.461± 0.180 29.625±0.11029.625± 0.110 −2.479±0.123-2.479± 0.123 38.922±0.16538.922± 0.165 −5.088±0.145-5.088± 0.145 EWC 40.321±0.18740.321± 0.187 −5.467±0.213-5.467± 0.213 30.215±0.14630.215± 0.146 −2.096±0.158-2.096± 0.158 39.058±0.15339.058± 0.153 −5.046±0.174-5.046± 0.174 GEM 41.613±0.28841.613± 0.288 −3.742±0.284-3.742± 0.284 29.220±0.29929.220± 0.299 −2.306±0.418-2.306± 0.418 40.064±0.27640.064± 0.276 −3.563±0.276-3.563± 0.276 C-FLAT 40.450±0.17140.450± 0.171 −3.945±0.108-3.945± 0.108 29.446±0.17429.446± 0.174 −2.032±0.115-2.032± 0.115 39.075±0.14339.075± 0.143 −3.706±0.086-3.706± 0.086 DNS 40.904±0.14940.904± 0.149 −4.912±0.174-4.912± 0.174 29.716±0.17029.716± 0.170 −2.455±0.100-2.455± 0.100 39.505±0.12639.505± 0.126 −4.605±0.149-4.605± 0.149 HMCLMR 43.891±0.08543.891± 0.085 −1.316±0.086-1.316± 0.086 36.300±0.11436.300± 0.114 −1.921±0.057-1.921± 0.057 42.942±0.07042.942± 0.070 −1.392±0.070-1.392± 0.070 HMCLCA 43.987±0.08743.987± 0.087 −1.305±0.090-1.305± 0.090 36.011±0.13636.011± 0.136 −2.046±0.099-2.046± 0.099 42.990±0.07342.990± 0.073 −1.398±0.078-1.398± 0.078 vs. Vanilla +9.3% +76.1% +21.6% +17.5% +10.5% +72.5% MERU-B Vanilla 43.326±0.24343.326± 0.243 −6.617±0.244-6.617± 0.244 30.535±0.09330.535± 0.093 −4.610±0.120-4.610± 0.120 41.727±0.22241.727± 0.222 −6.366±0.218-6.366± 0.218 EWC 43.472±0.21043.472± 0.210 −6.490±0.217-6.490± 0.217 30.965±0.08630.965± 0.086 −4.291±0.172-4.291± 0.172 41.909±0.19041.909± 0.190 −6.215±0.189-6.215± 0.189 GEM 44.880±0.27244.880± 0.272 −3.803±0.093-3.803± 0.093 32.010±0.37332.010± 0.373 −3.009±0.238-3.009± 0.238 43.271±0.22843.271± 0.228 −3.704±0.056-3.704± 0.056 C-FLAT 42.561±0.23642.561± 0.236 −4.700±0.146-4.700± 0.146 31.459±0.16131.459± 0.161 −4.300±0.237-4.300± 0.237 41.174±0.21141.174± 0.211 −4.650±0.124-4.650± 0.124 DNS 43.769±0.16343.769± 0.163 −6.265±0.225-6.265± 0.225 31.423±0.07331.423± 0.073 −4.138±0.193-4.138± 0.193 42.226±0.15142.226± 0.151 −5.999±0.217-5.999± 0.217 HMCLMR 45.736±0.04245.736± 0.042 −3.448±0.112-3.448± 0.112 36.456±0.09036.456± 0.090 −1.392±0.162-1.392± 0.162 44.576±0.03644.576± 0.036 −3.191±0.089-3.191± 0.089 HMCLCA 45.746±0.07845.746± 0.078 −3.159±0.134-3.159± 0.134 36.522±0.08336.522± 0.083 −1.229±0.116-1.229± 0.116 44.593±0.07144.593± 0.071 −2.918±0.111-2.918± 0.111 vs. Vanilla +5.6% +52.3% +19.6% +73.3% +6.9% +54.2% HyCoCLIP-B Vanilla 40.988±0.16040.988± 0.160 −4.421±0.097-4.421± 0.097 62.034±0.18262.034± 0.182 −2.544±0.172-2.544± 0.172 43.618±0.15243.618± 0.152 −4.186±0.087-4.186± 0.087 EWC 41.121±0.12941.121± 0.129 −4.385±0.122-4.385± 0.122 63.367±0.14863.367± 0.148 −1.423±0.091-1.423± 0.091 43.901±0.11843.901± 0.118 −4.014±0.107-4.014± 0.107 GEM 42.111±0.31842.111± 0.318 −3.236±0.353-3.236± 0.353 62.586±0.12562.586± 0.125 −1.449±0.121-1.449± 0.121 44.670±0.26844.670± 0.268 −3.013±0.315-3.013± 0.315 C-FLAT 40.951±0.11640.951± 0.116 −3.699±0.097-3.699± 0.097 62.010±0.04262.010± 0.042 −1.839±0.078-1.839± 0.078 43.583±0.10443.583± 0.104 −3.466±0.093-3.466± 0.093 DNS 41.356±0.10541.356± 0.105 −4.065±0.076-4.065± 0.076 62.536±0.18962.536± 0.189 −2.068±0.169-2.068± 0.169 44.004±0.10844.004± 0.108 −3.815±0.077-3.815± 0.077 HMCLMR 44.828±0.06444.828± 0.064 −0.635±0.069-0.635± 0.069 72.801±0.08772.801± 0.087 −0.879±0.091-0.879± 0.091 48.325±0.05348.325± 0.053 −0.666±0.069-0.666± 0.069 HMCLCA 45.108±0.03145.108± 0.031 −0.388±0.061-0.388± 0.061 72.772±0.07372.772± 0.073 −0.937±0.054-0.937± 0.054 48.566±0.02448.566± 0.024 −0.456±0.051-0.456± 0.051 vs. Vanilla +10.1% +91.2% +17.3% +63.2% +11.3% +89.1% HMCL gives the strongest stability–plasticity balance across backbones. Table 1 shows that both complete HMCL variants outperform every external baseline on all three backbones in final classification and retrieval performance as well as in the Overall score. Their BWT values also move substantially toward zero, so higher final scores accompany stronger retention rather than sacrificing earlier tasks. HMCLCAHMCL_CA gives the highest Overall score in every backbone block and consistently exceeds task-balanced GEM. DNS improves only modestly over Vanilla, suggesting that Euclidean null-space protection alone does not fully capture the constraints of hyperbolic representations. The agreement across classification and retrieval is informative because the former tests category discrimination, whereas the latter depends on preserving correspondence between modalities. Improvements in both settings are consistent with protecting their shared representation geometry, rather than benefiting only one prediction rule. Appendix H.1 provides the complete per-task results. CA and MR expose complementary members of the admissible family. On the unified 16-task stream, CA generally gives higher classification and Overall scores, while MR is stronger on retrieval for MERU-L and HyCoCLIP-B and gives the best Overall BWT on MERU-L. This division is not universal: in the modality-extended stream below, CA also gives the stronger retrieval result on all three backbones. The evidence therefore supports a task-dependent stability–plasticity trade-off within one admissible family, rather than assigning a fixed empirical role to either variant. Figure 3: Stage-wise relative task-performance change for HyCoCLIP-B on seed 42 in the unified 16-task stream. Columns correspond to the completed training stage t, and rows correspond to evaluated task i in stream order. Each observed cell reports 100(Pi,t−Pi,i)/Pi,i100(P_i,t-P_i,i)/P_i,i, where Pi,iP_i,i is the score immediately after task i is learned; thus, the outlined diagonal is zero, red denotes relative degradation, and blue denotes improvement. The gray upper triangle contains tasks not yet introduced. All panels share one color scale, the outlined rightmost column gives the final-stage changes, and (R) marks retrieval tasks evaluated with symmetric R@5; the remaining tasks use top-1 accuracy. HMCL remains stable throughout the task stream. Figure 3 normalizes each task by its own task-end score. On the displayed paired seed, Vanilla exhibits pronounced degradation on earlier tasks, whereas both HMCL variants keep the trajectories closer to their task-end reference. The contrast is visible across intermediate stages as well as at the final checkpoint, suggesting that the retention gains reflect sustained protection during adaptation rather than recovery only at the end of the stream. These old-task trajectories provide a stage-wise view of the five-seed BWT gains in Table 1; the displayed seed illustrates the temporal pattern rather than replacing the multi-seed comparison. The gains persist under alternative task orders. Appendix G.2 evaluates two shuffled HyCoCLIP-B streams with the main-table configurations fixed and no order-specific retuning. Across the paired seeds, both HMCL variants retain higher final classification, retrieval, and Overall scores and less negative BWT than Vanilla under either permutation. Their relative ranking changes with the order: MR leads in Overall on one permutation, whereas CA leads on the other. These results support robustness beyond the canonical sequence while reinforcing that the preferred admissible representative depends on the task stream. Table 2: Robustness to new modalities in a separate protocol with one audio and one thermal retrieval task (mean ± standard deviation over three paired seeds). Cls. is top-1 accuracy, Ret. is symmetric R@5, Overall averages all final task scores, and BWT is final minus immediate performance. Both HMCL variants use after-AdamW correction and step pullback (β=0.015β=0.015). Boldface marks the best result per backbone and metric. Backbone Metric Vanilla HMCLMR HMCLCA MERU-L Cls. 42.514±0.09242.514± 0.092 43.894±0.13043.894± 0.130 43.188±0.04643.188± 0.046 Ret. 16.221±0.04816.221± 0.048 16.278±0.10516.278± 0.105 18.224±0.07718.224± 0.077 Overall 36.327±0.06436.327± 0.064 37.396±0.09937.396± 0.099 37.315±0.02637.315± 0.026 BWT −4.876±0.054-4.876± 0.054 −1.650±0.038-1.650± 0.038 −1.110±0.036-1.110± 0.036 MERU-B Cls. 44.156±0.19044.156± 0.190 46.646±0.21346.646± 0.213 45.815±0.05045.815± 0.050 Ret. 14.868±0.13114.868± 0.131 17.000±0.07617.000± 0.076 18.482±0.05318.482± 0.053 Overall 37.265±0.11637.265± 0.116 39.670±0.14539.670± 0.145 39.384±0.02839.384± 0.028 BWT −7.293±0.191-7.293± 0.191 −4.354±0.120-4.354± 0.120 −2.871±0.067-2.871± 0.067 HyCoCLIP-B Cls. 42.869±0.15442.869± 0.154 44.563±0.18044.563± 0.180 45.348±0.07745.348± 0.077 Ret. 32.970±0.12332.970± 0.123 33.506±0.12333.506± 0.123 35.407±0.08135.407± 0.081 Overall 40.540±0.14340.540± 0.143 41.961±0.15541.961± 0.155 43.009±0.07243.009± 0.072 BWT −3.859±0.062-3.859± 0.062 −2.603±0.056-2.603± 0.056 −1.658±0.017-1.658± 0.017 HMCL remains effective when the stream introduces new modalities. Table 2 reports a separate protocol that adds one audio and one thermal retrieval task to the original stream. Both complete HMCL variants improve the Overall score and BWT over matched Vanilla training on every backbone. CA consistently improves retrieval R@5 while substantially reducing forgetting across the backbones. MR gives a slightly higher Overall score on the two MERU backbones, whereas CA is strongest on HyCoCLIP-B. The result shows that the shared geometric constraint remains useful when previously unseen modality types enter the stream. This extension is a distinct stress test: adaptation now accommodates additional cross-modal relations while retaining those learned earlier. The consistent retrieval gains suggest that geometric protection remains useful beyond the original image–text setting, although the changing MR–CA ranking cautions against treating either realization as uniformly preferable. Separate evaluation of earlier image–text tasks distinguishes retention of existing correspondences from new-modality gains. The retrieval gains persist at fine granularity. Figure 4 shows final COCO and Flickr30K retrieval across backbones, directions, and recall cutoffs; Appendix H.4 gives the complete numerical results. Under the same modality-extension protocol, HMCLCA exceeds Vanilla and gives the highest score in all 36 backbone–dataset–direction–cutoff comparisons. Its advantage therefore spans both retrieval directions and every recall cutoff, not only symmetric R@5. Figure 4: Fine-grained final retrieval performance on COCO and Flickr30K in the separate stream augmented with audio and thermal tasks. Each panel reports the mean over three paired seeds for one backbone; T2I and I2T denote text-to-image and image-to-text Recall@1,5,10\1,5,10\, respectively. Gray, blue, and green bars denote Vanilla, HMCLMR, and HMCLCA. Appendix H.4 reports exact means and standard deviations. Higher values indicate better retrieval. 7.3 Quantitative Representation Drift (RQ2) Figure 5: Dataset-wise drift for MERU-L (seed 1024) on the unified 16-task stream, measured from each task-end checkpoint to the final checkpoint using 50 fixed test anchors per old task. The 15 plotted rows exclude SST-2 because it is the final task and hence has no post-task drift interval. Gray circles, blue diamonds, and green squares denote Vanilla, HMCLMRHMCL_MR, and HMCLCAHMCL_CA, respectively; shaded rows identify retrieval tasks. Lower values indicate better preservation. In-panel percentages report each HMCL variant’s relative reduction in mean drift over the 15 plotted tasks compared with Vanilla. To answer RQ2, we go beyond task-level BWT, which records predictive change but does not identify which parts of the hyperbolic representation have moved. We compare each old task at its task-end checkpoint and at the final checkpoint using four complementary drift measures: • Radial drift is the mean absolute change of the time-like coordinate, averaged over image and text anchors. It tests hierarchical preservation in Condition (P3). • Angular drift is the mean absolute change of the normalized spatial Gram matrix, averaged over the image and text blocks. It removes magnitude effects and tracks the intra-modal relations in Condition (P1). • Cross-modal drift is the mean absolute change of the normalized spatial image–text Gram block and therefore measures the inter-modal relations in Condition (P2). • Paired-distance drift is the mean change in Lorentz geodesic distance between matched image–text anchors. It tests cross-modal alignment in Condition (P2) while remaining sensitive to hierarchy-related radial changes in Condition (P3). Figure 5 shows lower drift for both HMCL variants across all 15 old tasks and all four measures. Both variants substantially suppress radial, angular, cross-modal, and paired-distance drift, with CA giving slightly lower mean drift throughout. For both variants, every paired comparison favors HMCL and is significant under an exact two-sided Wilcoxon signed-rank test (p=6.1×10−5p=6.1× 10^-5; Bonferroni-adjusted p=2.4×10−4p=2.4× 10^-4 over four measures). The agreement between MR and CA shows that the stability gain is not specific to the zero-rotation realization: both retain radial hierarchy, within-modality relations, and cross-modal geometry throughout the stream. The joint behavior of these measures is more informative than any one measure alone. Small angular changes would not exclude radial contraction, and stable within-modality relations would not establish that image–text correspondence is retained. Lower paired-distance drift complements the normalized Gram measurements by tracking matched examples in the original Lorentz geometry. Taken together, the observations support preservation of both hierarchical and relational structure. They are consistent with the proposed mechanism, while remaining an empirical representation-space diagnostic rather than a proof that drift reduction alone causes the task-level gains. 7.4 ImageNet–WordNet Hierarchy (RQ3) WordNet Evaluation Protocol. To answer RQ3 quantitatively, ImageNet measures new-task capacity and WordNet [67] measures whether the learned semantic hierarchy survives later tasks. Following the HyCoCLIP protocol [8] and standard hierarchical-classification evaluation [68], we report top-1 accuracy, tree-induced error (TIE), lowest-common-ancestor distance (LCA), ancestor Jaccard, and hierarchical precision and recall. Lower TIE and LCA are better; higher values are better for all other metrics. These metrics separate prediction correctness from semantic error severity: top-1 counts every mistake equally, whereas TIE and LCA distinguish nearby errors from cross-branch confusions; ancestor Jaccard measures shared ancestry, and hierarchical precision and recall summarize the correctness and coverage of predicted ancestor paths. Their joint use prevents an apparent hierarchy gain from being attributed to a change in accuracy or only one aspect of the tree path. We therefore compute the hierarchy scores from the same final predictions as top-1 and interpret improvement only when predictive retention and semantic proximity move consistently. Appendix H.1 gives the split, hierarchy mapping, retained ranks, and configuration details. Figure 6: Radial geometry of the complete 20-class ImageNet primate WordNet subtree for HyCoCLIP-B (seed 42). (a) Geometry immediately after learning ImageNet; (b–d) geometry at the final checkpoint of the 16-task stream. Final representations are globally aligned to the task-end reference by a proper spatial rotation before display, so the remaining change reflects radial and relational distortion rather than an arbitrary global orientation. Node color denotes WordNet depth, and edges show the fixed subtree relations. “rad.” is the mean absolute radial change normalized by the task-end mean radius; “pair.” is the mean absolute change in pairwise Lorentz distance normalized by the task-end mean distance. Both quantities are computed in the original representation space rather than from the two-dimensional drawing. Table 3: Final ImageNet accuracy and WordNet hierarchy metrics on the 16-task HyCoCLIP-B stream (five-seed means). HMCL uses the configurations in Table 1; shaded rows identify HMCL. Method Top-1 ↑ TIE ↓ LCA ↓ Jacc. ↑ H-Prec. ↑ H-Rec. ↑ Vanilla 39.404 3.598 2.217 0.7869 0.8539 0.8560 EWC 40.045 3.549 2.199 0.7901 0.8560 0.8583 C-FLAT 38.934 3.634 2.237 0.7849 0.8521 0.8552 DNS 39.494 3.594 2.222 0.7875 0.8538 0.8568 HMCLMR 45.70045.700 3.1373.137 2.0422.042 0.81620.8162 0.87430.8743 0.87570.8757 HMCLCA 45.59045.590 3.1563.156 2.0512.051 0.81520.8152 0.87350.8735 0.87510.8751 HMCL preserves the ImageNet hierarchy. Table 3 reports the final HyCoCLIP-B checkpoints from Table 1. Both HMCL variants outperform all external baselines in top-1 accuracy and every hierarchy metric. Lower TIE and LCA together with higher ancestor Jaccard, hierarchical precision, and hierarchical recall show that their remaining errors preserve more of the correct WordNet ancestry. These concurrent gains show that hierarchy retention is not achieved by sacrificing classification performance. MR performs best on the ImageNet–WordNet measures, whereas CA gives the highest Overall score on the full stream, indicating a modest trade-off between hierarchy retention and aggregate performance. Thus, selecting a model solely by Overall can obscure differences in how well semantic ancestry survives continual adaptation. The radial view makes hierarchy retention visible. Figure 6 shows a representative checkpoint trajectory alongside the five-seed metrics. Vanilla contracts the depth rings and disrupts branch layout, whereas both HMCL variants stay close to the task-end geometry. Their smaller radial and pairwise-distance errors indicate preservation of both depth organization and relations among concepts. Because the views are aligned by a global spatial rotation, this contrast is not simply a difference in drawing orientation. The visualization complements the hierarchy metrics in Table 3: the former reveals changes in representation geometry, whereas the latter evaluates semantic ancestry in the final predictions. Appendix H.2 reports modality-specific norm distributions over all 15 old tasks. 7.5 Representation-Space Case Study (RQ3) HMCL (ours) Vanilla hat volleyball fashion interest style fashion [ROOT] [ROOT] HMCL (ours) Vanilla relaxation relaxation vacation aloof peace sunken [ROOT] [ROOT] HMCL (ours) Vanilla defeat workout workout pilates peace race [ROOT] [ROOT] Figure 7: Hierarchical concept retrieval along image-to-origin geodesics. For each Flickr30K example, we move from the image representation toward the manifold origin [ROOT] and display the nearest text concept along the resulting specific-to-generic path. HMCL gives more coherent sequences in these examples, such as hat → fashion → style, while Vanilla more often shifts to less related concepts after sequential training. Construction of the Concept Set. To complement the quantitative answer to RQ3, we study hierarchy retention through image traversals on Flickr30K. From the original captions, nouns and adjectives are collected to form a text vocabulary with different levels of semantic abstraction. Appendix G.3 details how the concept set is prepared. Traversal Protocol. For each image, we trace its embedding toward the manifold origin [ROOT], which represents the generic concept. We sample 20 locations in the origin’s tangent space, map them back to the Lorentz hyperboloid, and retrieve the nearest text representation at each location. Qualitative Finding. Figure 7 shows a recurring contrast in the displayed examples. HMCL maintains local semantic continuity along the geodesic—for example, hat → fashion → style and relaxation → vacation → peace—whereas Vanilla introduces abrupt off-path transitions such as volleyball → interest and aloof → sunken. Because the HMCL paths become progressively more generic before reaching [ROOT], the result suggests preservation of intermediate abstraction ordering rather than only convergence to the shared endpoint. This qualitative evidence complements the WordNet and radial-geometry results in Section 7.4, thereby providing a representation-level answer to RQ3. Appendix G.3 provides additional Flickr30K and COCO traversals. 7.6 Ablation Study (RQ4) To answer RQ4, we isolate where the geometric correction is applied and whether task anchoring is added, while keeping the HyCoCLIP-B stream and evaluation metrics fixed. Table 4: Focused update-placement ablation on the 16-task HyCoCLIP-B stream (five-seed mean ± sample standard deviation). Each placement reports both MR and CA. Pre and Post apply the correction to the raw gradient and realized AdamW displacement, respectively; Post+Pullback adds task anchoring. Update Corr. Overall ↑ BWT ↑ ImageNet ↑ Vanilla – 43.618±0.15243.618± 0.152 −4.186±0.087-4.186± 0.087 39.404±0.05839.404± 0.058 Pre MR 46.397±0.12446.397± 0.124 −2.907±0.070-2.907± 0.070 41.303±0.09541.303± 0.095 CA 47.172±0.12947.172± 0.129 −2.932±0.058-2.932± 0.058 40.871±0.00740.871± 0.007 Post MR 47.780±0.10647.780± 0.106 −1.548±0.073-1.548± 0.073 43.760±0.03243.760± 0.032 CA 47.785±0.10247.785± 0.102 −1.532±0.075-1.532± 0.075 43.759±0.04343.759± 0.043 Post + Pullback MR 48.053±0.07248.053± 0.072 −0.239±0.057-0.239± 0.057 46.257±0.03346.257± 0.033 CA 48.566±0.02448.566± 0.024 −0.456±0.051-0.456± 0.051 45.585±0.05245.585± 0.052 Table 4 shows that all corrected configurations outperform Vanilla in the Overall score, BWT, and ImageNet accuracy. The consistent advantage of Post over Pre for both MR and CA shows that the constraint is most effective when applied to AdamW’s realized displacement rather than to the raw gradient. Pullback then complements the geometric correction by limiting cumulative within-task deviation, improving retention without removing the flexibility needed for the current task. Post+Pullback CA attains the highest Overall score, whereas Post+Pullback MR gives the strongest BWT and ImageNet retention. The placement comparison separates optimizer alignment from the choice of admissible representative. Both MR and CA benefit from correcting the realized displacement, consistent with the fact that AdamW’s momentum and adaptive scaling can change the direction proposed by the raw gradient. Task anchoring addresses a different source of deviation: locally corrected updates can still accumulate change over a task. The further retention gains with Pullback therefore support its complementary role within the integrated update, rather than an alternative to geometric correction. The MR ablation uses stronger task anchoring than its main-table configuration. As detailed in Appendix H.8, this favors retention, whereas the main-table setting favors the Overall score; the configurations therefore represent different stability–plasticity trade-offs within the same update. 8 Conclusion We presented a continual-learning formulation for hyperbolic multimodal representations and characterized the updates that retain within-modal relations, cross-modal alignment, and semantic hierarchy. HMCL translates these representation-space conditions into a shared admissible family with two complementary realizations: CA permits a common rotation, whereas MR selects the identity representative. Across backbones, both variants improve final performance and retention over the external baselines; CA achieves the highest Overall scores, while MR gives the best ImageNet–WordNet hierarchy scores. The stage-wise, drift, modality-extension, and hierarchy analyses show that these gains coincide with more stable radial and relational structure rather than reflecting one aggregate metric alone. The component study further shows that geometric correction and task anchoring address complementary sources of change. Together, the results support shared geometric admissibility as a practical stability–plasticity principle for continual adaptation in non-Euclidean representation spaces. References [1] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021, p. 8748–8763. [2] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in CVPR, 2020, p. 9729–9738. [3] D. Yu, X. Zhang, Y. Chen, A. Liu, Y. Zhang, P. S. Yu, and I. King, “Recent advances of multimodal continual learning: A comprehensive survey,” arXiv preprint arXiv:2410.05352, 2024. [4] J. Han, R. Zhang, W. Shao, P. Gao, P. Xu, H. Xiao, K. Zhang, C. Liu, S. Wen, Z. Guo et al., “ImageBind-LLM: Multi-modality instruction tuning,” arXiv preprint arXiv:2309.03905, 2023. [5] S. Chen, H. Li, Q. Wang, Z. Zhao, M. Sun, X. Zhu, and J. Liu, “VAST: A vision-audio-subtitle-text omni-modality foundation model and dataset,” NeurIPS, vol. 36, p. 72 842–72 866, 2023. [6] X. Liu, X. Xia, S.-K. Ng, and T.-S. Chua, “Principled multimodal representation learning,” arXiv preprint arXiv:2507.17343, 2025. [7] K. Desai, M. Nickel, T. Rajpurohit, J. Johnson, and S. R. Vedantam, “Hyperbolic image-text representations,” in International Conference on Machine Learning. PMLR, 2023, p. 7694–7731. [8] A. Pal, M. van Spengler, G. M. D. di Melendugno, A. Flaborea, F. Galasso, and P. Mettes, “Compositional entailment learning for hyperbolic vision-language models,” in The Thirteenth International Conference on Learning Representations, 2025. [9] P. Mandica, L. Franco, K. Kallidromitis, S. Petryk, and F. Galasso, “Hyperbolic learning with multimodal large language models,” in European Conference on Computer Vision. Springer, 2024, p. 382–398. [10] W. Kim, S. Chun, T. Kim, D. Han, and S. Yun, “Hype: Hyperbolic entailment filtering for underspecified images and texts,” in European Conference on Computer Vision. Springer, 2024, p. 247–265. [11] J. Liu, M. Shen, X. Liu, R. Ying, M. Yang, T.-S. Chua, and I. King, “Hyperbolic multimodal continual learning,” in International Conference on Machine Learning, 2026. [12] D. Krioukov, F. Papadopoulos, M. Kitsak, A. Vahdat, and M. Boguná, “Hyperbolic geometry of complex networks,” Physical Review E—Statistical, Nonlinear, and Soft Matter Physics, vol. 82, no. 3, p. 036106, 2010. [13] M. Nickel and D. Kiela, “Poincaré embeddings for learning hierarchical representations,” Advances in neural information processing systems, vol. 30, 2017. [14] —, “Learning continuous hierarchies in the Lorentz model of hyperbolic geometry,” in International Conference on Machine Learning. PMLR, 2018, p. 3779–3788. [15] O. Ganea, G. Bécigneul, and T. Hofmann, “Hyperbolic neural networks,” Advances in neural information processing systems, vol. 31, 2018. [16] R. Shimizu, Y. Mukuta, and T. Harada, “Hyperbolic neural networks++,” arXiv preprint arXiv:2006.08210, 2020. [17] I. Chami, Z. Ying, C. Ré, and J. Leskovec, “Hyperbolic graph convolutional neural networks,” Advances in neural information processing systems, vol. 32, 2019. [18] M. Yang, M. Zhou, T. Zhang, J. Liu, Z. Li, L. Pan, H. Xiong, and I. King, “Hyperbolic graph neural networks: A review of methods and applications,” arXiv preprint arXiv:2202.13852, 2022. [19] M. Yang, H. Verma, D. C. Zhang, J. Liu, I. King, and R. Ying, “Hypformer: Exploring efficient transformer fully in hyperbolic space,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, p. 3770–3781. [20] P. Mettes, M. Ghadimi Atigh, M. Keller-Ressel, J. Gu, and S. Yeung, “Hyperbolic deep learning in computer vision: A survey,” International Journal of Computer Vision, vol. 132, no. 9, p. 3484–3508, 2024. [21] J. Liu, M. Yang, and I. King, “Hyperbolic learning for structured data, knowledge, and memory: A tutorial,” in Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2026, p. 13 351–13 355. [22] N. He, J. Liu, B. Zhang, N. Bui, A. Maatouk, M. Yang, I. King, M. Weber, and R. Ying, “Position: Beyond Euclidean–foundation models should embrace Non-Euclidean geometries,” in Proceedings of the Fourth Learning on Graphs Conference, ser. Proceedings of Machine Learning Research, vol. 338, 2025. [23] J. Liu, M. Yang, M. Zhou, S. Feng, and P. Fournier-Viger, “Enhancing hyperbolic graph embeddings via contrastive learning,” NeurIPS Workshop on Self-Supervised Learning: Theory and Practice, 2022. [24] Y. Zhang, H. Zhu, M. Yang, J. Liu, R. Ying, I. King, and P. Koniusz, “Understanding and mitigating hyperbolic dimensional collapse in graph contrastive learning,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025, p. 1984–1995. [25] Y. Fu, J. Li, J. Liu, Q. Xing, Q. Wang, and I. King, “HC-GLAD: Dual hyperbolic contrastive learning for unsupervised graph-level anomaly detection,” Neural Networks, p. 109009, 2026. [26] M. Yang, J. Liu, I. King, and R. Ying, “UHCone: Universal hyperbolic cone for implicit hierarchical learning,” in ICML Workshop on Geometry-grounded Representation Learning and Generative Modeling, 2024. [27] M. Yang, M. Zhou, J. Liu, D. Lian, and I. King, “HRCF: Enhancing collaborative filtering via hyperbolic geometric regularization,” in Proceedings of the ACM Web Conference 2022, 2022, p. 2462–2471. [28] M. Yang, Z. Li, M. Zhou, J. Liu, and I. King, “HICF: Hyperbolic informative collaborative filtering,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, p. 2212–2221. [29] Z. Qiu, J. Liu, Y. Chen, and I. King, “HiHPQ: Hierarchical hyperbolic product quantization for unsupervised image retrieval,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, p. 4614–4622. [30] V. Khrulkov, L. Mirvakhabova, E. Ustinova, I. Oseledets, and V. Lempitsky, “Hyperbolic image embeddings,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, p. 6417–6427. [31] S. Liu, J. Chen, L. Pan, C.-W. Ngo, T.-S. Chua, and Y.-G. Jiang, “Hyperbolic visual embedding learning for zero-shot recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, p. 9270–9278. [32] S. Ge, S. Mishra, S. Kornblith, C.-L. Li, and D. Jacobs, “Hyperbolic contrastive learning for visual representations beyond objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, p. 6840–6849. [33] G. Moreira, M. Marques, J. P. Costeira, and A. Hauptmann, “Hyperbolic vs Euclidean embeddings in few-shot learning: Two sides of the same coin,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, p. 2071–2079. [34] S. Ibrahimi, M. G. Atigh, N. Van Noord, P. Mettes, and M. Worring, “Intriguing properties of hyperbolic embeddings in vision-language models,” Transactions on Machine Learning Research, 2024. [35] S. Ramasinghe, V. Shevchenko, G. Avraham, and A. Thalaiyasingam, “Accept the modality gap: An exploration in the hyperbolic space,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, p. 27 263–27 272. [36] F. Kong, Y. Chen, J. Cai, and D. Modolo, “Hyperbolic learning with synthetic captions for open-world detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, p. 16 762–16 771. [37] Z. Peng, Z. Xu, Z. Zeng, C. Wen, Y. Huang, M. Yang, F. Tang, and W. Shen, “Understanding fine-tuning CLIP for open-vocabulary semantic segmentation in hyperbolic space,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, p. 4562–4572. [38] M. Yang, J. Chen, J. Tao, Y. Zhang, J. Liu, J. Zhang, Q. Ma, H. Verma, R. Zhang, M. Zhou, I. King, and R. Ying, “Low-rank adaptation for foundation models: A comprehensive review,” arXiv preprint arXiv:2501.00365, 2025. [39] M. Yang, R. B, A. Feng, B. Xiong, J. Liu, I. King, and R. Ying, “Hyperbolic fine-tuning for large language models,” in Advances in Neural Information Processing Systems, vol. 38, 2025. [40] T. Poppi, T. Kasarla, P. Mettes, L. Baraldi, and R. Cucchiara, “Hyperbolic safety-aware vision-language models,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, p. 4222–4232. [41] À. P. Vidal, K. Nasrollahi, T. B. Moeslund, and S. Escalera, “Machine unlearning in hyperbolic vs. Euclidean multimodal contrastive learning: Adapting alignment calibration to MERU,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, p. 1644–1653. [42] D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,” Advances in neural information processing systems, vol. 30, 2017. [43] P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara, “Dark experience for general continual learning: a strong, simple baseline,” Advances in neural information processing systems, vol. 33, p. 15 920–15 930, 2020. [44] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences, vol. 114, no. 13, p. 3521–3526, 2017. [45] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” arXiv preprint arXiv:1606.04671, 2016. [46] A. Mallya and S. Lazebnik, “PackNet: Adding multiple tasks to a single network by iterative pruning,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, p. 7765–7773. [47] A. Mallya, D. Davis, and S. Lazebnik, “Piggyback: Adapting a single network to multiple tasks by learning to mask weights,” in Proceedings of the European conference on computer vision (ECCV), 2018, p. 67–82. [48] Z. Ni, L. Wei, S. Tang, Y. Zhuang, and Q. Tian, “Continual vision-language representation learning with off-diagonal information,” in International Conference on Machine Learning. PMLR, 2023, p. 26 129–26 149. [49] J. Li, Y. Li, Y. Fu, J. Liu, Y. Liu, M. Yang, and I. King, “CLIP-powered domain generalization and domain adaptation: A comprehensive survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 5, p. 5405–5424, 2026. [50] Z. Zheng, M. Ma, K. Wang, Z. Qin, X. Yue, and Y. You, “Preventing zero-shot transfer degradation in continual learning of vision-language models,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, p. 19 125–19 136. [51] J. Yu, Y. Zhuge, L. Zhang, P. Hu, D. Wang, H. Lu, and Y. He, “Boosting continual learning of vision-language models via mixture-of-experts adapters,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, p. 23 219–23 230. [52] A. Bian, W. Li, H. Yuan, M. Wang, Z. Zhao, A. Lu, P. Ji, T. Feng et al., “Make continual learning stronger via C-FLAT,” Advances in Neural Information Processing Systems, vol. 37, p. 7608–7630, 2024. [53] J. Liu, Z. Qiu, Z. Li, Q. Dai, W. Yu, J. Zhu, M. Hu, M. Yang, T.-S. Chua, and I. King, “A survey of personalized large language models: Progress and future directions,” arXiv preprint arXiv:2502.11528, 2025. [54] J. Liu, W. Yu, Q. Dai, Z. Li, J. Zhu, M. Yang, T.-S. Chua, and I. King, “PerFit: Exploring personalization shifts in representation space of LLMs,” in International Conference on Learning Representations, 2026. [55] J. Liu, W. Yu, Z. Qiu, M. Yang, and I. King, “Memory has geometry: Non-uniform geometric memory for long-horizon personalized AI,” arXiv preprint arXiv:2609.17969, 2026. [56] J. Liu, R. S. B. B, X. Fu, M. Yang, W. Zhang, R. Ying, and I. King, “FlatLand: Personalized graph federated learning via tailored Lorentz space,” in International Conference on Machine Learning, 2026. [57] M. Ayoughi, M. G. Atigh, M. M. Derakhshani, C. G. Snoek, P. Mettes, and P. Groth, “Continual hyperbolic learning of instances and classes,” arXiv preprint arXiv:2506.10710, 2025. [58] T. Doan, S. Behpour, X. Li, W. He, L. Gou, and L. Ren, “A streamlined approach to multimodal few-shot class incremental learning for fine-grained datasets,” arXiv preprint arXiv:2403.06295, 2024. [59] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017. [60] G. Bécigneul and O. Ganea, “Riemannian adaptive optimization methods,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. [61] G. Mishne, Z. Wan, Y. Wang, and S. Yang, “The numerical stability of hyperbolic representation learning,” in International Conference on Machine Learning. PMLR, 2023, p. 24 925–24 949. [62] W. Chen, X. Han, Y. Lin, H. Zhao, Z. Liu, P. Li, M. Sun, and J. Zhou, “Fully hyperbolic neural networks,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, p. 5672–5686. [63] X. Liu, X. Xia, S.-K. Ng, and T.-S. Chua, “Continual multimodal contrastive learning,” arXiv preprint arXiv:2503.14963, 2025. [64] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in European Conference on Computer Vision. Springer, 2014, p. 740–755. [65] P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,” Transactions of the association for computational linguistics, vol. 2, p. 67–78, 2014. [66] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition, 2009, p. 248–255. [67] G. A. Miller, “WordNet: A lexical database for English,” Communications of the ACM, vol. 38, no. 11, p. 39–41, 1995. [68] A. Kosmopoulos, I. Partalas, E. Gaussier, G. Paliouras, and I. Androutsopoulos, “Evaluation measures for hierarchical classification: A unified view and novel approaches,” Data Mining and Knowledge Discovery, vol. 29, no. 3, p. 820–865, 2015. [69] R. Sarkar, “Low distortion Delaunay embedding of trees in hyperbolic plane,” in International symposium on graph drawing. Springer, 2011, p. 355–366. [70] T.-Y. Lam, Introduction to quadratic forms over fields. American Mathematical Soc., 2005, vol. 67. [71] V. Moretti, “The interplay of the polar decomposition theorem and the Lorentz group,” arXiv preprint math-ph/0211047, 2002. [72] A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, Tech. Rep., 2009. [73] L. Fei-Fei, R. Fergus, and P. Perona, “Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 2004, p. 178–178. [74] L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101 – mining discriminative components with random forests,” in European Conference on Computer Vision. Springer, 2014, p. 446–461. [75] M.-E. Nilsback and A. Zisserman, “Automated flower classification over a large number of classes,” in 2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing, 2008, p. 722–729. [76] P. Helber, B. Bischke, A. Dengel, and D. Borth, “EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 7, p. 2217–2226, 2019. [77] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, p. 3606–3613. [78] S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi, “Fine-grained visual classification of aircraft,” arXiv preprint arXiv:1306.5151, 2013. [79] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, p. 2278–2324, 1998. [80] B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, and M. Welling, “Rotation equivariant CNNs for digital pathology,” arXiv preprint arXiv:1806.03962, 2018. [81] J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. Girshick, “CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, p. 2901–2910. [82] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2013, p. 1631–1642. Appendix Supplementary derivations, implementation details, and extended results Contents Appendix A Notation Table Table 5 collects the symbols and conventions used in the main paper and this supplement. Table 5: Summary of the notation used in this work. Symbol Description Hyperbolic geometry and the Lorentz model d Intrinsic hyperbolic dimension; the Lorentz ambient space has dimension d+1d+1. K Squared hyperboloid radius (K>0K>0); the sectional curvature is −1/K-1/K. ℍKdH^d_K d-dimensional hyperbolic space with curvature −1/K-1/K. G Minkowski metric tensor, =diag(−1,1,…,1)∈ℝ(d+1)×(d+1)G=diag(-1,1,…,1) ^(d+1)×(d+1). ⟨⋅,⋅⟩ℒ ·,· _L Lorentzian inner product, ⟨,⟩ℒ=⊤ ,y _L=x Gy. dℒK(,)d_L^K(x,y) Geodesic distance, Karcosh(−⟨,⟩ℒ/K) Karcosh(- ,y _L/K). o Origin of the Lorentz model, (K,0,…,0)⊤∈ℍKd( K,0,…,0) ^d_K. x0,[1:d]x_0,x_[1:d] Time-like coordinate and d space-like coordinates of x. SO+(1,d),SO(d)SO^+(1,d),SO(d) Proper orthochronous Lorentz group and spatial rotation group. Multimodal learning M Number of modalities. m,m′m,m Modality indices, m,m′∈1,2,…,Mm,m ∈\1,2,…,M\. N Number of paired samples in the current context. mX^m Data space of modality m. m,m′X^m,m Paired dataset (im,im′)i=1N\(x_i^m,x_i^m )\_i=1^N. im,imx_i^m,z_i^m The i-th input and its hyperbolic embedding for modality m. Continual learning framework t Task or stage index. J Total number of optimizer steps within a task. k Within-task optimizer-step index, k∈0,…,J−1k∈\0,…,J-1\. Tm(t)T_m^(t) Lorentz transformation for modality m at task t. m,tW^m,t Final Lorentz-head parameters at task t. km,t=[k,0m,tk,sm,t]W_k^m,t=[\,w_k,0^m,t\;\;W_k,s^m,t\,] Parameters after step k, partitioned into the time-like input column and spatial input block; 0m,t=m,t−1W_0^m,t=W^m,t-1 and Jm,t=m,tW_J^m,t=W^m,t. ~k+1m,t W^m,t_k+1 Optimizer-proposed parameter candidate before HMCL correction. km,tg_k^m,t Raw gradient at step k. XtmX_t^m Input data for task t and modality m. qm,∗Z_q^m,* Frozen pretrained Lorentz representations of task-q data from modality m. qm,tZ_q^m,t The same data after the stage-t Lorentz head. f(,⋅)f(W;·) Lorentz layer in (22). Similarity measures and objectives m→m′S^m→ m Similarity matrix from modality m to m′m , with m→m′∈ℝN×NS^m→ m ^N× N. Sijm→m′S_ij^m→ m Entry (i,j)(i,j) of m→m′S^m→ m , equal to −dℒK(im,jm′)-d_L^K(z_i^m,z_j^m ). τ Temperature parameter for contrastive learning. ℒm→m′L_m→ m Unidirectional contrastive loss from modality m to m′m . ℒcontrastL_contrast Symmetric contrastive loss 12(ℒm→m′+ℒm′→m) 12(L_m→ m +L_m → m). aper()aper(z) Entailment-cone aperture at embedding z. ext(,′)ext(z,z ) Exterior angle between embeddings z and ′z . κ Boundary constant for entailment cones (typically κ=0.1κ=0.1). ℒentailL_entail Entailment loss for hierarchical relationships. Geometric analysis and parameter updates L General Lorentz transformation, ∈SO+(1,d)L ^+(1,d). R Spatial Lorentz rotation diag(1,~)diag(1, R). ~ R Spatial rotation matrix in SO(d)SO(d). m=[0msm] δ^m=[\, δ_0^m\;\; δ_s^m\,] Candidate parameter displacement, partitioned by input coordinate. kopt,m,t δ^opt,m,t_k Optimizer-proposed displacement ~k+1m,t−km,t W_k+1^m,t-W_k^m,t before HMCL correction. k⋆,m,t δ ,m,t_k Closest admissible displacement obtained from kopt,m,t δ_k^opt,m,t. kmA_k^m Cumulative displacement km,t−0m,tW_k^m,t-W_0^m,t. ℰℒ,t−1E_L,t-1 Defect from a shared first-order Lorentz rotation on protected representations. t−1,t−1ℒA_t-1,P^L_t-1 Zero-defect admissible family and its Frobenius closest-point map. t−1,t−1V_t-1,P_t-1 Protected-subspace basis and projector, with t−1=t−1t−1⊤P_t-1=V_t-1V_t-1 . β Pullback coefficient toward the task-start parameters. Appendix B Additional Preliminaries B.1 Why Hyperbolic Geometry Represents Hierarchies Hyperbolic space is a homogeneous geometry with constant negative curvature. Its volume expands exponentially with radius: in n dimensions and curvature −1-1, a radius-r ball has volume Vℍn(r)∝∫0rsinhn−1(t)t,V_H^n(r) _0^r ^n-1(t)\,dt, which is asymptotically proportional to e(n−1)re^(n-1)r. Euclidean volume, in contrast, grows only as rnr^n. In the two-dimensional case, for instance, the hyperbolic disk area is 2π(coshr−1)2π( r-1) rather than the Euclidean value πr2π r^2. This exponential capacity closely matches the growth of tree-structured data, which is why hyperbolic embeddings can represent taxonomies, trees, and knowledge graphs with low distortion [13, 69]. B.2 Formal Description of the Lorentz Model The Lorentz model embeds a d-dimensional hyperbolic manifold as one sheet of a hyperboloid in (d+1)(d+1)-dimensional Minkowski space. Definition 1 (Minkowski Space and Metric). Minkowski space is ℝd+1R^d+1 equipped with the indefinite metric =diag(−1,1,…,1)∈ℝ(d+1)×(d+1).G=diag(-1,1,…,1) ^(d+1)×(d+1) [rgb]0,0,0. Here K>0K>0 denotes the squared hyperboloid radius used below, so the corresponding sectional curvature is −1/K-1/K. Its signature (−,+,…,+)(-,+,…,+) distinguishes one time-like direction from d space-like directions. Definition 2 (Lorentzian Inner Product). For ,∈ℝd+1x,y ^d+1, their Lorentzian (Minkowski) inner product is ⟨,⟩ℒ=⊤=−x0y0+∑i=1dxiyi. ,y _L=x Gy=-x_0y_0+ _i=1^dx_iy_i. Definition 3 (Lorentz Manifold). The Lorentz realization of d-dimensional hyperbolic space with curvature −1/K-1/K is ℍKd=∈ℝd+1:⊤=−K,x0>0.H_K^d= \x ^d+1:x Gx=-K,\;x_0>0 \. We write ℒKd=(ℍKd,)L_K^d=(H_K^d,G) when referring to the manifold together with its metric. Remark 1. Expanding the manifold constraint gives −x02+∑i=1dxi2=−K,x02−∑i=1dxi2=K.-x_0^2+ _i=1^dx_i^2=-K, x_0^2- _i=1^dx_i^2=K. This equation describes a two-sheeted hyperboloid; x0>0x_0>0 selects its connected upper sheet. For any ∈ℍKdx _K^d, x0=K+∥[1:d]∥2≥K,x_0= K+\|x_[1:d]\|^2≥ K, where [1:d]=(x1,…,xd)⊤x_[1:d]=(x_1,…,x_d) contains the space-like coordinates. Definition 4 (Lorentzian Distance). The length of the geodesic between ,∈ℒKdx,y _K^d is dℒK(,)=Karcosh(−⟨,⟩ℒK)=Karcosh(−⊤K).d_L^K(x,y)= Karcosh\! (- ,y _LK )= Karcosh\! (- x GyK ). (21) Remark 2. For points on the upper hyperboloid, ⟨,⟩ℒ≤−K ,y _L≤-K, with equality if and only if =x=y. The argument of arcosharcosh is therefore at least one, so (21) is well defined. Substituting =x=y gives zero distance. Remark 3 (Relation to Ambient Euclidean Distance). Far from the origin, geodesic distance increases approximately logarithmically with ambient Euclidean distance. This growth allows hyperbolic space to represent tree-like structures with low distortion. The linear symmetries of the Lorentz model form a group of transformations that leave the Lorentzian inner product unchanged. Definition 5 (Proper Orthochronous Lorentz Group). The proper orthochronous Lorentz group is SO+(1,d)=∈ℝ(d+1)×(d+1):⊤=,det()=1,L00≥1.SO^+(1,d)= \L ^(d+1)×(d+1):L GL=G,\; (L)=1,\;L_00≥ 1 \. Remark 4. The metric constraint makes L an isometry, the determinant condition excludes orientation-reversing reflections, and L00≥1L_00≥ 1 preserves the time orientation of the upper hyperboloid. The group can also map any point on ℍKdH_K^d to any other point: for ,∈ℍKdx,y _K^d, some ∈SO+(1,d)L ^+(1,d) satisfies =Lx=y. B.3 Lorentz Neural Layers Definition 6 (Lorentz Transformation Layer [62]). Let ∈ℝd+1z ^d+1 be a Lorentz representation, with time-like component z0>0z_0>0 and spatial block s∈ℝdz_s ^d. A Lorentz transformation layer is defined by f(,)=[K+‖22],∈ℝd×(d+1).f(W;z)= bmatrix K+\|Wz\|_2^2\\ Wz bmatrix, ^d×(d+1). (22) Remark 5. The time-like output in (22) is reconstructed from the spatial output so that the result satisfies the Lorentz manifold constraint. For the block-wise analysis, we partition the weights as =[0s],0∈ℝd×1,s∈ℝd×d,W=[\,w_0\;\;W_s\,], _0 ^d× 1, _s ^d× d, (23) where 0w_0 multiplies the time-like input and sW_s acts on the space-like input coordinates. Appendix C Rationale for the Preservation Objectives Continual learning aims to incorporate the current task while retaining information acquired earlier. For hyperbolic representations, that information is represented by geometry: pairwise relations encode semantic similarity, while radial position encodes semantic specificity and hierarchy [7]. We therefore use the following three requirements for representations of earlier data: (P1) Intra-modal preservation: t−1m,t(t−1m,t)⊤=t−1m,t−1(t−1m,t−1)⊤,∀m∈1,…,M;Z_t-1^m,tG(Z_t-1^m,t) =Z_t-1^m,t-1G(Z_t-1^m,t-1) , ∀ m∈\1,…,M\; (P2) Inter-modal preservation: t−1m,t(t−1m′,t)⊤=t−1m,t−1(t−1m′,t−1)⊤,∀m≠m′;Z_t-1^m,tG(Z_t-1^m ,t) =Z_t-1^m,t-1G(Z_t-1^m ,t-1) , ∀ m≠ m ; (P3) Hierarchical preservation: ∥(t−1m,t)[1:d]∥=∥(t−1m,t−1)[1:d]∥,∀m∈1,…,M.\|(z_t-1^m,t)_[1:d]\|=\|(z_t-1^m,t-1)_[1:d]\|, ∀ m∈\1,…,M\. Table 6: Links between the desired semantic properties and the hyperbolic quantities that encode them. Semantic Property Geometric Quantity Condition −- Directly constrained Semantic similarity dℒ(,)∝⟨,⟩ℒd_L(x,y) ,y _L (P1), (P2) Semantic specificity dℒ(,)∝∥[1:d]∥d_L(x,o) \|x_[1:d]\| (P3) −- Implied consequence (Proposition 2) Concept partial order aper(),ext(,)aper(x),ext(x,y) Implied by the conditions Table 6 links each preservation goal to the geometric quantity that represents it. Conditions (P1) and (P2) fix Lorentzian inner products. Since the distance in (21) is a monotone function of these products, the associated semantic similarities remain unchanged. Condition (P3) fixes the spatial norm. On the hyperboloid, dℒK(,)=Karcosh(1+∥[1:d]∥22/K).d_L^K(x,o)= Karcosh\! ( 1+\|x_[1:d]\|_2^2/K ). Preserving the spatial norm therefore keeps a concept at the same level of specificity. Together, these invariants are also sufficient to retain the entailment relation used to encode concept order. Proposition 2 (Invariance of the Concept Partial Order). Let im,im′∈ℍKdz_i^m,z_i^m _K^d be hyperbolic embeddings. If ∥(im)[1:d]∥2\|(z_i^m)_[1:d]\|_2 and ⟨im,im′⟩ℒ _i^m,z_i^m _L do not change, then the entailment relation im⊒im′z_i^m _i^m is unchanged as well. Proof. Entailment holds when ℒentail(im,im′)=0L_entail(z_i^m,z_i^m )=0, or equivalently ext(im,im′)≤aper(im).ext(z_i^m,z_i^m ) (z_i^m). The two angles are aper(im) (z_i^m) =sin−1(2κK∥(im)[1:d]∥), = ^-1\! ( 2κ K\|(z_i^m)_[1:d]\| ), (24) ext(im,im′) (z_i^m,z_i^m ) =cos−1((im′)0+(im)0⟨im,im′⟩ℒ/K∥(im)[1:d]∥(⟨im,im′⟩ℒ/K)2−1). = ^-1\! ( (z_i^m )_0+(z_i^m)_0 _i^m,z_i^m _L/K\|(z_i^m)_[1:d]\| ( _i^m,z_i^m _L/K )^2-1 ). (25) The aperture depends only on the spatial norm, while the exterior angle depends on that norm and the Lorentzian inner product. The time-like coordinate follows from the manifold constraint. If these quantities are preserved, neither side of the entailment inequality changes, so the corresponding partial-order relation is invariant. ∎ Appendix D Proofs for the Geometric Characterization Definition 7 (Joint Embedding Matrix). Consider N paired examples from task t∈t , evaluated at stage s∈s for modalities m1m_1 and m2m_2. Given tm,s∈ℝN×(d+1)Z_t^m,s ^N×(d+1) with rows t,im,s∈ℍKdz_t,i^m,s _K^d, define t,alls:=(tm1,stm2,s)∈ℝ2N×(d+1).Z_t,all^s:= pmatrixZ_t^m_1,s\\[2.0pt] Z_t^m_2,s pmatrix ^2N×(d+1). The ordering in this vertical concatenation retains the pairing between the two modalities. Definition 8 (Extended Lorentz Gram Matrix). For =diag(−1,1,…,1)G=diag(-1,1,…,1), the extended Gram matrix of the joint embeddings is t,exts _t,ext^s :=t,alls(t,alls)⊤ :=Z_t,all^sG(Z_t,all^s) (26) =(tm1,s(tm1,s)⊤tm1,s(tm2,s)⊤tm2,s(tm1,s)⊤tm2,s(tm2,s)⊤). = pmatrixZ_t^m_1,sG(Z_t^m_1,s) &Z_t^m_1,sG(Z_t^m_2,s) \\[2.0pt] Z_t^m_2,sG(Z_t^m_1,s) &Z_t^m_2,sG(Z_t^m_2,s) pmatrix. (27) Its diagonal blocks contain within-modality Lorentzian products, while the off-diagonal blocks record the cross-modal geometry. Definition 9 (Lorentz Boost). For ∈ℝdv ^d with ‖<1\|v\|<1 and γ=(1−∥2)−1/2γ=(1-\|v\|^2)^-1/2, the boost associated with v is =[γ−γ⊤−γ+γ21+γ⊤].B= bmatrixγ&- \\ - &I+ γ^21+γvv bmatrix. It mixes the time-like coordinate with the direction v without independently rotating the spatial axes. Definition 10 (Lorentz Rotation). A spatial Lorentz rotation has block form =[1⊤~],~⊤~=,det(~)=1.R= bmatrix1&0 \\ 0& R bmatrix, R R=I, ( R)=1. Thus ~∈SO(d) R (d) rotates only the spatial coordinates. D.1 Shared Isometry Induced by Gram Preservation Lemma 1 (Witt’s Extension Theorem [70]). Let (,Q)(V,Q) be a non-degenerate quadratic space. Any isometry ϕ:→′φ:U between subspaces ,′⊆U,V extends to an isometry of all of V. Remark 6. In matrix notation, if 11⊤=22⊤Z_1QZ_1 =Z_2QZ_2 for a non-degenerate symmetric Q, then an element T of O()=:⊤=O(Q)=\T:T QT=Q\ maps the first set of rows to the second, i.e., 2=1⊤Z_2=Z_1T . Lemma 2 (Hyperboloid-Preserving Isometries). Let ∈O(1,d)L∈ O(1,d) be an orientation-preserving isometry that maps the upper hyperboloid ℍKd=:⊤=−K,x0>0H_K^d=\x:x Gx=-K,\;x_0>0\ onto itself. Then ∈SO+(1,d)L ^+(1,d). Proof. Because the upper sheet is invariant, applying L to =(K,0,…,0)⊤o=( K,0,…,0) gives ()0=KL00>0(Lo)_0= KL_00>0. The identity ⊤=L GL=G further implies −L002+∑i=1dLi02=−1,-L_00^2+ _i=1^dL_i0^2=-1, so L002=1+∑i=1dLi02≥1L_00^2=1+ _i=1^dL_i0^2≥ 1 and therefore L00≥1L_00≥ 1. Orientation preservation gives det()=1 (L)=1. Together with ∈O(1,d)L∈ O(1,d), these are the defining conditions of SO+(1,d)SO^+(1,d). ∎ Theorem 3 (Uniqueness of the Lorentzian Isometry). Let t,alls,t′,alls′∈ℝ2N×(d+1)Z_t,all^s,Z_t ,all^s ^2N×(d+1). Suppose (1) their extended Gram matrices agree, t,exts=t′,exts′G_t,ext^s=G_t ,ext^s ; and (2) both joint matrices have full column rank d+1d+1; and (3) the stage-to-stage correspondence is induced by a continuous path of full-column-rank configurations with the same extended Gram matrix, beginning at the first configuration. Then there is a unique ∈SO+(1,d)L ^+(1,d) satisfying t′,alls′=t,alls⊤.Z_t ,all^s =Z_t,all^sL . Proof. Full column rank means that the rows of both joint matrices span the ambient Lorentzian space. Associate each row t,im,sz_t,i^m,s with the corresponding row t′,im,s′z_t ,i^m,s . Equality of the extended Gram matrices gives ⟨t,im,s,t,jm′,s⟩ℒ=⟨t′,im,s′,t′,jm′,s′⟩ℒ _t,i^m,s,z_t,j^m ,s _L= _t ,i^m,s ,z_t ,j^m ,s _L for every pair of samples and modalities. This correspondence is therefore an isometry on a generating set. Lemma 1 extends it to a global element ∈O(1,d)L∈ O(1,d) such that t′,alls′=t,alls⊤.Z_t ,all^s =Z_t,all^sL . It remains to identify the connected component. Let (λ)Z(λ) denote the path in condition (3), with (0)=t,allsZ(0)=Z_t,all^s. Full rank makes the corresponding isometry unique, and (λ)⊤=((0)⊤(0))−1(0)⊤(λ)L(λ) = (Z(0) Z(0) )^-1Z(0) Z(λ) makes its continuity explicit. Thus (0)=L(0)=I and (1)=L(1)=L. Since the determinant of an element of O(1,d)O(1,d) is either +1+1 or −1-1, continuity gives det(λ)=+1 (λ)=+1 throughout the path. All configurations remain on the forward sheet, so applying (λ)L(λ) to the origin gives L00(λ)>0L_00(λ)>0; the Lorentz constraint then gives L00(λ)≥1L_00(λ)≥ 1. Hence ∈SO+(1,d)L ^+(1,d). For uniqueness, assume that 1L_1 and 2L_2 both satisfy the relation. Then t,alls(1⊤−2⊤)=.Z_t,all^s(L_1 -L_2 )=0. Since t,allsZ_t,all^s has column rank d+1d+1, its right null space contains only zero; hence 1=2L_1=L_2. ∎ Corollary 4 (A Common Isometry for All Modalities). Under Theorem 3, the same ∈SO+(1,d)L ^+(1,d) acts on both modalities: t′m1,s′=tm1,s⊤,t′m2,s′=tm2,s⊤.Z_t ^m_1,s =Z_t^m_1,sL , _t ^m_2,s =Z_t^m_2,sL . If, in addition, each modality-specific block has full column rank, then any modality-specific maps 1L_1 and 2L_2 satisfying these two relations must obey 1=2=L_1=L_2=L. Proof. Stacking the two modality-specific relations would produce t′,alls′=(tm1,s1⊤tm2,s2⊤).Z_t ,all^s = pmatrixZ_t^m_1,sL_1 \\[2.0pt] Z_t^m_2,sL_2 pmatrix. Theorem 3 gives the unique transformation for the full stacked matrix, so both blocks admit the same restriction. Under the additional blockwise full-rank condition, subtracting the common relation from each modality-specific relation gives tmi,s(i⊤−⊤)=Z_t^m_i,s(L_i -L )=0. The right null space is then trivial, and hence i=L_i=L for i∈1,2i∈\1,2\. ∎ Remark 7. Conditions (P1) and (P2) are jointly equivalent to t−1,extt=t−1,extt−1.G_t-1,ext^t=G_t-1,ext^t-1. Theorem 3 and Corollary 4 therefore show that preserving both within-modal and cross-modal relations determines one Lorentz transformation shared by all modalities. D.2 Removing the Boost Component Lemma 3 (Generation of SO+(1,d)SO^+(1,d) [71]). Every element of SO+(1,d)SO^+(1,d) can be written as a finite composition of Lorentz rotations and boosts. Remark 8. Boosts and spatial rotations are intrinsic linear transformations of the Lorentz model: if ∈ℍKdx _K^d, then both Bx and Rx remain on ℍKdH_K^d. Theorem 4 (Elimination of Boosts; Restatement of Theorem 1). Suppose ∈SO+(1,d)L ^+(1,d) maps every old representation according to t−1m,t=t−1m,t−1z_t-1^m,t=Lz_t-1^m,t-1, and the stacked old-representation matrix has full column rank. If Condition (P3) holds for all such representations, then =[1⊤~],~∈SO(d),L= bmatrix1&0 \\ 0& R bmatrix, R (d), i.e., L contains no nontrivial boost. Proof. For every old representation on ℍKdH_K^d, Condition (P3) fixes the spatial norm. The manifold constraint z0=K+∥[1:d]∥22z_0= K+\|z_[1:d]\|_2^2 then also fixes the time-like coordinate. In row-matrix form, t−1,allt−1(⊤0−0)=,0=(1,0,…,0)⊤.Z_t-1,all^t-1(L e_0-e_0)=0, _0=(1,0,…,0) . Full column rank makes the right null space trivial, so ⊤0=0L e_0=e_0. Writing L in time–space blocks and using ⊤=L GL=G therefore gives =diag(1,~),~⊤~=.L=diag(1, R), R R=I. Finally, det=1 =1 implies det~=1 R=1. Thus ~∈SO(d) R (d) and no boost component remains. ∎ D.3 First-Order Hierarchy Preservation The next lemma relates perturbations of the spatial and time-like coordinates on the Lorentz hyperboloid and leads directly to Corollary 1. Lemma 4 (First-Order Change of the Time-Like Coordinate). Let ∈ℝd+1z ^d+1 obey z02−∥[1:d]∥22=Kz_0^2-\|z_[1:d]\|_2^2=K with z0>0z_0>0. For an infinitesimal perturbation Δ , Δz0=([1:d]⊤z0)Δ[1:d]. z_0= ( z_[1:d] z_0 ) _[1:d]. (28) Proof. Define the manifold constraint ϕ()=z02−∥[1:d]∥22−K=0.φ(z)=z_0^2-\|z_[1:d]\|_2^2-K=0. (29) For ~=+Δ z=z+ , a first-order Taylor expansion gives ϕ(~)=ϕ()+∇ϕ()⊤Δ+O(∥Δ∥22).φ( z)=φ(z)+∇φ(z) +O(\| \|_2^2). Both z and ~ z lie on the manifold to first order, so ∇ϕ()⊤Δ=0.∇φ(z) =0. (30) Since ∇ϕ()=[2z0−2[1:d]],∇φ(z)= bmatrix2z_0\\ -2z_[1:d] bmatrix, Equation (30) becomes 2z0Δz0−2[1:d]⊤Δ[1:d]=02z_0 z_0-2z_[1:d] _[1:d]=0. Dividing by 2z0>02z_0>0 proves (28). ∎ Corollary 5 (Restatement of Corollary 1). Let Δ=t−1m,t−t−1m,t−1 =z_t-1^m,t-z_t-1^m,t-1 denote the update of an old-task embedding. If Condition (P3) makes its time-like coordinate invariant to first order, then [1:d]⊤Δ[1:d]=0.z_[1:d] _[1:d]=0. (31) Proof. Condition (P3) sets Δz0=0 z_0=0 to first order. Lemma 4 simultaneously requires Δz0=[1:d]⊤Δ[1:d]z0. z_0= z_[1:d] _[1:d]z_0. Because z0>0z_0>0, combining the two statements yields (31). ∎ Appendix E Derivations for the HMCL Update E.1 Proof of Proposition 1 Proposition 3 (Restatement: Admissible Parameter Changes). Consider the Lorentz layer in Section 3.1, with m,t=[0m,tsm,t]W^m,t=[\,w_0^m,t\;\;W_s^m,t\,]. Assume that Theorem 1 holds at stage t−1t-1. Define Δm=m,t−m,t−1,‖Δm‖F→0. ^m=W^m,t-W^m,t-1, \| ^m\|_F→ 0. The collection of modal displacements is admissible to first order if and only if one skew-symmetric satisfies, simultaneously for every modality, t−1m,∗(Δm)⊤=t−1m,t−1[:d]⊤,⊤=−.Z_t-1^m,*( ^m) =Z_t-1^m,t-1[1\!:\!d] , =- . (32) Proof. The spatial output of the Lorentz layer is linear in its parameters. Its block form is f[1:d](m,t;t−1m,∗)=t−1m,∗[0](0m,t)⊤+t−1m,∗[1:d](sm,t)⊤.f_[1:d](W^m,t;Z_t-1^m,*)=Z_t-1^m,*[0](w_0^m,t) +Z_t-1^m,*[1:d](W_s^m,t) . (33) Consequently, the applied displacement changes the old spatial output by t−1m,t[:d] _t-1^m,t[1\!:\!d] =t−1m,t−1[:d]+t−1m,∗[0](Δ0m)⊤+t−1m,∗[1:d](Δsm)⊤. =Z_t-1^m,t-1[1\!:\!d]+Z_t-1^m,*[0]( _0^m) +Z_t-1^m,*[1:d]( _s^m) . (34) Combining the two update terms gives the compact form t−1m,t[1:d]=t−1m,t−1[1:d]+t−1m,∗(Δm)⊤.Z_t-1^m,t[1:d]=Z_t-1^m,t-1[1:d]+Z_t-1^m,*( ^m) . (35) Theorem 1 also states that an admissible change of the old representations is a spatial rotation. In a neighborhood of the identity, write =++(‖F2),⊤=−.R=I+ +O(\| \|_F^2), =- . It follows that t−1m,t[:d] _t-1^m,t[1\!:\!d] =t−1m,t−1[:d]⊤ =Z_t-1^m,t-1[1\!:\!d]R =t−1m,t−1[:d]+t−1m,t−1[:d]⊤+(∥F2). =Z_t-1^m,t-1[1\!:\!d]+Z_t-1^m,t-1[1\!:\!d] +O(\| \|_F^2). (36) Equating the first-order terms of (35) and (36) yields t−1m,∗(Δm)⊤=t−1m,t−1[:d]⊤,⊤=−.Z_t-1^m,*( ^m) =Z_t-1^m,t-1[1\!:\!d] , =- . (37) The same follows from Theorem 1 for all modalities. Conversely, (37) matches the tangent motion of that common rotation. Its spatial norm is unchanged to first order, and the reconstructed time-like coordinate in (4) is therefore unchanged as well. ∎ The implementation fixes the time-input column and realizes the admissible output motion through Δsm _s^m. Such a representative exists exactly when col(t−1m,t−1[1:d]⊤)⊆col(t−1m,∗[1:d]).col\! (Z_t-1^m,t-1[1:d] ) \! (Z_t-1^m,*[1:d] ). (38) The regularity condition in Appendix E.2 makes the two column spaces equal, so (38) holds for every admissible rotation. This gives the blockwise form in (16). E.2 Derivation of Corollary 2 Corollary 6 (Restatement: Closest-Admissible Update). Given candidate modal displacements m=[0msm]m=1M\ δ^m=[\, δ_0^m\;\; δ_s^m\,]\_m=1^M, HMCL-CA jointly selects their closest admissible displacements and one shared rotation by solving minimizeΔm, \ ^m\,\, minimize 12∑m=1M‖Δm−m‖F2 12 _m=1^M\| ^m- δ^m\|_F^2 (39) subject to to ⊤=−,Δ0m=, =- , _0^m=0, t−1m,∗[1:d](Δsm)⊤=t−1m,t−1[1:d]⊤,m=1,…,M. _t-1^m,*[1:d]( _s^m) =Z_t-1^m,t-1[1:d] , m=1,…,M. Under the regularity condition below, the modal displacements are unique and are obtained from one Lyapunov equation for the common ⋆ . We derive the joint HMCL-CA solution and make the shared rotation explicit. For each modality, let m=t−1m,∗[1:d],m=t−1m,t−1[1:d].X_m=Z^m,*_t-1[1:d], _m=Z^m,t-1_t-1[1:d]. (40) Assume that mY_m has full column rank and that col(m)=col(m)col(X_m)=col(Y_m). Then m=mm,m=m†m,X_m=Y_mC_m, _m=Y_m X_m, (41) where mC_m is invertible. Proposition 1 now gives m(Δsm)⊤=⊤⟺Δsm=m−⊤.C_m( _s^m) = _s^m= C_m^- . (42) The same occurs for every m. Substituting (42) into the closest-admissible problem yields min⊤=−12∑m=1M‖m−⊤−sm‖F2. _ =- 12 _m=1^M \| C_m^- - δ_s^m \|_F^2. (43) Define =∑m=1Mm−⊤m−1,=∑m=1Msmm−1.H= _m=1^MC_m^- C_m^-1, = _m=1^M δ_s^mC_m^-1. (44) Since every mC_m is invertible, H is positive definite. Stationarity of (43) over the skew-symmetric matrices gives the Lyapunov equation ⋆+⋆=2Skew(),Skew()=12(−⊤). H+H =2Skew(F), (F)= 12(F-F ). (45) Positive definiteness of H gives a unique skew-symmetric solution. The closest admissible displacements are therefore sCA,m=⋆m−⊤,0CA,m=,m=1,…,M. δ_s^CA,m= C_m^- , δ_0^CA,m=0, m=1,…,M. (46) This construction solves for one rotation from all modal candidates. In the optimizer-consistent update, each sm δ_s^m is the spatial block of the displacement proposed by the chosen optimizer. E.3 Proof of Corollary 3 Corollary 7 (HMCL-MR: Minimal-Rotation Update (Restatement)). Let sm δ_s^m be an unconstrained candidate displacement and impose Proposition 1 while penalizing the induced rotation. In the minimum-rotation limit, ⋆= =0 and t−1m,∗[1:d](Δsm)⊤=,Δ0m=.Z_t-1^m,*[1:d]( _s^m) =0, _0^m=0. (47) The canonical solution is the null-space projection Δsm=sm(−t−1),t−1=(t−1m,∗[1:d])⊤(t−1m,∗[1:d](t−1m,∗[1:d])⊤)†t−1m,∗[1:d]. _s^m= δ_s^m(I-P_t-1), _t-1=(Z_t-1^m,*[1:d]) (Z_t-1^m,*[1:d](Z_t-1^m,*[1:d]) ) Z_t-1^m,*[1:d]. (48) Proof. Begin with the general admissible family t−1m,∗[1:d](Δsm)⊤=t−1m,t−1[:d]⊤,⊤=−.Z_t-1^m,*[1:d]( _s^m) =Z_t-1^m,t-1[1\!:\!d] , =- . (49) Among these displacements, consider the rotation-regularized closest point to sm δ_s^m: minΔsm,12‖Δsm−sm‖F2+λ2‖F2s.t.(49). _ _s^m, 12\| _s^m- δ_s^m\|_F^2+ λ2\| \|_F^2 .t. eq:app_admissible_general. (50) With multiplier , the Lagrangian is ℒ =12‖Δsm−sm‖F2+λ2‖F2 = 12\| _s^m- δ_s^m\|_F^2+ λ2\| \|_F^2 +⟨,t−1m,∗[1:d](Δsm)⊤−t−1m,t−1[:d]⊤⟩. + ,Z_t-1^m,*[1:d]( _s^m) -Z_t-1^m,t-1[1\!:\!d] . (51) Stationarity gives Δsm _s^m =sm−⊤t−1m,∗[1:d], = δ_s^m- Z_t-1^m,*[1:d], (52) =1λSkew((t−1m,t−1[:d])⊤), = 1λSkew\! ((Z_t-1^m,t-1[1\!:\!d]) ), (53) where Skew()=(−⊤)/2Skew(A)=(A-A )/2. Substituting (52) into (49) yields t−1m,∗[1:d](t−1m,∗[1:d])⊤=t−1m,∗[1:d](sm)⊤+(λ−1).Z_t-1^m,*[1:d](Z_t-1^m,*[1:d]) =Z_t-1^m,*[1:d]( δ_s^m) +O(λ^-1). (54) In the limit λ→∞λ→∞, the rotation term vanishes and =(t−1m,∗[1:d](t−1m,∗[1:d])⊤)†t−1m,∗[1:d](sm)⊤. = (Z_t-1^m,*[1:d](Z_t-1^m,*[1:d]) ) Z_t-1^m,*[1:d]( δ_s^m) . (55) Inserting this expression into (52) produces Δsm=sm[−(t−1m,∗[1:d])⊤(t−1m,∗[1:d](t−1m,∗[1:d])⊤)†t−1m,∗[1:d]]. _s^m= δ_s^m [I-(Z_t-1^m,*[1:d]) (Z_t-1^m,*[1:d](Z_t-1^m,*[1:d]) ) Z_t-1^m,*[1:d] ]. Equation (53) simultaneously gives ⋆= =0. Combining this spatial projection with Δ0m= _0^m=0 proves the stated canonical update. ∎ Appendix F Optimizer-Consistent HMCL: Derivations This appendix derives the direction–displacement mismatch, Lorentz defect, finite-step preservation result, feasible accumulation, and low-rank residual bound for the unified HMCL update. Throughout a fixed task, we omit the stage index t and retain the optimizer-step index k; in (k,0)(k,0) and (k,s)(k,s), the second entry identifies the time-like input column (00) or the spatial input block (s), whereas 0m,tW_0^m,t denotes the task-start parameters. As established in the main paper, a common spatial rotation preserves the three HMCL conditions: CA estimates its skew generator , while MR selects the identity member = =0. Both corrections act on the fully formed candidate displacement. F.1 A Concrete Adaptive-Optimizer Mismatch Let ⊆ℝqS ^q be the linear subspace of admissible vectorized parameter displacements and let be its orthogonal projector. For a positive diagonal matrix optD_opt, consider the preconditioned component of an AdamW step, pre=−ηopt,∈.d_pre=- _optu, . (56) Proposition 4 (Direction–Displacement Mismatch). Suppose AdamW maps an admissible first-moment direction ∈u to AdamW=−ηopt−ηω,d^AdamW=- _optu-ηω θ, (57) where optD_opt is a positive diagonal preconditioner and ω is the decoupled weight-decay coefficient. The preconditioned component remains admissible for every ∈u if and only if (−)opt=.(I- )D_opt =0. (58) The decay component is admissible exactly when ω=0ω=0 or ∈ θ . Proof of Proposition 4. Because projects onto S, every admissible direction can be written as =u= a for some a. The preconditioned direction remains in S for every admissible input if and only if its component in ⟂S vanishes: (−)opt (I- )D_optu =(−)opt=for every . =(I- )D_opt a=0 every a. (59) This is equivalent to (−)opt=(I- )D_opt =0. The decoupled decay term is −ηω-ηω θ. It lies in S exactly when ω=0ω=0 or ∈ θ . Adding either non-admissible component to an admissible one does not restore the constraint in general. ∎ A two-dimensional counterexample. Let =span([1,1]⊤)S=span([1,1] ) and choose the admissible direction =[1,1]⊤u=[1,1] . With opt=diag(1,2)D_opt=diag(1,2) and no decay, AdamW-like scaling gives pre=−η[1,2]⊤,d_pre=-η[1,2] , which is not in S. Equivalently, the normal vector ⟂=[1,−1]⊤v_ =[1,-1] satisfies ⟂⊤=0v_ u=0 but ⟂⊤pre=η≠0v_ d_pre=η≠ 0. The example is intentionally small: the same non-commutativity occurs between a low-rank representation projector and AdamW’s element-wise preconditioner in the Lorentz head. F.2 Lorentz Strain of a Realized Optimizer Candidate We now specialize the mismatch to the Lorentz representation geometry. For modality m, let m∈ℝNm×dY^m ^N_m× d contain the old spatial coordinates as rows, and let Δm ^m contain their first-order changes under the realized optimizer candidate. Define m ^m =argmin‖Δm−m⊤‖F2, = _B\| ^m-Y^mB \|_F^2, (60) m ^m =Δm−m(m)⊤. = ^m-Y^m(B^m) . Thus Δm=m(m)⊤+m ^m=Y^m(B^m) +E^m. Write m=m+mB^m=S^m+ ^m, where m=Sym(m)S^m=Sym(B^m) and m=Skew(m) ^m=Skew(B^m). For a row i, we use the equivalent column-vector form Δim=mim+im _i^m=B^my_i^m+e_i^m. Proposition 5 (Lorentz Defect of a Realized Step). Under the rank condition of Theorem 1, a realized first-order change preserves Conditions (P1)–(P3) if and only if m=,m=,m=m′for every m,m′.E^m=0, ^m=0, ^m= ^m every m,m . (61) Equivalently, the change is induced by one shared skew-symmetric generator. Proof. Suppose first that the realized change preserves Conditions (P1)–(P3) to first order. Proposition 1 then gives one skew-symmetric generator shared by all modalities, so Δm=m⊤for every m. ^m=Y^m every m. (62) Under the stated rank condition, the least-squares generator in (60) is unique. Hence m=E^m=0, m=B^m= , and Sym(m)=Sym(B^m)=0 for every modality. Conversely, suppose (61) holds. Then all modalities satisfy (62) for the same skew-symmetric matrix . For an old spatial coordinate y, its induced time-like derivative under the reconstruction in (4) is z˙0=⊤K+‖22=0. z_0= y y K+\|y\|_2^2=0. (63) The first-order change of a spatial inner product is likewise (i)⊤j+i⊤(j)=i⊤(⊤+)j=0.( y_i) y_j+y_i ( y_j)=y_i ( + )y_j=0. (64) The same cancellation applies across modalities because the generator is shared. The Lorentz Gram matrices and spatial radii are therefore stationary to first order, establishing Conditions (P1)–(P3). ∎ A compact measure of the departure from this shared tangent direction is ℒ=∑m(‖m‖F2+‖m‖F2)+∑m<m′‖m−m′‖F2.D_L= _m (\|S^m\|_F^2+\|E^m\|_F^2 )+ _m<m \| ^m- ^m \|_F^2. (65) It vanishes exactly for a shared first-order rotation under the same assumptions. Radial and cone-aperture drift. Let r=‖2r=\|y\|_2 and =++a=Sy+ y+e. A Taylor expansion and ⊤=0y y=0 give Δr=⊤(+)r+(‖22). r= y (Sy+e)r+O(\|a\|_2^2). (66) For h(r)=sin−1(2κ/r)h(r)= ^-1(2κ/r), h′(r)=−2κrr2−4κ2,h (r)=- 2κr r^2-4κ^2, (67) and therefore Δaper()=−2κrr2−4κ2Δr+((Δr)2). (y)=- 2κr r^2-4κ^2 r+O(( r)^2). (68) In particular, decoupled decay has local generator decay=−ηωB_decay=-η . It produces Δr=−ηωr+((ηω)2) r=-ηω r+O((ηω)^2) and therefore increases the cone aperture to first order. This calculation is a geometric interpretation of the decay term, not an explanation of the main-table results, which use zero weight decay; Appendix H.6 separately evaluates nonzero decay. Lorentz Gram drift. Let i=[zi,0;i]z_i=[z_i,0;y_i] and ˙i=[z˙i,0;i] z_i=[ z_i,0;a_i]. The first variation of an intra- or inter-modal Lorentz product is dϵ(i+ϵ˙i)⊤(j+ϵ˙j)|ϵ=0=˙i⊤j+i⊤˙j. ddε . (z_i+ε z_i ) G (z_j+ε z_j ) |_ε=0= z_i Gz_j+z_i G z_j. (69) For one shared skew generator, both time-like derivatives vanish and the spatial terms cancel. Symmetric strain, residual motion, or different modality generators remove this cancellation. Equation (69) therefore links the optimizer defect in (65) directly to the similarity matrices used by the contrastive objective. F.3 Closest Projection of the Realized Step For one modality, omit the superscript m and split the optimizer candidate at step k into the time-like input column k,0opt δ^opt_k,0 and spatial input block k,sopt δ^opt_k,s. Let =⊤P=VV be an orthogonal projector and =−Q=I-P. We write k=[k,0k,s] δ_k=[\, δ_k,0\;\; δ_k,s\,] for a generic displacement at the same step. On the upper sheet of ℍKdH_K^d, zero Lorentz geodesic distance is equivalent to equality of the two points. Because the spatial block of (4) is linear in its input and the time-input parameter block is fixed, the canonical zero-motion condition used in (19) is equivalent to k,s= δ_k,sP=0 and k,0= δ_k,0=0. The following theorem and corollary establish the closest-point and finite-step preservation claims used in Theorem 2. Theorem 5 (Closest Identity-Isometry Representative). Among displacements satisfying k,s= δ_k,sP=0 and k,0= δ_k,0=0, the unique displacement closest to the realized optimizer candidate is k,s⋆=k,sopt(−),k,0⋆=. δ _k,s= δ^opt_k,s(I-P), δ _k,0=0. (70) Proof of Theorem 5. Every spatial matrix has the orthogonal decomposition k,sopt=k,sopt+k,sopt. δ^opt_k,s= δ^opt_k,sP+ δ^opt_k,sQ. (71) The two terms are orthogonal in the Frobenius inner product because =PQ=0. A feasible spatial displacement k,sD_k,s satisfies k,s=D_k,sP=0 and therefore k,s=k,sD_k,s=D_k,sQ. Hence ‖k,s−k,sopt‖F2 \|D_k,s- δ^opt_k,s\|_F^2 =‖k,s−k,sopt‖F2+‖k,sopt‖F2. =\|D_k,s- δ^opt_k,sQ\|_F^2+\| δ^opt_k,sP\|_F^2. (72) The second term is independent of k,sD_k,s, while the first is uniquely minimized by k,s=k,soptD_k,s= δ^opt_k,sQ. The time-like constraint has the unique closest value k,0=d_k,0=0. Combining the two blocks gives (70). ∎ This proof is independent of how the optimizer constructs kopt δ^opt_k. Learning rates, moment estimates, preconditioners, and decay affect the candidate, while the Lorentz-derived projection solves the same closest-point problem for the displacement actually proposed. Corollary 8 (Exact Protected Lorentz Invariants). If an old frozen feature x lies in the range of t−1P_t-1, the update in (70) leaves its complete Lorentz representation unchanged after each finite step: k+1m()=km().z_k+1^m(x)=z_k^m(x). (73) Hence its hyperbolic relations, spatial radius, and entailment-cone aperture are unchanged. Proof of Corollary 8. Let =t−1x=P_t-1x. Equation (70) gives k,s⋆=k,sopt(−t−1)t−1=, δ _k,sx= δ^opt_k,s(I-P_t-1)P_t-1x=0, (74) and the time-like input block is fixed. The spatial output of the Lorentz layer is therefore identical before and after the corrected step. Its reconstructed first coordinate, which is a deterministic function of the spatial norm, is identical as well. Thus each protected Lorentz point is unchanged. Every intra- and inter-modal Lorentz Gram entry, hyperbolic distance, spatial radius, and cone aperture formed from these points is consequently unchanged. ∎ F.4 Task-Anchored Feasible Contraction For a protected representation system fixed throughout task t, let t−1A_t-1 be the joint admissible family in (17). It is linear because its defining constraints are linear in the modal displacements and their shared skew generator. For a modal tuple =(m)m=1MU=(U^m)_m=1^M, define ‖⊕F=(∑m‖m‖F2)1/2\|U\|_ F= ( _m\|U^m\|_F^2 )^1/2. Here umaxu_ bounds the realized AdamW displacement rather than the raw gradient. Its matrix-wide value can grow with the number of modal parameters even when the per-parameter step is small; the result requires only a finite bound, not umax≤1u_ ≤ 1, although a larger value makes the numerical accumulation bound looser. Corollary 9 (Within-Stage Admissibility and Bounded Accumulation). Let 0m=A_0^m=0 for every modality m denote the initial displacement from the task-start parameters. Given optimizer candidates kopt,t δ_k^opt,t, let kt=t−1ℒ(kopt,t)=(km)m=1MD_k^t=P^L_t-1( δ_k^opt,t)=(D_k^m)_m=1^M and define k+1m=(1−β)(km+km),0≤β<1.A_k+1^m=(1-β)(A_k^m+D_k^m), 0≤β<1. (75) Then kmm=1M∈t−1\A_k^m\_m=1^M _t-1 for every k. Relative to the uncontracted candidate, the task-start distance is reduced by 1−β1-β. The projection is non-expansive: ‖kt‖⊕F≤‖kopt,t‖⊕F\|D_k^t\|_ F≤\| δ_k^opt,t\|_ F. If the latter is at most umaxu_ for all k, then ‖Jt‖⊕F \|A_J^t\|_ F ≤∑j=0J−1(1−β)J−j‖jt‖⊕F ≤ _j=0^J-1(1-β)^J-j\|D_j^t\|_ F (76) ≤Jumax,β=0,(1−β)[1−(1−β)J]βumax,0<β<1, ≤ casesJu_ ,&β=0,\\[2.0pt] (1-β)[1-(1-β)^J]β\,u_ ,&0<β<1, cases ≤1−βumax(0<β<1). ≤ 1-β\,u_ (0<β<1). Proof of Corollary 9. The set t−1A_t-1 is a linear subspace, so projection places every ktD_k^t in it and induction keeps every ktA_k^t in it. As the orthogonal projection onto this subspace, t−1ℒP^L_t-1 is non-expansive in the joint Frobenius norm. Unrolling the recurrence gives Jt=∑j=0J−1(1−β)J−jjt.A_J^t= _j=0^J-1(1-β)^J-jD_j^t. The triangle inequality gives the data-dependent bound in (76); substituting the proposal bound umaxu_ and summing the geometric series gives its two stated cases. ∎ F.5 Residual Drift under Low-Rank PCA Let ∈ℝn×pX ^n× p contain the old frozen features, and let =⊤/n =X X/n have eigenvalues λ1≥⋯≥λp≥0 _1≥·s≥ _p≥ 0. Let rP_r be the orthogonal projector onto the top-r eigenspace and r=−rQ_r=I-P_r. For the zero-rotation restriction, k,s=k,srD_k,s=D_k,sQ_r, so 1n‖k,s⊤‖F2 1n\|XD_k,s \|_F^2 =1n‖rk,s⊤‖F2 = 1n\|XQ_rD_k,s \|_F^2 (77) ≤‖k,s‖221n‖r‖F2 ≤\|D_k,s\|_2^2 1n\|XQ_r\|_F^2 (78) =‖k,s‖22∑i>rλi. =\|D_k,s\|_2^2 _i>r _i. (79) Thus the update is exact on the retained subspace, and its mean-squared effect outside that subspace is bounded by the PCA spectral tail. This is an average-energy statement. It does not bound the worst example or the worst earlier task. The task-anchored update retains the same subspace property. The joint bound also applies to every modal block; in particular, J,s=J,srA_J,s=A_J,sQ_r. Combining (76) with (79) gives, for 0<β<10<β<1, 1n‖J,s⊤‖F2≤(1−βumax)2∑i>rλi. 1n\|XA_J,s \|_F^2≤ ( 1-βu_ )^2 _i>r _i. (80) The geometric correction controls the retained subspace exactly, whereas the anchor and the spectral tail jointly bound the average residual motion outside that subspace. F.6 Pseudocode, Optimizer State, and Complexity Algorithms 1 and 2 instantiate the same post-optimizer, task-anchored update. They differ only in the closest-admissible correction: HMCL-MR fixes the shared rotation to zero, whereas HMCL-CA estimates one rotation jointly from all modal candidates. Algorithm 1 Optimizer-Consistent HMCL-MR Input: Current-task data tD_t; task-start Lorentz heads 0m,tm=1M\W^m,t_0\_m=1^M; protected projector t−1P_t-1; optimizer OptOpt; anchoring coefficient β. Output: Final task parameters Jm,tm=1M\W^m,t_J\_m=1^M. for optimizer step k=0,…,J−1k=0,…,J-1 do Compute the HMCL loss on tD_t and gradients km,tm=1M\g^m,t_k\_m=1^M; Let OptOpt update its state and propose ~k+1m,tm=1M\ W^m,t_k+1\_m=1^M; for m=1,…,Mm=1,…,M do kopt,m,t←~k+1m,t−km,t δ^opt,m,t_k← W^m,t_k+1-W^m,t_k; k,0MR,m,t← δ^MR,m,t_k,0 0, k,sMR,m,t←k,sopt,m,t(−t−1) δ^MR,m,t_k,s← δ^opt,m,t_k,s(I-P_t-1); ^k+1m,t←km,t+kMR,m,t W^m,t_k+1 ^m,t_k+ δ^MR,m,t_k; k+1m,t←0m,t+(1−β)(^k+1m,t−0m,t)W^m,t_k+1 ^m,t_0+(1-β)( W^m,t_k+1-W^m,t_0); return Jm,tm=1M\W^m,t_J\_m=1^M. Algorithm 2 Optimizer-Consistent HMCL-CA Input: Current-task data tD_t; task-start Lorentz heads 0m,tm=1M\W^m,t_0\_m=1^M; protected matrices m,mm=1M\X_m,Y_m\_m=1^M from Appendix E.2; optimizer OptOpt; anchoring coefficient β. Output: Final task parameters Jm,tm=1M\W^m,t_J\_m=1^M. for m=1,…,Mm=1,…,M do m←m†mC_m _m X_m; ←∑m=1Mm−⊤m−1H← _m=1^MC_m^- C_m^-1; for optimizer step k=0,…,J−1k=0,…,J-1 do Compute the HMCL loss on tD_t and gradients km,tm=1M\g^m,t_k\_m=1^M; Let OptOpt update its state and propose ~k+1m,tm=1M\ W^m,t_k+1\_m=1^M; for m=1,…,Mm=1,…,M do kopt,m,t←~k+1m,t−km,t δ^opt,m,t_k← W^m,t_k+1-W^m,t_k; k←∑m=1Mk,sopt,m,tm−1F_k← _m=1^M δ^opt,m,t_k,sC_m^-1; Solve k⋆+k⋆=2Skew(k) _k H+H _k =2Skew(F_k); for m=1,…,Mm=1,…,M do k,0CA,m,t← δ^CA,m,t_k,0 0, k,sCA,m,t←k⋆m−⊤ δ^CA,m,t_k,s← _k C_m^- ; ^k+1m,t←km,t+kCA,m,t W^m,t_k+1 ^m,t_k+ δ^CA,m,t_k; k+1m,t←0m,t+(1−β)(^k+1m,t−0m,t)W^m,t_k+1 ^m,t_0+(1-β)( W^m,t_k+1-W^m,t_0); return Jm,tm=1M\W^m,t_J\_m=1^M. For k,s∈ℝd×pW_k,s ^d× p, the HMCL-MR correction k,s←k,s−(k,s)⊤ δ_k,s← δ_k,s-( δ_k,sV)V costs (dpr)O(dpr) and stores (pr)O(pr) values. The dense HMCL-CA solver adds one d×d× d Lyapunov equation per step, with (d3)O(d^3) time; a rank-r realization reduces this solve to (r3)O(r^3). The PCA basis and the task-level CA factors are estimated once per stage. The optimizer state remains the state produced by the raw gradients; for AdamW, this state includes the first- and second-moment estimates. The guarantees concern the realized parameter trajectory rather than a transported optimizer state. Appendix G Additional Implementation Details G.1 Dataset Details and Task Stream The unified journal benchmark contains 16 datasets and mixes image–text classification with bidirectional cross-modal retrieval. G.1.1 Classification Datasets The classification portion includes CIFAR-10 and CIFAR-100 [72], Caltech-101 [73], ImageNet-1K [66], Food-101 [74], Flowers-102 [75], EuroSAT [76], DTD [77], FGVC Aircraft [78], MNIST [79], PCAM [80], Country211 [1], CLEVR [81], and SST-2 [82]. Collectively, these benchmarks cover coarse object categories, fine-grained recognition, textures, scenes, and language sentiment. We regard each dataset as a separate task and evaluate it using the standard image–text classification protocol. G.1.2 Retrieval Datasets COCO [64] and Flickr30K [65] provide the retrieval stages. Both contain paired images and captions and are evaluated in the image-to-text and text-to-image directions. G.1.3 Task Stream Construction The datasets follow one fixed sequential order shared by every method and random seed. At a given stage, non-replay methods optimize only on the current dataset and do not revisit examples from completed tasks. Replay-based baselines additionally access earlier tasks through their explicitly bounded memory buffers, whose capacities are reported with the method configurations. Classification and retrieval are interleaved so that the stream requires the model to alternate between different multimodal objectives. G.1.4 Dataset Statistics Table 7 reports the train/test sizes, number of categories where applicable, and the evaluation measure for every stage. Table 7: Composition and statistics of the 16-stage continual multimodal benchmark. Dataset Task Classes Train Test Evaluation FGVC Aircraft [78] Classification 100 3,334 3,333 Accuracy Caltech-101 [73] Classification 102 2,448 6,084 Accuracy ImageNet-1K [66] Classification 1,000 1,281,167 50,000 Accuracy CIFAR-10 [72] Classification 10 45,000 10,000 Accuracy CIFAR-100 [72] Classification 100 45,000 10,000 Accuracy CLEVR [81] Classification 8 4,500 5,000 Accuracy Country211 [1] Classification 211 31,650 21,100 Accuracy DTD [77] Classification 47 1,880 1,880 Accuracy EuroSAT [76] Classification 10 5,000 5,000 Accuracy Flowers-102 [75] Classification 102 1,020 6,149 Accuracy Food-101 [74] Classification 101 68,175 25,250 Accuracy MNIST [79] Classification 10 48,000 10,000 Accuracy PCAM [80] Classification 2 262,144 32,768 Accuracy SST-2 [82] Classification 2 6,920 1,821 Accuracy COCO [64] Retrieval – 118,287 5,000 Recall@K Flickr30K [65] Retrieval – 29,000 1,000 Recall@K G.2 Task-Order Robustness We permute all tasks in the unified 16-task benchmark and keep the HyCoCLIP-B backbone, optimizer, training schedule, and main-table HMCL configurations fixed. The task-order RNG is independent of the five paired training seeds, and no hyperparameter is selected on either shuffled stream. The two orders, generated once using order seeds 2608311 and 2608312, are: seq2: Flickr30K → MNIST → PCAM → Food-101 → EuroSAT → Caltech-101 → SST-2 → COCO → Flowers-102 → CIFAR-10 → CLEVR → Aircraft → DTD → CIFAR-100 → Country211 → ImageNet. seq3: DTD → ImageNet → CLEVR → CIFAR-10 → Flickr30K → PCAM → Flowers-102 → EuroSAT → COCO → CIFAR-100 → Country211 → Aircraft → MNIST → Food-101 → Caltech-101 → SST-2. Table 8 reports the direct five-seed results without relative-difference rows. On seq2, HMCLMR gives the highest Overall score and BWT, while HMCLCA gives the highest final retrieval score. On seq3, HMCLCA gives the highest Overall score, while HMCLMR gives the strongest retrieval score and marginally better Overall BWT. Both complete variants retain a clear advantage over Vanilla under each permutation, showing that the main result is not tied to the canonical task order. The separate AdamW weight-decay sweep is reported in Appendix H.6. Table 8: Task-order robustness on HyCoCLIP-B using two fixed permutations of the unified 16-task stream (mean ± sample standard deviation over five paired seeds). Classification uses top-1 accuracy, retrieval uses symmetric R@5, and BWT is final minus immediate performance. Both HMCL variants use their fixed main-table configurations; no hyperparameter is retuned for either order. Classification Retrieval Overall Order Method Acc ↑ A↑BWT_A R@5 ↑ R5↑BWT_R5 Overall ↑ BWT ↑ seq2 Vanilla 42.095±0.12842.095± 0.128 −3.890±0.223-3.890± 0.223 62.673±0.09562.673± 0.095 −10.002±0.092-10.002± 0.092 44.668±0.10444.668± 0.104 −4.654±0.187-4.654± 0.187 seq2 HMCLMR 45.441±0.04945.441± 0.049 −0.198±0.054-0.198± 0.054 71.263±0.08271.263± 0.082 −3.997±0.101-3.997± 0.101 48.669±0.04748.669± 0.047 −0.673±0.051-0.673± 0.051 seq2 HMCLCA 45.222±0.02045.222± 0.020 −0.324±0.058-0.324± 0.058 71.351±0.07171.351± 0.071 −4.139±0.057-4.139± 0.057 48.488±0.01648.488± 0.016 −0.801±0.054-0.801± 0.054 seq3 Vanilla 41.299±0.16841.299± 0.168 −4.686±0.051-4.686± 0.051 62.551±0.09062.551± 0.090 −5.551±0.193-5.551± 0.193 43.956±0.14743.956± 0.147 −4.794±0.050-4.794± 0.050 seq3 HMCLMR 45.278±0.06245.278± 0.062 −0.733±0.088-0.733± 0.088 72.613±0.08172.613± 0.081 −2.141±0.117-2.141± 0.117 48.695±0.05348.695± 0.053 −0.909±0.065-0.909± 0.065 seq3 HMCLCA 45.430±0.07045.430± 0.070 −0.684±0.124-0.684± 0.124 72.397±0.05272.397± 0.052 −2.579±0.034-2.579± 0.034 48.801±0.06448.801± 0.064 −0.920±0.113-0.920± 0.113 G.3 Supplementary Materials for Case Study We further visualize how continual adaptation changes the visual–semantic hierarchy learned on Flickr30K. Nouns and adjectives extracted from the dataset captions augment the candidate text pool with concepts at different abstraction levels. Starting from an image embedding, we sample 20 locations on the geodesic leading to [ROOT], the manifold origin, and compare the nearest concepts recovered by HMCL and Vanilla. Examples (1)–(3) search a joint pool of captions, nouns, and adjectives; examples (4)–(6) use captions alone. Blue rows show traversals immediately after the Flickr30K stage, while green rows show the same analysis after the full continual sequence. In these examples, HMCL gives a clearer movement from image-specific descriptions toward more general concepts. Vanilla shows less consistent results after later tasks, which is consistent with greater semantic and hierarchical drift. After Flickr30K After the full stream (1) HMCL (ours) Vanilla A dog runs on the green grass near a wooden fence . A black and white dog is running in a grassy garden surrounded by a white fence . dog handsome sincerity fondness [ROOT] [ROOT] HMCL (ours) Vanilla A dog runs on the green grass near a wooden fence . A black and white dog is running in a grassy garden surrounded by a white fence . dog aloof sincerity pleasure [ROOT] [ROOT] (2) HMCL (ours) Vanilla The man with pierced ears is wearing glasses and an orange hat . The man with pierced ears is wearing glasses and an orange hat . hat fashion fashion style [ROOT] [ROOT] HMCL (ours) Vanilla The man with pierced ears is wearing glasses and an orange hat . The man with pierced ears is wearing glasses and an orange hat . fashion kind cousin new [ROOT] [ROOT] (3) HMCL (ours) Vanilla A girl in a pink shirt slides down an inflatable fun slide . Young girl sliding down an inflated slide . summer summer conventional quality [ROOT] [ROOT] HMCL (ours) Vanilla A girl in a pink shirt slides down an inflatable fun slide . Young girl sliding down an inflated slide . summer summer quick material [ROOT] [ROOT] After Flickr30K After the full stream (4) HMCL (ours) Vanilla Numerous Asian people working behind a counter . Numerous Asian people working behind a counter . An Asian food market where a woman is picking out food . An Asian food market where a woman is picking out food . Employees at a sushi restaurant prepare for dinner time rush . A cyclist wearing a black helmet is riding by some black vans . [ROOT] [ROOT] HMCL (ours) Vanilla Numerous Asian people working behind a counter . Employees at a sushi restaurant prepare for dinner time rush . An Asian food market where a woman is picking out food . A cyclist wearing a black helmet is riding by some black vans . A cyclist wearing a black helmet is riding by some black vans . Two wet dogs run into the surf at sunset . [ROOT] [ROOT] (5) HMCL (ours) Vanilla 2 boys in the foreground in a karate competition and coaches in background looking on with another coach sitting at table . A young boy demonstrates karate in a gymnasium . A young boy demonstrates karate in a gymnasium . A little girl dressed in yellow splashes in a shallow pool . Man standing by a poster of religious beliefs . A child plays at a playground . [ROOT] [ROOT] HMCL (ours) Vanilla 2 boys in the foreground in a karate competition and coaches in background looking on with another coach sitting at table . Two people are sitting outdoors on a blanket near a tree . A young boy demonstrates karate in a gymnasium . A surfer jumps a wave . Man standing by a poster of religious beliefs . A couple enjoying a glass of white wine . [ROOT] [ROOT] (6) HMCL (ours) Vanilla A young boy is sitting on the floor trying to take off his boots . A woman wearing glasses and an orange dress is standing with another woman wearing glasses and a green dress . A young boy puts on his boots at the top of a flight of stairs . Three young men wearing hooded sweatshirts , standing at a bench . A young boy demonstrates karate in a gymnasium . A couple enjoying a glass of white wine . [ROOT] [ROOT] HMCL (ours) Vanilla A young boy is sitting on the floor trying to take off his boots . An Indian girl playing a guitar . A young boy puts on his boots at the top of a flight of stairs . A couple enjoying a glass of white wine . couple enjoying a glass of white wine . Two wet dogs run into the surf at sunset . [ROOT] [ROOT] Appendix H Extended Experimental Protocol and Results H.1 ImageNet–WordNet Protocol and Per-Task Results We use the unified 16-task stream and metric convention defined in the main paper; Appendix G.1 gives the complete task order and dataset statistics. ImageNet-1K [66] uses its official training split and 50,000-image validation split, and each class is associated with a WordNet synset [67]. The hierarchy evaluation additionally reports TIE, LCA, ancestor Jaccard, hierarchical precision, and hierarchical recall. The per-task and pullback analyses below use HyCoCLIP-B. Its cumulative protected basis follows the 80% explained-variance rule, with retained rank after each completed task 2,12,29,29,29,29,32,32,32,32,32,33,33,33,29.2,12,29,29,29,29,32,32,32,32,32,33,33,33,29. For both HMCL variants, we select one fixed configuration on seed 42 using the Overall–BWT trade-off and reuse it for the remaining reported seeds. The pullback analysis below varies only its coefficient after fixing the correction family and all other training choices. Table 9: Final performance for every task in the extended stream (five-seed mean ± sample standard deviation). COCO and Flickr30K use symmetric R@5; all other tasks use top-1 accuracy. The two HMCL rows use the complete post-AdamW task-pullback configurations from Table 1; shaded cells denote HMCL, and boldface marks the best result for each task. Method Aircraft Caltech ImageNet C10 C100 CLEVR COCO Country211 Vanilla 4.94±0.104.94± 0.10 74.26±0.1574.26± 0.15 39.40±0.0639.40± 0.06 87.80±0.1287.80± 0.12 48.52±0.3348.52± 0.33 14.33±0.2214.33± 0.22 48.84±0.2048.84± 0.20 4.48±0.104.48± 0.10 EWC 5.02±0.125.02± 0.12 74.63±0.0874.63± 0.08 40.04±0.0640.04± 0.06 88.17±0.1288.17± 0.12 49.47±0.2949.47± 0.29 14.40±0.2014.40± 0.20 50.17±0.1850.17± 0.18 4.56±0.094.56± 0.09 GEM 4.99±0.194.99± 0.19 75.09±0.2675.09± 0.26 39.25±0.1239.25± 0.12 88.71±0.3188.71± 0.31 50.60±0.4650.60± 0.46 15.00±0.9815.00± 0.98 49.36±0.2149.36± 0.21 4.62±0.094.62± 0.09 C-FLAT 4.83±0.124.83± 0.12 74.03±0.2274.03± 0.22 38.93±0.0138.93± 0.01 87.70±0.1387.70± 0.13 48.87±0.1948.87± 0.19 16.80±0.3816.80± 0.38 49.08±0.1449.08± 0.14 4.60±0.064.60± 0.06 HMCLMR 5.77±0.115.77± 0.11 76.63±0.1476.63± 0.14 45.70±0.0345.70± 0.03 89.52±0.0789.52± 0.07 56.31±0.1156.31± 0.11 24.49±0.8924.49± 0.89 60.12±0.0960.12± 0.09 5.92±0.015.92± 0.01 HMCLCA 5.80±0.085.80± 0.08 76.91±0.1176.91± 0.11 45.59±0.0545.59± 0.05 89.57±0.0889.57± 0.08 56.77±0.2056.77± 0.20 25.86±0.5025.86± 0.50 60.10±0.0960.10± 0.09 6.09±0.046.09± 0.04 Method DTD EuroSAT Flickr30K Food101 MNIST Flowers PCAM SST-2 Vanilla 24.49±0.1724.49± 0.17 41.00±0.6541.00± 0.65 75.23±0.1975.23± 0.19 56.78±0.2056.78± 0.20 28.47±1.2328.47± 1.23 31.68±0.3331.68± 0.33 62.97±0.1762.97± 0.17 54.70±0.3554.70± 0.35 EWC 24.57±0.3024.57± 0.30 40.64±0.8740.64± 0.87 76.57±0.1576.57± 0.15 56.99±0.1456.99± 0.14 28.45±0.9328.45± 0.93 31.92±0.3731.92± 0.37 61.92±0.1661.92± 0.16 54.90±0.2654.90± 0.26 GEM 25.96±0.4425.96± 0.44 47.99±1.5647.99± 1.56 75.81±0.1975.81± 0.19 56.85±0.6056.85± 0.60 30.04±1.4630.04± 1.46 31.87±0.6531.87± 0.65 63.58±0.9163.58± 0.91 55.00±0.2355.00± 0.23 C-FLAT 24.35±0.3924.35± 0.39 40.57±0.7840.57± 0.78 74.94±0.1774.94± 0.17 56.34±0.1656.34± 0.16 27.47±1.1427.47± 1.14 31.13±0.2331.13± 0.23 63.79±0.2263.79± 0.22 53.89±0.3353.89± 0.33 HMCLMR 27.71±0.2827.71± 0.28 55.87±0.5055.87± 0.50 85.48±0.1185.48± 0.11 61.10±0.1161.10± 0.11 29.52±0.4729.52± 0.47 32.74±0.1532.74± 0.15 60.80±0.0860.80± 0.08 55.52±0.0855.52± 0.08 HMCLCA 27.89±0.1927.89± 0.19 56.44±0.3356.44± 0.33 85.45±0.1385.45± 0.13 61.46±0.0661.46± 0.06 30.49±0.5430.49± 0.54 32.48±0.1532.48± 0.15 60.79±0.0960.79± 0.09 55.35±0.1355.35± 0.13 The two HMCL variants attain the highest final score on 15 of the 16 tasks: HMCLCAHMCL_CA leads on ten classification tasks, while HMCLMRHMCL_MR leads on ImageNet-1K, both retrieval datasets, Flowers102, and SST-2. C-FLAT remains strongest on PCAM. This per-task breakdown agrees with the aggregate comparison: after matching both the optimizer-step budget and the global replay capacity, HMCL’s gain is distributed across classification and retrieval rather than being driven by a small subset of tasks. The complete per-run matrices, aggregation outputs, and source provenance accompany the extended experiment artifact. H.2 Modality-Specific Radial Contraction As a modality-specific complement to Figure 6 in the main paper, we examine MERU-L at seed 1024 under the fixed 16-task protocol. We include every dataset with a post-task forgetting interval: all 15 old tasks in the sequence, excluding only the final SST-2 task because no later learning follows it. For each old task, we sample 500 fixed test anchors and compare their spatial norms immediately after learning that task with those at the final checkpoint after the complete sequence. This gives 7,500 image and 7,500 matched text anchors per method. Figure 8 pools these anchors across all 15 old tasks; the dashed lines mark the corresponding pooled task-end medians. Figure 8: Modality-specific spatial-norm distributions after the complete MERU-L continual stream (seed 1024). The distributions pool 500 fixed test anchors from each of the 15 old tasks in the 16-task sequence; SST-2 is excluded only because it is last and has no post-task forgetting interval. Bars show final-checkpoint norms of old-task image and text embeddings, and color-matched dashed lines show their pooled task-end medians. The inset values report the median per-anchor ratio ∥final,space∥2/∥task-end,space∥2 _final,space _2/ _task -end,space _2. Vanilla retains only 0.87×0.87× of the image norm and 0.79×0.79× of the text norm at the median. The contraction appears in the task-level medians for 14 of 15 image tasks and all 15 text tasks. In contrast, HMCLMRHMCL_MR retains 1.00×1.00× and 0.99×0.99×, respectively, and no task-level median falls below 0.9×0.9× for either modality. Thus, the radial component of forgetting is a systematic contraction, especially for text, rather than merely nondirectional displacement. We avoid calling it a collapse to the origin because the Vanilla distributions remain nondegenerate. Scope of the guarantee. The after-AdamW MR correction exactly preserves only the retained subspace represented by t−1V_t-1. Because PCA controls average residual energy rather than the worst individual task, small drift can remain outside that subspace; the correction also leaves AdamW’s moment states unchanged. These limits explain why the empirical hierarchy drift is strongly reduced rather than identically zero. H.3 Stage-Wise Stability–Plasticity Diagnostics Figure 9: Stage-wise stability and plasticity diagnostics replotted from the original CMCL analysis of Liu et al. [63]. At stage t, panel (a) reports the mean deviation of previously learned modality-pair alignment scores from their preceding reference values; deviations close to zero indicate stable old-pair alignment. For t>1t>1, panel (b) reports the mean higher-order term used to assess the source plasticity condition o(η)/η≤0o(η)/η≤ 0; nonpositive values satisfy that condition. These theorem-specific quantities are not BWT or task-score gains. The old-pair alignment deviations remain between −0.074-0.074 and 0.0020.002 and approach zero at several stages. The higher-order terms are nonpositive throughout; after the two largest early-stage terms, their magnitudes fall below 0.060.06. Because this diagnostic uses the original CMCL protocol, it is included here as a theorem-specific consistency check rather than as a result on the current 16-task HMCL benchmark. H.4 Fine-Grained Retrieval on the Modality-Extended Stream Figure 4 in the main paper summarizes final COCO and Flickr30K retrieval in the separate modality-extension protocol, which adds one audio and one thermal retrieval task to the original continual benchmark. Table 10 provides the corresponding text-to-image and image-to-text Recall@1,5,10\1,5,10\ means and standard deviations for all three backbones and methods. This protocol is reported separately from the unified 16-task ImageNet experiment. Table 10: Fine-grained final retrieval performance in the separate protocol augmented with audio and thermal tasks (mean ± standard deviation over three paired seeds). T2I and I2T denote text-to-image and image-to-text retrieval, respectively. The methods and configurations match Table 2. Boldface marks the best result for each backbone and metric; shaded cells denote HMCL. (a) COCO Backbone Method T2I I2T R@1 R@5 R@10 R@1 R@5 R@10 MERU-L Vanilla 9.303±0.0729.303± 0.072 23.844±0.06523.844± 0.065 32.835±0.10832.835± 0.108 15.113±0.11415.113± 0.114 33.633±0.22033.633± 0.220 43.773±0.24743.773± 0.247 HMCLMR 8.848±0.0678.848± 0.067 22.787±0.18322.787± 0.183 31.955±0.24531.955± 0.245 15.847±0.12115.847± 0.121 34.573±0.54934.573± 0.549 45.353±0.18645.353± 0.186 HMCLCA 10.725±0.12010.725± 0.120 26.557±0.14326.557± 0.143 36.344±0.03236.344± 0.032 18.040±0.25018.040± 0.250 37.293±0.23937.293± 0.239 47.913±0.34847.913± 0.348 MERU-B Vanilla 9.491±0.0389.491± 0.038 23.635±0.11323.635± 0.113 32.550±0.19532.550± 0.195 13.067±0.52213.067± 0.522 29.293±0.44029.293± 0.440 38.740±0.59638.740± 0.596 HMCLMR 10.782±0.04510.782± 0.045 26.439±0.04826.439± 0.048 36.166±0.13736.166± 0.137 15.413±0.14015.413± 0.140 34.113±0.09034.113± 0.090 44.540±0.15944.540± 0.159 HMCLCA 12.071±0.06712.071± 0.067 28.605±0.06828.605± 0.068 38.408±0.06738.408± 0.067 17.360±0.21617.360± 0.216 36.973±0.04636.973± 0.046 47.780±0.20347.780± 0.203 HyCoCLIP-B Vanilla 21.074±0.21221.074± 0.212 42.901±0.33442.901± 0.334 54.318±0.10254.318± 0.102 34.900±0.18034.900± 0.180 60.640±0.32260.640± 0.322 71.847±0.15371.847± 0.153 HMCLMR 20.219±0.06320.219± 0.063 42.571±0.26442.571± 0.264 54.019±0.27754.019± 0.277 36.927±0.39036.927± 0.390 62.660±0.27862.660± 0.278 73.660±0.22373.660± 0.223 HMCLCA 23.843±0.13823.843± 0.138 47.056±0.16447.056± 0.164 58.643±0.07758.643± 0.077 40.767±0.25040.767± 0.250 66.833±0.16366.833± 0.163 76.827±0.12976.827± 0.129 (b) Flickr30K Backbone Method T2I I2T R@1 R@5 R@10 R@1 R@5 R@10 MERU-L Vanilla 12.713±0.04612.713± 0.046 29.227±0.26629.227± 0.266 38.187±0.28038.187± 0.280 19.500±0.26519.500± 0.265 39.100±0.17339.100± 0.173 49.933±0.40449.933± 0.404 HMCLMR 11.147±0.13011.147± 0.130 27.200±0.33327.200± 0.333 36.073±0.48936.073± 0.489 21.667±0.30621.667± 0.306 41.067±0.05841.067± 0.058 50.933±0.25250.933± 0.252 HMCLCA 14.600±0.16014.600± 0.160 33.040±0.36533.040± 0.365 43.147±0.08143.147± 0.081 22.633±0.40422.633± 0.404 44.500±0.10044.500± 0.100 54.833±0.15354.833± 0.153 MERU-B Vanilla 11.813±0.06111.813± 0.061 27.773±0.19227.773± 0.192 36.613±0.42836.613± 0.428 15.400±0.26515.400± 0.265 34.633±0.75134.633± 0.751 43.267±0.32143.267± 0.321 HMCLMR 13.967±0.14213.967± 0.142 31.780±0.14031.780± 0.140 41.300±0.30041.300± 0.300 18.733±0.49318.733± 0.493 40.367±0.50340.367± 0.503 50.767±0.60350.767± 0.603 HMCLCA 16.673±0.22716.673± 0.227 35.000±0.08735.000± 0.087 44.833±0.15144.833± 0.151 20.367±0.05820.367± 0.058 43.367±0.56943.367± 0.569 54.567±0.25254.567± 0.252 HyCoCLIP-B Vanilla 45.073±0.39145.073± 0.391 70.940±0.30870.940± 0.308 79.693±0.06179.693± 0.061 64.433±0.80264.433± 0.802 86.100±0.17386.100± 0.173 92.200±0.10092.200± 0.100 HMCLMR 45.860±0.33345.860± 0.333 72.207±0.46972.207± 0.469 81.147±0.22581.147± 0.225 66.433±0.81466.433± 0.814 87.100±0.26587.100± 0.265 92.900±0.26592.900± 0.265 HMCLCA 49.613±0.14749.613± 0.147 76.160±0.22376.160± 0.223 84.240±0.14084.240± 0.140 69.800±0.34669.800± 0.346 89.167±0.35189.167± 0.351 94.000±0.20094.000± 0.200 H.5 Reproducibility Notes Before continual training, we validated the complete cached feature store against the expected sample counts, feature and label shapes, and label ranges. The canonical task order is Aircraft, Caltech-101, ImageNet-1K, CIFAR-10, CIFAR-100, CLEVR, COCO, Country211, DTD, EuroSAT, Flickr30K, Food-101, MNIST, Flowers-102, PCAM, and SST-2. The paired seeds are 42, 123, 456, 789, and 1024. All main results use cached 512-dimensional backbone features. ImageNet stores 1,281,167 image features and one seven-prompt text prototype per class; class prototypes are paired to image features by label inside each minibatch. For classification tasks, AdamW uses learning rate 5×10−45× 10^-4, zero weight decay, 10 epochs, batch size 1024, and entailment weight 0.2. Retrieval tasks use learning rate 5×10−55× 10^-5, zero weight decay, 15 epochs, batch size 1024, and the same entailment weight. “One optimizer step per epoch” means that gradients of all minibatch losses in that epoch are accumulated before a single AdamW update; gradients are cleared only at the epoch boundary. EWC accumulates its regularized loss in the same way. GEM likewise accumulates the current and memory gradients, applies its projection once when the epoch gradient violates a constraint, and then performs one AdamW step. Thus every method receives the same epoch-level parameter-update budget. Let ρ denote retained PCA variance. The fixed HMCL settings are MRCAbackboneρβkminρβkminMERU-L.60.148.65.138MERU-B.70.0758.70.075/.2011HyCoCLIP-B.80.1258.75.151 array[]c|c|c& @intercol @intercol @intercol& @intercol @intercol\\ backbone&ρ&β&k_ &ρ&β&k_ \\ -L&.60&.14&8&.65&.13&8\\ MERU-B&.70&.075&8&.70&.075/.20&11\\ HyCoCLIP-B&.80&.125&8&.75&.15&1 array where the MERU-B CA entry .075/.20 gives classification/retrieval pullback coefficients. These configurations are fixed after selection on seed 42 and then reused without change for the other four seeds. EWC uses coefficient 10−410^-4 and memory size 512. GEM uses margin 0.5, the exact quadprog solver, and a task-balanced global buffer capped at 512 feature pairs; after all 16 tasks it contains 32 pairs per task. C-FLAT uses Co2L contrastive distillation with power 1.0 and memory size 512, perturbation radius ρ=0.2ρ=0.2, and first-order coefficient λ=0.2λ=0.2; AdamW is its wrapped base optimizer, with base AdamW on task 1 and C-FLAT activated from task 2. DNS uses its dual-sided Euclidean null-space projection under the same task order, cached inputs, schedule, and update budget. Each run writes a task-end checkpoint and partial performance matrix, and is accepted only when all 16 tasks are complete and the matrix is a finite 16×1616× 16 lower-triangular array. The experiment artifact retains the per-run configurations, matrices, checkpoints, source hashes, exclusion audit, and aggregation outputs. H.6 Robustness to AdamW Weight Decay The main-table experiments use zero weight decay. To test whether the conclusions depend on this choice, we sweep λ∈0,10−4,10−3,10−2λ∈\0,10^-4,10^-3,10^-2\ on HyCoCLIP-B while holding all other training conditions fixed. The MRpreMR_pre control applies the minimal-rotation correction to the raw gradient before AdamW and uses no pullback. We use the distinct notation MRpreMR_pre because this control is not the complete post-AdamW, pullback-equipped HMCLMRHMCL_MR reported in the main comparison. HMCLCAHMCL_CA uses the fixed complete configuration from the main comparison. Table 11: Robustness to AdamW weight decay on HyCoCLIP-B (mean ± sample standard deviation over five paired seeds). Overall averages the final scores of 14 classification and two retrieval tasks; signed BWT is final minus immediate performance, so higher is better. All conditions share the task order, learning-rate schedule, epochs, batch size, loss, and update budget; only the weight-decay coefficient λ changes. MRpreMR_pre is the raw-gradient, pre-AdamW minimal-rotation control, whereas HMCLCAHMCL_CA uses the complete post-AdamW correction. λ Overall ↑ BWT ↑ Vanilla MRpreMR_pre HMCLCAHMCL_CA Vanilla MRpreMR_pre HMCLCAHMCL_CA 00 43.618±0.15243.618± 0.152 46.400±0.18246.400± 0.182 48.566±0.02448.566± 0.024 −4.186±0.087-4.186± 0.087 −2.929±0.106-2.929± 0.106 −0.456±0.051-0.456± 0.051 10−410^-4 43.618±0.15143.618± 0.151 46.398±0.18146.398± 0.181 48.565±0.02348.565± 0.023 −4.185±0.088-4.185± 0.088 −2.931±0.104-2.931± 0.104 −0.458±0.050-0.458± 0.050 10−310^-3 43.616±0.15143.616± 0.151 46.399±0.18346.399± 0.183 48.565±0.02248.565± 0.022 −4.186±0.090-4.186± 0.090 −2.928±0.106-2.928± 0.106 −0.461±0.054-0.461± 0.054 10−210^-2 43.606±0.15143.606± 0.151 46.384±0.18046.384± 0.180 48.563±0.02548.563± 0.025 −4.191±0.088-4.191± 0.088 −2.940±0.103-2.940± 0.103 −0.459±0.051-0.459± 0.051 Table 11 shows negligible sensitivity throughout the tested range. From λ=0λ=0 to 10−210^-2, the Overall score of MRpreMR_pre remains between 46.384 and 46.400, with BWT between −2.940-2.940 and −2.928-2.928; the Overall score of HMCLCAHMCL_CA remains between 48.563 and 48.566, with BWT between −0.461-0.461 and −0.456-0.456, and it is strongest in every condition. The relatively mild aggregate forgetting of MRpreMR_pre is concentrated in the 14 classification tasks (BWT approximately −2.69-2.69), whereas its two retrieval tasks have BWT approximately −4.65-4.65. Hence this control confirms that minimal rotation already protects much of the old classification geometry, while the complete CA correction gives the more consistent cross-task result. Most importantly, the ordering among Vanilla, MRpreMR_pre, and HMCLCAHMCL_CA is unchanged by AdamW weight decay. H.7 Clean Single-Stage Training Time Figure 10: Clean ImageNet-1K training time at the third continual stage (seed 42). Each bar is one complete synchronized fit_task timing on a common RTX A6000 under the matched schedule; values are seconds. Jobs are serialized, and accepted runs contain no foreign GPU compute PID. We measure method-side cost on ImageNet-1K, the third task after sequentially learning Aircraft and Caltech-101. For every backbone–method pair, seed 42 uses the same cached features, 10-epoch schedule, batch size, and epoch-level optimizer-step budget as the main comparison. The timer brackets the complete fit_task call with CUDA synchronization. It therefore includes cached-tensor transfer to the device, optimization and, for HMCL, geometric correction, pullback, and the method’s task-end state update, while excluding disk cache loading, evaluation, and checkpoint serialization. The nine displayed jobs are run serially on the same RTX A6000. The dispatcher first waits for the preceding experiment queue to exit and then requires six consecutive five-second samples with no compute process, at most 2% utilization, and at most 32 MiB allocated memory. During a run, it records the GPU compute PIDs, utilization, memory, temperature, and power approximately once per second. The presence of any compute PID other than the timed process invalidates that attempt and triggers a fresh idle wait and retry. No displayed attempt was invalidated or retried. Figure 10 shows that Vanilla and the two HMCL variants remain in the same computational regime: all measurements lie between 172.4 and 194.6 seconds. Averaging the within-backbone ratios to Vanilla gives 1.022×1.022× for HMCLMRHMCL_MR and 1.037×1.037× for HMCLCAHMCL_CA. Thus, in this clean single-run diagnostic, the complete HMCL corrections add only a small average cost relative to Vanilla. Because there is one timing run per pair, the differences below 10% should not be interpreted as a statistically resolved ranking. H.8 Pullback Hyperparameter Analysis We vary β only after fixing the post-optimizer MR correction, task order, protected subspace, and all training hyperparameters. The selected setting uses β=0.125β=0.125 from a screening grid spanning 0.05 to 0.25. Relative to the matched β=0.20β=0.20 setting, reducing β to 0.125 improves the Overall score by 0.6% and mean classification accuracy by 1.2%, while retrieval R@5 decreases by 2.1%. It also worsens forgetting, BWT, and final ImageNet retention. The pullback coefficient therefore selects the stability–plasticity operating point and is not treated as a method component in the main ablation.