Paper deep dive

Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 104

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%

Last extracted: 7/5/2026, 2:03:04 AM

Summary

The paper proposes Rank-Aware Hyperbolic Alignment (RAHA), a novel vision-language dataset distillation (VLDD) framework. Unlike existing methods that use Euclidean space and uniform alignment, RAHA utilizes hyperbolic geometry to better capture the hierarchical nature of multimodal semantics. The method decomposes cross-modal correlation into a low-rank 'range' subspace (capturing dominant semantic coupling) and a 'residual' subspace (capturing modality-private diversity). RAHA optimizes a joint objective consisting of a hyperbolic contrastive loss (hITC) and two specialized losses: a range-matching loss to transfer dominant semantic structures and a residual loss to regularize and preserve modality-specific information, ensuring robust transfer and high-quality synthetic pair generation.

Entities (9)

CovMatch · method · 100%EDGE · method · 100%Hyperbolic Geometry · mathematical-concept · 100%Lorentz model · mathematical-model · 100%LoRS · method · 100%MTT-VL · method · 100%Rank-Aware Hyperbolic Alignment · method · 100%RepBlend · method · 100%Vision-Language Dataset Distillation · task · 100%

Relation Signals (4)

Rank-Aware Hyperbolic Alignment → addresses → Vision-Language Dataset Distillation

confidence 100% · We thus propose a rank-aware hyperbolic alignment (RAHA)... for Vision-Language Dataset Distillation

Rank-Aware Hyperbolic Alignment → implements → Hyperbolic Geometry

confidence 100% · RAHA lifts multimodal representations to hyperbolic space

Rank-Aware Hyperbolic Alignment → uses → Lorentz model

confidence 100% · Lorentz model and lifting. Let c > 0 denote the curvature parameter. A point on the Lorentz hyperboloid is...

Rank-Aware Hyperbolic Alignment → improvesupon → LoRS

confidence 90% · LoRS relaxes alignment at the similarity level... We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets. Most existing methods match expert trajectories or cross-modal statistics, yet still enforce full-dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank-deficient image--text correlation, with shared semantics concentrated in a low-dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low-rank factorization, but does not explicitly control dominant alignment capacity and structure in the representation space. We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes distilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality-private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

103,448 characters extracted from source content.

Expand or collapse full text

Rank-Aware Hyperbolic Alignment for Vision–Language Dataset Distillation Jongoh Jeong 1 , Sun-Kyung Lee 2 , and Kuk-Jin Yoon 1 1 Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea 2 Electronics and Telecommunications Research Institute (ETRI), Republic of Korea jeong2, kjyoon@kaist.ac.kr, sklee2014@etri.re.kr Project Page Abstract. Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets. Most existing methods match expert trajectories or cross-modal statistics, yet still enforce full-dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank- deficient image–text correlation, with shared semantics concentrated in a low-dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low-rank factorization, but does not explicitly control domi- nant alignment capacity and structure in the representation space. We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes dis- tilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality-private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets. Keywords: Vision–language·Dataset distillation·Hyperbolic geometry 1 Introduction Vision–language models (VLMs) have become a foundational layer of modern AI, connecting perception with language to support retrieval, captioning, grounding, and multimodal reasoning [2–4,34,38,61,75]. Their rapid progress is driven by large paired image–text datasets and scalable contrastive objectives, amplified by careful data mixture design, filtering, deduplication, and hard-negative effects in batch training. As paired collections grow from millions to billions of pairs [8, 24,36,58,63,73], capability improves, but the associated burdens escalate. These burdens include privacy exposure from unconsented personal data [5,62], uncertain licensing across countries and platforms [42], limited provenance traceability and consent management [43], and vulnerability to data poisoning and content drift [9]. These constraints increasingly shape how multimodal models can be built, audited, shared, and deployed. arXiv:2606.29464v1 [cs.CV] 28 Jun 2026 2Jeong et al. This tension motivates a practical question: can we replace web-scale paired data with compact and auditable surrogates that retain multimodal training utility? Dataset distillation (D) offers a principled direction by synthesizing a small set of examples that approximates the learning signal of a much larger dataset [33,40, 67,74]. Beyond efficiency, distilled surrogates can reduce the footprint of sensitive data, enable controlled redistribution when raw pairs cannot be shared, and accelerate research iteration through cheaper ablations and faster model selection. In this sense, distillation is not only a systems optimization, but also a practical tool for developing VLMs under real constraints. Extending D to vision–language data is harder than the unimodal case as a distilled set must preserve cross-modal relational structure, not only uni- modal diversity. Table 1 organizes prior VLDD methods by the supervision signal extracted from real data and the spaces in which they distill images and text. Existing approaches broadly fall into three families: (i) Trajectory-based methods [70,72,76] match training dynamics but incur high cost and can inherit architectural bias from the teacher; (i) Generative methods [81] leverage diffu- sion priors to synthesize pairs, trading direct control of alignment structure for scalability; and (i) Distribution statistics-based methods [32] match cross-modal moments efficiently, yet they operate in the Euclidean space and still impose a largely uniform alignment constraint across feature directions. Across these families, a common limitation persists: current methods provide limited explicit control over which representation directions should be aligned and which should remain modality-private, especially under the extreme compression. This limitation is consequential because image–text correlation is often effec- tively low-rank [72]. A compact shared subspace captures dominant semantic factors and coarse relations that should align across modalities, while the re- maining directions form a weakly correlated residual component that absorbs modality-specific cues, annotation artifacts, and nuisance variation. Under tight budgets, enforcing alignment uniformly in this residual component can suppress complementary information and harm transfer. Separately, multimodal semantics are naturally hierarchical, spanning entities, attributes, and relations at different abstraction levels. Euclidean geometry offers limited inductive bias for preserving such nested structure when only a small distilled set must carry the training signal. We thus address these limitations altogether by proposing RAHA (Rank- Aware Hyperbolic Alignment), a geometry-aware distillation framework that controls what is aligned and how. As shown in Table 1, RAHA couples hyperbolic contrastive alignment with an explicit range–residual treatment of cross-modal correlation. RAHA estimates an adaptive rank decomposition of real batch cou- pling, matches synthetic relevance in the shared range component, and regularizes the complementary residual component so that weak interactions do not dominate under compression. These objectives complement a base hyperbolic contrastive term that maintains pairwise discriminability on the synthetic set. We evaluate RAHA on standard VLDD benchmarks and analyze retrieval, cross-architecture transfer, robustness to perturbations, and qualitative fidelity of synthesized pairs, RAHA3 Table 1: Comparison in the approach, categorized by source of supervision from real data, latent, image/text spaces each method operates on with the respective focus. MethodSource of supervision Latent space Image space Text spaceApproach/Focus MTT-VL [70] Training Trajectory (Expert) Euclidean PixelEnc. output Matching Training Trajectory LoRS [72]PixelEnc. output + Low-rank similarity RepBlend [76]PixelEnc. output + Modality collapse mitigation EDGE [81]Diffusion Model (SD [56])DM latentCaptionGenerative CovMatch [32]Distribution (Statistics)PixelEnc. input Image-text cross-covariance OursDistribution (Statistics)HyperbolicPixelEnc. inputExplicit subspace decomposition with the goal of understanding when structured relevance distillation is most beneficial. Contributions. We make the following main contributions in this paper: –Rank-aware hyperbolic formulation for VLDD. We formulate VLDD as hyperbolic contrastive learning with tangent-space relevance distillation that exposes an explicit range–residual decomposition of cross-modal correlation. –Range–residual relevance distillation objective. We propose an objec- tive that matches real and synthetic relevance in the shared dominant range component while regularizing the residual component to preserve modality- private variability under tight budgets. – Evaluation with component ablations. We evaluate RAHA on standard retrieval benchmarks and provide ablations that isolate the contributions of hyperbolic contrast, range matching, and residual regularization, alongside analyses of cross-architecture transfer, robustness, and qualitative samples. 2 Related Work Unimodal dataset distillation (D) synthesizes a small surrogate set that preserves the training utility of a much larger corpus [67]. Early kernel-based formulations [47, 48] offer analytical insight but scale poorly. More practical approaches match training signals or feature statistics: gradient matching [77,79], distribution matching [66, 78, 80], and efficient parameterizations of synthetic data [27,31,41,44,83]. In parallel, trajectory matching aligns full optimization paths [10] and has been extended with memory-efficient scaling [13] and im- proved stability [19,23,37,82]. Recent work broadens the toolkit further with implicit-gradient views [45], information-theoretic criteria [59], optimal-transport objectives [14,39], and diffusion-based generation [11,60,65]. We further refer to [12,33,74] for comprehensive surveys. These unimodal advances provide the methodological foundation, but the field is transitioning to the multimodal regime, where distilled sets must additionally preserve cross-modal correspondence. Vision–language dataset distillation (VLDD) extends D to paired image– text data, where a compact synthetic set must maintain both intra-modal diversity and cross-modal alignment. MTT-VL [70] initiated the field with trajectory match- ing and retrieval-based evaluation. LoRS [72] improves efficiency by distilling low-rank similarity structure within the same paradigm. RepBlend [76] addresses 4Jeong et al. multimodal-specific pathologies such as modality collapse through representation blending and balanced supervision. Moving beyond expert trajectories, distribu- tion matching approaches align cross-modal statistics via geodesic kernel energy on a unit hypersphere [25] and cross-covariance [32] with within-modality reg- ularization, where [32] is the first to train the text encoder jointly. In contrast, EDGE [81] leverages diffusion priors to generate pairs with correlation and diver- sity objectives. While these existing methods operate in Euclidean space with uniform alignment pressure, our method is trajectory-free and geometry-aware: we perform rank-aware hyperbolic alignment that decomposes cross-modal corre- lation into a shared range and a residual subspace, selectively enforcing alignment only where it carries semantic content while preserving modality-private diversity. Hyperbolic geometry for vision–language. Hyperbolic spaces are Rieman- nian manifolds with constant negative curvature that embed tree-like hierarchies with low distortion [50,52]. MERU [17] maps image–text features onto the Lorentz model and reveals an interpretable radial layout in which generic concepts lie near the origin, suggesting that the modality gap can reflect abstraction-level differences rather than mere misalignment. Ramasinghe et al. [55] argue that alignment objectives should respect, not collapse, unimodal hierarchies. On the data-centric side, HDD [35] studies hyperbolic centroid matching for unimodal distillation, and recent hyperbolic VL methods exploit entailment-like asymmetry through hierarchy-aware objectives [28,51,53]. We build on these foundations but introduce an explicit range–residual decomposition within hyperbolic space, enabling selective alignment along shared semantic directions while suppressing information-deficient residual components to preserve hierarchical cross-modal structure under extreme compression. 3 Proposed Method We propose Rank-Aware Hyperbolic Alignment (RAHA), a vision–language dataset distillation method that learns a compact synthetic set by optimizing two objectives jointly: (i) a hyperbolic contrastive loss on synthetic image–text pairs, and (i) a rank-aware relevance distillation loss that transfers cross-modal rank structure from real to synthetic data through a range–residual decomposition of batch-level statistics, as highlighted in Table 2. 3.1 Preliminaries and Distillation Protocol Scope and protocol. Dataset distillation seeks a small synthetic set that preserves the training utility of a much larger dataset [10, 67, 78]. In vision– language learning, paired datasets are large and training must preserve cross- modal alignment. Since retrieval-oriented VLMs use contrastive objectives, the distilled set must preserve bidirectional ranking behavior (image-to-text and text-to-image). VLDD is evaluated by training a retrieval model on the synthetic set and reporting Recall@K in both directions. RAHA5 Table 2: Comparison of structural information preserved and missed in the existing methods. Left: matched schematic targets. Right: description of targeted structures versus our proposed rank-aware hyperbolic alignment. ∗ Not in particular order. (a) Matching target structures (b) Matching target by method Method Matched Target Structure CovMatch [32] Cross-covariance (+ per-modality feature reg.) • Preserves: 2 nd -order cross-modal correlation • Misses: Explicit pairwise ranking / higher-order structure LoRS [72] Explicit similarity matrix ̃ S with low-rank factorization • Preserves: Pairwise similarity via low-rank structure • Misses: Geometry- /hierarchy-aware structure Ours Rank-aware image-text relevance by subspace • Preserves: Relative similarity order + hierarchical structure Vision–language dataset distillation. Let the real paired dataset beD real = (x i ,t i ) N i=1 , wherex i ∈Xis an image andt i ∈Tis a caption. We synthesize a compact setD syn =( ̃x j , ̃ t j ) M j=1 withM ≪ Nsuch that training onD syn yields comparable bidirectional retrieval to training onD real . The retrieval model is Ψ=E v (·;θ v ), E t (·;θ t ), π v (·;ω v ), π t (·;ω t ), whereE v ,E t are modality-specific encoders andπ v ,π t are linear projection heads that map encoder outputs to a shared d-dimensional space. For a pair (x,t), the projected embeddings are z v = π v (E v (x;θ v );ω v )∈R d , z t = π t (E t (t;θ t );ω t )∈R d .(1) RAHA operates on these projected features without cosine normalization. Synthetic data parameterization. Each synthetic image is a learnable tensor ̃x j ∈R 3×H×W . Discrete text tokens are not differentiable, so we follow [32] to parameterize each synthetic caption by its input-layer token embeddings and an attention mask: ̃ t j ≡ ( ̃ E j , ̃ m j ),where ̃ E j ∈R L×d e , ̃ m j ∈0, 1 L ,(2) whereLis the maximum sequence length andd e is the token embedding dimension. The mask ̃ m j is fixed at initialization, only marking valid-token/padding positions, while ̃ E j carries learnable linguistic content. This parameterization keepsD syn differentiable with respect to both image pixels and text embeddings. 3.2 Hyperbolic Contrastive Alignment Transition from Euclidean to hyperbolic InfoNCE. Standard contrastive alignment uses InfoNCE with a Euclidean similarity (dot product or cosine) to push matched pairs together and mismatched pairs apart within a batch [54]. Given a batch of paired embeddings(z v i ,z t i ) ̃ B i=1 , Euclidean image–text con- trastive loss constructs logitsℓ ij =s(z v i ,z t j )/τwith temperatureτ >0 and 6Jeong et al. similarity functions, then applies symmetric cross-entropy over both directions. RAHA replaces the Euclidean similarity with a geodesic distance on the Lorentz hyperboloid. We base our intuition from the fact that cross-modal semantics that exhibit a hierarchical structure by which captions refine coarse visual concepts into specific attributes, and the hyperbolic geometry accommodates such nested relations through its exponential geometry [21,49,50]. Lorentz model and lifting. Letc >0 denote the curvature parameter. A point on the Lorentz hyperboloid isu= (u 0 , ̄u)∈R d+1 satisfying−u 2 0 +∥ ̄u∥ 2 2 =−1/c withu 0 >0. Given a tangent vectorw ∈R d at the origin with normr=∥w∥ 2 , the exponential map produces the spatial component Exp c o (w) = sinh( √ cr) √ cr w ∈R d ,(3) and the time coordinate is recovered asu 0 = p 1/c +∥Exp c o (w)∥ 2 2 . Given a scale parameter s > 0, we lift projected embeddings from Eq. 1 to the hyperboloid: h v = Exp c o (sz v ), h t = Exp c o (sz t ). (4) The geodesic distance between two pointsu= (u 0 , ̄u) andv= (v 0 , ̄v) on the hyperboloid is d c (u,v) = 1 √ c arcosh −c⟨u,v⟩ L ,(5) where ⟨u,v⟩ L =−u 0 v 0 + ̄u ⊤ ̄v is the Lorentzian inner product. Hyperbolic image–text contrastive loss (hITC). For a synthetic batch of size ̃ B , we define logits as the negative geodesic distance scaled by temperature τ > 0 (i.e., 0.07): ℓ ij =− d c (h v i ,h t j ) τ . (6) The hITC loss applies symmetric cross-entropy: L hITC = 1 2 ̃ B ̃ B X i=1 " − log exp(ℓ i ) P j exp(ℓ ij ) − log exp(ℓ i ) P j exp(ℓ ji ) # . (7) This loss is computed on synthetic pairs only and does not involve real data. 3.3 Distilling Cross-Relevance via Range and Residual Subspaces L hITC enforces alignment within each synthetic pair but does not transfer the relative cross-modal ranking structure observed in real data. To distill this cross- modal structure, RAHA decomposes the cross-covariance of real tangent-space features into a low-rank range subspace, capturing the adaptive top-k dominant image–text coupling directions, and a complementary residual subspace containing the remaining interactions. Relevance distributions in each subspace are then matched from real to synthetic via entropy-regularized optimal transport with Sinkhorn iterations. The final loss groups each subspace’s matching term with its regularizer, yielding a clean three-term objective: hITC, range, and residual. RAHA7 Tangent-space decomposition. Both real and synthetic projected features are lifted to the hyperboloid via Eq. 3 and mapped back to the tangent space at the origin with the logarithmic map: x v = Log c o (h v ), x t = Log c o (h t ).(8) This round-trip (Euclidean→hyperboloid→tangent space) applies a nonlinear warping controlled bycandsthat concentrates features according to their hyperbolic norm, so that subsequent linear operations respect the underlying geometry. Hyperbolic tangent-space cross-covariance. For a real batchBof size B, letX v ,X t ∈R B×d stack the tangent features row-wise. Define batch means μ v = 1 B P i x v i , μ t = 1 B P i x t i and the cross-covariance C real = 1 B− 1 (X v − 1μ ⊤ v ) ⊤ (X t − 1μ ⊤ t ) ∈R d×d .(9) C syn ∈R d×d is computed identically from the synthetic batch. Real features are treated as stop-gradient targets throughout. Range–residual subspace decomposition. We factorize C real via SVD: C real = UΣV ⊤ , Σ = diag(σ 1 ,...,σ d ), σ 1 ≥·≥ σ d ≥ 0,(10) where each singular valueσ i measures the strength of cross-modal coupling along that direction. We select the effective rankkas the smallest integer whose top-k singular values capture at least a fractionρ∈(0,1) of the total squared energy: k = min ( k ′ : P k ′ i=1 σ 2 i P d i=1 σ 2 i ≥ ρ ) . (11) LetU k ∈R d×k andV k ∈R d×k denote the firstkcolumns ofUandV. These define the range basis:U k spans the image-side coupling directions andV k spans the text-side directions. Given a tangent featurex v ∈R d , its range coordinates and residual component are: a v = x v U k ∈R k , x v res = x v − a v U ⊤ k ∈R d ,(12) and analogouslya t =x t V k ,x t res =x t −a t V ⊤ k on the text side. The basis (U k ,V k ) is computed from the real batch and reused to project both real and synthetic features. Subspace similarity matrices. LetA v ,A t stack the range coordinates and X v res ,X t res stack the residual components for a batch. We form separate cross- modal similarity matrices: G range = A v (A t ) ⊤ ∈R n×n , G res = X v res (X t res ) ⊤ ∈R n×n ,(13) where n is B for real or ̃ B for synthetic data. 8Jeong et al. Row-wise relevance distributions. Each similarity matrix is converted into row- wise probability distributions using a relevance temperatureτ r >0. The raw similarities in Eq. 13 are divided byτ r to form logits, and the softmax within the transport loss (below) divides byτ r again, giving an effective temperature ofτ 2 r : P range = softmax G range real τ 2 r , P res = softmax G res real τ 2 r ,(14) with synthetic counterpartsQ range ,Q res fromG range syn andG res syn . Real distributions P are stop-gradient targets. Entropy-regularized set matching. Given a synthetic row distributionq i (the i-th row ofQ) and a real row distributionp j (thej-th row ofP), we measure discrepancy by KL divergence: D(q i ,p j ) = X m q im log q im p jm . (15) Stacking all pairwise divergences yields a cost matrixΓ ∈R ̃ B×B withΓ ij = D(Q i,: ,P j,: ). Since there is no prescribed one-to-one correspondence between synthetic and real rows, we compute a soft couplingT ∈R ̃ B×B via entropy- regularization [15]: T = arg min T≥0 ⟨T,Γ⟩− εH(T ) s.t. T1 = 1 ̃ B 1, T ⊤ 1 = 1 B 1,(16) whereε >0 is the regularization strength andH(T) =− P i,j T ij logT ij . This is solved by Sinkhorn–Knopp iterations [15] on the Gibbs kernelK ij =exp(−Γ ij /ε). The couplingTis computed under stop-gradient onΓ, so gradients flow only through the KL cost terms. The transport-weighted cost gives a one-directional loss: L → (G real ,G syn ) = τ 2 r X i,j T ij Γ ij ,(17) where theτ 2 r factor compensates for the temperature scaling applied during distribution computation. We enforce bidirectionality by averaging the image-to- text and text-to-image directions: L match (G real ,G syn ) = 1 2 L → (G real ,G syn ) +L → (G ⊤ real ,G ⊤ syn ) . (18) Applying Eq. 18 to range and residual similarities yields L range match and L res match . Range loss. The range loss combines relevance matching with an energy regu- larizer that prevents the synthetic range component from collapsing. Define the projected cross-covariance blocks b C range real =U ⊤ k C real V k and b C range syn =U ⊤ k C syn V k , both inR k×k , and their mean squared energies e real = 1 k 2 ∥ b C range real ∥ 2 F , e syn = 1 k 2 ∥ b C range syn ∥ 2 F . (19) The regularizer penalizes synthetic energy only when it falls below the real energy: L range reg = max(0, e real − e syn ) e real + ε , (20) RAHA9 whereε >0 is a small constant for numerical stability. This one-sided penalty ensures the synthetic data maintains at least as much coupling energy as the real data in the top-ksubspace, without penalizing cases where synthetic energy exceeds the real. The grouped range loss is then: L range =L range match +L range reg .(21) Residual loss. The residual subspace captures cross-modal interactions outside the top-krange. Matching these interactions can sharpen ranking margins, but if residual energy dominates range energy, the distilled data over-represents weakly correlated directions at the expense of the dominant structure. The residual loss thus pairs a matching term with a compression regularizer. We form the residual cross-covariance of synthetic data by projecting out the range on both sides: C res syn = (I − U k U ⊤ k )C syn (I − V k V ⊤ k ) ∈R d×d ,(22) whereIis thed×didentity. We let the residual energy defined ase res = 1 d 2 ∥C res syn ∥ 2 F and the ratior=e res /(e syn +ε), measuring how much synthetic coupling energy lies outside the range. The compression regularizer is defined as: L residual reg = r + max(0, r− 1).(23) The first term provides a steady gradient shrinkingrtoward zero, while the second activates an additional penalty whenr >1 (residual energy exceeds range energy). We then define the grouped residual loss in Eq. 24 whereλ comp controls the relative strength of compression versus matching within the residual subspace. L residual =L residual match + λ comp L residual reg .(24) 3.4 Final Distillation Training Objective The total distillation objective is then defined as: L total =L hITC + λ range L range + λ residual L residual .(25) Here,λ range andλ residual weight the range and residual subspace losses relative to the contrastive term. Expanding Eq. 21 and Eq. 24, the three scalar hyperpa- rameters (λ range ,λ residual ,λ comp ) control four loss components: range matching and its regularizer shareλ range , residual matching is scaled byλ res , and residual compression by λ residual λ comp . By default, λ range =0.8, λ residual =0.4, λ comp =0.1. Training procedure. Only the synthetic parameters ( ̃x j , ̃ E j ) are updated by gradient descent by Eq. 25. The encoder weights (θ v ,θ t ,ω v ,ω t ) are initialized from pretrained checkpoints and held fixed during the synthetic update step. Following the online distillation protocol of [32], each distillation iteration consists of multiple outer-loop steps. Within each outer step: (1) a real batch is sampled and its features are computed with stop-gradient, (2) a synthetic sub-batch of size ̃ B ≤ Mis sampled , (3) distillation loss is computed and backpropagated through the synthetic pathway only, and (4) the synthetic parameters are updated by SGD. For each outer-loop step, the network is updated for one inner-loop step on real data, providing a slowly evolving latent feature landscape for enhanced generalization in the next outer step. Alg. 1 summarizes the procedure. 10Jeong et al. Algorithm 1 Distilling data with Rank-Aware Hyperbolic Alignment Require: Real dataset D real , synthetic budget M, curvature c, scale s, outer steps N out , inner steps N in , iterations T, batch sizes B, ̃ B 1: Initialize D syn =( ̃x j , ̃ E j , ̃ m j ) M j=1 from real samples 2: Initialize encoder Ψ from pretrained weights 3: for t = 1 to T do 4: for ℓ = 1 to N out do 5:Synthetic update: 6:Sample real batch B and synthetic sub-batch ̃ B 7:Compute projected features (z v ,z t ) for both batches via ΨEq. 1 8:Lift synthetic features to hyperboloid via Exp, compute L hITC Eqs. 4–7 9:Re-map all features to tangent space via LogEq. 8 10:Compute C real ,C syn , SVD of C real , and rank kEqs. 9–11 11:Project into range and residual, form G range ,G res Eqs. 12–13 12:Compute relevance distributions and Sinkhorn couplingEqs. 14–16 13:Compute L range and L res Eqs. 21, 24 14:Update D syn by SGD on L total Eq. 25 15:Model update: Train Ψ for N in steps on real data 16: end for 17: end for 18: return D ∗ syn 4 Experiments 4.1 Experimental Setup Datasets and splits. We evaluate multimodal dataset distillation on three canoni- cal image–caption benchmarks for bidirectional retrieval: Flickr8k [24], Flickr30k [73], and MS COCO [36]. We use the standard retrieval splits popularized by Karpathy et al. [26]. Unless otherwise noted, COCO results are reported on the 5k-image test split (rather than the 1k subset), which is the most common setting in recent MDD evaluations. Dataset statistics are summarized in Table 3. Table 3: Dataset statistics for image–text retrieval. Splits follow the standard retrieval protocol [26]. Each image is annotated with five human-written captions. DatasetYear #Images (Train/Val/Test) Caps/img Caption characteristics Flickr8k [24]20136k / 1k / 1k5Short, single-sentence human captions describing everyday scenes; relatively clean and small-scale. Flickr30k [73] 201429k / 1k / 1k5Crowd-sourced, concise descriptions with broader visual diversity (people, objects, actions, relations). MS COCO [36] 2014113k / 5k / 5k5 Large-scale, object-centric captions with richer compositionality and vocabulary; the most diverse of the three. Task and metrics. We use bidirectional image–text retrieval as the main probe of cross-modal alignment. We report Recall@K forK ∈ 1,5,10in both directions: text-to-image (T→I, denoted IR@K) and image-to-text (I→T, denoted TR@K). Retrieval similarities are computed by dot product in the shared embedding space. For fair comparison, we evaluate all methods using the same similarity function. RAHA11 Synthetic budget. We distill a compact synthetic set ofNimage–text queries withN ∈100,200,500. Each query consists of (i) one image of size 3×224×224 optimized in pixel space and (i) one continuous 768-dimensional text embedding optimized directly in embedding space (i.e., we synthesize embeddings rather than discrete tokens), following the standard embedding-level MDD setup [70,72]. These budgets correspond to strong compression regimes (e.g.,N =100 is below 1% of COCO training images). Baselines and network architecture. We compare RAHA to (i) coreset selection under the same budget (Random, Herding, K-Center, Forgetting) [20, 64, 69], and (i) vision–language distillation methods, including trajectory-matching approaches ( [70], [13], [72], [76]) and distribution/statistics matching [32]. All methods use the same budget, architecture, and evaluation protocol. We use the network composed from NFNet [7] and BERT [18], each followed by a lightweight linear projection head to a sharedd-dimensional embedding space [32,70,72,76]. Distillation protocol. We optimize the synthetic images and synthetic text em- beddings while using an online surrogate model to compute distillation gradients. Each outer distillation iteration alternates between two stages: (i) an offline synthetic update stage, where we updateD syn forN out (e.g., 50) gradient steps using the current model state, and (i) online model update stage forN in (e.g., 1), where we update the trainable retrieval model for one step on real data. This alternating schedule intentionally varies the surrogate model state during distilla- tion, exposing the synthetic set to a shifting encoder geometry and improving the generality of the distilled pairs across latent-space configurations. RAHA computes its relevance matching terms in the hyperbolic representation induced by Lorentz lifting and tangent-space operations, and we allow a structured radial modality gap, consistent with the observation that text concepts are often more generic than images [17]. After distillation, we train a retrieval model from scratch on the distilled set for 100 epochs and evaluate on the real test split. 4.2 Main Results Image–Text Retrieval. We report bidirectional retrieval performance on Flickr8k, Flickr30k, and COCO for budgetsN ∈ 100,200,500in Table 4. Across all datasets and budgets, RAHA consistently outperforms coreset selection under the same budget, highlighting the benefit of optimizing synthetic pairs rather than selecting real pairs. Compared to trajectory-matching baselines (MTT-VL, TESLA, LoRS, RepBlend), RAHA is competitive across budgets while operating in a distribution-matching regime that does not require storing expert trajectories. On Flickr8k, RAHA scales strongly withNand attains the best overall mean recall atN =500, indicating that subspace-conditioned relevance matching preserves retrieval-relevant ranking structure even under extreme compression. On Flickr30k and COCO, RAHA remains comparable to strong trajectory-based baselines and improves with increasingN, suggesting 12Jeong et al. Table 4: Image-text retrieval for 100, 200, and 500 synthetic pairs using the coreset methods (left) and representative distillation methods (right). The compression rate for Flickr8k, Flickr30k, and COCO datasets are approximately 1.7%, 0.3%, 0.8‰, 3.3%, 0.7%, 1.7‰, 8.3%, 1.7%, 4.4‰ for 100, 200, and 500 pairs, respectively. Coreset samplingDistillation methods Matching Training Trajectories (MTT)Distribution Matching (DM) RandomHerding [69] K-Center [20] Forgetting [64] MTT-VL [70] TESLA WBCE [13] LoRS [72] CovMatch [32]Ours Data # Pairs IR TR Mean IR TR Mean IR TR Mean IR TR Mean IR TR Mean IR TR Mean IR TR Mean IR TR Mean IR TR Mean Flickr8k 100 3.5 7.85.7 4.7 8.36.5 5.0 8.66.8 4.1 4.95.1 3.9 6.25.1 4.6 14.19.4 17.3 21.519.4 18.4 22.420.4 19.0 21.920.4 200 7.8 11.89.8 5.5 11.98.7 8.8 12.810.8 6.6 9.48.0 7.0 10.18.6 4.8 18.511.7 19.5 24.722.1 17.4 20.318.8 22.7 27.925.3 500 12.6 18.1 15.4 12.0 16.314.1 12.8 17.915.4 15.1 19.717.4 12.7 17.315.0 8.5 18.513.5 20.7 29.225.0 23.8 28.025.9 27.7 33.630.7 Flickr30k 100 7.4 9.98.6 7.9 9.58.7 7.5 9.68.6 2.9 5.04.0 15.0 25.820.4 2.5 18.010.2 22.5 32.327.4 20.9 24.622.8 18.7 22.720.7 200 11.1 15.113.1 10.9 14.312.6 10.7 14.612.6 4.2 6.75.5 15.4 26.921.2 1.3 10.25.8 23.5 35.529.5 20.2 23.722.0 23.5 27.925.7 500 19.7 25.622.6 19.5 24.221.8 20.4 26.923.7 8.9 11.710.3 18.9 31.025.0 7.0 14.710.9 26.8 36.331.6 26.3 31.528.9 30.0 35.932.9 COCO 100 2.9 4.03.4 3.0 4.03.5 3.3 4.23.8 1.4 2.72.1 5.4 9.47.4 1.0 7.74.4 7.0 11.79.4 6.5 7.47.0 6.6 7.77.2 200 4.7 6.45.6 4.8 6.55.6 5.1 6.75.9 2.8 3.93.3 6.8 11.59.2 0.3 3.01.7 9.1 13.711.4 7.4 9.28.3 9.3 11.010.2 500 8.6 11.510.1 8.6 11.09.8 8.9 12.010.5 4.9 7.86.4 9.1 16.112.6 3.7 5.94.8 9.7 17.213.5 9.9 8.311.2 12.6 14.913.7 A man stares at a lady Two people in rafting gear standing on a dry, rocky riverbank, pointing at the river A surfer jumps a wave as one paddles to the wave Girl touches young man’s face as young man in pink shirt observes, leafy background Eight people are standing on a hill above the clouds A child is jumping off a platform into a pool A little girl touches the face of an adult woman Two people in rafting gear standing on a dry, rocky riverbank, pointing at the river A surfer rides a medium sized wave A dog runs through the grass A black dog plays with a toy on the grass The large beige dog is running through the grass A little brown dog runs through the grass A white dog sits on a rocky lawn Two tan dogs play with a blue toy in a green field of grass A brown and white dog runs through the green grass A white dog wearing a blue collar runs for a green ball A tan dog rolls in the grass Fig. 1: Qualitative synthesized pairs. Representative samples at initialization (left), after CovMatch (middle), and after RAHA distillation (right). Please zoom in for details and view in color. that relevance distillation benefits from additional synthetic capacity on more diverse datasets. See comparison with EDGE [81] in Appendix. Relative to the strongest distribution/statistics matching baseline, Cov- Match [32], RAHA trades some absolute retrieval performance in the smallest- budget regime for a different inductive bias. CovMatch directly aligns Euclidean cross-covariances and feature-level statistics, which is highly effective whenN is extremely small. RAHA instead matches row-wise relevance distributions af- ter extracting a rank-adaptive range/residual decomposition from real batch structure, explicitly controlling how alignment capacity is allocated across cou- pled and weakly-coupled directions. This distinction is reflected most clearly in cross-architecture transfer and robustness (Table 5), and is further supported by qualitative differences in the synthesized pairs (Fig. 1). Qualitative comparison. Fig. 1 contrasts representative synthesized pairs. Cov- Match outputs often retain visually salient residual artifacts such as high- frequency speckling and banding, even when coarse layout is plausible. Such artifacts are consistent with satisfying global second-order alignment through structured noise that perturbs features without improving perceptual realism. In contrast, RAHA yields cleaner textures and more natural edges, suggesting that range–residual relevance matching suppresses spurious degrees of freedom that manifest as visible artifacts. RAHA also better preserves fine-grained image–text RAHA13 Table 5: Comparison on Flickr8k [24]: (i) Cross-architecture generalization aver- aged over IR/TR@K=1,5,10. Source-model results marked with ‘ ∗ ’ are not averaged; best average results are in boldface, and runner-up inunderline. (a)–(d): NFNet, NF-ResNet, NF-RegNet, ViT-B. (i) Robustness to noise (δ) by modality: (a)–(c): JPEG [22] (75%)/Bit Quantization [71] (4-bit), Additive White Gaussian Noise (AWGN) (σ = 0.01), 10-PGD [46] (step size=2). # Pairs (i) Cross-architecture generalization(i) Robustness to noise Text ε BERT [18]DistilBERT [57]Image-side δText-side δ Image ε (a) (b) (c) (d) (a) (b) (c) (d)Mean (a) (b) (c)Mean (a) (b) (c)Mean 100 CovMatch [32] 20.4 ∗ 6.4 6.3 7.7 9.6 6.8 6.3 7.87.3 6.0 6.0 1.94.6 3.8 10.1 0.04.6 Ours20.4 ∗ 5.15.46.39.96.76.78.06.94.95.61.84.13.18.10.03.7 200 CovMatch [32] 18.8 ∗ 7.2 6.3 6.7 9.3 7.2 6.7 7.27.2 5.0 5.7 1.94.2 3.3 8.7 0.04.0 Ours25.3 ∗ 7.47.29.110.58.07.910.88.75.86.82.45.03.69.80.04.5 500 CovMatch [32] 25.9 ∗ 8.1 7.3 8.9 12.3 8.4 7.3 9.18.7 8.0 8.3 3.06.4 3.9 11.5 0.05.1 Ours30.7 ∗ 5.411.314.615.412.512.816.912.78.09.53.06.84.415.00.06.4 semantics in several cases, where CovMatch drifts to mismatched captions (e.g., incorrect scenes/actions), while RAHA retains the correct relational content and discriminative attributes. Overall, the qualitative evidence aligns with RAHA’s design goal: preserve dominant coupled structure while explicitly controlling weakly-coupled residual interactions. 4.3 Architectural Transfer and Robustness To test whether distilled pairs capture transferable cross-modal relevance rather than overfitting the source encoders used during distillation, we follow a cross- architecture protocol. We distill with a fixed source configuration and then retrain/evaluate retrieval models that replace either the image backbone or the text encoder while keeping the distilled set unchanged. We also evaluate robustness via the degradation metricδunder common perturbations applied to either modality. Across budgets. AtN =100, RAHA is competitive with CovMatch on cross- architecture transfer while improving robustness to both image- and text-side perturbations, reducing the average degradationδon both modalities. AtN =200, RAHA yields a clear gain in mean transfer (from 7.2 to 8.7), with improvements that are consistent across both BERT and DistilBERT targets and across vision backbones. AtN =500, transfer and robustness become less capacity-limited, and both methods generally improve; importantly, RAHA remains competitive in this higher-budget regime, indicating that the relevance structure distilled by RAHA does not rely on a single encoder geometry and continues to generalize as synthetic capacity grows. Taken together, transfer improves strongly with N, while robustness does not improve uniformly at the highest budget. These trends are consistent with the intended role of RAHA: distilling relevance in a rank-adaptive shared range while regulating residual interactions, which can improve stability under perturbations and reduce overfitting to a particular encoder configuration. 14Jeong et al. Fig. 2: Ablation study for Flickr8kN=100 setting with each component added, demon- strating the synergy of the two subspace losses. 5 Auxiliary Discussion Ablation study. Using only hyperbolic contrastL hITC provides a strong baseline, confirming that geodesic InfoNCE on synthetic pairs already yields retrieval-relevant alignment (Fig. 2). Adding the range relevance term further improves mean retrieval, supporting the role of the range component as the dominant carrier of cross-modal coupling distilled from real data. Using the residual term alone is weaker, consistent with residual interactions being harder to match reliably without anchoring to the dominant coupled structure. Combining range and residual relevance matching with their subspace regularizers yields the best performance, indicating that residual matching is most effective when explicitly controlled rather than optimized in isolation. Distillation cost. For completeness, we report a coarse per-iter wall-clock breakdown on an RTX A6000 into (i) data loading, (i) model init., and (i) the synthetic update (distillation step), as a complementary metric that contextualizes the computational profile of each objective. With identical data (0.67s) and model (0.3s) initialization, the main difference arises in the distillation step: at a batch size of 1, both are similar at≈25 s, while the difference grows with batch size due to lifting and decomposition particularly (e.g., at 64,≈55 s vs.≈400 s), reflecting the stronger batch-scaling cost of RAHA’s structure-aware update. We do not use runtime as the primary comparison axis, since our evaluation focuses on retrieval accuracy, transfer, and robustness under fixed synthetic budgets. In sum, RAHA distills structured image–text relevance through a rank- aware shared subspace. It does not claim uniform dominance in every extreme- compression cell. RAHA is competitive at100pairs and outperforms [32] at 200/500. Here, RAHA brings to the surface an important difficulty in VLDD: dataset-scale semantics can be abstracted into decomposed range/residual bases, but a very small synthetic set may not have enough capacity to materialize those semantic modes. At extreme rates, e.g., on Flickr30k/MS COCO, the selected basis can be stable, but too few pairs may not populate the decomposed structure. Thus, MTT-style [72] can remain strong at the smallest budgets. As # pairs↑, RAHA better materializes these modes. Structurally, RAHA uses the conceptual text hierarchy [17,28] as an inductive bias for VLDD, since hyperbolic geometry can keep generic concepts close to many descendants while keeping specific instances separable. RAHA15 On transfer and robustness comparison. [76] builds on the MTT-based [72] line by addressing modality collapse, whereas RAHA follows the distribution- matching line of [32], with hyperbolic range–residual relevance modeling in a methodologically complementary, rather than mutually exclusive, direction. # PairsMethod Trainable Text Enc. Cross-arch (Mean) Image-δ (Mean) Text-δ (Mean) 100/200/500 LoRS [72]✗10.2 / 10.8 / 11.00.5 / 0.5 / 0.5 0.5 / 0.5 / 0.6 RepBlend [76]✗8.1 / 8.9 / 10.70.6 / 0.5 / 0.5 0.5 / 0.6 / 0.5 CovMatch [32]✓7.3 / 7.2 / 8.74.6 / 4.2 / 6.4 4.6 / 4.0 / 5.1 Ours✓6.9 / 8.7 / 12.74.1 / 5.0 / 6.8 3.7 / 4.5 / 6.4 Rank selection & subspace stability. RAHA selects the effective rank not manually but based on the smallestkwhose cumulative squared singular-value energy ofC real =UΣV ⊤ reachesρ, with numerical-rank clipping andk max (Eq.11, §A3.2). This defines the range as the minimal energy-bearing image–text coupling subspace and treats the complement as residual structure. Fig. 3(a) shows that larger real batches improve retrieval by stabilizing the cross-covariance estimate, rather than causingkto simply follow the algebraic ceilingrank(C real )≤ B−1. Fig. 3(b) shows thatρ=0.95 is the best energy threshold, while 1.0 degrades by absorbing the low-energy residual tail. Fig. 3(c) further shows that the selected rank remains stably within a dataset-specific bound over iterations, supporting stable batch-wise SVD in practice. 15 20 Fig. 3: Batch, hyperparameter ρ ablations (a,b) and rank by dataset (c). 6 Conclusion In this work, we present a hyperbolic geometry-aware objective for VLDD, termed RAHA, that lifts image–text representations to hyperbolic space and enforces rank-consistent cross-modal relevance matching, improving stability and cross-architecture transfer under tight compression budgets. Limitations. As with [25,32,70,72,76], RAHA remains bounded by the expres- sivity of chosen teacher encoders, and performance may degrade under domain- shifted or noisy captions. RAHA is driven by inherent hierarchical structures, and thus datasets with weak hierarchy may see limited gains (§A5). Disclosure of LLM Use. LLM was used only for editing (e.g., formatting) and did not contribute to the underlying technical ideas or experimental results. Acknowledgments This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2024-00457882, AI Research Hub Project) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2026-25473963). We thank Jaehyuk Jang for providing key insights. Rank-Aware Hyperbolic Alignment for Vision–Language Dataset Distillation — Appendix — Jongoh Jeong 1 , Sun-Kyung Lee 2 , and Kuk-Jin Yoon 1 1 Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea 2 Electronics and Telecommunications Research Institute (ETRI), Republic of Korea jeong2, kjyoon@kaist.ac.kr, sklee2014@etri.re.kr Project Page This appendix provides additional analysis, implementation details, and extended experiments that support the paper, to be added after the main text. Specifically, it covers five parts: the scope of image–text dataset distillation, motivation and additional details of RAHA, experimental details, further analyses and discussion, and broader societal impact as follows: 1. Scope of image–text dataset distillation (Sec. §A1) 2. Motivation and additional method details (Sec. §A2) 3. Experimental details (Sec. §A3) 4. Further analyses and discussion (Sec. §A4) 5. Broader societal impact (Sec. §A5) A1 Scope of Vision-Language Dataset Distillation In this section, we clarify the scope of image–text dataset distillation relative to unimodal condensation, and position RAHA as a method for cross-modal relevance compression that prioritizes the modality-shared structure. Scope. Image–text dataset distillation aims to compress a large paired training set into a much smaller synthetic set that remains effective for learning cross- modal models. In our setting, a model is trained from scratch on the distilled set and evaluated primarily by bidirectional image-text retrieval, with additional evaluation on prompted image classification in Sec. A3.5. Unlike unimodal dataset distillation, success depends not only on preserving within- modality diversity, but also on retaining the cross-modal alignment structure that determines which captions should be retrieved for an image and which images should be retrieved for a caption. We further examine whether this distilled structure transfers across architectures and remains robust under noise. The goal is therefore not merely dataset compression, but preservation of the shared structure most relevant for retrieval and generalization under a limited synthetic budget. Existing image–text distillation methods preserve this structure only indirectly. Trajectory-based approaches transfer optimization dynamics from real-data train- ing, which can improve fidelity but require storing long trajectories and do not RAHA: Appendix17 explicitly identify which components of image–text correspondence matter most. Distribution-based approaches are more storage-efficient, but typically operate in Euclidean feature space and align statistics more uniformly across representation directions. As a result, they may preserve broad feature similarity without suf- ficiently emphasizing the dominant shared relations most critical for retrieval. Generative approaches provide another route to scalable multimodal compression, but they offer less direct control over how image–text correspondence is preserved in the final distilled set. Across these families, the common limitation is that, under strong compression, not all directions in the joint representation space are equally informative, yet most existing objectives do not explicitly prioritize the most retrieval-relevant shared structure. Our approach. RAHA is designed for this setting. Our starting point is that image–text correspondence is structured rather than isotropic: some directions encode dominant shared relevance, whereas others reflect weaker or noisier residual interactions. To model this more effectively, we redesign the alignment space through a hyperbolic lifting, using geometry as an inductive bias for the non-uniform semantic structure often present in image–text data. We do not assume that paired image–text data form a literal tree; rather, the hyperbolic space provides a more suitable geometry for preserving uneven shared structure than a purely Euclidean formulation. Within this lifted space, RAHA emphasizes the dominant shared component of cross-modal relevance while regulating weaker residual structure, so that limited synthetic capacity is used more effectively for retrieval, transfer, and robustness. From this perspective, image–text dataset distillation is best viewed as a problem of cross-modal relevance compression. The objective is not to reproduce every aspect of the original multimodal feature distribution, but to preserve the shared structure that is most important for efficient and transferable retrieval learning in the smallest possible synthetic set. A2 Motivation and Additional Method Details In this section, we describe our motivations for the two core design decisions: why RAHA introduces hyperbolic lifting, and why it transfers only the information- rich shared structure. RQ: Why introduce hyperbolic lifting? Image–text semantics often have a hierar- chical character [17]. Captions move across levels of abstraction, from scene-level concepts to activities, attributes, and fine-grained object relations. This structure is not naturally flat. Hyperbolic geometry provides a more suitable inductive bias because it can represent coarse-to-fine variation more naturally than Euclidean similarity alone. In RAHA, hyperbolic lifting is not intended to impose a literal tree. Its role is to provide a geometry that better accommodates hierarchical semantic variation. Hyperbolic lifting is also motivated by the fact that image and text need not coincide even when they are semantically matched. Captions are often more abstract than the images they describe. A geometry that allows structured radial 18Jeong et al. separation is therefore preferable to one that encourages all matched pairs to collapse into a single flat region. This is especially important in distillation, where aggressive collapse can remove useful structure when only a small number of synthetic pairs is available. RQ: Why decompose features in the hyperbolic tangent space? Hyperbolic ge- ometry provides the right inductive bias, but decomposition is more stable and interpretable in the tangent space. RAHA therefore performs the decomposition on tangent-space features induced by hyperbolic lifting. This retains the geometry of the lifted representation while allowing dominant and residual directions to be estimated with standard linear operations. The reason for this decomposition is intuitive. Not all image–text interactions are equally useful for distillation. The strongest shared directions usually carry the most reliable retrieval signal, while the weaker complement often contains low-energy, unstable, or modality-specific structure. Under strong compression, matching all of these interactions equally can waste synthetic capacity and reduce transferability. Logarithmic map at the origin. The tangent-space decomposition in §3.3 relies on the logarithmic mapLog c o , the inverse of the exponential map defined in Eq. 3. Given a point on the hyperboloid with spatial component ̄u ∈R d , the time coordinate isu 0 = p 1/c +∥ ̄u∥ 2 2 . The logarithmic map at the origin recovers the tangent vector: Log c o ( ̄u) = arcosh(u 0 ) √ c∥ ̄u∥ 2 ̄u ∈R d .(A.1) This maps hyperboloid points back to the tangent plane, where Euclidean linear algebra (centering, SVD, projection) can be applied. The round-trip compositionLog c o ◦ Exp c o is not the identity: it applies a radial reweighting f(r) =arcosh(cosh( √ cr))/( √ cr) that compresses large-norm features more than small-norm ones, concentrating the distribution near the origin in a curvature- dependent manner. In the limitc→0, this reweighting approaches the identity and the tangent features reduce to the original Euclidean embeddings. RQ: Why prioritize distilling the information-rich shared structure? RAHA is built on the intuition that distilled pairs should preserve useful shared structure, not full feature correspondence. The dominant shared component carries the most stable image–text relevance information. Matching this component directly encourages the synthetic set to preserve retrieval-oriented structure. In contrast, uniformly matching weak residual directions can amplify noise or modality-specific artifacts that do not help ranking. This selectivity is most important when the synthetic budget is very small. A compact synthetic set cannot reproduce every degree of freedom in the real data. It should therefore prioritize the dominant shared signal and regulate weaker residual structure. This is the central intuition behind RAHA. Hyperbolic lifting provides a geometry aligned with hierarchical semantics, and tangent-space decomposition provides a practical way to preserve the most retrieval-relevant shared structure. RAHA: Appendix19 Relation to prior methods. From this perspective, RAHA differs from methods that match Euclidean second-order statistics more uniformly across feature space, and from methods that emphasize low-rank similarity without explicitly separating dominant shared directions from weaker residual ones. The advantage of RAHA is not only its geometry. It is its selective allocation of alignment capacity to the most informative shared structure. Notation summary. For reference throughout the paper and the appendix, we organize and report key notations in Table A1 below. Table A1: Summary of notations. NotationDescription D real , D syn Real and synthetic paired datasets N, MNumber of real and synthetic pairs B, ̃ BReal and synthetic batch sizes E v , E t Image and text encoders π v , π t Image and text projection heads z v , z t Projected embeddings (∈R d ) h v , h t Hyperboloid spatial components after lifting x v , x t Tangent-space features after Log map cLorentz curvature parameter sLifting scale parameter τhITC temperature τ r Relevance temperature C real , C syn Cross-covariance matrices (∈R d×d ) U k , V k Range basis matrices (∈R d×k ) ρEnergy threshold for SVD rank selection kEffective coupling rank a v , a t Range coordinates (∈R k ) x v res , x t res Residual components (∈R d ) G range , G res Range and residual similarity matrices ΓCost matrix TSinkhorn coupling matrix εSinkhorn entropic regularization εNumerical stability constant (10 −8 ) λ range , λ residual , λ comp Loss weighting factors A3 Experimental Details In this section, we provide full implementation details, hyperparameter settings, dataset-selection rationale, extended quantitative comparisons including a repro- ducibility analysis of the baseline [32], and prompted classification evaluation with their qualitative results. 20Jeong et al. A3.1 Workbench Environment All experiments were conducted on a Linux workstation with an Intel Xeon Silver 4210R CPU and a single NVIDIA RTX A6000 GPU. The software environment used Python 3.10, PyTorch 2.6.0, and Torchvision 0.21.0, with CUDA 11.8 and cuDNN 9.1.0. Unless otherwise specified, all compared methods follow the same default training setup, including batch sizes, optimizer parameters, and evaluation protocol as in [32] for fair comparisons. For RAHA, we use the same settings across datasets and synthetic budgets, unless explicitly stated otherwise. A3.2 Implementation Details We follow the standard image–text distillation setup in which synthetic images are optimized directly in pixel space, while synthetic text is optimized in continuous embedding space. Each distilled query consists of one 3×224×224 synthetic image and one 768-dimensional text embedding. The retrieval model uses an NFNet image encoder and a BERT text encoder, each followed by a trainable projection head into a shared embedding space. Unless otherwise stated, all experiments follow the distillation-evaluation protocols as in [32], including finetuning text encoder layers. As with existing methods [32,70,72,76], synthetic images and synthetic text embeddings are up- dated by an SGD [6] optimizer with learning rate 1, respectively, and momentum 0.5. The online retrieval model is also optimized with SGD: the image encoder and text encoder use a learning rate of 0.01, while the image and text projection heads use a learning rate of 0.1. We perform at most 200 distillation iterations, with outer-/inner-loop steps of 50/1, respectively. The distilled set is evaluated every 50 iterations, and each evaluation reports the average over 5 runs. After distillation, a retrieval model is trained from scratch on the distilled set for 100 epochs and evaluated on the real test split. The default optimization and evaluation settings used throughout the experiments are summarized in Table A2. RAHA-specific hyperparameters. Table A3 lists the RAHA-specific hyper- parameters used throughout the experiments. These values are fixed across all datasets and budgets unless stated otherwise. In particular, we distinguish two small constants used in the paper,εandε. The symbolε= 0.05 denotes the entropic regularization strength in the Sinkhorn transport solver (Eq. 16), which controls the smoothness of the coupling matrix T. The symbolε= 10 −8 is a numerical stability floor used in denominators and clamping operations. Scope of baselines. RepBlend [76] operates within the trajectory-matching paradigm, inheriting expert-trajectory construction and LoRS-style low-rank sim- ilarity learning. It is therefore not a direct counterpart to distribution-matching methods such as CovMatch or RAHA. We include LoRS [72] as the represen- tative trajectory-based baseline with explicit low-rank modeling. RepBlend’s contributions are orthogonal and could in principle be combined with distribution- matching frameworks. RAHA: Appendix21 Table A2: Default optimization and evaluation settings used throughout the experi- ments unless otherwise specified. HyperparameterValue Synthetic optimizerSGD lr image / text1.0 / 1.0 Synthetic momentum0.5 Online optimizerSGD lr enc image / proj0.01 / 0.10 lr enc text / proj0.01 / 0.10 Online momentum0.9 Weight decay 5× 10 −4 Max iterations200 Outer / inner loop iteration 50 / 1 Batch size64 / 64 Eval runs5 Eval train epochs100 Table A3: RAHA-specific hyperparameters used in all experiments. HyperparameterNotation Value Lorentz curvaturec1.0 Lifting scales1.0 hITC temperatureτ0.07 Relevance temperatureτ r 0.07 Energy threshold (rank selection)ρ0.95 Sinkhorn regularizationε0.05 Sinkhorn iterations–20 Numerical stability constantε10 −8 Range loss weightλ range 0.8 Residual loss weightλ residual 0.4 Compression weight (within residual) λ comp 0.1 Synthetic data initialization. By default, we initialize the synthetic set by samplingMimage–text pairs uniformly at random fromD real . Each synthetic image ̃x j is initialized as the pixel tensor of a randomly selected real image, and each synthetic text embedding ̃ E j is initialized by passing the corresponding real caption through the text encoder’s embedding layer including positional encoding, before the Transformer encoder layers. Model initialization. At the start of each distillation iteration, the encoder Ψis reset to its pretrained weights before the outer loop begins. This follows the offline-online training protocol practiced in CovMatch [32] and prevents the encoder from drifting into a configuration that overfits the current synthetic set. Without this per-iteration reset, the encoders can adapt to exploit synthetic- specific artifacts (e.g., high-frequency texture patterns), potentially yielding a distilled set with poor transferability when a fresh encoder is trained from scratch 22Jeong et al. during evaluation. The reset ensures that the distillation gradient is computed under a diverse set of encoder states (as the inner-loop updates produce a slightly different trajectory each iteration), which encourages the synthetic data to generalize across encoder configurations. Numerical stability of Singular Value Decomposition (SVD). The tangent-space cross-covarianceC real can be ill-conditioned, particularly early in training or with small batch sizes. To ensure stable rank selection, we apply a jittered SVD by computingSVD(C real +δI) withδ= 10 −6 ·∥C real ∥ F /dandI being a rectangular identity of matching dimensions, usingtorch.linalg.svd. In case of SVD computation failure due to ill-conditioning, we alternatively com- pute the SVD indirectly via the eigendecomposition ofC real C ⊤ real andC ⊤ real C real , symmetrized to avoid numerical asymmetry. Sinkhorn cost (Γ) normalization. Before computing the Gibbs kernelK= exp(−Γ/ε), we normalize the cost matrix by its mean:Γ ← Γ/ ̄ Γ, where ̄ Γ= 1 ̃ B P ij Γ ij . This prevents kernel entries from collapsing to zero when absolute cost values are large relative toε, and makes the effective regularization strength approximately invariant to the scale of KL divergences across different training stages. A3.3 Deliberate Selection of Datasets by Scale We evaluate on Flickr8k, Flickr30k, and COCO using the standard Karpathy-style retrieval splits. These datasets cover a useful range of scales (refer to Table 3 in the main paper), from a relatively small and clean benchmark to substantially larger and more diverse image–text collections. This choice allows to gauge whether the same distillation principle remains effective across dataset scale and diversity. Flickr8k provides a small-scale setting where coverage is limited and overfitting is a practical concern. Flickr30k is a standard mid-scale benchmark. COCO is larger and more diverse, with broader compositional variation. Together, these datasets provide a useful picture of how each method behaves as the source distribution becomes more complex. This evaluation is particularly informative for RAHA, whose advantage is expected to grow as synthetic capacity increases. The results in the main paper confirm this trend: RAHA remains competitive at small budgets and improves steadily as N grows. Sensitivity to real Euclidean cross-covariance. CovMatch is a strong Euclidean statistics-matching baseline, but its behavior depends on how strongly the real Euclidean cross-covariance target is enforced during distillation. In its released implementation, the covariance term is written as∥C syn − ρC real ∥ 2 F , whileλcontrols the relative strength of auxiliary feature-matching losses. Thus,ρ rescales the real cross-covariance target, whereasλcontrols the balance between covariance alignment and feature-level regularization. In CovMatch, they report different (ρ,λ) settings across budgets, which indicates that its performance is sensitive to this balance. In contrast, RAHA does not rely on rescaling the real cross-covariance target across settings. Instead, it imposes structured supervision through an RAHA: Appendix23 explicit decomposition into range and residual subspaces, so that dominant shared structure and weaker residual structure are treated differently by design. This reduces dependence on pair-level tuning and aligns more naturally with our geometric motivation. A3.4 Extended Quantitative Results On Reproducibility. (Table A4). Exact reproduction of CovMatch from the released public artifacts is not straightforward. The public repository provides script entry points for multiple dataset–budget settings, but the released hyper- parameters are not fully synchronized with the values reported in the paper. Specifically, the public code usesρ= 2 for the 200-pair setting where the paper reportsρ= 1, andλ= 0.5 for the 500-pair setting where the paper reports λ= 0.6. The code repository indicates that only the Flickr30k 100-pair synthetic set is directly released; other configurations must be re-run from scratch. Table A4 summarizes the published-to-reproduced gap alongside RAHA’s performance at each budget. Under the reproduced protocol, which we consider the fairest publicly verifiable comparison, RAHA outperforms CovMatch at 200 and 500 pairs on all three datasets and remains competitive at 100 pairs. This pattern is consistent across Flickr8k (where the published and reproduced CovMatch numbers are closer), Flickr30k, and COCO. Higher-scale comparison on C3M-595K-LLaVA. The relatively smaller gains on Flickr30k and COCO at low budgets reflect properties of these bench- marks rather than a fundamental limitation of hyperbolic geometry. Both datasets contain multiple captions per image with substantial semantic overlap, creating a regime in which precise instance discrimination can dominate over global hier- archy recovery. This can reduce the visible benefit of a hierarchy-aware geometry even when the underlying structure remains present. The trend reverses at larger budgets, where RAHA’s advantage becomes clear across all three datasets. In this light, we complement the Flickr30k and COCO evaluations with an additional evaluation in Table A5 on C3M-595K-LLaVA [1], a subset of C3M due to the inaccessibility of the C3M dataset in full from the original source. C3M-595K-LLaVA (595k in scale) stands as a broader and noisier image–text source than the standard retrieval benchmarks, providing a useful stress test at larger scale. At 100 pairs, RAHA matches CovMatch as with the Flickr8k case. At 200 pairs, RAHA tops CovMatch. At 500 pairs, the gain becomes substantial, with the mean score improving from 5.1 to 8.1. These results indicate that the selective preservation of information-rich shared structure becomes increasingly beneficial as the source distribution becomes broader and the synthetic budget becomes sufficiently expressive. Larger-budget comparison. Table A6 reports the 1000-pair comparison. RAHA improves clearly over CovMatch on Flickr8k and Flickr30k in mean retrieval. On COCO, RAHA also improves the mean score over CovMatch. These results show that the advantage of geometry-aware relevance distillation becomes more pronounced once the synthetic set has enough capacity to retain structured shared information. 24Jeong et al. Table A4: Gap between published and reproduced CovMatch mean retrieval scores for Flickr30k and COCO.∆denotes the shortfall of the reproduced result relative to the published number. Under the reproducible protocol (|B|= 64), RAHA outperforms CovMatch at 200 and 500 pairs on both datasets and remains competitive at 100 pairs. Dataset # Pairs Published Reproduced∆ RAHA Flickr30k 10030.522.8 −7.720.7 20034.422.0 −12.4 25.7 50038.428.9 −9.5 32.9 COCO 10011.57.0 −4.57.2 20014.88.3 −6.5 10.2 50019.611.2 −8.4 13.7 Table A5: Comparison on C3M-595K-LLaVA across 100, 200, and 500 pairs. IR and TR denote mean image-retrieval and text-retrieval scores averaged over K@1,5,10, and Mean is their average. RAHA matches CovMatch at 100 pairs and becomes clearly stronger as the synthetic budget increases. Dataset# Pairs Method IR TR Mean C3M-595K-LLaVA [1]100 CovMatch [32] 3.6 4.3 4.0 Ours3.74.34.0 C3M-595K-LLaVA [1]200 CovMatch [32] 3.9 4.3 4.1 Ours5.15.75.4 C3M-595K-LLaVA [1]500 CovMatch [32] 4.8 5.4 5.1 Ours7.68.68.1 Comparison with EDGE (Table A6). EDGE [81] reports larger-budget results beyond the standard low-budget setting, making it a useful point of comparison at higher synthetic capacity. This comparison is complementary to CovMatch because the two baselines reflect different design choices. CovMatch [32] emphasizes Euclidean statistics matching with a trainable text encoder, whereas EDGE is more generation-oriented by employing Stable Diffusion (SD) v1.5 [56]. RAHA differs from both by focusing on the selective preservation of retrieval- relevant shared structure through geometry-aware matching without an explicitly pretrained generative model like SD. We note that EDGE operates in a generative paradigm (latent diffusion synthesis of images plus discrete caption generation) and uses a fundamentally different data representation than RAHA, making the comparison complementary rather than strictly head-to-head. The comparison in Table A6 at the budgets where EDGE numbers are available shows that RAHA outperforms EDGE on all three datasets at both 500 and 1000 pairs. A3.5 Evaluation on Prompted Classification Prompted image classification task results. To assess whether RAHA’s distilled pairs preserve structure useful beyond retrieval, we evaluate prompted RAHA: Appendix25 Table A6: Extended 500/1000-pair image-text retrieval comparison. IR and TR denote mean image-retrieval and text-retrieval scores, and Mean is their average. RAHA improves over CovMatch on all three datasets at this larger synthetic budget. Dataset# Pairs MethodIR TR Mean Flickr8k [24] 500 EDGE [81] N/A N/A N/A CovMatch [32] 23.8 28.0 25.9 Ours27.733.630.7 1000 EDGE [81] N/A N/A N/A CovMatch [32] 28.0 33.2 30.6 Ours33.640.737.1 Flickr30k [73] 500 EDGE [81] 19.4 32.1 25.8 CovMatch [32] 26.3 31.5 28.9 Ours30.035.932.9 1000 EDGE [81] 26.2 34.8 30.5 CovMatch [32] 26.5 30.2 28.4 Ours34.541.538.0 COCO [36] 500 EDGE [81] 6.5 9.4 7.9 CovMatch [32] 9.9 12.6 11.2 Ours12.614.913.7 1000 EDGE [81] 9.6 12.6 11.1 CovMatch [32] 9.6 22.1 14.2 Ours16.420.818.6 Table A7: Prompted image classification results on CIFAR-100 [30] (32×32), CUB-200-2011 [68], Stanford Cars [29], and ImageNet-1K [16] (224×224). We use the dataset-specific prompts as used in CLIP [54]. DatasetImage Resolution # Pairs Method Top-1 Acc. (%) CIFAR-100 [30]32×32100 CovMatch [32]14.55 Ours15.60 CUB-200-2011 [68] 224×224100 CovMatch [32]12.53 Ours14.43 Stanford Cars [29] CovMatch [32]5.63 Ours7.80 ImageNet-1K [16] CovMatch [32]7.62 Ours7.88 zero-shot classification on four benchmarks following the CLIP prompting tech- nique [54]. Table A7 reports Top-1 accuracy atN =100. RAHA improves over CovMatch on CIFAR-100 (+1.05 p), CUB-200-2011 (+1.90 p), and Stanford Cars (+2.17 p), and matches on ImageNet-1K (+0.26 p). The gains are largest on fine-grained benchmarks (CUB, Cars), where the hierarchical inductive bias of hyperbolic geometry is expected to be most beneficial, as these datasets 26Jeong et al. A photo of a troutA photo of a rabbit A photo of a otterA photo of a cockroach A photo of a lizard A photo of a castle A photo of a lizardA photo of a lizard A photo of a troutA photo of a rabbit A photo of a otterA photo of a cockroach Fig. A1: Distilled data comparison on CIFAR-100. Left: initial samples. Middle: CovMatch-distilled [32]. Right: RAHA-distilled (Ours). RAHA preserves visual fidelity and image–text consistency more reliably across samples. A photo of a Savannah SparrowA photo of a Scott Oriole A photo of a ForstersTern A photo of a Green Violetear A photo of a Florida JayA photo of a Mallard A photo of a Gadwall A photo of a Mockingbird A photo of a Florida JayA photo of a Mallard A photo of a Gadwall A photo of a Mockingbird Fig. A2: Distilled data comparison on CUB-200-2011. Left: initial samples. Middle: CovMatch-distilled [32]. Right: RAHA-distilled (Ours). A photo of a Dodge Sprinter Cargo Van 2009 A photo of a Audio S6 Sedan 2011 A photo of a Volkswagen Beetle Hatchback 2012 A photo of a Bentley Continental Supersports Conv. Convertible 2012 A photo of a BMW 1 Series Coupe 2012 A photo of a Nissan Leaf Hatchback 2012 A photo of a Hyundai Accent Sedan 2012 A photo of a Audi RS 4 Convertible 2008 A photo of a HUMMER H2 SUT Crew Cab 2009 A photo of a Audi 100 Sedan 1994 A photo of a Volkswagen Gold Hatchback 1991 A photo of a Bentley Continental Supersports Conv. Convertible 2012 Fig. A3: Distilled data comparison on Stanford Cars. Left: initial samples. Middle: CovMatch-distilled [32]. Right: RAHA-distilled (Ours). A photo of a pizza A photo of a EnglishSpringer A photo of a shoppingbasket A photo of a seacucumber A photo of a dough A photo of a Englishfoxhound A photo of a barn spider A photo of a siamang A photo of a banana A photo of a EnglishSpringer A photo of a shoppingbasket A photo of a sea slug Fig. A4: Distilled data comparison on ImageNet-1K. Left: initial samples. Middle: CovMatch-distilled [32]. Right: RAHA-distilled (Ours). RAHA: Appendix27 exhibit taxonomic structure that aligns naturally with coarse-to-fine semantic organization. A3.6 Additional Qualitative Results The qualitative comparison complements the retrieval tables by showing how distilled image–text pairs evolve during optimization and how well different methods preserve cross-modal consistency. Relative to the strongest Euclidean statistics-based baseline, RAHA tends to produce cleaner visual structure and better preserve fine-grained image–text agreement. This is consistent with our de- sign goal: preserving dominant shared structure while controlling weaker residual interactions. Here, we additionally provide qualitative comparison of 100-pair distilled data on CIFAR-100, CUB-200-2011, Stanford Cars, and ImageNet datasets between the initial sample, CovMatch [32]-distilled and RAHA-distilled synthetic samples in Figs. A1, A2, A3, and A4. Broadly, we observe a similar pattern to Fig. 1 of the main paper: CovMatch leaves a significant portion of trailing noise-like artifacts in various regions of the distilled images, with occasional mismatched attributes in the distilled text. By contrast, RAHA-distilled samples produce substantially more realistic distilled data that are geometrically consistent (without noise-like artifacts) and match prompted label text with the initialization while achieving higher classification accuracy than the baseline. Interestingly, on CIFAR-100 containing coarse 32×32 resolution images, Cov- Match completely diverges from being realistic in visuals. Moreover, the text descriptions (labels) are far from the initialization, suggesting that the second- order statistics matching mechanism in CovMatch [32] fails to optimize distilled data toward realism. In contrast, our method largely maintains realistic distilla- tion of the real data without undesirable visual artifacts or semantic perturbations in the text. A4 Further Analyses and Discussion In this section, we further discuss sensitivity to the hyperparameter, compute profile with a contextualized cost defense, and performance at small budgets accompanied by the reproducibility analysis. Feature-association visualization. We also include a feature visualization in Euclidean and Lorentz spaces to support the geometric motivation of RAHA (Fig. A5). The goal is not to claim exact tree reconstruction. Instead, the vi- sualization tests whether the distilled image and text features retain a more structured relative organization. We focus on three properties: cleaner grouping of semantically related pairs, more stable coarse-to-fine organization, and reduced overlap caused by noisy residual interactions. 28Jeong et al. Fig. A5(a) is particularly informative because it separates hierarchical-depth mismatch from total cross-modal discrepancy. For each matched pair of Lorentz- lifted points(x img i ,x txt i ), we plot the radial gap ∆r i = r(x img i )− r(x txt i ) against the matched hyperbolic distance d H x img i ,x txt i = 1 √ c arcosh max − c D x img i , x txt i E L , 1 + ε . In this pair distance formulation,r(x) =d H (o,x)is the geodesic radius from the origin, so∆r i measures whether the paired image and text embeddings are placed at comparable semantic depth in hyperbolic space. Since radius itself is a distance from the origin, the triangle inequality gives: |∆r i | =|d H (o,x img i )− d H (o,x txt i )| ≤ d H (x img i ,x txt i ), which means that large radial mismatch necessarily induces a large matched pair distance, regardless of any angular agreement. Thus, panel (a) is not merely descriptive: it directly diagnoses whether the method places matched image–text pairs at compatible hierarchical levels. The plot shows that CovMatch exhibits a broader and more negatively shifted distribution of∆r i , indicating that text features are often pushed farther from the origin than their matched image features. These pairs also populate the higher- distance regime, revealing that substantial radial mismatch is coupled with poorer cross-modal association. In contrast, RAHA concentrates pairs much closer to ∆r i ≈0and simultaneously in a lowerd H regime. This means that RAHA reduces modality-dependent radial drift and places matched image–text pairs at more compatible semantic depths, not only making them closer in hyperbolic space, but also organizing them more consistently with the intended coarse-to-fine hierarchy. We view this as direct geometric evidence that the rank-aware range–residual supervision preserves hierarchy-aware cross-modal relevance more faithfully than uniform Euclidean-style alignment. Fig. A5(b) provides a complementary manifold-level view of the distilled fea- tures in the Lorentz model. In contrast to (a) quantifying matched-pair consistency through∆r i andd H (x img i ,x txt i ), (b) shows whether these local improvements translate into a more coherent global cross-modal geometry. In CovMatch, the image and text embeddings appear more unevenly organized, with a less com- patible radial arrangement and stronger modality-dependent displacement. In RAHA, the two modalities exhibit a more consistent joint configuration: their placements are more geometrically compatible, radial progression is more stable, and cross-modal organization is less distorted by weak residual interactions. This does not imply exact recovery of an underlying semantic tree. Rather, it indicates that RAHA better preserves relative coarse-to-fine structure across modalities, which is precisely the geometric behavior that the hyperbolic formulation is intended to induce. RAHA: Appendix29 Fig. A5: (a) Radial gap and matched hyperbolic pair distance. The radial gap measures how differently a matched image and text pair are placed with respect to the hyperbolic origin. RAHA yields a tighter radial-gap distribution and lower matched hyperbolic distances than CovMatch, indicating more consistent cross-modal placement at similar hierarchical depth. (b) Euclidean and Lorentz views of the distilled features. Compared with CovMatch, RAHA yields a more compatible global image–text configuration in hyperbolic space, with more stable radial ordering and less irregular modality-dependent displacement, consistent with preserving hierarchy-aware cross-modal relevance. A4.1 Hyperparameter Sensitivity and Ablations Taken together with Fig. 2 in the main paper, Fig. A6 provides a component- wise view of where RAHA’s gain comes from. Fig. 2 isolates the contribution of the range–residual decomposition: starting from the base hyperbolic contrastive objectiveL hITC , adding the range termL range yields the larger single-component improvement, whereas using the residual termL residual alone is weaker. The full objective nevertheless performs best, indicating that the dominant shared range carries the primary retrieval signal, while the residual branch is most useful as a controlled secondary refinement once it is anchored to the range term and regularized by compression. Fig. A6(a)–(c) supports the same interpretation quantitatively, where stronger range supervision is beneficial for the range branch as opposed to and mild residual compression being sufficient for the residual branch. Fig. A6(d) further isolates the role of geometry while keeping the same Sinkhorn-based relevance-matching pipeline. An all-Euclidean formulation per- forms poorly (IR/TR/RMean = 1.4/3.0/2.2), showing that the explicit subspace decomposition is not effective when both contrastive alignment and relevance matching remain in Euclidean space. Replacing only the contrastive term by its Euclidean counterpart while keeping the rank-aware relevance matching in the hyperbolic pipeline (eITC) recovers most of the performance (IR/TR/RMean = 18.1/21.7/19.9), and the fully hyperbolic variant (hITC) further improves to 19.0/21.9/20.4. Thus, the gain of RAHA is not attributable to a single scalar weight or a single loss term in isolation. It comes from the interaction between selective range–residual supervision and hyperbolic lifting: the latter is necessary for the decomposition to become effective, while lifting the contrastive term as well provides an additional consistent gain over the mixed-geometry variant. 30Jeong et al. (a) λ range (b) λ residual (c) λ comp (d) Euclidean vs. Hyperbolic space Fig. A6: Hyperparameter sensitivity and geometry ablation of RAHA at the Flickr8k 100-pair setting. (a)–(c) RAHA is most stable when the dominant shared range remains the primary alignment target, residual matching is weighted moderately, and residual compression is kept mild. (d) Keeping the same Sinkhorn-based relevance-matching pipeline and varying only the geometry shows that an all-Euclidean formulation performs poorly (IR/TR/RMean = 1.4/3.0/2.2), a mixed variant with Euclidean ITC and hyperbolic relevance matching (eITC) reaches 18.1/21.7/19.9, and the fully hyperbolic variant (hITC) performs best at 19.0/21.9/20.4. A4.2 Compute Profile RAHA follows the line of distribution matching branch for VLDD and avoids the storage overhead of expert-trajectory approaches, which require full parameter checkpoints at each expert step. At the same time, each distillation step is more expensive than CovMatch because it includes hyperbolic lifting and log-map operations (O(Bd) element-wise operations), tangent-space cross-covariance and SVD (O(d 2 B+d 3 )), range–residual projection (O(Bdk)), and Sinkhorn iterations (O(n iter · ̃ B· B)). Table A8 reports a per-component breakdown on a single RTX A6000 under the default Flickr8kN =100 setting with batch size 64. All timing values for the distillation step are reported per synthetic image within a batch of 64, and the per-iteration wall-clock time is obtained by multiplying by the synthetic batch size ̃ B. Table A8: Per-component cost breakdown (Flickr8k 100-pair setting on 1×RTX A6000). Distillation-step timings are reported per synthetic image at batch size 64. StageCovMatch [32] RAHA (Ours) Data initialization (per run)≈0.69 s Model initialization (per iter)≈0.27 s Distillation step (per image, ̃ B=64) ≈0.78 s≈7.42 s Distillation step (per iter, ̃ B=64)≈55 s≈400 s Peak GPU VRAM≈9.3 GB RAHA: Appendix31 The distillation-step overhead is dominated by the SVD computation on the d×dcross-covariance (withd=2304) and the Sinkhorn iterations on the ̃ B×Bcost matrix. At batch size 1, the per-image distillation times are comparable (≈25 s each), confirming that the overhead scales with batch-level matrix operations rather than per-sample computation. Peak GPU memory is identical because the additional intermediates (cross-covariance, SVD factors, coupling matrix) are small relative to the model and batch tensors stored by both methods. Under the default setting ofT =200 iterations withN out =50 outer-loop steps each, a full RAHA distillation run on Flickr8kN =100 completes in approximately 1.3 GPU-hours on a single RTX A6000, compared to approximately 0.14 GPU- hours for CovMatch. For each distillation iteration, both CovMatch and RAHA occupy a peak GPU memory of approximately 9.3 GB with batch sizes of 64 for real and synthetic data. A detailed per-component cost breakdown with contextualization is provided Table A8. Contextualizing the overhead. Three considerations place this cost in practical perspective. (i) Distillation is a one-time offline cost: the resulting synthetic set is reused across all downstream training runs, architecture searches, and ablations. The amortized cost per downstream experiment is therefore negligible once the distilled set is produced. (i) RAHA avoids the storage burden of trajectory- matching methods (e.g., MTT-VL [70], LoRS [72]), which must checkpoint full encoder parameters at every expert step. For a model with∼90 M parameters saved at 50 expert steps, this amounts to∼18 GB of storage per distillation run—a cost that RAHA does not incur. (i) The overhead is concentrated in batch-level matrix operations (SVD, Sinkhorn) rather than per-sample computation. As shown in the main paper (§4.4), at batch size 1 both methods take≈25 s per image; the gap emerges because RAHA’s structure-aware operations scale with batch size while CovMatch’s element-wise covariance matching does not. Taken together, these factors indicate that RAHA’s compute profile is prac- tical for a distillation method. We note that the compute overhead is not fundamental to the hyperbolic formulation for VLDD, but rather to the explicit subspace decomposition and permutation-invariant matching that constitute the core algorithmic contribution. As shown in Fig. A6(d), this explicit subspace decomposition does not perform well in Euclidean space, which underscores the importance of the hyperbolic lifting. A5 Broader Societal Impact In this section, we discuss the societal implications of image–text dataset distillation, including benefits, inherited risks, and RAHA-specific considerations regarding bias in the preserved subspace. Image–text dataset distillation can have both positive and negative down- stream effects. On the positive side, compact distilled sets reduce storage, transfer, and repeated training cost. This can make experimentation more accessible and improve reproducibility. Distillation may also reduce the need to redistribute raw 32Jeong et al. paired datasets in settings where privacy, licensing, or provenance concerns limit direct sharing of the original data. At the same time, distillation does not remove issues present in the source data. A compact synthetic set can still encode harmful correlations, stereotypes, geographic imbalance, or annotation artifacts inherited from the original collection. In multimodal settings, these risks may be amplified by the interaction between image and language. Distillation therefore changes the form of the data, but not automatically its fairness or safety properties. A consideration specific to RAHA is that the rank-aware decomposition parti- tions cross-modal correlation into a dominant range subspace and a complemen- tary residual component, with the distillation objective prioritizing preservation of the former. If harmful correlations, such as stereotypical associations between visual attributes and textual captions, are concentrated in the dominant singular directions, the range-preserving objective will retain them in the distilled set. Conversely, if such correlations reside primarily in the residual subspace, the compression regularizer may suppress them, but this suppression is incidental rather than by design. In neither case does RAHA provide an explicit mechanism for bias detection or removal. We view this as an important direction for future work: combining structure-aware distillation with explicit fairness constraints or post-hoc auditing of the preserved subspace could yield distilled sets that are both efficient and more equitable. RAHA also has technical limitations that matter for responsible use. As with prior multimodal distillation methods, performance depends on the pretrained encoders used during distillation. It can degrade under strong domain shift, noisy captions, or abstract descriptions that do not exhibit stable hierarchical structure. In addition, the rank-aware prior is most suitable when the shared cross-modal signal is concentrated in a dominant subspace. If useful signal is distributed across many weak directions, excessive residual suppression can remove task-relevant information. The cost overhead is derived primarily from lifting features to hyperbolic space and taking the tangent, alongside performing SVD on thed×dcross-covariance matrix. and Sinkhorn on the ̃ B×B cost matrix, not from the Lorentz lift. This is a one-time offline distillation cost, and the distilled set can be reused for downstream training. RAHA also avoids trajectory-checkpoint storage required by [70,72,76]. Thus, [32] is preferable when wall-clock efficiency is primary, while RAHA is justified when structured relevance modeling, transfer, and higher-budget scaling are prioritized. These bottlenecks are optimizable through truncated/randomized SVD, cached basis updates, low-rank or warm-start Sinkhorn, fewer iters, and fused GPU kernels. For these reasons, we view RAHA as a method for efficient and structured compression, not as a guarantee of fairness, neutrality, or robustness beyond the tested setting. Future work should study whether structure-aware distillation can be combined with explicit bias auditing, provenance constraints, and safer data filtering. RAHA: Appendix33 References 1.Llava-c3m-pretrain-595k dataset. Hugging Face Datasets (2023),https://huggin gface.co/datasets/liuhaotian/LLaVA-C3M-Pretrain-595K 2.Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023) 3. Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al.: Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35, 23716– 23736 (2022) 4. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al.: Qwen technical report. arXiv preprint arXiv:2309.16609 (2023) 5.Birhane, A., Prabhu, V., Han, S., Boddeti, V.N., Luccioni, A.S.: Into the LAIONs den: Investigating hate in multimodal datasets. In: Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track (2023) 6.Bottou, L.: Stochastic gradient descent tricks. In: Neural networks: tricks of the trade: second edition, p. 421–436. Springer (2012) 7. Brock, A., De, S., Smith, S.L., Simonyan, K.: High-performance large-scale image recognition without normalization. In: International conference on machine learning. p. 1059–1071. PMLR (2021) 8. Byeon, M., Park, B., Kim, H., Lee, S., Baek, W., Kim, S.: Coyo-700m: Image-text pair dataset. https://github.com/kakaobrain/coyo-dataset (2022) 9.Carlini, N., Jagielski, M., Choquette-Choo, C.A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., Tramèr, F.: Poisoning web-scale training datasets is practical. In: IEEE Symposium on Security and Privacy (SP) (2024) 10.Cazenavette, G., Wang, T., Torralba, A., Efros, A.A., Zhu, J.Y.: Dataset distillation by matching training trajectories. In: CVPR (2022) 11. Cazenavette, G., Wang, T., Torralba, A., Efros, A.A., Zhu, J.Y.: Generalizing dataset distillation via deep generative prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 3739–3748 (2023) 12. Cui, J., Wang, R., Si, S., Hsieh, C.J.: Dc-bench: Dataset condensation benchmark. Advances in Neural Information Processing Systems 35, 810–822 (2022) 13.Cui, J., Wang, R., Si, S., Hsieh, C.J.: Scaling up dataset distillation to imagenet-1k with constant memory. In: International Conference on Machine Learning. p. 6565–6590. PMLR (2023) 14.Cui, X., Qin, Y., Zhou, W., Li, H., Li, H.: Optical: Leveraging optimal transport for contribution allocation in dataset distillation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 15245–15254 (2025) 15. Cuturi, M.: Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems 26 (2013) 16. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. p. 248–255. Ieee (2009) 17.Desai, K., Nickel, M., Rajpurohit, T., Johnson, J., Vedantam, R.: Hyperbolic image-text representations. In: ICML (2023) 18.Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). p. 4171–4186 (2019) 34Jeong et al. 19.Du, J., Jiang, Y., Tan, V.Y., Zhou, J.T., Li, H.: Minimizing the accumulated trajectory error to improve dataset distillation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 3749–3758 (2023) 20.Farahani, R.Z., Hekmatfar, M.: Facility location: concepts, models, algorithms and case studies. Springer Science & Business Media (2009) 21.Ganea, O., Bécigneul, G., Hofmann, T.: Hyperbolic neural networks. Advances in neural information processing systems 31 (2018) 22.Guo, C., Rana, M., Cisse, M., Van Der Maaten, L.: Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117 (2017) 23.Guo, Z., Wang, K., Cazenavette, G., Li, H., Zhang, K., You, Y.: Towards loss- less dataset distillation via difficulty-aligned trajectory matching. arXiv preprint arXiv:2310.05773 (2023) 24.Hodosh, M., Young, P., Hockenmaier, J.: Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research 47, 853–899 (2013) 25.Jeong, J., Kwon, H., Kim, M., Yoon, K.J.: Multimodal distribution matching for vision-language dataset distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2026) 26.Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 3128–3137 (2015) 27.Kim, J.H., Kim, J., Oh, S.J., Yun, S., Song, H., Jeong, J., Ha, J.W., Song, H.O.: Dataset condensation via efficient synthetic-data parameterization. In: ICML (2022) 28.Kim, W., Chun, S., Kim, T., Han, D., Yun, S.: Hype: Hyperbolic entailment filtering for underspecified images and texts. In: European Conference on Computer Vision (ECCV) (2024). https://doi.org/10.48550/arXiv.2404.17507 29.Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: Proceedings of the IEEE international conference on computer vision workshops. p. 554–561 (2013) 30.Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009) 31.Lee, H.B., Lee, D.B., Hwang, S.J.: Dataset condensation with latent space knowledge factorization and sharing. arXiv preprint arXiv:2208.10494 (2022) 32.Lee, Y., Chung, H.W.: Covmatch: Cross-covariance guided multimodal dataset distillation with trainable text encoder. arXiv preprint arXiv:2510.18583 (2025) 33.Lei, S., Tao, D.: A comprehensive survey of dataset distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence 46(1), 17–32 (2023) 34.Li, J., Li, D., Xiong, C., Hoi, S.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: International Conference on Machine Learning. p. 12888–12900. PMLR (2022) 35.Li, W., Li, G., Maeda, K., Ogawa, T., Haseyama, M.: Hyperbolic dataset distillation. In: Advances in Neural Information Processing Systems (NeurIPS) (2025), poster 36. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. p. 740–755. Springer (2014) 37.Liu, D., Gu, J., Cao, H., Trinitis, C., Schulz, M.: Dataset distillation by automatic training trajectories. In: European Conference on Computer Vision. p. 334–351. Springer (2024) 38. Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems 36, 34892–34916 (2023) RAHA: Appendix35 39.Liu, H., Li, Y., Xing, T., Wang, P., Dalal, V., Li, L., He, J., Wang, H.: Dataset dis- tillation via the wasserstein metric. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 1205–1215 (2025) 40.Liu, P., Du, J.: The evolution of dataset distillation: Toward scalable and generaliz- able solutions. arXiv preprint arXiv:2502.05673 (2025) 41.Liu, S., Wang, K., Yang, X., Ye, J., Wang, X.: Dataset distillation via factorization. In: NeurIPS (2022) 42.Longpre, S., Mahari, R., Chen, A., Obeng-Marnu, N., Sileo, D., Brannon, W., Muennighoff, N., Khazam, N., Kabbara, J., Perisetla, K., et al.: The data provenance initiative: A large scale audit of dataset licensing & attribution in AI. arXiv preprint arXiv:2310.16787 (2023) 43.Longpre, S., Mahari, R., Lee, A., Lund, C., Oderinwale, H., Brannon, W., Saxena, N., Obeng-Marnu, N., South, T., Hunter, C., Klyman, K., et al.: Consent in crisis: The rapid decline of the AI data commons. In: Advances in Neural Information Processing Systems (NeurIPS) (2024) 44. Loo, N., Hasani, R., Amini, A., Rus, D.: Efficient dataset distillation using random feature approximation. In: NeurIPS (2022) 45.Loo, N., Hasani, R., Lechner, M., Rus, D.: Dataset distillation with convexified implicit gradients. arXiv preprint arXiv:2302.06755 (2023) 46.Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017) 47. Nguyen, T., Chen, Z., Lee, J.: Dataset meta-learning from kernel ridge-regression. arXiv preprint arXiv:2011.00050 (2020) 48.Nguyen, T., Novak, R., Xiao, L., Lee, J.: Dataset distillation with infinitely wide convolutional networks. In: NeurIPS (2021) 49. Nickel, M., Kiela, D.: Poincaré embeddings for learning hierarchical representations. arXiv preprint arXiv:1705.08039 (2017) 50. Nickel, M., Kiela, D.: Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In: Proceedings of the 35th International Conference on Machine Learning (ICML). Proceedings of Machine Learning Research (PMLR), vol. 80, p. 3779–3788 (2018) 51. Pal, A., van Spengler, M., D’Amely di Melendugno, G.M., Flaborea, A., Galasso, F., Mettes, P.: Compositional entailment learning for hyperbolic vision-language models. In: International Conference on Learning Representations (ICLR) (2025), oral 52.Peng, W., Varanka, T., Mostafa, A., Shi, H., Zhao, G.: Hyperbolic deep neural networks: A survey. arXiv preprint arXiv:2101.04562 (2021) 53. Poppi, T., Kasarla, T., Mettes, P., Baraldi, L., Cucchiara, R.: Hyperbolic safety- aware vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025).https://doi.org/10 .48550/arXiv.2503.12127 54.Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. p. 8748–8763. PmLR (2021) 55.Ramasinghe, S., Shevchenko, V., Avraham, G., Thalaiyasingam, A.: Accept the modality gap: An exploration in the hyperbolic space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 27263–27272 (June 2024) 36Jeong et al. 56.Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 10684–10695 (2022) 57.Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019) 58. Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al.: Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 35, 25278–25294 (2022) 59.Shang, Y., Yuan, Z., Yan, Y.: Mim4d: Mutual information maximization for dataset distillation. arXiv preprint arXiv:2312.16627 (2023) 60.Su, D., Hou, J., Gao, W., Tian, Y., Tang, B.: D^4m: Dataset distillation via disentangled diffusion model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 5809–5818 (2024) 61.Team, G., Georgiev, P., Lei, V.I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024) 62. Thiel, D.: Identifying and eliminating CSAM in generative ML training data and models. Tech. rep., Stanford Internet Observatory (2023).https://doi.org/10.2 5740/kh752sm9123, https://purl.stanford.edu/kh752sm9123 63. Thomee, B., Shamma, D.A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., Li, L.J.: Yfcc100m: The new data in multimedia research. Communications of the ACM 59(2), 64–73 (2016) 64. Toneva, M., Sordoni, A., Combes, R.T.d., Trischler, A., Bengio, Y., Gordon, G.J.: An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 (2018) 65.Wang, H., Zhao, Z., Wu, J., Shang, Y., Liu, G., Yan, Y.: Cao 2 : Rectifying inconsis- tencies in diffusion-based dataset distillation (2025),https://arxiv.org/abs/25 06.22637 66.Wang, K., Zhao, B., Peng, X., Zhu, Z., Yang, S., Wang, S., Huang, G., Bilen, H., Wang, X., You, Y.: Cafe: Learning to condense dataset by aligning features. In: CVPR (2022) 67.Wang, T., Zhu, J.Y., Torralba, A., Efros, A.A.: Dataset distillation. arXiv preprint arXiv:1811.10959 (2018) 68.Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P.: Caltech-ucsd birds 200 (2010) 69.Welling, M.: Herding dynamical weights to learn. In: Proceedings of the 26th Annual International Conference on Machine Learning. p. 1121–1128 (2009) 70.Wu, X., Zhang, B., Deng, Z., Russakovsky, O.: Vision-language dataset distillation (2024), https://openreview.net/forum?id=2y8XnaIiB8, tMLR 2024 71.Xu, W., Evans, D., Qi, Y.: Feature squeezing: Detecting adversarial examples in deep neural networks. In: NDSS (2018).https://doi.org/10.14722/ndss.2018.23295, https://w.ndss-symposium.org/ndss-paper/feature-squeezing-detectin g-adversarial-examples-in-deep-neural-networks/ 72.Xu, Y., Lin, Z., Qiu, Y., Lu, C., Li, Y.L.: Low-rank similarity mining for multimodal dataset distillation. In: Proceedings of the 41st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 235, p. 55144–55161. PMLR (2024), https://proceedings.mlr.press/v235/xu24q.html 73. Young, P., Lai, A., Hodosh, M., Hockenmaier, J.: From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics 2, 67–78 (2014) RAHA: Appendix37 74.Yu, R., Liu, S., Wang, X.: Dataset distillation: A comprehensive review. IEEE transactions on pattern analysis and machine intelligence 46(1), 150–170 (2023) 75. Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language image pre- training. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 11975–11986 (2023) 76.Zhang, X., Zhang, Z., Du, J., Liu, Z., Zhou, J.T.: Beyond modality collapse: Rep- resentations blending for multimodal dataset distillation. arXiv arXiv:2505.14705 (2025) 77. Zhao, B., Bilen, H.: Dataset condensation with differentiable siamese augmentation. In: ICML (2021) 78.Zhao, B., Bilen, H.: Dataset condensation with distribution matching. In: WACV (2023) 79.Zhao, B., Mopuri, K.R., Bilen, H.: Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929 (2020) 80.Zhao, G., Li, G., Qin, Y., Yu, Y.: Improved distribution matching for dataset condensation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 7856–7865 (2023) 81.Zhao, Z., Wang, H., Wu, J., Shang, Y., Liu, G., Yan, Y.: Efficient multimodal dataset distillation via generative models. arXiv preprint arXiv:2509.15472 (2025) 82. Zhong, W., Tang, H., Zheng, Q., Xu, M., Hu, Y., Guan, W.: Towards stable and storage-efficient dataset distillation: Matching convexified trajectory. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 25581–25589 (2025) 83.Zhou, Y., Nezhadarya, E., Ba, J.: Dataset distillation using neural feature regression. arXiv preprint arXiv:2206.00719 (2022)

Instant research discovery