Paper deep dive
Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration
Matteo Barbieri, Giammarco La Barbera, Juan Pablo De La Plata, Sabine Sarnacki, Isabelle Bloch, Pietro Gori
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive benchmark of intra-patient 3D multimodal deformable registration methods across three datasets covering different anatomical regions and difficulty levels, including both synthetic deformation recovery and real clinical scenarios. We evaluate classical optimization-based approaches and modern learning-based methods, including recent deep learning and foundation models, using complementary metrics: Average Dice similarity coefficient (DSC), average 95th-percentile Hausdorff distance (HD95), and a modality-independent structural similarity measure based on the MIND self-similarity context (MIND-SSC). Results show high variability across datasets, with learning-based methods demonstrating superior performance on large synthetic benchmarks, while only limited improvements are observed in real pelvic registration. A key finding of this study is the consistent disagreement between geometric metrics (DSC, HD95) and image-based similarity metrics (MIND-SSC), highlighting that improved overlap does not necessarily imply better global multimodal correspondence. Furthermore, anatomy-guided approaches achieve the highest overlap scores but exhibit degraded performance outside of segmented regions, revealing a trade-off between label-driven alignment and global structural coherence. Overall, our results indicate that no current method achieves robust performance across anatomies and modalities. We demonstrate that intra-patient 3D multimodal registration requires multi-criteria evaluation, including deformation-based metrics, and remains an open problem.
Tags
Links
- Source: https://arxiv.org/abs/2609.15669v1
- Canonical: https://arxiv.org/abs/2609.15669v1
Trouble viewing inline? Open PDF directly →
Full Text
106,224 characters extracted from source content.
Expand or collapse full text
Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration Journal: Computerized Medical Imaging and Graphics Matteo BARBIERI Affiliation: IMAG2, Institut IMAGINE, Université Paris Cité, Paris, France Giammarco LA BARBERA Affiliation: IMAG2, Institut IMAGINE, Université Paris Cité, Paris, France Affiliation: Replico SAS, Paris, France Juan Pablo DE LA PLATA Affiliation: IMAG2, Institut IMAGINE, Université Paris Cité, Paris, France Affiliation: Replico SAS, Paris, France Sabine SARNACKI Affiliation: IMAG2, Institut IMAGINE, Université Paris Cité, Paris, France Affiliation: Université Paris Cité, Department of Pediatric Surgery, Hôpital Necker Enfants-Malades, Assistance Publique–Hôpitaux de Paris (AP-HP), Paris, France Isabelle BLOCH Affiliation: IMAG2, Institut IMAGINE, Université Paris Cité, Paris, France Affiliation: Sorbonne Université, CNRS, LIP6, Paris, France Affiliation: LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France Pietro GORI Email: pietro.gori@telecom-paris.fr Corresponding author: Corresponding author Affiliation: LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France Abstract Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive benchmark of intra-patient 3D multimodal deformable registration methods across three datasets covering different anatomical regions and difficulty levels, including both synthetic deformation recovery and real clinical scenarios. We evaluate classical optimization-based approaches and modern learning-based methods, including recent deep learning and foundation models, using complementary metrics: Average Dice similarity coefficient (DSC¯ DSC), average 95th-percentile Hausdorff distance (HD95¯ HD95), and a modality-independent structural similarity measure based on the MIND self-similarity context (MIND-SSC). Results show high variability across datasets, with learning-based methods demonstrating superior performance on large synthetic benchmarks, while only limited improvements are observed in real pelvic registration. A key finding of this study is the consistent disagreement between geometric metrics (DSC¯ DSC, HD95¯ HD95) and image-based similarity metrics (MIND-SSC), highlighting that improved overlap does not necessarily imply better global multimodal correspondence. Furthermore, anatomy-guided approaches achieve the highest overlap scores but exhibit degraded performance outside of segmented regions, revealing a trade-off between label-driven alignment and global structural coherence. Overall, our results indicate that no current method achieves robust performance across anatomies and modalities. We demonstrate that intra-patient 3D multimodal registration requires multi-criteria evaluation, including deformation-based metrics, and remains an open problem. This benchmark provides guidance for future research toward more generalizable, anatomically consistent, and clinically relevant registration frameworks. Keywords: Multimodal registration , Deep Learning , Deformable registration 1 Introduction Image registration is one of the building blocks for achieving personalized precision medicine, as it enables the consistent spatial alignment of anatomical structures across images acquired using different modalities, at different time points or from different patients. By establishing spatial correspondences, registration plays a critical role in numerous clinical and research applications, including longitudinal disease monitoring [19, 49], population analysis [29, 27], image-guided interventions [77], as well as fundamental research in the understanding of the human organs, such as the brain, where complementary anatomical information and functional information are integrated into a common reference frame [64, 75, 78]. Formally, image registration consists in estimating an optimal spatial transformation — typically represented as a dense displacement field or a parametric mapping — that aligns a moving image onto a fixed reference image. Transformation models are commonly divided into linear (e.g., rigid or affine), which apply global transformations governed by a small number of parameters, and deformable and non-linear models, which allow for spatially varying deformations and are essential for capturing complex anatomical variability [16]. Deformable registration is often further constrained through regularization terms to ensure smoothness, topology preservation, or diffeomorphic properties, thereby enforcing anatomically plausible mappings [3, 15]. From an algorithmic perspective, registration approaches are traditionally classified into classical iterative methods and learning-based (data-driven) methods. Classical methods formulate registration as an energy minimization problem combining a similarity measure and a regularization term, solved iteratively using numerical optimization techniques. These approaches benefit from well-established theoretical foundations and explicit control over transformation properties [64]. By contrast, learning-based methods aim to directly infer transformation using neural networks, trading explicit modeling for statistical learning and enabling real-time inference once trained [11, 78]. In addition to these general categorizations, several application-dependent factors critically influence the choice of a registration strategy. Among them, the imaging modalities of the input images (e.g., Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US)) and whether they correspond to the same modality (monomodal) or different modalities (multimodal) is particularly important, as it determines the validity of assumptions regarding intensity correspondence. Other factors include the dimensionality of the data (e.g. 2D vs. 3D), the anatomical region of interest (e.g. brain, pelvis, abdomen, etc.), and whether the task involves intra-subject (longitudinal) or inter-subject (population-level) alignment [23, 28, 26]. These considerations directly drive the design of similarity measures [8], transformation models, and learning strategies. Since the advent of digital medical imaging, research has focused mainly on monomodal registration, achieving significant improvement of similarity and structure-based metrics even in cases with complex geometrical mismatch. Multimodal registration, used for multiparametric studies or intra-operative visualization, still represents a challenging problem, as the assessment of registration quality through superimposition of anatomical features can no longer pass through intensity-based metrics. While some authors have attempted to provide better suited similarity metrics such as MIND [36] or MIND-SSC [37], others have developed learning-based similarity approximations such as REE-UNET [55] and used them in registration pipelines [35, 54, 69]. This work specifically focuses on deformable intra-patient multimodal registration. Our aim is to provide a methodological and clinically-relevant benchmark on two publicly available datasets and one private dataset for a selected number of state-of-the-art multimodal deformable registration strategies. This benchmark evaluates two distinct capabilities: (i) recovery of known transformations under modality shifts, and (i) estimation of true correspondence in realistic multimodal settings. Based on synthetic and real misaligned data, we perform a comparative study of the registration ability of different methods in setups of increasing difficulty, while varying modalities and regions of interest. Eventually, we evaluate the current state of multimodal deformable registration and provide the research community with guidelines for further work. The paper is organized as follows. After summarizing related work in Section 2, a brief introduction to the functioning of each evaluated method is provided in Section 3. The specifics of the benchmarking methodology — such as datasets, protocols and evaluation metrics — are described in Section 4, and associated results are presented in Section 5. Eventually, a discussion presenting the outcomes and limitations of the current benchmark, as well as guidelines for future work on deformable multimodal image registration is provided in Section 6. A conclusion summarizing the findings is provided in Section 7. 2 Related work Let IT,IM:Ω→ℝI_T,I_M: denote the target (i.e. reference) and moving images, respectively, defined on a domain of the d-dimensional real space (Ω⊂ℝd ^d). Deformable image registration aims to estimate a spatial transformation ϕ:Ω→Ωφ: → such that IM∘ϕ≈ITI_M φ≈~I_T, so that the moving image IMI_M, warped by the transformation ϕφ, should match the target image ITI_T. This problem is commonly formulated as the minimization of an energy functional of the following form: ℰ(ϕ)=(IT,IM∘ϕ)+λℛ(ϕ),E(φ)=D (I_T,I_M φ)+λ\,R(φ), (1) where D is a similarity measure between images, ℛR is a regularization functional enforcing smoothness or physical plausibility, and λ>0λ~>~0 balances the two terms. Different registration methods can be distinguished by their parameterization of ϕφ, the choice of similarity metric D, and the form of regularization ℛR. To better understand the difficulties of multimodal registration, and since some monomodal registration methods are sometimes directly used for multimodal registration, we present in the following section registration methods for both the monomodal and multimodal registration problem. Please note that we will consider only methods accounting for geometric transformations, without intensity/iconographic modifications, thereby excluding frameworks such as metamorphosis [21, 22, 52]. 2.1 Monomodal image registration Monomodal registration is characterized by the assumption that voxels representing the same point in the physical space within moving and target images have similar intensity distributions. In this setting, intensity-based similarity metrics such as sum of squared differences (SSD) or cross-correlation (C) are well suited, enabling accurate alignment in applications such as longitudinal analysis, where one can assume that moving IMI_M and target ITI_T images only differ by Gaussian noise (SSD) or that there is a linear relationship between them (C). Among the classical approaches, Advanced Normalization Tools (ANTs) [3] and Elastix [44] have emerged as widely adopted frameworks for medical image registration. In particular, the Symmetric Normalization (SyN) algorithm [3], based on a diffeomorphic formulation with a symmetric energy functional, is often regarded as the reference method for brain MRI registration due to its robustness, inverse consistency, and strong performance in benchmark evaluations. For applications involving large, spatially heterogeneous, and highly non-linear deformations (such as those encountered in thoracic or abdominal imaging), dense non-parametric approaches remain essential. In particular, free-form deformation (FFD) models based on B-splines [61] provide flexible local control, while variational diffeomorphic frameworks such as Large Deformation Diffeomorphic Metric Mapping (LDDMM) [27, 2, 4, 6, 8] offer strong theoretical guarantees on smoothness and invertibility. In parallel, algorithms based on optical-flow inspired updates such as Deformable demons and its diffeomorphic extensions remain highly competitive due to their computational efficiency and ability to handle large displacement fields [48, 71]. More recently, deep learning-based methods have significantly transformed monomodal registration by replacing iterative optimization with feedforward inference. Early frameworks such as VoxelMorph [5] demonstrated that Convolutional Neural Networks (CNN) can learn to predict dense deformation fields in an unsupervised manner using differentiable spatial transformers. Subsequent extensions introduced probabilistic formulations and diffeomorphic constraints via stationary velocity fields [15], improving both robustness and topology preservation. Alternatively, methods such as GradICON [66] enforce inverse consistency through gradient-based regularization, leading to improved numerical stability and more consistent mappings. Meanwhile weakly-supervised frameworks incorporate anatomical priors by means of keypoints or full organ segmentations to enforce reliable anatomical displacement fields [40, 41]. A significant difference between the developments of deep learning methods with respect to the previously described classical methods is that the expression of physical properties (such as diffeomorphism) is not formally enforced but rather implicitly encouraged through the tuning of parameters that balance competing criteria. Building on these convolutional approaches, more recent architectures leverage transformer-based designs to better capture long-range spatial dependencies. Unlike CNNs, which rely on local receptive fields and hierarchical feature aggregation, transformers exploit global context through self-attention mechanisms, enabling direct interactions between distant regions of the image. This property is particularly advantageous in medical image registration, where large anatomical displacements and weak local correspondences can challenge purely local models. Transformer-based registration methods such as TransMorph [9], Vit-v-Net [10] and others reported by Ramadan et al. in [59] typically combine attention modules with multi-scale representations, allowing them to jointly encode fine-grained details and global structural alignment. Finally, hybrid methods have emerged to bridge the gap between purely data-driven approaches and classical variational frameworks. These approaches integrate learned components, such as similarity metrics or feature embeddings, into explicit energy minimization schemes. For example, learned similarity metrics have been shown to outperform handcrafted metrics in challenging registration scenarios [32, 56, 62]. More recently, learned semantic feature representations have been incorporated into optimization-based registration frameworks to improve robustness across modalities while retaining the interpretability and regularization properties of classical formulations [14, 33]. 2.2 Multimodal image registration By contrast, multimodal image registration remains significantly more challenging due to the absence of direct intensity correspondence between modalities such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI) or ultrasound. Early approaches addressed this issue using similarity metrics derived from information theory, most notably Mutual Information (MI) [51], which quantifies statistical dependence between image intensities and remains a strong baseline for both rigid and deformable registration, alongside early gradient-based formulations [30]. However, MI suffers from limited spatial specificity and sensitivity to interpolation artifacts. To overcome these limitations, a second class of methods introduced modality-invariant descriptors that rely on structural (rather than intensity-based) information. A prominent example is the Modality Independent Neighborhood Descriptor (MIND-SSC) [37], which encodes local self-similarity patterns and has become a standard for multimodal deformable registration. Other descriptors, such as binary gradient angle representations [42] or self-similarity-based features [37, 73], further exploit the structural consistency of anatomical patterns across modalities. Deep learning has had a particularly strong impact in the multimodal setting, as it enables the learning of modality-invariant representations directly from data. Early works explored weakly-supervised and adversarial learning strategies [20, 40, 76], as well as deep similarity metrics [32], to bridge modality gaps. Other approaches relied on image-to-image translation [43, 45, 74], although their effectiveness has been shown to be modality-dependent and sometimes inferior to direct registration methods [50]. More recent methods leverage contrastive learning and shared embedding spaces [54, 18, 57], cycle-consistent training strategies [43], cross-modal attention mechanisms [63], or self-supervised anatomical representations [46]. In parallel, differentiable similarity learning frameworks such as DISA [60] have been proposed to generalize similarity modeling across modalities. Contemporary state-of-the-art approaches focus on end-to-end deformation prediction with strong geometric constraints. Examples include reinforcement learning-based formulations [39], geometry-consistent adversarial models [47], and transformer-based architectures [9]. Recent advances also explore foundation models and large-scale pretraining, such as multiGradICON [17], which extends inverse-consistent diffeomorphic frameworks to multimodal settings, and universal matching models [34]. Additional recent contributions include anatomy-aware architectures such as MAIRNet [24] and strategies that explicitly reduce modality gaps via mono-modalization [12]. Emerging directions further include diffusion-based similarity modeling [69], implicit neural representations [72], and modality-agnostic feature learning from large pretrained models [35]. 2.3 Limitations of existing benchmarks and methods Overall, while classical methods based on MI and handcrafted descriptors remain competitive in controlled or low-variability scenarios, learning-based approaches increasingly define the state of the art in multimodal deformable registration, particularly in settings involving large modality gaps, heterogeneous datasets, and real-time constraints. The field is progressively moving toward unified, generalizable frameworks capable of handling diverse modalities, anatomies, and acquisition conditions within a single model. However, the effectiveness of both classical and learning-based multimodal deformable registration methods remains limited in many clinical applications where sub-millimetric to millimetric accuracy (typically <1<1–2 m) is required, such as stereotactic radiotherapy, neurosurgical navigation, and image-guided interventions. Despite substantial progress, extensive evaluations consistently report residual Target Registration Errors (TRE) in the range of 1–3 m for state-of-the-art deformable registration methods, even in relatively controlled settings such as the Learn2Reg challenge [38]. The Learn2Reg challenge editions (2020–2024) provide a comprehensive multi-anatomy benchmarking framework that reveals persistent task-dependent gaps [31, 38]: median TRE values of approximately 1–2 m are achievable in brain registration, whereas abdominal registration remains substantially harder, with 95th-percentile Hausdorff distances reaching up to 20 m even for top-ranked methods. These results highlight the impact of respiratory motion, sliding organ interfaces, and large non-linear deformations that introduce fundamental ambiguities in defining voxel-level correspondence, particularly in homogeneous or biomechanically complex regions [64, 13]. Furthermore, among the best-performing methods in Learn2Reg 2024, intra-method variability in TRE can exceed 1 m across tasks, underscoring that no single architecture generalizes robustly across anatomies and modalities [31]. In the case of larger structures, the lack of quality is also visible from the Dice results. For instance, on large-scale brain MRI benchmarks (e.g. OASIS, ABIDE, ADHD200), mean Dice Similarity Coefficient (DSC) scores for cortical and subcortical structures cluster around 75–78% for modern approaches such as VoxelMorph and its diffeomorphic variants [15, 5]. The challenge becomes even more pronounced in scenarios involving highly heterogeneous image appearances, large unconstrained fields of view, or significant modality-specific artifacts — such as speckle noise in ultrasound or bias fields in MRI. In registration of prostate MRI with Transrectal Ultrasound (TRUS), one of the most clinically relevant and extensively studied multimodal tasks, state-of-the-art methods — including learning-based approaches — consistently report TRE values in the range of 4–11 m, with only marginal improvements over classical iterative techniques despite the focus on small regions of interest during prediction [32]. This is particularly concerning, given that the clinically tolerated targeting error for prostate biopsy guidance is often cited as ≤ 2.5 m [67]. More broadly, recent surveys confirm that ultrasound-based multimodal registration constitutes one of the most challenging subfields, reporting an average TRE of 4.95 m for the best considered method on a manually curated dataset [73]. Even methods specifically designed for CT–ultrasound or MRI–ultrasound settings, including those leveraging self-similarity descriptors [36, 37] or cross-modal attention [63], have not demonstrated consistent sub-3 m accuracy in prospective clinical evaluations, reflecting the fundamental difficulty of establishing reliable correspondences across such dissimilar intensity distributions. In addition, the performance and generalization of deep learning-based registration methods are strongly constrained by the scarcity of large-scale annotated datasets with accurate voxel-wise correspondence ground truth. Unlike for the monomodal settings, multimodal registration inherently lacks “ground truth” deformation fields, necessitating weakly-supervised, synthetic, or self-supervised training strategies that introduce additional uncertainty [40, 78]. This issue is further compounded by domain shift across scanners, acquisition protocols, and patient populations, which have been shown to degrade TRE performance by several millimeters when models are evaluated outside their training distribution [38]. Concretely, methods achieving median TRE values below 2 m on in-distribution test sets have been observed to exceed 4–5 m on out-of-distribution data [31, 38], a degradation that is especially acute for learning-based methods that overfit to scanner-specific contrast profiles or resolution characteristics. Recent large-scale approaches, including inverse-consistent diffeomorphic frameworks such as multiGradICON [17], aim to address these challenges through improved regularization, symmetry constraints, and scalability to diverse training distributions. While these methods demonstrate improved stability and competitive benchmark performance, evaluations on population-scale datasets, such as UK Biobank [65], reveal persistent variability in alignment quality, with residual errors still exceeding clinically desirable thresholds (Dice coefficient > 0.8, TRE > 5 m) [7] in a non-negligible fraction of cases, highlighting the gap between controlled benchmark performance and real-world clinical deployment. 3 Evaluated registration methods Based on the previous analysis of related work, we focus on largely used, state-of-the-art methods. For better representativeness, we evaluate methods spanning classical optimization, handcrafted multimodal descriptors, and modern deep learning approaches including CNN- and foundation-model-based registration frameworks. The methods were selected on the basis of how widely they were used and the availability of ready-to-use packages or github implementations. Given these constraints, Table 1 summarizes the methods investigated in this study, reporting their original publications and how to retrieve them, while a brief presentation of each of them is provided below. Given the common trend of using handcrafted similarity-based metrics when training DL methods, we chose to use the MIND-SSC loss metric (ℒMIND−SSCL_MIND-SSC) as a common ground when training models. This also includes the pretrained version of the multiGradICON foundation model, which was already trained using MIND-SSC loss by the authors [17]. Anatomically-guided registration In addition to the proposed experiments, an assessment of the performance of anatomically-guided registration is proposed and implemented for the ANTs SyN and GradICON methods, respectively called ANTS SyN seg. and GradICON seg. in the following, representing the classical and deep-learning side of the optimization strategies. For classical methods, such as ANTs SyN, this is performed by giving as input the segmentation masks of homologous structures in IMI_M and ITI_T instead of the target and moving images. The resulting deformation map is then applied to the full image. For GradICON, the modified implementation consists in the introduction of an additional soft Dice loss term, evaluating the matching of the registered (linear interpolation) and target segmentations, thus favoring the alignment of segmented structures during training. Table 1: Considered multimodal deformable registration methods for benchmark. Method Reference publication Category Code repository ANTs SyN [3] Classic https://antspyx.readthedocs.io/en/latest/ Elastix [44] Classic https://github.com/InsightSoftwareConsortium/ITKElastix Deforming demons [71, 48] Classic https://github.com/InsightSoftwareConsortium/ITKElastix VoxelMorph [5] CNN https://github.com/htwin/voxelmorph_torch/tree/master GradICON [66] CNN https://github.com/MICV-yonsei/M2M-Reg M2M-GradICON [12] CNN https://github.com/MICV-yonsei/M2M-Reg multiGradICON [17] Foundation model https://github.com/uncbiag/uniGradICON ANTs SyN ANTs SyN [3] performs deformable registration in the space of diffeomorphisms, ensuring smoothness and invertibility of the estimated mappings. The transformations are generated by time-dependent velocity fields through forward ϕMT _MT and backward ϕTM _TM flows, defined by: ∂ϕMT(x,t)∂t ∂ _MT(x,t)∂ t =vMT(ϕMT(x,t),t), =v_MT( _MT(x,t),t), ϕMT(x,0) _MT(x,0) =x, =x, (2) ∂ϕTM(x,t)∂t ∂ _TM(x,t)∂ t =vTM(ϕTM(x,t),t), =v_TM( _TM(x,t),t), ϕTM(x,0) _TM(x,0) =x. =x. Here, ϕMT(x,t) _MT(x,t) (resp. ϕTM(x,t) _TM(x,t)) denotes the position at time t of a point initially located at x, i.e., a pointwise evaluation of the flow, whereas ϕMT(⋅,t) _MT(·,t) (resp. ϕTM(⋅,t) _TM(·,t)) denotes the full deformation map at time t, i.e., a diffeomorphism from Ω to Ω . SyN adopts a symmetric formulation by introducing two velocity fields vMT(x,t)v_MT(x,t) and vTM(x,t)v_TM(x,t), defined for t∈[0,12]t∈[0, 12], which transport the moving image IMI_M and the target image ITI_T toward a common midpoint. The corresponding energy functional is ℰ(vMT,vTM)=∫01/2(‖LvMT(⋅,t)‖22+‖LvTM(⋅,t)‖22)t+(IM∘ϕMT(⋅,12),IT∘ϕTM(⋅,12)),E(v_MT,v_TM)= _0^1/2 (\|Lv_MT(·,t)\|_2^2+\|Lv_TM(·,t)\|_2^2 )\,dt+D (I_M _MT(·, 12),\,I_T _TM(·, 12) ), (3) where L is a differential operator enforcing regularity and D is a similarity measure between images. The final transformation mapping IMI_M to ITI_T is obtained as ϕ=ϕMT(⋅,12)∘ϕTM−1(⋅,12).φ= _MT(·, 12) _TM^-1(·, 12). (4) This symmetric construction makes the method invariant to the choice of reference image and improves inverse consistency. Deforming Demons Deforming Demons estimates a dense displacement field ϕMT _MT through iterative updates derived from optical flow principles. At each iteration, the update is computed as ϕMTk+1=ϕMTk+(IT−IM∘(Id+ϕMTk))∇IT(IT−IM∘(Id+ϕMTk))2+‖∇IT‖22, _MT^k+1= _MT^k+ (I_T-I_M (Id+ _MT^k))∇ I_T(I_T-I_M (Id+ _MT^k))^2+\|∇ I_T\|_2^2, (5) which aligns image intensities using local gradient information. The displacement field is regularized via Gaussian smoothing to enforce spatial coherence. Extensions to diffeomorphic Demons introduce symmetrization or parametrize the transformation by a stationary velocity field vMTv_MT, and compute ϕMT=exp(vMT) _MT= (v_MT), ensuring invertibility while retaining the same local update mechanism. Elastix parametric registration Elastix implements parametric registration by modeling the transformation as a linear combination of basis functions. The deformation is expressed as ϕMTθ(x)=x+∑i=1pθiβi(x), _MT^θ(x)=x+ _i=1^p _i _i(x), (6) where βi _i are typically B-spline basis functions defined on a control point grid, and p is the number of such functions. The parameters θi _i are optimized by minimizing the energy functional given in Equation 1. This formulation constrains the deformation to a low-dimensional space, enabling efficient optimization and explicit control over smoothness. VoxelMorph VoxelMorph formulates registration as a supervised or unsupervised learning problem, where a neural network fθf_θ predicts a dense deformation field from image pairs: ϕMT=fθ(IT,IM). _MT=f_θ(I_T,I_M). (7) The network is trained by minimizing the energy functional of Equation 1 with ℛ=‖∇ϕMT‖22R=\|∇ _MT\|_2^2. Diffeomorphic variants predict a stationary velocity field vMTv_MT and compute ϕMT=exp(vMT) _MT= (v_MT) using scaling-and-squaring integration, ensuring invertibility of the transformation. GradICON GradICON learns bidirectional transformations between image pairs and enforces inverse consistency during training. The model predicts forward and backward deformations ϕTM _TM and ϕMT _MT, and minimizes a loss function that includes the inverse-consistency constraint ℒICL_IC ℒIC=‖∇[ϕTM∘ϕMT]−Id‖22,L_IC=\|∇[ _TM _MT]-Id\|_2^2, (8) as an additional regularizer ℛR, which penalizes deviations from perfect inversion. The full objective function also includes an image similarity term and smoothness regularization. Spatial image gradients are incorporated in the learning process to stabilize correspondence estimation and improve alignment near edges. M2M-GradICON Figure 1: Schematic cyclic registration from M2M-GradICON. M2M-GradICON addresses multimodal registration by first mapping the input images into a shared, mono-modal representation. To enforce consistency between the two modalities, the authors introduce a four-image cyclic registration scheme, illustrated in Figure 1. Specifically, in addition to the original reference and moving images, ITI_T and IMI_M, the authors also consider their corresponding (bridge) images in the shared representation, IT′I_T and IM′I_M . Deformation fields are estimated between the original and shared representations, namely ϕMM′ _M and ϕM′M _M M between IMI_M and IM′I_M , and ϕTT′ _T and ϕT′T _T T between ITI_T and IT′I_T . Two weighting hyperparameters, λcan _can and λIC _IC, control the contributions of the canonical consistency regularizer ℒcanL_can and the inverse-consistency regularizer ℒICL_IC, respectively. The resulting cost is : ℒ=ℒsim+λcanℒcan+λICℒIC,L=L_sim+ _canL_can+ _ICL_IC, (9) with ℒsim= _sim= ((IM∘ϕMM′),IM′)+((IM′∘ϕM′M),IM)+ \! ((I_M _M ),I_M )+D\! ((I_M _M M),I_M )+ ((IT∘ϕTT′),IT′)+((IT′∘ϕT′T),IT) \! ((I_T _T ),I_T )+D\! ((I_T _T T),I_T ) ℒcan _can =‖∇[ϕMT∘ϕTM′∘ϕM′T′∘ϕT′M−Id]‖22 =\|∇[ _MT _TM _M T _T M-Id]\|_2^2 ℒIC _IC =‖∇[ϕTM∘ϕMT]−Id‖22 =\|∇[ _TM _MT]-Id\|_2^2 This ensures transitive consistency of the deformation fields. The optimization is performed over a shared network that produces pairwise mappings, resulting in a globally coherent transformation space across the dataset. Note: The present description matches the code provided by the github link in Table 1, which is slightly different from the one described in [12]. multiGradICON multiGradICON incorporates multi-scale and group-wise registration within an inverse-consistent learning framework. The deformation is represented as a composition of transformations across resolution levels: ϕ=ϕ(1)∘⋯∘ϕ(L),φ=φ^(1) ·s φ^(L), (10) where each ϕ(l)φ^(l) captures deformations at a specific scale. Training enforces similarity, smoothness, and consistency constraints at each level, improving robustness to large deformations and reducing local minima. The hierarchical structure enables coarse-to-fine alignment while maintaining global coherence across multiple images. 4 Benchmarking methodology This section describes the experimental framework used to evaluate and compare multimodal deformable registration methods. The benchmark is designed to assess performance across multiple anatomical regions, imaging modalities, and registration scenarios. We first present the datasets and the synthetic deformation protocol used to establish quantitative ground truth. We then describe the training strategy, evaluation metrics, and benchmarking procedures adopted to ensure a fair and reproducible comparison between classical and learning-based approaches. 4.1 Datasets The benchmark study is conducted on three datasets that cover healthy and pathological subjects on different anatomical regions and multimodal configurations. The first dataset is the publicly available BRATS dataset [1, 53], containing pre-registered T1 and T2-weighted images of patients with gliomas, including low-grade and high-grade brain tumors (e.g., astrocytomas and glioblastomas). The second dataset is the publicly available Human Connectome Project (HCP) [70], already pre-processed as described in [25] and containing pre-registered T2-weighted and diffusion b0b_0 images of healthy young adults with no major neurological abnormalities affecting brain anatomy. The third dataset, called IMAG2 (Pelvis), is a private one containing multimodal pelvic MRI of 80 pediatric subjects from the Hôpital Necker Enfants-Malades, acquired under the ClinicalTrials ID NCT06224985. It contains a mixture of control patients (10%) and patients with pelvic tumors (40%) or pelvic malformations (50%). More information about this dataset can be found in [58]. For each patient, it contains unregistered coronal T2-weighted MRI (reconstruction voxel size 0.86×0.88×0.86 mm3m^3) and multi-gradient axial DWI image (acquisition voxel size 3.3×2.3×3.6 mm3m^3, NEX 2, 1 b0b_0 image, 25 directions at b-value 600 s/mm2s/m^2) acquired on a 3T GE Discovery MR700 during the same procedure. Diffusion-weighted images were preprocessed using MRtrix3 [68], including denoising, Gibbs ringing artifact removal, motion and eddy-current correction, and bias field homogenization. The b0b_0 image was subsequently generated by extracting the only non-diffusion-weighted (b-value = 0 s/mm2s/m^2) volume from the preprocessed diffusion dataset. These datasets include brain and pelvic anatomies of control and tumor/malfomation subjects, enabling evaluation across distinct anatomical structures and deformation regimes ranging from controlled synthetic transformations to real intra-patient multimodal misalignment. All datasets are preprocessed using a standardized pipeline. For the BRATS and HCP datasets, the images are exploited at their respective resampled resolution of 1×1×11× 1× 1 m and 1.25×1.25×1.251.25× 1.25× 1.25 m, while the IMAG2 (pelvis) data uses a resampling of 1.25×1.25×3.51.25× 1.25× 3.5 m corresponding to the mean resolution of all available images. Intensity normalization is performed using z-scoring, and a cropping to a common Field-Of-View (FOV) is performed using the information present in the DICOM headers. Each dataset is exploited at a specific cropping size to balance computational efficiency and FOV: BRATS and HCP are cropped at 192×192×192192× 192× 192 to minimize the impact of dark background on the computation cost while the IMAG2 (pelvis) dataset is of size 192×256×128192× 256× 128, to provide a trade-off between keeping additional anatomical information and computation time. Segmentation labels for the BRATS and HCP datasets are provided by the respective project leaders. For the IMAG2 (pelvis) dataset, manual T2w image segmentations of the Hipbones, Sacrum, L5 vertebra and Bladder were provided by a medical expert and checked by at least two other experts, including an experienced radiologist. Given the difficulty of segmenting bone structures in b0b_0 images, the segmentation was obtained under the assumption that bones are rigid structures and do not change between the consecutive T2w and b0b_0 images. Hence, a rigid registration from the T2w to the b0b_0 image was performed and checked manually to align the T2w mask with visible anatomy in the b0b_0 image. The bladder was very visible in the b0b_0 and manually segmented from scratch. For each dataset, 20% of the subjects are held out as a test set, while maintaining the proportions of healthy vs. pathological cases as close as possible to the one of the entire set. The remaining 80% are split into training and validation subsets using an 80/20 ratio, resulting in 64% training, 16% validation, and 20% testing. Learning-based methods use the validation set to select the best model during the training process based on the minimum validation loss value, whereas classical registration methods use it for grid-search of optimal parameters. The test set is strictly held out and used exclusively for final evaluation across all methods. The number of subjects and split distribution are reported in Table 2. Table 2: Dataset description. Dataset # Cases # Train-val-test Availability BRATS [53] 1251 801-200-250 Public HCP [70] 708 454-113-141 Public IMAG2 (pelvis) 80 52-12-16 Proprietary 4.2 Synthetic Deformation Protocol For datasets containing pre-registered pairs (e.g. BRATS and HCP), synthetic transformations are generated to enable a quantitative evaluation of the registration results. For each pair of registered source and target modality images, random B-spline displacement fields are sampled independently using a coarse control grid and interpolated over the full target modality image domain. The control point grid size is fixed to (4×4×4)(4× 4× 4), corresponding to the minimum grid size imposed to support cubic B-splines (degree (3,3,3)(3,3,3)). Such a coarse grid induces smooth, low-frequency deformations, as each control point influences a large spatial region through the compact support of the spline basis functions, thereby limiting the representation of high-frequency spatial variations. The resulting displacement field is then applied to the target modality image to obtain a synthetically deformed target image. Two samples from the BRATS and HCP datasets are shown in Figure 2. Control point displacements ic_i are resampled independently from a uniform distribution: i∼(−s,s)3,c_i (-s,s)^3, (11) where s=15s=15 voxels (15-19 m) defines the maximum displacement amplitude along each spatial dimension, thus providing an overall spectrum of small and large patient displacements, corresponding to slipping or intentional motion, representing a range of plausible anatomical intra-subject displacements observed in clinical practice. The dense displacement field is obtained via BSpline interpolation of the control grid over normalized spatial coordinates in [0,1]3[0,1]^3, yielding spatially smooth deformation fields without additional explicit regularization. The target modality image (resp. segmentation) is then warped using trilinear (resp. nearest neighbor) interpolation according to the resulting displacement field to create a synthetic target image. Part of the resulting images were checked for anatomical consistency by a biomedical engineer with more than 7 years of experience in medical imaging. Subject 1Subject 2 Figure 2: Synthetic deformation examples on the BRATS and HCP datasets. Each row corresponds to a different patient sample. The grid overlay illustrates the global deformation pattern. 4.3 Training protocol For all the methods that require training, the paradigm applied is the one described in their respective papers and, where applicable, the similarity metric is replaced with the MIND-SSC loss. For consistency and fair comparison, the training is unsupervised for all methods. We accumulate gradients to match an effective batch size of 8 samples and perform validation every 5 training epochs. All models are trained with an Adam optimizer with an initial learning rate of 10−410^-4 and a learning rate scheduler stepping based on the validation loss with a patience of 10 epochs and decrease factor of 0.5. Early stopping is called when the learning rate falls below 10−710^-7. Furthermore, all trainings were performed on the IDRIS Jean-Zay cluster using the pytorch-gpu/py3/2.8.0 environment. VoxelMorph, and GradICON trainings were performed on a single NVIDIA Tesla V100 SXM2 with 32 GB VRAM. M2M-GradICON required more memory and used a single NVIDIA A100 SXM4 with 80 GB VRAM. 4.4 Evaluation Metrics The registration performance is evaluated using overlap-based and boundary-based metrics computed on raw images and anatomical segmentations. As a result, five complementary accuracy metrics are presented hereafter. The Dice similarity coefficient (DSC) is used to measure volumetric overlap between registered and reference segmentations, and is reported as the average across all studied anatomical structures. In the following, it will be noted as DSC¯ DSC for simplicity. Boundary accuracy is assessed using the 95th-percentile Hausdorff Distance (HD95), averaged across all studied anatomical structures and noted as HD95¯ HD95 in the following. The value of the MIND-SSC loss is also reported as a surrogate measure of structural similarity to assess the global image improvement. The MIND-SSC loss is defined as follows. Given a local neighborhood N, an image I, a global noise estimate σ and SSD the Sum of Squared Differences, the self-similarity descriptor between two patches x and y in N is given by: S(I,x,y)=exp(−SSD(x,y)σ2)S(I,x,y)=exp (- SSD(x,y)σ^2 ) (12) Given I and J two images defined on an image domain Ω and x∈Ωx∈ the center of a homologous patch within these two images, the MIND-SSC loss ℒMIND−SSCL_MIND-SSC is obtained as: ℒMIND−SSC=1||∑x∈Ω∑y∈|S(I,x,y)−S(J,x,y)|.L_MIND-SSC= 1|N| _x∈ _y |S(I,x,y)-S(J,x,y)|. (13) In addition, the determinant of the Jacobian of the displacement field is computed to obtain the folding-ratio (f_\_ratio) percentage of the proposed transformation. Finally, given that the registration on the BRATS and HCP datasets was computed on synthetically displaced images and that the original displacement field is available, it is interesting to report the relative error of the Root-Mean-Square-Error (DVF rel. error) between the original displacement field and the one predicted for the registration. The result is provided as a percentage value of the difference with respect to the norm of the original displacement field. A detailed description of these metrics is given in Table 3 where a and b denote binary segmentations; AlA_l and BlB_l denote binary segmentation of multilabel segmentations A and B restrained to label l∈[1;N]l∈[1;N]; d(⋅,⋅)d(·,·) is the surface distance; P95P_95 denotes the 95th-percentile; ϕφ is the estimated transformation; JϕJ_φ its Jacobian matrix; Ω the image domain; and ϕpred _pred and ϕgt _gt the predicted and ground-truth displacement vector fields (DVFs), respectively. Table 3: Summary of the evaluation metrics. Metric Definition DSC DSC(a,b)=2|a∩b||a|+|b| (a,b)= 2|a∩ b||a|+|b| DSC¯ DSC DSC¯(A,B)=1N∑l=1NDSC(Al,Bl) DSC(A,B)= 1N _l=1^NDSC(A_l,B_l) HD95 HD95(a,b)=max(P95(d(a,b)),P95(d(b,a))) 95(a,b)= \! (P_95\! (d(a,b) ),P_95\! (d(b,a) ) ) HD95¯ HD95 HD95¯(A,B)=1N∑l=1NHD95(Al,Bl) HD95(A,B)= 1N _l=1^NHD95(A_l,B_l) ℒMIND−SSCL_MIND-SSC ℒMIND−SSC=1||∑x∈Ω∑y∈|S(I,xi,y)−S(J,xj,y)|. _MIND-SSC= 1|N| _x∈ _y |S(I,x_i,y)-S(J,x_j,y)|. f_ratio 100⋅#x:det(Jϕ(x))<0#Ω 100· \#\x: (J_φ(x))<0\\# DVF rel. error 100⋅‖ϕpred−ϕgt‖2‖ϕgt‖2 100· \| _pred- _gt\|_2\| _gt\|_2 All metrics are computed by propagating segmentation labels from the moving image using the estimated transformation and comparing them to the reference labels in the target image space. Results are reported as mean and standard deviation among subjects. In addition, we report computational efficiency metrics such as the runtime, throughput, CPU and GPU usage. Runtime and throughput were measured over 50 repeated inference runs on the same hardware platform. CPU utilization is normalized over the total available system capacity. GPU memory corresponds to peak allocated GPU memory during inference. All reported metrics were computed using a Linux WSL2 environment (Linux 6.6.87.2) using Python 3.12.3 on a 32-core x86_64 CPU system equipped with 31.2 GB of RAM and a single NVIDIA GeForce RTX 5090 GPU with 31.8 GB VRAM running CUDA 13.0. 4.5 Benchmarking Protocol All methods are evaluated under a unified benchmarking framework to ensure fair comparison. Identical preprocessing steps, image pairs, and intra-patient registration settings are used across all methods. The same evaluation pipeline is applied uniformly to all outputs. For each classical registration method, the main set of tunable hyper-parameters was selected and optimized on a subset of 10 cases from the validation set. Hyper-parameter configurations were ranked according to the average Dice similarity coefficient (DSC¯ DSC), and the best-performing configuration was subsequently used for all evaluations on the corresponding dataset. The selected hyper-parameters for each method and dataset are reported in Table 4. As for learning-based methods the large training cost makes it intractable to perform extensive hyper-parameter grid-search, we thus decided to evaluate them using their default hyper-parameters, as proposed in their respective articles. Unlike the other learning-based methods, the pretrained multiGradICON model imposes a fixed input size of 175×175×175175× 175× 175 voxels. Consequently, all images evaluated with this model were cropped to satisfy this constraint while conserving the maximum amount of segmented organs within the bounds of the test images. The resulting performance should therefore be interpreted in light of this architectural limitation. Table 4: Best hyper-parameter configurations selected for each classical registration method and dataset. Method Hyper-parameter BRATS HCP IMAG2 (pelvis) ANTs SyN grad_step 0.2 0.2 0.1 flow_sigma 3.0 2.0 3.0 total_sigma 0.5 0.0 0.0 reg_iterations (80,40,20) (80,40,20) (80,40,20) Elastix maximum_number_of_iterations 512 256 512 number_of_resolutions 3 3 4 number_of_spatial_samples 2048 2048 2048 bspline_final_grid_spacing (m) 32.0 16.0 32.0 Log-Demons n_iter 30 80 30 sigma_update 1.5 1.5 1.5 sigma_field 1.0 1.0 1.0 5 Results This section presents the quantitative and qualitative results of the benchmark described in Section 4. The quantitative section focuses on the analyses of the metrics reported in Section 4.4 applied to all the registration methods, including their anatomy-based versions described in paragraph “Anatomically guided registration” of Section 3. Qualitative results are then presented in Section 5.2. 5.1 Registration Accuracy This section presents the quantitative results for all registration approaches defined in Section 3. Given the difference in input data between the anatomy-guided and image-guided registration experiments, the presentation of the respective results is divided in Sections 5.1.1 and 5.1.2 Table 5: Registration accuracy results on BRATS. Method DSC¯ DSC (%) ↑ HD95¯ HD95 (m) ↓ ℒMIND−SSCL_MIND-SSC ↓ DVF rel. error (%)↓ f_ratio ⋅102· 10^2 (%) ↓ Unregistered 42.6 ± 13.5 7.6 ± 2.1 47.6 ± 5.5 100 – ANTs SyN 59.2 ± 19.0 7.1 ± 17.8 50.8 ± 15.8 94.3 ± 2.3 0.0003 ± 0.0012 Elastix 74.8 ± 9.6 2.4 ± 1.0 44.9 ± 9.5 99.0 ± 14.3 0.0000 ± 0.0000 Demons 41.8 ± 13.3 7.7 ± 2.1 38.5 ± 5.2 99.4 ± 0.3 0.0001 ± 0.0004 VoxelMorph 62.3 ± 11.1 3.9 ± 1.4 10.8 ± 2.4 88.5 ± 2.5 2.5562 ± 0.4611 GradICON 77.0 ± 8.0 1.9 ± 0.5 12.2 ± 3.2 81.3 ± 4.7 3.2633 ± 0.7565 M2M_GradICON 55.1 ± 12.5 4.9 ± 1.7 23.0 ± 3.8 89.5 ± 2.4 0.0018 ± 0.0046 multiGradICON 64.0 ± 11.9 3.9 ± 1.8 20.7 ± 4.8 81.2 ± 6.0 0.0061 ± 0.0063 ANTs SyN seg. 93.3 ± 3.9 1.1 ± 0.3 87.3 ± 12.0 99.7 ± 0.3 0.0000 ± 0.0000 GradICON seg. 76.7 ± 9.9 2.0 ± 0.7 16.5 ± 3.6 82.0 ± 4.6 0.0268 ± 0.0321 Table 6: Registration accuracy results on HCP. Method DSC¯ DSC (%) ↑ HD95¯ HD95 (m) ↓ ℒMIND−SSCL_MIND-SSC ↓ DVF rel. error (%)↓ f_ratio ⋅102· 10^2 (%) ↓ Unregistered 19.7 ± 4.6 9.5 ± 1.5 116.2 ± 9.3 100 – ANTs SyN 17.0 ± 11.6 15.4 ± 4.8 120.1 ± 9.0 99.5 ± 0.5 0.0135 ± 0.0135 Elastix 44.7 ± 9.8 5.9 ± 2.2 121.6 ± 7.6 211.3 ± 38.2 0.0169 ± 0.0707 Demons 29.7 ± 8.2 8.4 ± 1.7 115.2 ± 9.5 98.6 ± 0.5 0.0017 ± 0.0022 VoxelMorph 54.7 ± 3.0 3.0 ± 0.3 8.7 ± 1.4 142.3 ± 8.3 6.4165 ± 0.6430 GradICON 57.0 ± 4.4 3.7 ± 0.5 17.7 ± 2.6 211.2 ± 14.3 4.9456 ± 0.2164 M2M_GradICON 45.0 ± 6.6 4.4 ± 4.2 91.8 ± 7.4 94.0 ± 3.1 0.0011 ± 0.0042 multiGradICON 22.3 ± 7.5 10.4 ± 2.1 67.4 ± 3.9 155.9 ± 14.8 0.6305 ± 0.2771 ANTs SyN seg. 86.3 ± 3.6 1.8 ± 0.5 139.1 ± 11.2 98.5 ± 0.8 0.0001 ± 0.0005 GradICON seg. 74.6 ± 3.3 1.6 ± 0.3 21.8 ± 2.0 162.2 ± 10.9 2.3012 ± 0.1990 Table 7: Registration accuracy results on IMAG2 (pelvis). Method DSC¯ DSC (%) ↑ HD95¯ HD95 (m) ↓ ℒMIND−SSCL_MIND-SSC ↓ DVF rel. error (%)↓ f_ratio ⋅102· 10^2 (%) ↓ Unregistered 67.0 ± 14.3 8.0 ± 4.2 17.3 ± 9.7 – – ANTs SyN 66.2 ± 12.8 7.9 ± 3.7 42.2 ± 21.3 – 0.0213 ± 0.0846 Elastix 59.2 ± 13.8 10.3 ± 5.2 26.9 ± 17.6 – 0.0134 ± 0.0536 Demons 61.3 ± 14.1 9.0 ± 4.1 23.8 ± 11.9 – 0.0912 ± 0.0781 VoxelMorph 65.1 ± 14.8 7.9 ± 3.7 14.6 ± 12.1 – 1.8370 ± 1.0131 GradICON 67.1 ± 14.3 8.0 ± 4.2 17.2 ± 9.7 – 0.0000 ± 0.0000 M2M_GradICON 61.6 ± 11.7 9.0 ± 4.2 23.4 ± 12.4 – 0.0000 ± 0.0000 multiGradICON 60.8 ± 10.4 8.7 ± 3.9 81.0 ± 31.2 – 0.1887 ± 0.1674 ANTs SyN seg. 81.2 ± 9.5 5.5 ± 4.4 40.0 ± 18.2 – 0.0025 ± 0.0087 GradICON seg. 67.1 ± 13.9 7.9 ± 4.0 19.4 ± 10.1 – 0.0010 ± 0.0023 5.1.1 Image-guided registration Tables 5–7 summarize the registration performance of all evaluated methods using the metrics defined in Section 4.4. Higher DSC¯ DSC values indicate better overlap, whereas lower HD95¯ HD95, ℒMIND−SSCL_MIND-SSC, DVF error, and fratiof_ratio values indicate superior performance. Note that DVF error is only available for BRATS and HCP because these datasets were generated using known synthetic deformations. The IMAG2 (pelvis) dataset corresponds to a real registration task and therefore does not provide a deformation ground truth. Across all datasets, substantial variability is observed between methods and anatomical regions, highlighting the difficulty of multimodal deformable registration without explicit anatomical supervision. Moreover, the different metrics reveal important trade-offs between segmentation overlap, image-based similarity, deformation recovery, and transformation regularity. On the BRATS dataset, all methods except Demons achieved substantial improvements over the initial misalignment. Among the classical approaches, Elastix obtained the strongest overlap-based performance, reaching a DSC¯ DSC of 74.8±9.674.8± 9.6 and an HD95¯ HD95 of 2.4±1.02.4± 1.0 m. However, its DVF error remained high (99.0±14.399.0± 14.3), indicating poor recovery of the underlying synthetic deformation despite good anatomical alignment. Demons achieved the lowest MIND-SSC loss among classical methods (38.5±5.238.5± 5.2) but failed to improve DSC¯ DSC and HD95¯ HD95 compared with the unregistered baseline. Among learning-based approaches, GradICON produced the best geometric registration accuracy, achieving the highest DSC¯ DSC (77.0±8.077.0± 8.0) and the lowest HD95¯ HD95 (1.9±0.51.9± 0.5 m). VoxelMorph achieved the lowest MIND-SSC loss overall (10.8±2.410.8± 2.4), suggesting excellent local structural alignment, while multiGradICON obtained the lowest DVF error (81.2±6.081.2± 6.0), closely followed by GradICON (81.3±4.781.3± 4.7). In terms of topology preservation, the classical methods produced almost perfectly diffeomorphic transformations with negligible folding ratios, whereas VoxelMorph and GradICON exhibited substantial folding (fratio=2.56f_ratio=2.56 and 3.263.26, respectively). M2M-GradICON represented a compromise between both families, achieving low folding (0.00180.0018) at the cost of reduced registration accuracy. The HCP dataset exhibits a similar pattern but with a greater separation between methods. ANTs SyN degraded alignment relative to the baseline, yielding lower DSC¯ DSC (17.0±11.617.0± 11.6), higher HD95¯ HD95 (15.4±4.815.4± 4.8 m), and increased MIND-SSC loss (120.1±9.0120.1± 9.0). Elastix improved overlap considerably (DSC¯ DSC 44.7±9.844.7± 9.8), but again showed very poor deformation recovery with a DVF error exceeding 200200. Among the learning-based methods, GradICON achieved the highest DSC¯ DSC (57.0±4.457.0± 4.4), whereas VoxelMorph obtained the lowest HD95¯ HD95 (3.0±0.33.0± 0.3 m) and lowest MIND-SSC loss (8.7±1.48.7± 1.4). By contrast, M2M-GradICON produced the most accurate recovery of the synthetic deformation, obtaining the lowest DVF error (94.0±3.194.0± 3.1), substantially outperforming all competing approaches. MultiGradICON showed limited improvements over the baseline, performing poorly across DSC¯ DSC, HD95¯ HD95, and MIND-SSC loss despite maintaining a relatively low folding ratio. The IMAG2 (pelvis) dataset proved considerably more challenging than the synthetic brain benchmarks. Most methods produced only marginal improvements relative to the unregistered baseline, and several degraded performance. Among the classical methods, ANTs SyN preserved the baseline overlap (DSC¯ DSC 66.2±12.866.2± 12.8) and HD95¯ HD95 (7.9±3.77.9± 3.7 m) but substantially increased the MIND-SSC loss (42.2±21.342.2± 21.3), suggesting poorer multimodal structural consistency. Elastix and Demons generally reduced overlap performance relative to the baseline. Among learning-based methods, GradICON achieved the highest DSC¯ DSC (67.1±14.367.1± 14.3), matching the baseline performance while maintaining a low MIND-SSC loss (17.2±9.717.2± 9.7). VoxelMorph produced the lowest MIND-SSC loss value overall (14.6±12.114.6± 12.1) and competitive HD95¯ HD95 (7.9±3.77.9± 3.7 m), but at the expense of a large folding ratio (1.84±1.011.84± 1.01). By contrast, GradICON and M2M-GradICON generated essentially fold-free transformations (fratio≈0f_ratio≈ 0) while maintaining comparable registration accuracy. MultiGradICON performed poorly on this dataset, yielding a substantially increased MIND-SSC loss (81.0±31.281.0± 31.2) and lower overlap than the baseline. Overall, the results demonstrate that registration performance depends strongly on the anatomical region and imaging characteristics. On the synthetic BRATS and HCP datasets, learning-based approaches outperformed classical methods in terms of overlap accuracy and image consistency. However, a distinction emerges between methods that best recover the known synthetic deformation and those that optimize image-based registration metrics. In particular, M2M-GradICON and multiGradICON generally achieved the lowest DVF errors and folding ratios, whereas GradICON and VoxelMorph achieved superior DSC¯ DSC, HD95¯ HD95, and MIND-SSC loss scores. This suggests that accurate reconstruction of the synthetic deformation does not necessarily translate into the best anatomical alignment under the chosen evaluation criteria. On the real IMAG2 (pelvis) dataset, where no deformation ground truth is available, all methods exhibited more modest gains, indicating that the registration task is substantially more difficult. Among the unsupervised methods, GradICON provided the most consistent behavior across datasets, combining strong overlap accuracy with low MIND-SSC loss, while maintaining near-diffeomorphic transformations on IMAG2 (pelvis). 5.1.2 Anatomy-guided registration The lower part of Tables 5–7 reports the performance of registration methods when anatomical segmentations are explicitly incorporated into the optimization process. Because these approaches exploit supervision unavailable to the unsupervised methods, they should be interpreted separately and may be viewed as an approximate upper bound on the achievable registration accuracy when anatomical annotations are available. On the BRATS dataset, ANTs SyN seg. achieved the strongest overlap-based performance, obtaining a DSC¯ DSC of 93.3±3.993.3± 3.9 and the lowest HD95¯ HD95 of 1.1±0.31.1± 0.3 m. However, these improvements were accompanied by a very large MIND-SSC loss (87.3±12.087.3± 12.0), substantially higher than all learning-based methods and even higher than the unregistered baseline. Furthermore, despite the excellent overlap performance, the DVF error remained high (99.7±0.399.7± 0.3), indicating poor recovery of the underlying synthetic deformation. By contrast, GradICON seg. achieved lower overlap (DSC¯ DSC 76.7±9.976.7± 9.9) but substantially improved image-based consistency, yielding a much lower MIND-SSC loss value (16.5±3.616.5± 3.6) than ANTs SyN seg. while remaining higher than the standard GradICON version. Its DVF error (82.0±4.682.0± 4.6) was also markedly lower than that of ANTs SyN seg., indicating a deformation closer to the ground-truth transformation. Topology preservation further distinguished the two approaches: ANTs SyN seg. remained essentially fold-free (fratio≈0f_ratio≈ 0), whereas GradICON seg. introduced a small amount of folding (0.0268±0.03210.0268± 0.0321). These results illustrate that maximizing anatomical overlap does not necessarily imply improved multimodal consistency or more accurate recovery of the true deformation. The same trend is observed on HCP. ANTs SyN seg. achieved the highest DSC¯ DSC (86.3±3.686.3± 3.6) and near-optimal topology preservation with an almost-zero folding ratio (0.0001±0.00050.0001± 0.0005). However, it also produced the highest MIND-SSC loss (139.1±11.2139.1± 11.2) among all evaluated methods and a DVF error of 98.5±0.898.5± 0.8. GradICON seg. achieved slightly lower overlap (DSC¯ DSC 74.6±3.374.6± 3.3) but produced the best HD95¯ HD95 (1.6±0.31.6± 0.3 m), indicating more accurate boundary alignment. More importantly, its MIND-SSC loss (21.8±2.021.8± 2.0) was dramatically lower than that obtained by ANTs SyN seg., suggesting substantially better preservation of multimodal structural information. Nevertheless, its DVF error remained high (162.2±10.9162.2± 10.9), indicating that neither anatomy-guided approach accurately recovered the known synthetic deformation despite achieving strong registration metrics. This is probably because, while the anatomical input provides a solid basis for the registration of the segmentations, no constraint is available outside of the segmented area. As such, any field transforming the partial field (on the segmented areas) into a near-diffeomorphic one is considered a correct solution, thus limiting their ability to recover the actual deformation.Compared with the unsupervised variants, anatomy-guided optimization primarily improved geometric alignment measures while providing less clear benefits in terms of deformation recovery. On the IMAG2 (pelvis) dataset, where no deformation ground truth is available, ANTs SyN seg. again produced the highest overlap accuracy, achieving a DSC¯ DSC of 81.2±9.581.2± 9.5 and the lowest HD95¯ HD95 (5.5±4.45.5± 4.4 m). However, this gain was accompanied by a relatively large MIND-SSC loss (40.0±18.240.0± 18.2), suggesting that the optimization focused strongly on aligning annotated structures. GradICON seg. achieved substantially lower overlap, matching the baseline but preserved multimodal consistency more effectively, obtaining a considerably lower MIND-SSC loss value (19.4±10.119.4± 10.1). Both methods produced nearly diffeomorphic transformations, with very small folding ratios, although GradICON seg. generated the smoother deformation field overall (0.0010±0.00230.0010± 0.0023 versus 0.0025±0.00870.0025± 0.0087). As observed on the synthetic datasets, the improvements obtained through segmentation guidance were much more pronounced on overlap-based metrics than on image-based similarity measures. Overall, anatomy-guided registration substantially improves geometric alignment metrics, particularly DSC¯ DSC and HD95¯ HD95, and enables very accurate matching of annotated anatomical structures. However, the results consistently reveal a trade-off between segmentation overlap and multimodal image consistency. ANTs SyN seg. systematically achieves the highest overlap scores across all datasets while maintaining nearly fold-free transformations, but often produces substantially worse MIND-SSC loss values and does not consistently recover the underlying synthetic deformation. Conversely, GradICON seg. generally yields lower overlap performance but produces markedly lower MIND-SSC loss and, on BRATS, substantially better DVF recovery. These findings suggest that optimizing directly for anatomical correspondence can bias the registration towards the labeled structures and may not necessarily yield globally consistent multimodal alignments. Consequently, anatomy-guided methods should be evaluated using multiple complementary metrics rather than overlap measures alone. 5.2 Qualitative results Figures 3, 4 and 5 show the qualitative registration results for the three employed datasets, compared across all the methods described in Section 3. The subjects selected for display are the ones showing — regardless of the registration method — the highest improvement (largest difference between pre- and post-registration) on global Dice score for each dataset. Figure 3 shows improved registration results for ANTs SyN and Elastix. The largest structure is globally matched and the smallest inner one partly overlaps. The result of the Demons registration remains questionable. As for the deep-learning based algorithms, similar results are obtained. General overlap of the brain as a whole and the largest structure is obtained for all methods with the exception of M2M_GradICON which has more limited success as summarized in the quantitative table. Considering ANTs SyN Seg., the structural overlap of all the visible object is striking and has the overall smallest differences. Figure 3: Qualitative registration result of case BraTS-GLI-01259-000 (BRATS) with segmentation masks of the whole tumor. Top row: Moving T2 image (blue segmentation), Fixed T1 image (red segmentation), Initial overlay, overlay of the fixed and registerd image for classic approaches. Bottom row: Overlay of fixed and registered images from deep learning approaches as well as the comparison of GradICON and Ants SyN with additional anatomical guidance. Figure 4 reports the registration results on the HCP dataset focusing on the brain stem. The displayed patient mostly shows limited improvement with unrealistic registration of the skull of the T2 image and a poor overlap of the other structures in the brain. The ANTs SyN Seg. and GradICON Seg. showcase the best improvement in the structural overlap even though a significant misalignment subsists on the brain stem on ANTs SyN seg. Figure 4: Qualitative registration result of case HCP-792564 (HCP) with segmentation masks of the brain stem. Top row: Moving T2 image (blue segmentation), Fixed T1 image (red segmentation), Initial overlay, overlay of the fixed and registerd image for classic approaches. Bottom row: Overlay of fixed and registered images from deep learning approaches as well as the comparison of GradICON and Ants SyN with additional anatomical guidance. Figure 5 presents the results on a case from the IMAG2 (pelvis) dataset. At first glance, it is clear that the misalignment between the structues is less explicit than for the other datasets. As such, most of the registrations do not seem to significantly improve the results and some like Elastix, Demons and multiGradICON deteriorate the alignment of the pelvis. ANTs SyN seg. is the only method that improves the alignment of all the labeled structures. Figure 5: Qualitative registration result of case 1-2-42_190415 of the IMAG2 (pelvis) dataset. Segmentations include the Pelvis, L5 vertebra, Sacrum and bladder. Top row: Moving T2 image (blue segmentation), Fixed T1 image (red segmentation), Initial overlay, overlay of the fixed and registered image for classic approaches. Bottom row: Overlay of fixed and registered images from deep learning approaches as well as the comparison of ANTS SyN seg. and GradICON seg. The visible reduced size of the multiGradICON image is caused by the cropping of the images to fit multiGradICON’s input size parameters. 5.3 Computational Performance Table 8 reports computational efficiency on the HCP dataset. Learning-based methods were substantially faster than iterative optimization-based registration algorithms. VoxelMorph achieved the lowest runtime at 0.0280.028 s per registration pair and the highest throughput of 35.535.5 registrations/s. multiGradICON also demonstrated high efficiency with 0.1790.179 s runtime and 5.595.59 registrations/s. By contrast, classical methods required substantially longer execution times. Elastix was the slowest approach with 22.722.7 s runtime, followed by ANTs SyN at 8.788.78 s. Demons achieved moderate runtime performance (4.384.38 s) but exhibited the highest CPU utilization (90.8%90.8\%). Learning-based methods relied on GPU acceleration and required between 7.607.60 GB and 24.5624.56 GB of GPU memory, whereas classical methods operated exclusively on CPU resources. Overall, the results highlight a clear trade-off between registration accuracy and computational efficiency. While segmentation-guided classical approaches achieved the highest accuracy, learning-based methods enabled inference times that were two to three orders of magnitude faster. Table 8: Computational performance comparison of registration methods on the HCP dataset. Method Runtime ↓ Throughput ↑ CPU ↓ GPU ↓ (s) (reg/s) (%) (GB) ANTs SyN 8.78 ± 0.29 0.114 ± 0.004 24.54 ± 0.81 0.00 ± 0.00 Elastix 22.7 ± 0.42 0.044 ± 0.000 45.5 ± 0.4 0.00 ± 0.00 Demons 4.38 ± 0.08 0.228 ± 0.004 90.8 ± 1.00 0.00 ± 0.00 VoxelMorph 0.028 ± 0.001 35.5 ± 0.7 3.20 ± 0.40 7.60 ± 0.00 GradICON 0.350 ± 0.019 2.86 ± 0.15 3.16 ± 0.07 9.68 ± 0.00 M2M-GradICON 0.713 ± 0.010 1.40 ± 0.02 3.18 ± 0.02 24.56 ± 0.00 multiGradICON 0.179 ± 0.003 5.59 ± 0.10 3.20 ± 0.10 15.24 ± 0.00 Metrics: Runtime corresponds to mean wall-clock inference time per registration pair. Throughput denotes registrations processed per second and is derived as the inverse runtime. CPU usage is normalized over all logical CPU cores. GPU peak corresponds to maximum allocated GPU memory during inference. 6 Discussion Multimodal deformable image registration remains a highly challenging problem. Unlike monomodal registration, multimodal registration cannot rely on direct intensity correspondence and must instead infer spatial relationships between images that encode the same anatomy through fundamentally different physical contrast mechanisms. Consequently, the registration quality depends simultaneously on the transformation model, similarity measure, regularization strategy, anatomical region and modality pair. The results of this benchmark demonstrate that no evaluated method achieves consistently superior performance across all datasets and evaluation criteria, and that registration in a real-world scenario with limited training data remains a challenging and unsolved problem in many clinically relevant settings.. A first major observation is the strong dependence of registration performance on anatomical region and modality pair. Classical methods exhibit particularly heterogeneous behavior. ANTs SyN achieves moderate improvements on BRATS but degrades performance on HCP, whereas Elastix performs competitively on the synthetic brain benchmarks but deteriorates substantially on IMAG2 (Pelvis). This variability suggests that handcrafted multimodal similarity measures remain highly sensitive to the appearance relationship between modalities and to the anatomical context. In this specific case, the low visibility of the skull in the b0 of the HCP dataset is assumed to be the cause of such a failure for ANTs SyN. By contrast, learning-based approaches provide more consistent performance across datasets, with GradICON and VoxelMorph generally outperforming classical methods on BRATS and HCP. However, even the best-performing methods exhibit only limited to no improvement on the IMAG2 (pelvis) dataset, indicating that robust multimodal registration remains highly dependent on the clinical setup. A second and perhaps more surprising finding concerns the relationship between deformation recovery and registration accuracy. On the synthetic BRATS and HCP benchmarks, the availability of ground-truth deformation fields made it possible to evaluate both the quality of the resulting registration and the accuracy with which the original deformation was recovered. First of all, for both classical and learning-based methods, the DVF percentage error remains over 80% and goes up to 211%, which outlines a poor recovery capability. DVF errors remain high, indicating that the recovered deformations differ substantially from the synthetic fields used for data generation. However, because image registration is inherently non-identifiable, deformation recovery should not be interpreted as a direct measure of registration accuracy. In addition, the results reveal that these two objectives are not equivalent. In several cases, methods achieving the lowest DVF errors did not produce the highest DSC¯ DSC or lowest HD95¯ HD95 values. For example, M2M-GradICON consistently achieved among the lowest deformation recovery errors while remaining substantially inferior to GradICON in terms of overlap-based metrics. Conversely, GradICON and VoxelMorph frequently obtained superior DSC¯ DSC, HD95¯ HD95, and MIND-SSC loss values while exhibiting larger deformation errors. This observation suggests that a deformation field capable of producing optimal anatomical correspondence according to registration metrics may differ significantly from the synthetic deformation used to generate the benchmark. More broadly, these results indicate that synthetic deformation recovery and multimodal anatomical registration should be regarded as related but distinct tasks. A third major observation is the trade-off between registration accuracy and transformation regularity. The folding-ratio results demonstrate that the methods producing the largest gains in overlap and boundary alignment are often those generating the largest number of local topology violations (although the maximum occurrence remains very limited). In particular, VoxelMorph and GradICON achieved some of the strongest registration performances on BRATS and HCP but also produced the highest folding ratios. Conversely, M2M-GradICON, multiGradICON, and the classical methods generated nearly fold-free transformations, albeit at the expense of reduced registration accuracy. This trade-off highlights a fundamental challenge in deformable registration. Maximizing image alignment alone can encourage transformations that are anatomically implausible, while strong regularization can limit the ability of the model to match complex deformations. Since local folding corresponds to non-invertible mappings that cannot be interpreted as physically meaningful anatomical transformations, deformation regularity should be considered a primary evaluation criterion rather than a secondary quality measure. The anatomy-guided experiments provide additional insight into the behavior of multimodal registration algorithms. Incorporating segmentation information dramatically improves overlap-based metrics, with ANTs SyN seg. achieving DSC¯ DSC values above 90% on BRATS and above 80% on both HCP and IMAG2 (Pelvis). However, these improvements do not translate consistently to the remaining evaluation criteria. In particular, anatomy-guided approaches often exhibit substantially worse MIND-SSC loss values than their unsupervised counterparts and do not systematically improve deformation recovery. This observation suggests that anatomy-guided registration should not be interpreted simply as a more accurate version of unsupervised registration. Instead, anatomical supervision changes the optimization objective itself by explicitly prioritizing agreement on annotated structures. Such a behavior may be desirable in applications focusing on contour propagation, treatment planning, or organ tracking. However, it also indicates that high segmentation overlap does not necessarily imply improved global multimodal correspondence throughout the image volume. The comparison between ANTs SyN seg. and GradICON seg. further highlights this distinction. ANTs SyN seg. consistently achieves the highest DSC¯ DSC values and among the lowest HD95¯ HD95 values, demonstrating its ability to maximize alignment of labeled structures. By contrast, GradICON seg. systematically produces substantially lower MIND-SSC loss and, on BRATS, markedly better deformation recovery. These results suggest that the two methods optimize different notions of registration quality. Whereas ANTs SyN seg. focuses on matching annotated structures as accurately as possible, GradICON seg. appears to preserve a broader notion of multimodal structural consistency. This is a direct consequence of the training protocol used for GradICON seg. which combined both segmentation overlap and MIND-SSC loss value in its optimization criterion whereas ANTs SyN seg. only focused on segmentation overlap. This distinction reinforces the importance of evaluating anatomy-guided methods using multiple complementary metrics rather than overlap measures alone. The behavior of multiGradICON deserves particular attention. As a foundation model trained on a large and diverse collection of registration tasks, one might expect it to generalize more effectively than task-specific approaches. However, the results demonstrate that multiGradICON does not provide consistent performance across datasets and neither consistently outperform the smaller GradICON models. In fact, it is frequently inferior in terms of DSC¯ DSC, HD95¯ HD95, and ℒMIND−SSCL_MIND-SSC, particularly on IMAG2 (pelvis). Several explanations are possible. First, the model was evaluated in a zero-shot setting without dataset-specific adaptation. Secondly, the training objective of multiGradICON emphasizes transformation consistency and robustness across tasks rather than optimization for any single registration benchmark. Regardless of the underlying cause, these results indicate that large-scale pretraining alone does not currently guarantee robust multimodal registration performance across anatomies and modality pairs. Perhaps the most clinically relevant finding of this benchmark is the discrepancy between performance on synthetic and real registration tasks. On BRATS and HCP, multiple methods substantially improve registration accuracy across most metrics. By contrast, on IMAG2 (Pelvis), virtually no unsupervised method achieves a meaningful improvement over the unregistered baseline. The best-performing methods do not modify the DSC¯ DSC and HD95¯ HD95 at all despite the use of state-of-the-art registration frameworks. This contrast suggests that current synthetic benchmarks may overestimate the capability of registration algorithms to solve real multimodal correspondence problems. While synthetic deformations provide a valuable controlled evaluation setting, they cannot reproduce all the uncertainty associated with anatomical variability, modality-specific visibility, partial volume effects, acquisition artifacts, and ambiguous correspondences encountered in clinical practice. These findings therefore highlight the need for larger collections of real multimodal datasets with expert annotations and clinically meaningful evaluation criteria. The benchmark also exposes a broader limitation of multimodal registration research: the absence of dense ground-truth correspondences. Each available evaluation strategy captures only a part of the problem. The segmentation overlap evaluates selected structures but ignores unlabeled anatomy. DVF recovery is only available for synthetic experiments and may not correlate with registration performance. Image-based similarity metrics such as MIND-SSC loss provide dense measurements but do not necessarily reflect clinically relevant structure alignment. Topology-preservation metrics quantify transformation plausibility but not registration accuracy. Consequently, the registration quality cannot be characterized by any single metric. The disagreement observed throughout this benchmark strongly supports the adoption of multi-criteria evaluation protocols incorporating geometric, photometric, deformation-based, and anatomical measures simultaneously. An additional challenge revealed by this benchmark concerns the interpretation of quantitative metrics themselves. Although numerical evaluation is essential for objective comparison, quantitative scores do not always reflect the perceived quality or practical usefulness of a registration. Across several datasets, visual inspection often suggested a more favorable alignment than indicated by the reported metrics. For example, in BRATS, the complex and irregular morphology of brain tumors can lead to moderate DSC¯ DSC values despite visually satisfactory alignment of the tumor boundaries. Similarly, the highly folded cortical anatomy can amplify local discrepancies captured by image-based similarity metrics such as MIND-SSC, even when the overall anatomical correspondence appears clinically acceptable. In HCP, the problem is further complicated by modality-specific visibility differences. Since structures outside the brain, including portions of the skull and facial anatomy, are hardly visible on b0b_0 images, registration methods may introduce substantial deformations in these regions that negatively affect quantitative measurements while having limited impact on the clinically relevant intracranial alignment. Finally, in IMAG2 (Pelvis), the lower image quality of the fixed image and the difficulty of establishing unambiguous multimodal correspondences result in only small visual differences between methods despite measurable quantitative variations. Depending on the intended application, such discrepancies may or may not be clinically meaningful. Several limitations of the present study should nevertheless be acknowledged. First, the brain experiments rely on synthetically generated deformations and therefore evaluate the recovery of known transformations rather than true multimodal correspondence estimation. Secondly, the benchmark remains dependent on the availability and quality of anatomical segmentations. Thirdly, although for classical methods we performed a dataset-specific hyperparameter tuning, learning-based approaches were evaluated using the hyperparameters proposed by their authors. This choice was made because performing a comparable grid search would have been prohibitively time-consuming. As a result, some performance differences may reflect hyperparameter sensitivity rather than intrinsic methodological superiority. Finally, the IMAG2 (pelvis) dataset remains relatively small and does not provide dense correspondence annotations, preventing a precise evaluation of voxel-level registration accuracy. Future progress in multimodal deformable registration will likely require advances beyond similarity metrics alone. In particular, the acquisition of large real-world multimodal datasets will probably determine the outcome of future research. Specific focus should be laid on providing datasets containing genuinely aligned multimodal acquisitions (through animal or human acquisitions on sedated patients) and segmentation labels distributed throughout the whole field of view to maximize the correspondence between the anatomical and similarity metrics. On a similar level, evaluation protocols tied to downstream clinical tasks may be an alternate solution to assess clinical validity of the registration strategies, as it helps to tie back the registration performance to a single relevant metric. On a second level, improved anatomical and biomechanical priors and stronger topology-preserving optimization strategies represent promising directions. More generally, the results of this benchmark suggest that future methods should not optimize exclusively for overlap accuracy or image similarity, but instead seek a balanced compromise between anatomical correspondence, multimodal consistency, accurate deformation estimation, and transformation plausibility. 7 Conclusion Overall, the results of this benchmark demonstrate that multimodal deformable registration remains an open and highly context-dependent problem. Anatomy-guided methods can achieve excellent alignment of labeled structures, but may not guarantee global multimodal consistency. Learning-based methods such as GradICON and VoxelMorph provide competitive and sometimes more balanced performance, but their robustness across anatomical regions and real clinical deformation scenarios remains limited. Classical methods remain useful baselines but show strong dataset dependence. These findings support the need for comprehensive, multi-metric evaluation and suggest that clinically reliable multimodal registration will require methods that jointly address anatomical accuracy, image-based consistency, physically plausible deformation and downstream clinical task performance. Acknowledgments This work has been funded and supported by Ligue contre le cancer. The data for the IMAG2 (Pelvis) dataset was acquired following the ClinicalTrials ID NCT06224985, APHP ID 2015-A01705-44. This project was provided with computing AI and storage resources by GENCI at IDRIS thanks to the grant 2025-AD010316730 on the supercomputer Jean Zay V100 and A100 partitions. References [1] M. Adewole, J. D. Rudie, A. Gbdamosi, O. Toyobo, C. Raymond, D. Zhang, O. Omidiji, R. Akinola, M. A. Suwaid, A. Emegoakor, et al. (2023) The BRAin Tumor Segmentation (BRATS) challenge 2023: glioma segmentation in sub-saharan africa patient population (BRATS-africa). ArXiv, p. arXiv–2305. Cited by: §4.1. [2] J. Ashburner (2011) Diffeomorphic registration using geodesic shooting and gauss-newton optimisation. NeuroImage 55 (3), p. 954–967. External Links: Document Cited by: §2.1. [3] B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis 12 (1), p. 26–41. Cited by: §1, §2.1, §3, Table 1. [4] B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis 12 (1), p. 26–41. External Links: Document Cited by: §2.1. [5] G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V. Dalca (2019) Voxelmorph: a learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging 38 (8), p. 1788–1800. Cited by: §2.1, §2.3, Table 1. [6] M. F. Beg, M. I. Miller, A. Trouvé, and L. Younes (2005) Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International Journal of Computer Vision 61 (2), p. 139–157. Cited by: §2.1. [7] K. K. Brock, S. Mutic, T. R. McNutt, H. Li, and M. L. Kessler (2017) Use of image registration and fusion algorithms and techniques in radiotherapy: report of the aapm radiation therapy committee task group no. 132. Medical physics 44 (7), p. e43–e76. Cited by: §2.3. [8] N. Charon, B. Charlier, J. Glaunes, P. Gori, and P. Roussillon (2019) Fidelity metrics between curves and surfaces: currents, varifolds, and normal cycles. In Riemannian Geometric Statistics in Medical Image Analysis, p. 441–477. External Links: Document Cited by: §1, §2.1. [9] J. Chen, E. C. Frey, Y. He, W. P. Segars, Y. Li, and Y. Du (2022) TransMorph: Transformer for unsupervised medical image registration. Medical Image Analysis 82, p. 102615. Cited by: §2.1, §2.2. [10] J. Chen, Y. He, E. C. Frey, Y. Li, and Y. Du (2021) Vit-V-Net: Vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv:2104.06468. Cited by: §2.1. [11] J. Chen, Y. Liu, S. Wei, Z. Bian, S. Subramanian, A. Carass, J. L. Prince, and Y. Du (2025) A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond. Medical Image Analysis 100, p. 103385. Cited by: §1. [12] K. Choo, H. Han, J. Kim, C. Yoon, and S. J. Hwang (2025) Mono-Modalizing extremely heterogeneous multi-modal medical image registration. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 433–443. Cited by: §2.2, §3, Table 1. [13] W. R. Crum, O. Camara, and D. L. Hill (2006) Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Transactions on Medical Imaging 25 (11), p. 1451–1461. Cited by: §2.3. [14] S. Czolbe, O. Krause, and A. Feragen (2021) Semantic similarity metrics for learned image registration. In Medical Imaging with Deep Learning, p. 105–118. Cited by: §2.1. [15] A. V. Dalca, G. Balakrishnan, J. Guttag, and M. R. Sabuncu (2019) Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Medical Image Analysis 57, p. 226–236. Cited by: §1, §2.1, §2.3. [16] B. D. De Vos, F. F. Berendsen, M. A. Viergever, H. Sokooti, M. Staring, and I. Išgum (2019) A deep learning framework for unsupervised affine and deformable image registration. Medical Image Analysis 52, p. 128–143. Cited by: §1. [17] B. Demir, L. Tian, H. Greer, R. Kwitt, F. Vialard, R. S. J. Estépar, S. Bouix, R. Rushmore, E. Ebrahim, and M. Niethammer (2024) MultiGradICON: A foundation model for multimodal medical image registration. In International Workshop on Biomedical Image Registration, p. 3–18. Cited by: §2.2, §2.3, Table 1, §3. [18] N. Dey, J. Schlemper, S. S. M. Salehi, B. Zhou, G. Gerig, and M. Sofka (2022) ContraReg: contrastive learning of multi-modality unsupervised deformable image registration. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 66–77. Cited by: §2.2. [19] S. Durrleman, X. Pennec, A. Trouvé, G. Gerig, and N. Ayache (2009) Spatiotemporal atlas estimation for developmental delay detection in longitudinal datasets. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009, Lecture Notes in Computer Science, Vol. 5761, p. 297–304. External Links: Document Cited by: §1. [20] J. Fan, X. Cao, Q. Wang, P. Yap, and D. Shen (2019) Adversarial learning for mono-or multi-modal registration. Medical Image Analysis 58, p. 101545. Cited by: §2.2. [21] A. François, P. Gori, and J. Glaunès (2021) Metamorphic image registration using a semi-lagrangian scheme. In Geometric Science of Information, p. 781–788. External Links: Document Cited by: §2. [22] A. François, M. Maillard, C. Oppenheim, J. Pallud, I. Bloch, P. Gori, and J. Glaunès (2022) Weighted metamorphosis for registration of images with different topologies. In Biomedical Image Registration, p. 8–17. External Links: Document Cited by: §2. [23] Y. Fu, Y. Lei, T. Wang, W. J. Curran, T. Liu, and X. Yang (2020) Deep learning in medical image registration: A review. Physics in Medicine & Biology 65 (20), p. 20TR01. Cited by: §1. [24] X. Gao, W. Zhong, R. Wang, A. F. Heimann, M. Tannast, and G. Zheng (2024) MAIRNet: weakly supervised anatomy-aware multimodal articulated image registration network. International Journal of Computer Assisted Radiology and Surgery 19 (3), p. 507–517. Cited by: §2.2. [25] M. F. Glasser, S. N. Sotiropoulos, J. A. Wilson, T. S. Coalson, B. Fischl, J. L. Andersson, J. Xu, S. Jbabdi, M. Webster, J. R. Polimeni, et al. (2013) The minimal preprocessing pipelines for the Human Connectome Project. Neuroimage 80, p. 105–124. Cited by: §4.1. [26] P. Gori, O. Colliot, L. M. Kacem, Y. Worbe, A. Routier, C. Poupon, A. Hartmann, N. Ayache, and S. Durrleman (2018) Double diffeomorphism: combining morphometry and structural connectivity analysis. IEEE Transactions on Medical Imaging 37 (9), p. 2033–2043. Cited by: §1. [27] P. Gori, O. Colliot, L. Marrakchi-Kacem, Y. Worbe, C. Poupon, A. Hartmann, N. Ayache, and S. Durrleman (2017) A bayesian framework for joint morphometry of surface and curve meshes in multi-object complexes. Medical Image Analysis 35, p. 458–474. External Links: Document Cited by: §1, §2.1. [28] P. Gori, O. Colliot, L. Marrakchi-Kacem, Y. Worbe, A. Routier, C. Poupon, A. Hartmann, N. Ayache, and S. Durrleman (2015) Joint morphometry of fiber tracts and gray matter structures using double diffeomorphisms. In Information Processing in Medical Imaging, Lecture Notes in Computer Science, Vol. 9123, p. 275–287. External Links: Document Cited by: §1. [29] P. Gori, O. Colliot, Y. Worbe, L. Marrakchi-Kacem, S. Lecomte, C. Poupon, A. Hartmann, N. Ayache, and S. Durrleman (2013) Bayesian atlas estimation for the variability analysis of shape complexes. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013, Lecture Notes in Computer Science, Vol. 8149, p. 267–274. External Links: Document Cited by: §1. [30] E. Haber and J. Modersitzki (2006) Intensity gradient based registration and fusion of multi-modal images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 726–733. Cited by: §2.2. [31] L. Hansen, W. Heyer, C. Großbröhmer, F. Madesta, T. Sentker, W. Jiazheng, Y. Zhang, H. Zhang, M. Liu, J. Wang, et al. (2025) Learn2Reg 2024: new benchmark datasets driving progress on new challenges. arXiv preprint arXiv:2509.01217. Cited by: §2.3, §2.3. [32] G. Haskins, J. Kruecker, U. Kruger, S. Xu, P. A. Pinto, B. J. Wood, and P. Yan (2019) Learning deep similarity metric for 3D MR–TRUS image registration. International Journal of Computer Assisted Radiology and Surgery 14 (3), p. 417–425. Cited by: §2.1, §2.2, §2.3. [33] G. Haskins, U. Kruger, and P. Yan (2020) Deep learning in medical image registration: a survey. Machine Vision and Applications 31 (1), p. 8. Cited by: §2.1. [34] X. He, H. Yu, S. Peng, D. Tan, Z. Shen, H. Bao, and X. Zhou (2025) MatchAnything: universal cross-modality image matching with large-scale pre-training. arxiv 2025. arXiv preprint arXiv:2501.07556. Cited by: §2.2. [35] Y. He, M. Liu, Q. Liu, J. Wang, Y. Wang, H. Zhang, and X. Chen (2025) SAMIR, an efficient registration framework via robust feature learning from SAM. arXiv preprint arXiv:2509.13629. Cited by: §1, §2.2. [36] M. P. Heinrich, M. Jenkinson, M. Bhushan, T. Matin, F. V. Gleeson, M. Brady, and J. A. Schnabel (2012) MIND: Modality Independent Neighbourhood Descriptor for multi-modal deformable registration. Medical Image Analysis 16 (7), p. 1423–1435. Cited by: §1, §2.3. [37] M. P. Heinrich, M. Jenkinson, B. W. Papież, S. M. Brady, and J. A. Schnabel (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 187–194. Cited by: §1, §2.2, §2.3. [38] A. Hering, L. Hansen, T. C. Mok, A. C. Chung, H. Siebert, S. Häger, A. Lange, S. Kuckertz, S. Heldmann, W. Shao, et al. (2022) Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning. IEEE Transactions on Medical Imaging 42 (3), p. 697–712. Cited by: §2.3, §2.3. [39] J. Hu, Z. Luo, X. Wang, S. Sun, Y. Yin, K. Cao, Q. Song, S. Lyu, and X. Wu (2021) End-to-end multimodal image registration via reinforcement learning. Medical Image Analysis 68, p. 101878. Cited by: §2.2. [40] Y. Hu, M. Modat, E. Gibson, W. Li, N. Ghavami, E. Bonmati, G. Wang, S. Bandula, C. M. Moore, M. Emberton, et al. (2018) Weakly-supervised convolutional neural networks for multimodal image registration. Medical Image Analysis 49, p. 1–13. Cited by: §2.1, §2.2, §2.3. [41] B. Jian, M. F. Azampour, F. De Benetti, J. Oberreuter, C. Bukas, A. S. Gersing, S. C. Foreman, A. Dietrich, J. Rischewski, J. S. Kirschke, et al. (2022) Weakly-supervised biomechanically-constrained CT/MRI registration of the spine. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 227–236. Cited by: §2.1. [42] D. Jiang, Y. Shi, D. Yao, Y. Fan, M. Wang, and Z. Song (2017) Multimodal image registration based on binary gradient angle descriptor. International Journal of Computer Assisted Radiology and Surgery 12 (12), p. 2157–2167. Cited by: §2.2. [43] B. Kim, D. H. Kim, S. H. Park, J. Kim, J. Lee, and J. C. Ye (2021) CycleMorph: cycle consistent unsupervised deformable image registration. Medical Image Analysis 71, p. 102036. Cited by: §2.2. [44] S. Klein, M. Staring, K. Murphy, M. A. Viergever, and J. P. Pluim (2009) Elastix: a toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging 29 (1), p. 196–205. Cited by: §2.1, Table 1. [45] L. Kong, C. Lian, D. Huang, Y. Hu, Q. Zhou, et al. (2021) Breaking the dilemma of medical image-to-image translation. Advances in Neural Information Processing Systems 34, p. 1964–1978. Cited by: §2.2. [46] F. Liu, K. Yan, A. P. Harrison, D. Guo, L. Lu, A. L. Yuille, L. Huang, G. Xie, J. Xiao, X. Ye, et al. (2021) SAME: deformable image registration based on self-supervised anatomical embeddings. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 87–97. Cited by: §2.2. [47] Y. Liu, W. Wang, Y. Li, H. Lai, S. Huang, and X. Yang (2023) Geometry-consistent adversarial registration model for unsupervised multi-modal medical image registration. IEEE Journal of Biomedical and Health Informatics 27 (7), p. 3455–3466. Cited by: §2.2. [48] H. Lombaert, L. Grady, X. Pennec, N. Ayache, and F. Cheriet (2014) Spectral log-demons: diffeomorphic image registration with very large deformations. International Journal of Computer Vision 107 (3), p. 254–271. Cited by: §2.1, Table 1. [49] M. Lorenzi and X. Pennec (2013) Efficient parallel transport of deformations in time series of images: a robust framework for longitudinal analysis. IEEE Transactions on Medical Imaging. Cited by: §1. [50] J. Lu et al. (2021) Is image-to-image translation the panacea for multimodal image registration? A comparative study. Medical Image Analysis. Cited by: §2.2. [51] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens (2002) Multimodality image registration by maximization of mutual information. IEEE transactions on Medical Imaging 16 (2), p. 187–198. Cited by: §2.2. [52] M. Maillard, A. Francois, J. Glaunes, I. Bloch, and P. Gori (2022) A deep residual learning implementation of metamorphosis. In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), p. 1–4. External Links: Document Cited by: §2. [53] B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest, et al. (2014) The multimodal BRAin Tumor image Segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging 34 (10), p. 1993–2024. Cited by: §4.1, Table 2. [54] T. C. Mok, Z. Li, Y. Bai, J. Zhang, W. Liu, Y. Zhou, K. Yan, D. Jin, Y. Shi, X. Yin, et al. (2024) Modality-agnostic structural image representation learning for deformable multi-modality medical image registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 11215–11225. Cited by: §1, §2.2. [55] L. Nascimento, Q. François, B. Duplat, S. Haliyo, and I. Bloch (2026) Image synthesis and transfer learning for estimating multi-modal brain registration error. In 2026 IEEE 23rd International Symposium on Biomedical Imaging (ISBI), p. 1–5. Cited by: §1. [56] M. Niethammer, R. Kwitt, and F. Vialard (2019) Metric learning for image registration. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 8455–8464. Cited by: §2.1. [57] N. Pielawski, E. Wetzer, J. Öfverstedt, J. Lu, C. Wählby, J. Lindblad, and N. Sladoje (2020) CoMIR: Contrastive Multimodal Image Representation for registration. Advances in neural information processing systems 33, p. 18433–18444. Cited by: §2.2. [58] L. Pio, R. Kassir, G. La Barbera, C. Lozach, E. Bonnot, T. Isla, J. P. de la Plata Alcalde, P. Gori, I. Bloch, and S. Sarnacki (2026) 3D imaging contribution in pediatric surgical oncology: a multi-stakeholder assessment study. Scientific Reports 16, p. 14264. External Links: Document Cited by: §4.1. [59] H. Ramadan, D. El Bourakadi, A. Yahyaouy, and H. Tairi (2024) Medical image registration in the era of transformers: A recent review. Informatics in Medicine Unlocked 49, p. 101540. Cited by: §2.1. [60] M. Ronchetti, W. Wein, N. Navab, O. Zettinig, and R. Prevost (2023) DISA: DIfferentiable Similarity Approximation for universal multimodal registration. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 761–770. Cited by: §2.2. [61] D. Rueckert, L. I. Sonoda, C. Hayes, D. L. Hill, M. O. Leach, and D. J. Hawkes (2002) Nonrigid registration using free-form deformations: application to breast MR images. IEEE Transactions on Medical Imaging 18 (8), p. 712–721. Cited by: §2.1. [62] M. Simonovsky, B. Gutiérrez-Becker, D. Mateus, N. Navab, and N. Komodakis (2016) A deep metric for multimodal registration. In International conference on medical image computing and computer-assisted intervention, p. 10–18. Cited by: §2.1. [63] X. Song, H. Chao, X. Xu, H. Guo, S. Xu, B. Turkbey, B. J. Wood, T. Sanford, G. Wang, and P. Yan (2022) Cross-modal attention for multi-modal image registration. Medical Image Analysis 82, p. 102612. Cited by: §2.2, §2.3. [64] A. Sotiras, C. Davatzikos, and N. Paragios (2013) Deformable medical image registration: A survey. IEEE Transactions on Medical Imaging 32 (7), p. 1153–1190. Cited by: §1, §1, §2.3. [65] C. Sudlow, J. Gallacher, N. Allen, V. Beral, P. Burton, J. Danesh, P. Downey, P. Elliott, J. Green, M. Landray, et al. (2015) UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS medicine 12 (3), p. e1001779. Cited by: §2.3. [66] L. Tian, H. Greer, F. Vialard, R. Kwitt, R. S. J. Estépar, R. J. Rushmore, N. Makris, S. Bouix, and M. Niethammer (2023) GradICON: approximate diffeomorphisms via gradient inverse consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 18084–18094. Cited by: §2.1, Table 1. [67] G. Tilak, K. Tuncali, S. Song, J. Tokuda, O. Olubiyi, F. Fennessy, A. Fedorov, T. Penzkofer, C. Tempany, and N. Hata (2015) 3T MR-guided in-bore transperineal prostate biopsy: A comparison of robotic and manual needle-guidance templates. Journal of Magnetic Resonance Imaging 42 (1), p. 63–71. Cited by: §2.3. [68] J. Tournier, R. Smith, D. Raffelt, R. Tabbara, T. Dhollander, M. Pietsch, D. Christiaens, B. Jeurissen, C. Yeh, and A. Connelly (2019) MRtrix3: a fast, flexible and open software framework for medical image processing and visualisation. Neuroimage 202, p. 116137. Cited by: §4.1. [69] N. Tursynbek, H. Greer, B. Demir, and M. Niethammer (2025) Guiding registration with emergent similarity from pre-trained diffusion models. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 240–251. Cited by: §1, §2.2. [70] D. C. Van Essen, S. M. Smith, D. M. Barch, T. E. Behrens, E. Yacoub, K. Ugurbil, W. H. Consortium, et al. (2013) The Wu-Minn Human Connectome Project: an overview. Neuroimage 80, p. 62–79. Cited by: §4.1, Table 2. [71] T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache (2009) Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45 (1), p. S61–S72. Cited by: §2.1, Table 1. [72] J. Wang, R. Liu, Y. Lei, M. J. Baine, and T. Liu (2025) Weakly supervised spatial implicit neural representation learning for 3D MRI-Ultrasound deformable image registration in HDR prostate brachytherapy: TU-130-202-10. Medical Physics 52 (10), p. e70059527–e70059528. Cited by: §2.2. [73] Y. Wang, T. Fu, C. Wu, J. Xiao, J. Fan, H. Song, P. Liang, and J. Yang (2023) Multimodal registration of ultrasound and mr images using weighted self-similarity structure vector. Computers in Biology and Medicine 155, p. 106661. Cited by: §2.2, §2.3. [74] D. Wei, S. Ahmad, J. Huo, W. Peng, Y. Ge, Z. Xue, P. Yap, W. Li, D. Shen, and Q. Wang (2019) Synthesis and inpainting-based MR-CT registration for image-guided thermal ablation of liver tumors. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 512–520. Cited by: §2.2. [75] H. Xiao, X. Teng, C. Liu, T. Li, G. Ren, R. Yang, D. Shen, and J. Cai (2021) A review of deep learning-based three-dimensional medical image registration methods. Quantitative Imaging in Medicine and Surgery 11 (12), p. 4895. Cited by: §1. [76] Z. Xu, J. Luo, J. Yan, R. Pulya, X. Li, W. Wells I, and J. Jagadeesan (2020) Adversarial uni-and multi-modal stream networks for multimodal image registration. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 222–232. Cited by: §2.2. [77] M. Zanello, R. Carron, S. Peeters, P. Gori, A. Roux, I. Bloch, C. Oppenheim, and J. Pallud (2021) Automated neurosurgical stereotactic planning for intraoperative use: a comprehensive review of the literature and perspectives. Neurosurgical Review 44 (2), p. 867–888. External Links: Document Cited by: §1. [78] J. Zou, B. Gao, Y. Song, and J. Qin (2022) A review of deep learning-based deformable medical image registration. Frontiers in Oncology 12, p. 1047215. Cited by: §1, §1, §2.3.