Paper deep dive
CARDIAG: A Dense Segment Classification Benchmark of Deep Learning Architectures for Coronary Angiography
Dominik Bernard Lau, Hubert Malinowski, Jerzy Szyjut, Adam Brzeski, Tomasz Dziubich, Radosław Targoński, Tomasz Figatowski, Natalia Zielińska
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/2/2026, 1:05:39 PM
Summary
The paper introduces CARDIAG, a multi-center, multi-label benchmark dataset for coronary angiography segmentation, addressing the lack of standardized evaluation protocols in the field. The authors evaluate 24 deep learning architectures, including CNNs, Vision Transformers, and State Space Models, on this dataset. The best single model is ConvNeXt V2 with DeepLab V3 Plus (macro F1=0.456), while an ensemble of ConvNeXt V2, Mamba U-Net, and Feature Pyramid Network achieves an F1 of 0.479. The study highlights the importance of high and low-resolution features, model calibration, and robustness across patient demographics and acquisition parameters.
Entities (10)
Relation Signals (9)
CARDIAG → isbenchmarkfor → Deep Learning Models
confidence 95% · demonstrate a new benchmark for the assessment of deep learning models
CARDIAG → provideslabelsfor → SYNTAX
confidence 95% · The data contains SYNTAX labels... densely classify pixels of coronary angiograms to one of SYNTAX classes
ConvNeXt V2 → ispartofensemble → Feature Pyramid Network
confidence 92% · ensemble with Mamba U-Net and Feature Pyramid Network
Feature Pyramid Network → ispartofensemble → ConvNeXt V2
confidence 92% · ensemble with Mamba U-Net and Feature Pyramid Network
Mamba U-Net → ispartofensemble → ConvNeXt V2
confidence 92% · ensemble with Mamba U-Net and Feature Pyramid Network
ConvNeXt V2 → ispartofensemble → Mamba U-Net
confidence 92% · ensemble with Mamba U-Net and Feature Pyramid Network
Gdańsk University of Technology → affiliatedwith → Dominik B. Lau
confidence 90% · Dominik B. Lau Gdańsk University of Technology
ConvNeXt V2 → outperforms → other_architectures
confidence 90% · we nominate ConvNeXt V2 encoder with DeepLab V3 Plus decoder as the best performing
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Accurate pixel-level classification of coronary angiograms is critical for cardiovascular disease assessment, yet the field lacks standardized evaluation protocols. In this work we demonstrate a new benchmark for the assessment of deep learning models which densely classify pixels of coronary angiograms to one of SYNTAX classes (or background). The evaluation covers 24 distinct architectures starting with classic convnets to recent state-space-based vision algorithms. We release CARDIAG - a multi-center, multi-label dataset which we carefully split to reliably compute metrics, accounting for diameter error, overlap, centerline quality and calibration. The data contains SYNTAX labels, binary, uncertainty and segmentation masks as well as intermediate frames together with the selected non-sensitive DICOM metadata. From the multitude of algorithms, we nominate ConvNeXt V2 encoder with DeepLab V3 Plus decoder as the best performing, achieving macro $F_1=0.456$, which we then ensemble with Mamba U-Net and Feature Pyramid Network, for an increased $F_1=0.479$. We demonstrate all the architectures to be well calibrated and determine the generalization of the top 5 methods, together with the data efficiency of these architectures. We highlight the importance of both high-resolution and low-resolution features in encoding. We also demonstrate the model correctness in the context of patient demographic, vessel sides and projection angle configurations. Overall the released benchmark allows for future studies to robustly and rigorously assess the proposals, not only for SYNTAX segmentation, but lesion detection and many more.
Tags
Links
- Source: https://arxiv.org/abs/2607.22139v1
- Canonical: https://arxiv.org/abs/2607.22139v1
Trouble viewing inline? Open PDF directly →
Full Text
70,018 characters extracted from source content.
Expand or collapse full text
CARDIAG: A Dense Segment Classification Benchmark of Deep Learning Architectures for Coronary Angiography Dominik B. Lau Gdańsk University of Technology NASK National Research Institute Correspondence: dominik.lau@nask,pg.edu.pl Hubert Malinowski Gdańsk University of Technology Jerzy Szyjut Gdańsk University of Technology Adam Brzeski Gdańsk University of Technology Tomasz Dziubich Gdańsk University of Technology Radosław Targoński Medical University of Gdańsk Tomasz Figatowski Medical University of Gdańsk Natalia Zielińska Medical University of Gdańsk Abstract Accurate pixel-level classification of coronary angiograms is critical for cardiovascular disease assessment, yet the field lacks standardized evaluation protocols. In this work we demonstrate a new benchmark for the assessment of deep learning models which densely classify pixels of coronary angiograms to one of SYNTAX classes (or background). The evaluation covers 24 distinct architectures starting with classic convnets to recent state-space-based vision algorithms. We release CARDIAG - a multi-center, multi-label dataset which we carefully split to reliably compute metrics, accounting for diameter error, overlap, centerline quality and calibration. The data contains SYNTAX labels, binary, uncertainty and segmentation masks as well as intermediate frames together with the selected non-sensitive DICOM metadata. From the multitude of algorithms, we nominate ConvNeXt V2 encoder with DeepLab V3 Plus decoder as the best performing, achieving macro F1=0.456F_1=0.456, which we then ensemble with Mamba U-Net and Feature Pyramid Network, for an increased F1=0.479F_1=0.479. We demonstrate all the architectures to be well calibrated and determine the generalization of the top 5 methods, together with the data efficiency of these architectures. We highlight the importance of both high-resolution and low-resolution features in encoding. We also demonstrate the model correctness in the context of patient demographic, vessel sides and projection angle configurations. Overall the released benchmark allows for future studies to robustly and rigorously assess the proposals, not only for SYNTAX segmentation, but lesion detection and many more. Keywords: Image segmentation, SYNTAX score, Coronary Angiography, Benchmark, Multi-center, Robustness 1 Introduction AI-based segmentation of the coronary vasculature enables automated, reproducible reconstruction of the coronary tree, which is fundamental for quantitative assessment of lumen geometry, stenosis severity, and plaque burden. This improves diagnostic accuracy and reduces inter‑observer variability compared to manual interpretation, while markedly decreasing analysis time and workload [18]. Hence, we formalize the posed "SYNTAX segmentation" task as a dense prediction problem. Given an input X (X-ray coronary angiogram), the goal is to find a model fθf_θ that assigns SYNTAX class [37] to a pixel (i,j)(i,j). The ground truth is conventionally one-hot encoded, while the model assigns to the pixel a probability distribution over the 26 distinct SYNTAX classes i.e. ^(,)=fθ(X,i,j)∈[0,1]26 y(i,j)=f_θ(X,i,j)∈[0,1]^26 (1) The task is challenging due to several factors, one of which being the long-tailed distribution of classes driven by anatomical variation, the other - a lack of ground truth data available to the community. Past SYNTAX segmentation efforts elaborate mainly on classic CNNs, as well as the ensembling of them. To start with, multiple approaches focus on segmenting a single selected part of the artery tree: Zhang et al. [48] design Progressive Perception Learning framework (PPL) in which they utilize features at different resolution levels (local, surrounding, semantic) for improved performance. However the approach is limited to highlighting left anterior descending (LAD), right coronary artery (RCA) and left circumflex (LCX). Other notable approaches are: Park et al. [29] proposal to also use ensembling as a way to improve Dice score of the segmentation, Jun et al. [17] introduce T-Net - a variation on U Net to account for multiple-size feature maps in a single block. While the findings are important, in the context of this study especially the use of multi-scale features proves crucial later, these approaches solve a task fundamentally easier than what we partake, as the main segments are more coarse-grained and cover bigger areas. Even a simple U Net can reach F1>0.80F_1>0.80 as shown by Xian et al. [41]. Furthermore, RCA, LAD and LCX are typically observed by a clinician from different perspectives, which makes the task closer to binary segmentation than multi-class labelling. The goal of binary segmentation is to assign all of the image pixels to two classes - either vessel (foreground) or non-vessel (background). While it is still simpler than multi-class task we propose, it might serve as a base for further methods such as stenosis detection [13]. For the foreground detection many approaches were proven successful such as PSPNet [55], ViT [45] or U-Net variants such as Se-RegUNet [4]. There have also been recent approaches to binary segmentation using state-space models such as HR-UMamba++ proposed by Zhang et al. [49], nevertheless to our knowledge no previous study applies Mamba to SYNTAX segmentation. We address the gap as the Mamba family is known to address long contexts well providing us with the hypothesis that in the task in question one might therefore benefit from learning long-range dependencies. As for the particular SYNTAX segmentation task, the domain remains underexplored with a very few approaches advancing the domain throughout the recent years. Many standard approaches providing a good baseline were proposed on behalf of the original ARCADE [31] challenge such as the ensembling of multiple segmentation models contributed by Bilal et al. [1], splitting the segmentation to multi-step pipeline where the arteries are first split to left and right-side specific models [20] or using a vessel map-guided YOLOv8 [22]. In the domain there have also been proposed generative approaches such as cGAN by Yang et al. [46], its main drawback being starting from a binary mask of unspecified source, or UENet [35] with multi-scale "E shaped" patch-GAN style discriminator and structural VGG feature extraction loss. Perhaps the most notable are approaches by Du et al [10] and Zhai et al. [47] where the utilized datasets consisted of respectively 12,323 and over 7000 samples. The datasets used in these studies were unfortunately not released to the public rendering the approaches irreproducible. There have also been proposed multiple approaches using graph neural networks and matching of the artery graphs such as AGMN [51], EAGMN [53], MGM [52], HAGMN-UQ [50], the last approach also attempting uncertainty quantification. These provide a greater inductive bias through giving the arteries a rigid graph representation. However it comes at a cost of multi-step extraction pipeline with possible downstream error propagation. As described, except for the graph-based approaches by Zhao et al. [51, 53, 52, 50], earlier studies typically focus on single architecture, it usually being YOLO or some variant of a U Net with or without the adversarial training component. To date there has not been proposed a wide multi-family architecture sweep to nominate the best baseline for further experiments and improvements. We conduct a rigorous benchmark of the available approaches to organize the domain and summarize what factors of the available methods are beneficial to the task. Another important matter is the availability of public datasets, which for SYNTAX segmentation is considered scarce - it might be attributed to significant annotation costs and the labeling being a non-negligable time investment imposed on the experts. Many of the studies mentioned earlier use in-house datasets ([48, 29, 17, 41, 55, 4, 49, 35, 10, 47, 51, 53, 52, 50]) making the results and proposed approaches hardly reproducible. ARCADE is the biggest public dataset so far, emerging throughout the years as the gold-standard benchmark for both segmentation and stenosis detection tasks. It contains 1500 expert-labeled images. It was collected in Almaty, Kazakhstan by a group of specialists from the Research Institute of Cardiology and Internal Diseases. There is nevertheless a high concern towards the quality of the labels as pointed out by Seo et al [34]. Furthermore, the authors do not attach neither the metadata, nor patient, center or examination id and as the authors suggest, there might be from 1 to 12 frames from the same patient present. In spite of the fact, no actions aimed at accounting for this potential leakage have been disclosed, whereas the data is spanning only two distinct centers. Due to that, treating the dataset as a generalization benchmark might be questionable. There are also many other coronary angiography datasets available, not for SYNTAX segmentation per se, such as: XCAD [23] and DCA1 [3] - both of which contain labels for binary segmentation, CoronaryDominance [19] which contains over 1500 multi-center studies with artery side dominance marked and CardioSyntax [30] - a dataset of 1,844 images paired with SYNTAX scores and 1,025 images with dominance specified. In the study we construct a contender dataset to the above which we name Coronary Angiography Roentgen Data for Image Analysis from Gdańsk (CARDIAG). We attach in CARDIAG extra data not necessarily present in ARCADE such as background frames, catheter mask, explicit binary mask, supplementary frames, acquisition metadata and uncertainty masks, all of which can be utilized by the community for the development of more robust architectures tacking multiple tasks exceeding segmentation and lesion detection. The dataset is multi-center with an anonymized identifier of each of the image source provided for the broader audience. Contribution statement The main contributions of this work are therefore summarized as follows: • Introduction of CARDIAG, a Comprehensive Multi-Center Dataset: We release a publicly available dataset designed to address the critical limitations of existing SYNTAX segmentation benchmarks. Sourced from five medical centers, CARDIAG uniquely provides explicit catheter masks, uncertainty annotations, intermediate frames, and comprehensive metadata to prevent data leakage and enable true generalization testing. • A Comprehensive Architectural Benchmark: We conduct a rigorous evaluation of 24 state-of-the-art vision models, spanning legacy and modern large-kernel CNNs, Vision Transformers, and State Space Models. We further assess real-world robustness through leave-one-center-out generalization validation and data efficiency analysis. • Clinical and Acquisition Meta-Analysis: We systematically investigate the influence of patient demographics and X-ray acquisition parameters on model performance, providing critical insights into algorithmic robustness across varying, real-world clinical conditions. • Uncertainty Quantification for Clinical Safety: We evaluate model calibration and uncertainty using inference-time dropout. This addresses a frequently omitted prerequisite for reliable clinical decision support. 2 Methods In this part of the study we cover the dataset acquisition protocols and contents as well as used architectures, training processes configuration and the setup of the conducted experiments. 2.1 Dataset The labelling process was performed using our AngioTagger software from 2019 to 2021 by 3 specialists in interventional cardiology with at least 3 years of experience. We applied SYNTAX score definitions [37], performing the analysis on a single angiographic frame with optimal contrast opacification of the coronary arteries. Angiographic images obtained from anonymized examinations of 114 patients. The mean age of the cohort was 64.6 years, with a median age of 65 years. Male participants accounted for 58% of the study population, with ages ranging from 34 to 87 years, while females represented 40% of the cohort, aged between 69 and 91 years. Examinations were performed using an Axiom Artis angiography system (Siemens Healthineers, Erlangen, Germany) in cardiology centres in Northern Poland. At the first stage, annotations were generated automatically and subsequently manually corrected. Prior to the annotation process, all participating experts underwent dedicated training in the use of Angiotagger software, and consensus meetings were conducted to harmonize the annotation criteria and ensure consistent interpretation of the labeling guidelines. Figure 1: Example comparison of annotations provided by two experts: (a) depicts a full segmentation of the vascular tree, while (b) illustrates segment-specific annotations (RCA mid). In the comparison visualizations, the first image shows the mask generated by expert 1, the second shows the mask generated by expert 2, the third highlights the differences between annotations (white indicates overlapping regions, while red and blue denote regions marked by only one of the observers), and the final image presents the original input data. To evaluate annotation quality and consistency, validation was performed on a randomly selected subset of 10 images. Representative examples of vascular tree segmentation are shown in Fig.1. For full vascular tree segmentation, inter-observer variability was assessed using the Dice score (DSC), with annotations provided by an expert with over 15 years of experience serving as the reference ground truth. The mean DSC across the evaluated subset was 0.87, indicating substantial agreement between observers. The most frequent discrepancies were observed at the distal ends of vessels, where one expert tended to exclude terminal branches considered clinically non-significant, while the other provided more detailed delineation of these structures. To address this source of variability, an uncertainty-aware annotation mechanism was introduced in the form of uncertainty masks, allowing experts to mark ambiguous or clinically irrelevant vessel fragments in a coarse manner. These regions were then excluded from agreement metric computation. Under this refined evaluation protocol, the mean DSC increased to 0.9072, reflecting improved consistency under uncertainty-aware annotation rules. For the assessment of stenosis labeling agreement, Fleiss’ kappa coefficient was used. Across the selected validation subset of 10 random cases, κ=0.7611, indicating substantial inter-rater agreement. Figure 2: Segment statistics including left-to-right ratio, frequencies and average sizes. Figure 3: Patient demographics statistics of the dataset. Each sample distributed by us as a part of CARDIAG, besides an x-ray image, contains: (1) metadata: acquisition parameters of the image, that is SID (source-image distance), SOD (source-object distance), spacing (vertical/horizontal distance between pixels), primary angle (α) and secondary angle (β) compliant with DICOM standard [26]. This allows for the reconstruction of detector plane position during the acquisition of the particular projection. The angles are measured in degrees, whereas SID, SOD and spacing are given in m. The projection angles cover the most common settings of the C-arm as depicted in Fig. 4. The metadata contains also an anonimized center ID and a sample id in the format <examination id>/<projection id> which allows for advanced stratified sampling utilized in the study. (2) binary mask: that is, such that for a given mask b and a pixel with coordinates (x,y)(x,y), b(x,y)=1⇔(x,y)∈vesselb(x,y)=1 (x,y) , otherwise b(x,y)=0b(x,y)=0. (3) segment masks: the directory contains also binary masks split further to particular segments. The segments, as mentioned before, follow the SYNTAX convention, also in the naming of the files. As an example take 4a.png, which will be a binary map m4am_4a, where m4a(x,y)=1⇔(x,y)∈segment 4am_4a(x,y)=1 (x,y) 4a, logically m4a(x,y)=0m_4a(x,y)=0 otherwise. (4) catheter binary mask: analogously to the labels before, we provide a mask pointing to where the catheter used to inject contrast dye is located on the image. (5) supplementary frames: we attach to each sample extra frames for each projection including background background.png, up to 30 previous frames and up to 10 following frames. (6) uncertainty masks: are yet another set of masks showing areas covering distal segments branching from the labelled arteries, where the specialists were unsure whether it should or should not be included and classified to one of the significant SYNTAX arteries. The available labels have been illustrated in Figure 5. The final bundle consists of 644 data points. Whereas the article focuses on the usage of the labeled data to SYNTAX + catheter segmentation, it could nevertheless be used for more challenging tasks including the following: • Stenosis detection (stenosis masks) - location and classification of the severity of the lesions visible on the x-ray. • Frame interpolation (supplementary frames) - inferring intermediate frames based on neighbouring images. • 3D reconstruction (SID, SOD, α, β) - the construction of 3D models of arteries based on the input 2D projections. Notably, the data we share is stripped off demographic data, to reduce the risk of patient re-identification. Label statistics are significant in reaching further conclusions in research based on our dataset. As seen in Figure 2 there is an imbalance in segment labels in a few domains: (1) there is more left sides than right sides - more videos are acquired for this particular side, it being harder to understand (2) the more distal the segment, the fewer the data samples including its mask - ultimately, some segments are not in the coronary trees of some patients, which is dependent on the heart dominance (i.e. segment 15 only appearing for people with left dominance). Second of all, the gathered data is not uniform in terms of age group magnitudes as mentioned i.e. there is naturally more people in age groups where the coronary artery disease is most common (50-80 years). The entire human population is not reflected in the data and thus the models using the dataset as base should only be utilized for the aforementioned age gap. This is visualized in Figure 3, not only that, there is more men across the samples than women - statistically speaking men are more at risk of CAD and develop the condition earlier (thus the imbalance) [28]. Figure 4: Labelled X-ray angiography images overlaid on (α, β)-coordinate system. Figure 5: Summary of the labels available for a single sample 2.2 Architectures and Inductive Biases In the study we identify four groups of architectures and associated inductive biases: (1) Standard CNNs including U-Net [32], Attention U-Net [27]s, SegResNet[25], nnUNet [15], PSPNet [54], FPN [21], UPerNet [42], DeepLabV3 [5] and DeepLabV3+ [6]. This is a well-known and studied approach to the task, with an implicit assumption that the nearby pixels are related. Furthermore, in this case, the models satisfy translation equivariance [2]. (2) ViTs (Vision Transformers): SegFormer [43] and SwinUNETR [12] capture long-range dependencies using self attention. These architectures mitigate to an extent the lack of standard inductive biases of CNNs using shifted windows (SwinUNETR) or patch merging and usage of CNN feature extraction (SegFormer). (3) Modern and Large-Kernel CNNs: ConvNeXtV2 [40] + DeepLab/Hamburger [11], MedNeXt [33], SegNeXt [11] and RepLKNet [8] (+ DeepLab/Hamburger) seek to reconcile the local efficiency of CNNs with large receptive fields typical to transformers. (4) State Space Models (SSMs): Mamba-UNet[39], SegMamba [44] these mitigate the bottlenecks of ViTs’ self-attention quadratic complexity and often hybridize CNN components to retain their priors. (5) Domain-Pretrained Baselines: XRayVision [7] (Encoder) + U-Net and RadImageNet [24] + DeepLab - these baselines test the hypothesis that inductive biases learned from massive medical imaging datasets (chest X-rays and multi-modal radiology scans) are more effective than standard ImageNet weights for the specific visual textures of angiography. We provide in brief implementation details for each of these architectures in Table 1. Table 1: Architectural and implementation details of the evaluated segmentation models. Model Implementation Details Model Implementation Details ConvNeXtV2 DeepLab ConvNeXtV2 encoder + ASPP head ConvNeXt DeepLab ConvNeXt encoder + ASPP head SegFormer Big MiT-B4 encoder + MLP decoder SegFormer MiT-B0 encoder + MLP decoder MambaUNet Pure State Space Model (SSM) UNet VM-UNet Vision Mamba-based UNet SwinUMamba Swin Transformer + Mamba blocks SwinUNETR Swin Transformer + CNN decoder RepLKNet Hamburger Large-kernel + Matrix decomp. head ConvNeXtV2 Hamburger ConvNeXtV2 + Matrix decomp. head RadImageNet DeepLab ResNet pretrained on RadImageNet XRayVision UNet DenseNet121 from TorchXRayVision DeepLabV3+ ResNet + ASPP with low-level skip DeepLab Standard ResNet + ASPP FPN ResNet50 + Feature Pyramid Network ConvNeXt FPN ConvNeXt + Feature Pyramid Network UPerNet ResNet + FPN & Pyramid Pooling ConvNeXt UPerNet ConvNeXt + UPerNet decoder PSPNet ResNet50 + Pyramid Pooling (PPM) SegNeXt Multi-scale Conv Attention (MSCA) MedNeXt ConvNeXt styled for medical imaging Attention UNet CNN with spatial attention gates UNet Standard symmetric CNN with skips RepLKNet DeepLab Large-kernel encoder + ASPP head Furthermore, we present an ensemble of ConvNext V2 + FPN + Mamba U Net with three different voting algorithms: (A) hard voting, which is a simple majority voting (B) soft voting, which is a mean of all probabilities of each classes (C) entropy weighted, which leverages the entropy of each model’s predicted probability distribution. In this strategy, the final prediction is calculated as a weighted average, where each model’s weight is inversely proportional to its entropy, thereby granting greater influence to the models that are most confident in their predictions for a given pixel. 2.3 Training We train each model representative of each of the architectures using the same set of losses, optimizer and scheduler. We set ℒ=ℒCE+ℒDiceL=L_CE+L_Dice (2) where ℒCEL_CE is a regular multi-class cross entropy loss and ℒDiceL_Dice is a multi-class Dice coefficient loss. We choose for the optimizer AdamW with β1=0.5 _1=0.5, β2=0.999 _2=0.999, λ=10−2λ=10^-2 and η=10−4η=10^-4. We use cosine annealing with warm restarts for the learning rate scheduler with T0=20T_0=20, multiplier of 11 and ηmin=10−6 _min=10^-6. The above hyperparameters have been chosen empirically through a set of pilot experiments and represent values commonly used in deep learning research, therefore they will not be subject to ablation in this study. To further stabilize the training process across our diverse set of architectures, we employ an Exponential Moving Average (EMA) of the model weights [16]. Instead of relying on the active parameters optimized via backpropagation, EMA maintains a separate copy of the weights. At each training step t, the model weights θt _t are updated by the optimizer, whereas EMA weights are subsequently updated using the combination θtEMA=αtθt−1EMA+(1−αt)θt _t^EMA= _t _t-1^EMA+(1- _t) _t (3) We apply a step-dependent decay rate αt=minαmax,1−1/t _t=min\ _max,1-1/t\, where αmax=0.999 _max=0.999. EMA acts as a low-pass filter that smooths out optimization noise and guides the models to more generalizable minima without incurring additional inference cost. Finally, we have optimized the models using early stopping mechanism with a patience of 100 epochs for a maximum of 700 epochs. 2.4 Experiments For a comprehensive evaluation of each model’s performance on the given task we use a set of metrics covering variants of F1F_1 score F1=2TP2TP+FP+FNF_1= 2TP2TP+FP+FN (4) We define F1(m)F_1^(m) as a mean of f1f_1 of every class (except background). Moreover, we compute F1(ci)F_1^(ci) - mean of segments of critical significance to diagnoses and F1(hi)F_1^(hi), F1(li)F_1^(li) for respectively high and low importance to the diagnosis. The per-importance segment split is as follows (convention: SYNTAX nomenclature [explanation]): (critical) 1 [RCA proximal], 5 [Left main/LM], 6 [LAD proximal], 11 [Proximal circumflex]; (high) 7 [LAD mid], 2 [RCA mid], 9 [LAD: First diagonal], 12 [LCX: intermediate/anterolateral], 13 [LCX: distal circumflex], 14 [LCX: left posterolateral], 3 [RCA distal]; (low) the rest of the SYNTAX segments [37]. Likewise, we use Hausdorff distance [38, 14] HD95(X<Y)=P95(miny∈Yd(x,y))HD_95(X<Y)=P_95( _y∈ Yd(x,y)) (5) where d is some distance metric in pixels and P95P_95 denotes 95th percentile of the distances distribution between the prediction and the closest pixel of the ground truth class of the regarded pixel. Metrics that we’ve defined above do not fully capture the quality of the reflected topologies of the arteries, thus requiring us to use clDiceclDice [36] clDice=2Tprec(SP,VG)⋅Tsens(SG,VP)Tprec(SP,VG)+Tsens(SG,VP)clDice= 2T_prec(S_P,V_G)· T_sens(S_G,V_P)T_prec(S_P,V_G)+T_sens(S_G,V_P) (6) where Tprec=|SP∩VG||SP|T_prec= |S_P∩ V_G||S_P| and Tsens=|SG∩VP||SG|T_sens= |S_G∩ V_P||S_G| represent topological precision adn sensitivity respectively. VPV_P and VGV_G denote predicted and ground truth segmentation mask respectively and S∗S_*, analogically, their skeletons. To further evaluate the clinical utility of the models, we additionally propose a metric related to the artery width (i.e. does the model reflect the stenoses properly), quantified as a diameter error (DEDE) DE=1|SG|∑x∈SG|DP(x)−DG(x)|DE= 1|S_G| _x∈ S_G|D_P(x)-D_G(x)| (7) where D∗(x)D_*(x) is a diameter at skeleton pixel x (obtained from a regular distance transform). Notably we measure FLOPS (floating operations / s) of each model and a calibration error defined as =AUCROC(U,E)C=AUC_ROC(U,E) (8) where we take 5 samples from each of the models, enabling dropout during inference, to measure uncertainty U of each model and the capability of the uncertainty map as a binary classifier of model errors E. We assume the probability of model making an error Prob(y^≠y)=UProb( y≠ y)=U. We split the dataset to a single training/validation/test fold, which we then use to train and test each of the models mentioned in section 2.2. To prevent data leakage we stratify the splits at both the institutional and patient levels. Specifically, the data is first grouped by the originating medical center, and then further aggregated by individual patient IDs. Within each medical center, we randomly allocate entire patients-rather than individual imaging samples-into training (70%), validation (10%), and test (20%) sets. This dual-level stratification ensures that all data belonging to a single patient remains strictly isolated within one specific fold, eliminating patient-level data leakage. Furthermore, applying this division independently within each center guarantees that the proportional representation of the different hospitals is preserved across the training, validation, and testing splits, mitigating center-specific bias and ensuring a reliable evaluation of the models. To measure the generalization of the models, we also perform a leave-one-center-out cross-validation. For this evaluation we select only the top-performing model from each group. To evaluate these models, we partition the data into single-center subsets, using one center for testing and the remaining ones for training. Each model is trained for 300 epochs, maintaining the same settings as in the general experiments. In addition, we conduct tests related to data efficiency. As with the generalization tests, we select several models, each of which is trained on a random subset of the training set. Specifically, we train the models on tdf∈0.2,0.4,0.5,0.8,1.0tdf∈\0.2,0.4,0.5,0.8,1.0\ of the training set. For each subset size, the data remains identical for every model. Unlike in the standard training phase, to ensure a fair comparison (so that each model converges), we do not set a maximum number of epochs; instead, the maximum number of iterations is fixed at 8,000, with early stopping disabled. All other settings remain consistent with the general training. Ultimately, each model is evaluated on the entire test set. 3 Results In this section we summarize the results of the conducted experiments. First we show a general benchmark of all of the architectures across different families. Then, for a select few architectures, we investigate more in-depth the generalization and data efficiency features. Moreover for a single model (ConvNext V2 + DeepLab V3 Plus decoder) we present the performance in context of common acquisition and demographic parameters including age, sex and projection angles. 3.1 Benchmark Table 2: Benchmark quantitative results. Model F1(m)F_1^(m) F1(ci)F_1^(ci) F1(hi)F_1^(hi) F1(li)F_1^(li) DEDE clDiceclDice C HD95HD_95 ConvNeXt V2 DeepLab 0.456±0.0270.456_± 0.027 0.694±0.0380.694_± 0.038 0.464±0.0450.464_± 0.045 0.273±0.0310.273_± 0.031 2.332±0.1152.332_± 0.115 0.810±0.0160.810_± 0.016 0.959±0.0060.959_± 0.006 57.662±5.65957.662_± 5.659 VMUnet 0.438±0.0230.438_± 0.023 0.674±0.0340.674_± 0.034 0.461±0.0420.461_± 0.042 0.255±0.0280.255_± 0.028 2.280±0.1142.280_± 0.114 0.810±0.0170.810_± 0.017 0.978±0.0030.978_± 0.003 57.649±4.66157.649_± 4.661 FPN 0.437±0.0250.437_± 0.025 0.668±0.0390.668_± 0.039 0.428±0.0420.428_± 0.042 0.266±0.0300.266_± 0.030 2.463±0.1252.463_± 0.125 0.800±0.0170.800_± 0.017 0.973±0.0050.973_± 0.005 56.904±4.85256.904_± 4.852 DeepLab V3+ 0.432±0.0240.432_± 0.024 0.680±0.0340.680_± 0.034 0.429±0.0430.429_± 0.043 0.242±0.0280.242_± 0.028 2.537±0.1142.537_± 0.114 0.789±0.0160.789_± 0.016 0.963±0.0050.963_± 0.005 57.686±4.89857.686_± 4.898 SegFormer (Big) 0.428±0.0250.428_± 0.025 0.678±0.0360.678_± 0.036 0.442±0.0450.442_± 0.045 0.241±0.0280.241_± 0.028 2.681±0.1302.681_± 0.130 0.776±0.0170.776_± 0.017 0.971±0.0050.971_± 0.005 58.728±4.99758.728_± 4.997 Swin-UMamba 0.427±0.0240.427_± 0.024 0.678±0.0350.678_± 0.035 0.450±0.0440.450_± 0.044 0.237±0.0270.237_± 0.027 2.226±0.1172.226_± 0.117 0.798±0.0160.798_± 0.016 0.960±0.0050.960_± 0.005 62.128±5.82562.128_± 5.825 Mamba-UNet 0.427±0.0230.427_± 0.023 0.683±0.0340.683_± 0.034 0.451±0.0440.451_± 0.044 0.241±0.0250.241_± 0.025 2.176±0.1112.176_± 0.111 0.804±0.0160.804_± 0.016 0.976±0.0030.976_± 0.003 59.702±5.30759.702_± 5.307 RepLKNet Hamburger 0.425±0.0240.425_± 0.024 0.660±0.0370.660_± 0.037 0.430±0.0440.430_± 0.044 0.238±0.0280.238_± 0.028 2.930±0.1292.930_± 0.129 0.750±0.0190.750_± 0.019 0.958±0.0050.958_± 0.005 54.428±4.75654.428_± 4.756 ConvNeXt FPN 0.424±0.0230.424_± 0.023 0.651±0.0340.651_± 0.034 0.426±0.0420.426_± 0.042 0.237±0.0270.237_± 0.027 2.975±0.0962.975_± 0.096 0.769±0.0170.769_± 0.017 0.975±0.0030.975_± 0.003 52.108±4.74952.108_± 4.749 ConvNeXt DeepLab 0.423±0.0260.423_± 0.026 0.681±0.0350.681_± 0.035 0.424±0.0440.424_± 0.044 0.240±0.0270.240_± 0.027 2.334±0.1122.334_± 0.112 0.810±0.0170.810_± 0.017 0.941±0.0080.941_± 0.008 61.436±5.72361.436_± 5.723 DeepLab 0.417±0.0250.417_± 0.025 0.671±0.0370.671_± 0.037 0.427±0.0430.427_± 0.043 0.236±0.0280.236_± 0.028 2.551±0.1092.551_± 0.109 0.799±0.0160.799_± 0.016 0.937±0.0070.937_± 0.007 59.754±5.20359.754_± 5.203 Rad ImageNet DeepLab 0.407±0.0240.407_± 0.024 0.652±0.0370.652_± 0.037 0.415±0.0440.415_± 0.044 0.228±0.0280.228_± 0.028 2.721±0.1282.721_± 0.128 0.765±0.0180.765_± 0.018 0.977±0.0040.977_± 0.004 61.415±4.92961.415_± 4.929 MedNeXt 0.404±0.0240.404_± 0.024 0.669±0.0360.669_± 0.036 0.430±0.0450.430_± 0.045 0.213±0.0250.213_± 0.025 2.384±0.1192.384_± 0.119 0.791±0.0170.791_± 0.017 0.957±0.0060.957_± 0.006 68.186±6.06068.186_± 6.060 Segformer 0.401±0.0230.401_± 0.023 0.638±0.0350.638_± 0.035 0.418±0.0430.418_± 0.043 0.212±0.0240.212_± 0.024 2.614±0.1122.614_± 0.112 0.783±0.0160.783_± 0.016 0.979±0.0030.979_± 0.003 68.714±5.28168.714_± 5.281 ConvNeXt V2 Hamburger 0.397±0.0230.397_± 0.023 0.642±0.0360.642_± 0.036 0.406±0.0430.406_± 0.043 0.217±0.0260.217_± 0.026 3.061±0.1233.061_± 0.123 0.742±0.0180.742_± 0.018 0.972±0.0030.972_± 0.003 59.630±5.01559.630_± 5.015 Attention UNet 0.396±0.0250.396_± 0.025 0.650±0.0360.650_± 0.036 0.422±0.0450.422_± 0.045 0.204±0.0270.204_± 0.027 2.442±0.1262.442_± 0.126 0.778±0.0170.778_± 0.017 0.956±0.0060.956_± 0.006 89.172±7.40889.172_± 7.408 SegNeXt 0.386±0.0230.386_± 0.023 0.644±0.0370.644_± 0.037 0.407±0.0430.407_± 0.043 0.208±0.0240.208_± 0.024 2.960±0.1162.960_± 0.116 0.760±0.0170.760_± 0.017 0.970±0.0050.970_± 0.005 61.459±4.92961.459_± 4.929 Swin-UNETR 0.372±0.0240.372_± 0.024 0.603±0.0400.603_± 0.040 0.387±0.0440.387_± 0.044 0.188±0.0240.188_± 0.024 2.493±0.1212.493_± 0.121 0.763±0.0170.763_± 0.017 0.941±0.0070.941_± 0.007 77.336±6.41677.336_± 6.416 PSPNet 0.362±0.0240.362_± 0.024 0.591±0.0380.591_± 0.038 0.361±0.0430.361_± 0.043 0.194±0.0240.194_± 0.024 3.271±0.1153.271_± 0.115 0.724±0.0200.724_± 0.020 0.977±0.0030.977_± 0.003 62.956±5.02462.956_± 5.024 XRay Vision UNet 0.360±0.0240.360_± 0.024 0.602±0.0410.602_± 0.041 0.355±0.0400.355_± 0.040 0.199±0.0240.199_± 0.024 2.544±0.1352.544_± 0.135 0.784±0.0170.784_± 0.017 0.968±0.0040.968_± 0.004 71.179±5.62971.179_± 5.629 UPerNet 0.356±0.0230.356_± 0.023 0.609±0.0410.609_± 0.041 0.361±0.0440.361_± 0.044 0.185±0.0230.185_± 0.023 2.680±0.1402.680_± 0.140 0.759±0.0180.759_± 0.018 0.973±0.0050.973_± 0.005 68.974±5.22368.974_± 5.223 UNet 0.350±0.0220.350_± 0.022 0.599±0.0370.599_± 0.037 0.342±0.0380.342_± 0.038 0.163±0.0200.163_± 0.020 2.784±0.1092.784_± 0.109 0.751±0.0140.751_± 0.014 0.942±0.0070.942_± 0.007 94.775±6.61294.775_± 6.612 ConvNeXt UPerNet 0.315±0.0220.315_± 0.022 0.562±0.0410.562_± 0.041 0.331±0.0400.331_± 0.040 0.153±0.0190.153_± 0.019 3.031±0.1513.031_± 0.151 0.736±0.0200.736_± 0.020 0.963±0.0090.963_± 0.009 70.901±5.49870.901_± 5.498 RepLKNet DeepLab 0.280±0.0170.280_± 0.017 0.507±0.0310.507_± 0.031 0.298±0.0350.298_± 0.035 0.126±0.0170.126_± 0.017 5.742±0.1535.742_± 0.153 0.515±0.0210.515_± 0.021 0.948±0.0040.948_± 0.004 60.907±4.94560.907_± 4.945 Hard Voting 0.479±0.0260.479_± 0.026 0.705±0.0360.705_± 0.036 0.493±0.0450.493_± 0.045 0.286±0.0310.286_± 0.031 2.428±0.1302.428_± 0.130 0.801±0.0180.801_± 0.018 0.981±0.0030.981_± 0.003 51.537±5.02251.537_± 5.022 Soft Voting 0.475±0.0250.475_± 0.025 0.708±0.0350.708_± 0.035 0.492±0.0460.492_± 0.046 0.285±0.0310.285_± 0.031 2.307±0.1212.307_± 0.121 0.815±0.0170.815_± 0.017 0.981±0.0030.981_± 0.003 52.032±5.03352.032_± 5.033 Entropy Weighted 0.468±0.0250.468_± 0.025 0.704±0.0350.704_± 0.035 0.483±0.0450.483_± 0.045 0.279±0.0300.279_± 0.030 2.296±0.1202.296_± 0.120 0.818±0.0170.818_± 0.017 0.981±0.0030.981_± 0.003 52.729±5.07852.729_± 5.078 Figure 6: Various model metrics w.r.t. GFLOPs, mind the log scale of X axis. Figure 7: Architectures performance across SYNTAX segments with highlighted ensembles. In Table 2, the results of the wide architecture sweep are presented. As seen, the encoder-decoder combination of ConvNeXt V2 + DeepLabV3+ offers the best overall results as a single architecture, dominating the F1F_1 score categories (F1(m)F_1^(m), F1(ci)F_1^(ci), F1(hi)F_1^(hi), F1(li)F_1^(li)) and tying for the best clDiceclDice. Considering other individual metrics, Mamba U-Net is the most accurate at estimating the vessel diameter (DEDE of 2.176), whereas Hausdorff distance is minimized by ConvNeXt FPN. On the other end of the spectrum, the large-kernel RepLKNet paired with DeepLab fails to exceed the performance of a basic U-Net, performing significantly worse than the vast majority of the tested architectures. Pairing the same RepLKNet backbone with a Hamburger decoder recovers the performance, suggesting a specific architectural incompatibility between RepLKNet and DeepLab for this dataset. Architecture family-wise, all of the Mamba-based models (VM-UNet, Swin-UMamba, and Mamba U-Net) show highly respectable results. However, they do not reach the absolute top, performing in the same range as classic strong baselines (e.g., ResNet + DeepLabV3+ or FPN). This indicates that whereas state-space models are highly capable, there may not be a strict benefit to using them (individually) over well-tuned classical CNN approaches for this specific segmentation task. Regarding Vision Transformer (ViT) based architectures, models such as SegFormer and SwinUNETR demonstrate varied performance. More specifically, the scaled-up SegFormer ("Segformer (Big)") achieves a competitive mean F1F_1 score of 0.428. Other Transformer approaches like SwinUNETR (F1(m)F_1^(m) of 0.372) fall towards the lower end of the benchmark. Overall, whereas ViTs are highly capable, they do not surpass the leading modern CNNs, suggesting that advanced convolutions may still hold an edge in terms of the inductive bias adjustment. The impact of domain-specific medical pre-training was evaluated utilizing RadImageNet and Xrayvision weights, as the strictly domain-aligned weights are not available. Whereas initializing a U-Net with Xrayvision weights provides a marginal improvement over the standard U-Net baseline (F1(m)F_1^(m) increases from 0.350 to 0.360, alongside a notable bump in clDiceclDice), applying RadImageNet weights to DeepLab surprisingly slightly underperforms the standard DeepLab configuration (0.4070.407 vs. 0.4170.417). This indicates that whereas medical pre-training can offer slight benefits to baseline architectures, it does not automatically guarantee superior feature extraction across all decoder variants. To test the boundaries, an ensemble of models (i.e., ConvNeXt V2, FPN, and Mamba U-Net) was evaluated. The ensemble pushes F1(m)F_1^(m) to a peak of 0.479, improving also the centerlines (clDiceclDice). The usage of different voting strategies is not critical to achieving these gains, as both Hard Voting and Soft Voting perform similarly. Entropy weighted voting is slightly worse - one possible explanation is that the strategy might not be beneficial, because it might jeopardize the decorrelation of the various model results. It brings us to the calibration argument. C reveals interesting fact about all of the arguments, that sampling from them acts as an efficient error prediction utility, offering extremely high AUCAUC, which might mean that the models are well-correlated, however we are not fully convinced of the metric C being the best indicator of that. Figure 6 puts the key performance metrics in the context of computational cost and model sizes. The plots reveal that scaling up verically does not strictly correlate with improved segmentation accuracy. The heaviest architectures (i.e. convnext upernet) fail to justify the computational demands yielding sub-par metrics compared to lighter models. Instead, the highest-performing architectures reside roughly in the middle of the X axis (ConvNeXt V2 + DeepLabV3+). Mamba U-Net emerges as exceptionally efficient; it achieves the lowest DEDE whereas being located at the lower end of the GFLOPsGFLOPs spectrum. Ultimately, the inductive biases such as advanced convolutions and state space models are more effective to the task than a simple parameter scaling. In addition, we also attach F1F_1 scores for each class (denoted F1(j)F_1^(j) where j - symbol class according to SYNTAX e.g. F1(4a)F_1^(4a)) and F1(catheter)F_1^(catheter) in Fig. 7. The catheter is segmented well by most of the architectures (all except the ReplKNet+DeepLab underperformer). The same is observed for main RCA segments: 01, 02 and LCA: 05. There is a steep error increase between the aforementioned and further branching segments i.e. 03. 06, 07. Finally there are also the distals with critically low scores which was in general established already in the above paragraphs: 16a, 16b, 16c, 9a, 10, 10a, 14, 15 - these are near zero in terms of F1F_1. These segments also exhibit substantial inter-observer variability among expert annotators and therefore may represent the intrinsic labeling ceiling rather than solely algorithmic failure. Ignoring the outlier ReplKNet Deeplab, throughout the architectures we see major improvements w.r.t. U-Net baseline for plenty segments, going from "yellow" to "green": 04, 16, 07, 08, 11 - these results prove once again that the inter-architectural differences in design choices are not to be underestimated. The quantitative metrics are qualitatively reflected in Figure 8. The figure provides a visual comparison of predictions from a representative selection of models. These specific architectures were chosen to illustrate qualitative differences across distinct families, as the quantitative margins between the absolute highest-scoring models were negligible. Figure 8: Qualitative comparison of inference results across five selected examples. The displayed models (RepLKNet, VM-UNet, ConvNeXt V2 Deeplab, and the Soft Voting Ensemble of VM-UNet, ConvNeXt V2 Deeplab and FPN) were chosen to represent distinct architectural families, as the quantitative differences among the top-performing models were marginal. Columns from left to right represent: the input X-ray, Ground Truth (GT) annotations, and the selected models. The visualizations highlight varying degrees of performance in capturing segment continuity and distal branches. 3.2 Generalization Table 3 presents the results of the generalization experiments conducted for each center in the dataset. For every model, the best and worst performances are highlighted. Additionally, the mean performance across multiple test sets are calculated to assess the variability of the results. The table also specifies the training and test set sizes for each experiment. While performance may initially appear to fluctuate across centers, this variance is largely attributable to differences in test set sizes. Specifically, the drop in performance observed for Center 2 corresponds to a test set that was larger than its available training data. In contrast, centers with more proportionate train-test splits yielded results consistent with the baseline averages. Therefore, the overall findings confirm that the trained models maintain strong generalization capabilities. Table 3: Generalization Qualitative Results. Hold-out Clinical Center Training Set Size Testing Set Size F1(m)F_1^(m) Convnextv2 Fpn Segformer Vmunet Center 1 606 39 0.539±0.0470.539_± 0.047 0.541±0.0540.541_± 0.054 0.466±0.0430.466_± 0.043 0.480±0.0480.480_± 0.048 Center 2 279 366 0.380±0.0160.380_± 0.016 0.398±0.0170.398_± 0.017 0.367±0.0160.367_± 0.016 0.387±0.0150.387_± 0.015 Center 3 497 148 0.472±0.0230.472_± 0.023 0.425±0.0240.425_± 0.024 0.428±0.0230.428_± 0.023 0.448±0.0240.448_± 0.024 Center 4 564 81 0.458±0.0300.458_± 0.030 0.400±0.0320.400_± 0.032 0.406±0.0300.406_± 0.030 0.390±0.0320.390_± 0.032 Center 5 634 11 0.488±0.0830.488_± 0.083 0.510±0.0590.510_± 0.059 0.459±0.0900.459_± 0.090 0.483±0.0900.483_± 0.090 Average 0.467 0.455 0.425 0.438 3.3 Data efficiency The results of the data efficiency experiment are presented in Figure 9. Across all subsets, the ConvNextv2+DeepLab model consistently outperforms other architectures. Notably, this model achieves performance on 50% of the training data that is comparable to the results reached by most other models using the full dataset. The sustained upward trend suggests that further increases in training data volume would likely lead to higher segmentation accuracy. These findings indicate that data availability, rather than model architecture, remains the primary bottleneck in improving segmentation performance. Figure 9: Impact of training data volume on the predictive accuracy of the evaluated models 3.4 Acquisition and patient-specific parameters Figure 10: F1 across various acquisition perspectives. Bubble size denotes cluster size. Figure 11: F1 across segments belonging to different sides. For LCA the performance is significantly worse, as expected, due to its entanglement. Figure 12: Patient demographic influencing results. Figure 12 puts results in the context of demographic. There seems to be a modest correlation of the general method performance (reflected as F1(m)F_1^(m)) to patient age and sex however we take note that these effects might be caused by the distribution of samples and specific cases along the population. An intuitive observation is that the performance is worse for the left side than for the right (see Figure 11) as the former is more convoluted and tortuous. Overlaying results on α, β angle-space, as is done in Figure 10, we also identify particular projection settings that underperform opposed to the entire population (RAO Cra, RAO Cau) as well as ones that overperform (RAO straight and LAO straights). 4 Discussion The comprehensive benchmark of architectures on this dataset reveals several insights worth discussing. Despite the recent shift towards SSM and ViTs and their popularity, modern CNNs remain the best choice for this task. Specifically, ConvNext V2 + DeepLabV3 achieving the best scores for most of the relevant metrics suggests that translational equivariance combined with multi-scale features of the ASPP is highly efficient for such vascular structures. Transformers being overperformed by the CNNs suggests the lack of the needed bias and the locality forcing the model to maintain the continuity of the vessels, which the models were unable to achieve due to limited data. Whereas missing the top scores in terms of centerline and vessel area overlap, SSMs achieved the smallest diameter error. The ability of Mamba to model long-range dependencies allows them to efficiently capture the consistent understanding of the artery shapes along the entire tree. This suggests the applicability of Mamba models for morphological measurements (e.g. stenosis assessment). From a clinical perspective, accurate vessel segmentation constitutes a prerequisite for automated quantitative coronary angiography (QCA) workflows and may facilitate future AI-assisted stenosis quantification. Beyond anatomical labeling alone, accurate preservation of vessel borders, centerlines, and local lumen diameter is essential for reliable estimation of lesion severity and disease burden. Consequently, improvements in segmentation quality may translate into more robust downstream measurements, including automated stenosis assessment and future end-to-end SYNTAX score computation. The findings reveal that the arbitrary pairing of encoders and decoders with each other, especially choosing the more recent architectures of which, does not necessarily guarantee any improvement. The most extreme observed case is pairing RepLKNet with DeepLab dropping the results below even a baseline U Net. These findings suggest, that the large kernels of RepLKNet clash with atrous convolutions of DeepLab. On the other hand, replacing the decoder with Hamburger, in which the dilated convolutions do not appear, recovers the performance. The argument about large kernels is supported by the RepLKNet DeepLab prediction shapes i.e. them showing really imprecise segmentation masks diameters, reflecting the limited resolution at which the model processes image. Counterintuively, pretraining on medical data provides either marginal or negative results compared to standard ImageNet pretraining. It follows that the general feature extractor is robust enough and that possibly coronary angiography images do not share common traits with neither chest x-rays (X-Ray Vision) nor CT, MRI and US (RadImageNet). Aligned results across multiple clinical centers demonstrate that the trained models exhibit strong generalization capabilities. These findings suggest that incorporating multi-center datasets, acquired using diverse imaging devices, is essential for developing models that achieve clinically acceptable performance and remain robust for real-world deployment. Experiments across various training subsets demonstrate that dataset size is perhaps the most critical factor in determining final model performance. While our comparison of diverse architectures indicates that certain models outperform others, the results suggest that the primary limitation in this field lies in data availability rather than architectural design. Consequently, to further enhance performance for clinical applications, we suggest that future efforts should increasingly focus on the acquisition of larger, more diverse datasets as a primary driver of improvement, alongside continued model development. The success of the hybrid architecture (CNN + Mamba) proves that SSM and CNN learn complementary features rather than embeddings merely corresponding to each other. As discussed eariler, CNNs, while in general the best, give way to Mamba in terms of reflecting the accurate diameter, however, oddly, it is not what is being improved by ensembling. Conversely, it is the performance on segments of high and low importance that show the biggest improvement. This might suggest however that Mamba adds to CNNs a long range hierarchical context necessary for better segmentation of such distal segments. Not for all of the projections the model performance is the same. As for the RAO Cranial view, the foreshortening of LAD and LCX causes certain arteries to overlap, which leads to the decreased F1F_1. On the other hand, views like LAO Straight, RAO Straight are often prioritized clinically for their clarity, lack of distortions, thus the training data likely contains more high quality examples of these, allowing the model to generalize better to them. We have also seen different age-performance trend, showing higher performance for males compared to females, which can be attributed to two facts: (1) males statistically have larger coronary artery diameters than females [9] (2) it is caused by the way data is distributed in terms of sex and age i.e. the training set consists of more male coronary arteries therefore the models generalize better. Overall, the results are still far from reaching the levels of robustness necessary for deployment in clinical applications. Whereas the model should be sufficient for the critical segments, the importance of hihi segments should not be underestimated - F1(hi)F_1^(hi) is particularly low, whereas the lesions of these segments are also significant and should by no means be omitted. High clDiceclDice however suggests the models successfully express the concept of a segment continuity and lastly, using inference-time dropout grants a successful quantitative metric of when the model is uncertain, which is a valuable trait all clinical production models should have. Limitations Of The Study While our benchmark evaluates frames independently, coronary angiography is a dynamic video sequence. Although we provide in CARDIAG the supplementary frames, the current models do not take the benefit of that and ultimately if applied to full sequences might exhibit a lack of smoothness. Furthermore, although our analysis revealed performance disparities across patient sex, age and different projection angles, we have not implemented specific mitigation strategies such as targeted data augmentation, noisy label learning and long tail learning to rectify these biases. Additionally, we have not calculated the final end-to-end metric (SYNTAX score) for the evaluation of models based on the segmentations - it would require also lesion detection which is out of the scope of this study. Lastly, we have not analyzed inter-annotation divergence therefore we do not know about aleatoirc uncertainty which might ultimately be a glass ceiling of any possible segmentation methods. Future works and Clinical Relevance The release of the CARDIAG dataset and our accompanying benchmarks establishes a foundational step toward fully automated clinical decision-support systems. Accurate semantic labeling of the coronary tree is a critical prerequisite for such systems, providing the anatomical context necessary for precise lesion localization within the standardized SYNTAX framework, integration with stenosis detection algorithms, and automated SYNTAX score calculation. Importantly, segmentation quality in this domain must be evaluated beyond conventional computer vision metrics like the F1 score; preserving vessel continuity, centerline topology, and lumen diameter directly influences subsequent clinical analyses, including stenosis severity estimation and quantitative coronary angiography (QCA). Future work can move beyond 2D spatial analysis by incorporating information from angiographic videos. Furthermore, the supplementary unlabelled frames included in the CARDIAG release present an opportunity to explore self-supervised pre-training for learning more robust initial weights. Ultimately, these technical advancements will facilitate an end-to-end pipeline capable of translating x-ray images into SYNTAX score estimation, and ultimately AI-assisted support for PCI versus CABG treatment planning. Author contributions D. B. Lau: Conceptualization, Methodology, Software, Validation, Investigation, Data curation, Writing - Original Draft. J. Szyjut: Methodology, Software, Validation, Investigation. H. Malinowski: Methodology, Software, Validation, Investigation. A. Brzeski: Conceptualization, Software, Validation, Data curation. T. Dziubich: Conceptualization, Writing - Review & Editing, Supervision, Validation, Data curation. T. Figatowski: Validation, Data curation, Writing - Review. N. Zielińska: Validation, Data curation, Writing - Review. R. Targoński: Validation, Supervision, Data curation, Writing - Review. Acknowledgment The authors would like to thank CI TASK for granting access to resources and CAISE - a cloud computing platform that allowed for the efficient training and reduced the experiments run time (notably four H100 GPUs). Dataset availability The dataset described in the study is available on Zenodo at https://zenodo.org/records/19958730 together with the weights of the models used in the benchmark. The source code is available at https://github.com/cvlab-ai/cardiag-benchmark Funding The work was supported in part by project “Cloud Artificial Intelligence Service Engineering (CAISE) platform to create universal and smart services for various application areas”, No. KPOD.05.10-IW.10-0005/24, as part of the European IPCEI-CIS program, financed by NRRP (National Recovery and Resilience Plan) funds. References [1] M. Bilal et al. (2023) Multivessel coronary artery segmentation and stenosis localisation using ensemble learning. arXiv preprint arXiv:2310.17954. Note: doi: 10.48550/ARXIV.2310.17954 Cited by: §1. [2] M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković (2021) Geometric deep learning: grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478. Note: doi: 10.48550/ARXIV.2104.13478 Cited by: §2.2. [3] F. Cervantes-Sanchez, I. Cruz-Aceves, A. Hernandez-Aguirre, M. A. Hernandez-Gonzalez, and S. E. Solorio-Meza (2019-12) Automatic segmentation of coronary arteries in X-Ray angiograms using multiscale analysis and artificial neural networks. Appl. Sci. 9 (24), p. 5507. Note: doi: 10.3390/app9245507 Cited by: §1. [4] S. Chang et al. (2024-03) Optimizing ensemble U-Net architectures for robust coronary vessel segmentation in angiographic images. Sci. Rep. 14 (1). Note: doi: 10.1038/s41598-024-57198-5 Cited by: §1, §1. [5] L. Chen, G. Papandreou, F. Schroff, and H. Adam (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587. Note: doi: 10.48550/ARXIV.1706.05587 Cited by: §2.2. [6] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. Eur. Conf. Comput. Vis. (ECCV), Munich, Germany, p. 833–851. Note: doi: 10.1007/978-3-030-01234-2_49 Cited by: §2.2. [7] J. P. Cohen et al. (2022) TorchXRayVision: a library of chest X-Ray datasets and models. In Proc. Med. Image Deep Learn. (MIDL), Zurich, Switzerland. Cited by: §2.2. [8] X. Ding, X. Zhang, Y. Zhou, J. Han, G. Ding, and J. Sun (2022) Scaling up your kernels to 31x31: revisiting large kernel design in CNNs. arXiv preprint arXiv:2203.06717. Note: doi: 10.48550/ARXIV.2203.06717 Cited by: §2.2. [9] J. T. Dodge, B. G. Brown, E. L. Bolson, and H. T. Dodge (1992-07) Lumen diameter of normal human coronary arteries. influence of age, sex, anatomic variation, and left ventricular hypertrophy or dilation.. Circulation 86 (1), p. 232–246. Note: doi: 10.1161/01.cir.86.1.232 Cited by: §4. [10] T. Du et al. (2021-05) Training and validation of a deep learning architecture for the automatic analysis of coronary angiography. EuroIntervention 17 (1), p. 32–40. Note: doi: 10.4244/eij-d-20-00570 Cited by: §1, §1. [11] M. Guo, C. Lu, Q. Hou, Z. Liu, M. Cheng, and S. Hu (2022) SegNeXt: rethinking convolutional attention design for semantic segmentation. In Adv. Neural Inform. Process. Syst. (NeurIPS), New Orleans, LA, USA. Cited by: §2.2. [12] Y. He, V. Nath, D. Yang, Y. Tang, A. Myronenko, and D. Xu (2023) SwinUNETR-V2: stronger swin transformers with stagewise convolutions for 3D medical image segmentation. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Vancouver, BC, Canada, p. 416–426. Note: doi: 10.1007/978-3-031-43901-8_40 Cited by: §2.2. [13] B. Huang et al. (2025-07) Deep learning model for coronary artery segmentation and quantitative stenosis detection in angiographic images. Med. Phys. 52 (7). Note: doi: 10.1002/mp.17970 Cited by: §1. [14] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge (1993) Comparing images using the Hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intell. 15 (9), p. 850–863. Note: doi: 10.1109/34.232073 Cited by: §2.4. [15] F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein (2020-12) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18 (2), p. 203–211. Note: doi: 10.1038/s41592-020-01008-z Cited by: §2.2. [16] P. Izmailov et al. (2018) Averaging weights leads to wider optima and better generalization. In Proc. Conf. Uncertainty Artif. Intell. (UAI), Monterey, CA, USA. Cited by: §2.3. [17] T. J. Jun, J. Kweon, Y. Kim, and D. Kim (2020-08) T-Net: nested encoder–decoder architecture for the main vessel segmentation in coronary angiography. Neural Netw. 128, p. 216–233. Note: doi: 10.1016/j.neunet.2020.05.002 Cited by: §1, §1. [18] S. Kaba, H. Haci, A. Isin, A. Ilhan, and C. Conkbayir (2023-07) The application of deep learning for the segmentation and classification of coronary arteries. Diagnostics 13 (13), p. 2274. Note: doi: 10.3390/diagnostics13132274 Cited by: §1. [19] I. Kruzhilov et al. (2025-02) CoronaryDominance: angiogram dataset for coronary dominance classification. Sci. Data 12 (1). Note: doi: 10.1038/s41597-025-04676-8 Cited by: §1. [20] J. Ku, Y. Lee, J. Shin, I. K. Lee, and H. Kim (2023) MPSeg : multi-phase strategy for coronary artery segmentation. arXiv preprint arXiv:2311.10306. Note: doi: 10.48550/ARXIV.2311.10306 Cited by: §1. [21] T. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie (2017-07) Feature pyramid networks for object detection. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, HI, USA, p. 936–944. Note: doi: 10.1109/CVPR.2017.106 Cited by: §2.2. [22] T. Liu, H. Lin, A. K. Katsaggelos, and A. Kline (2023) YOLO-Angio: an algorithm for coronary anatomy segmentation. arXiv preprint arXiv:2310.15898. Note: doi: 10.48550/ARXIV.2310.15898 Cited by: §1. [23] Y. Ma et al. (2021-10) Self-supervised vessel segmentation via adversarial learning. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Montreal, QC, Canada, p. 7516–7525. Note: doi: 10.1109/ICCV48922.2021.00744 Cited by: §1. [24] X. Mei et al. (2022-09) RadImageNet: an open radiologic deep learning research dataset for effective transfer learning. Radiol. Artif. Intell. 4 (5). Note: doi: 10.1148/ryai.210315 Cited by: §2.2. [25] A. Myronenko (2019) 3D MRI brain tumor segmentation using autoencoder regularization. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI) Brainlesion Workshop (BrainLes), Granada, Spain, p. 311–320. Note: doi: 10.1007/978-3-030-11726-9_28 Cited by: §2.2. [26] National Electrical Manufacturers Association (NEMA) (2026) Digital imaging and communications in medicine (DICOM). Technical report NEMA, Rosslyn, VA, USA. Note: Standard PS3 / ISO 12052. [Online]. Available: https://dicom.nema.org/medical/dicom/2026b/ Cited by: §2.1. [27] O. Oktay et al. (2018) Attention U-Net: learning where to look for the pancreas. In Proc. Med. Image Deep Learn. (MIDL), Amsterdam, The Netherlands. Cited by: §2.2. [28] L. P. Palaniappan et al. (2026-03) 2026 heart disease and stroke statistics: a report of US and global data from the American Heart Association. Circulation 153 (9). Note: doi: 10.1161/cir.0000000000001412 Cited by: §2.1. [29] J. Park et al. (2023-06) Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography. Med. Phys. 50 (12), p. 7822–7839. Note: doi: 10.1002/mp.16554 Cited by: §1, §1. [30] A. Ponomarchuk et al. (2025-02) CardioSyntax: end-to-end SYNTAX score prediction - dataset, benchmark and method. In Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), Tucson, AZ, USA, p. 5873–5883. Note: doi: 10.1109/WACV61041.2025.00573 Cited by: §1. [31] M. Popov et al. (2024-01) Dataset for automatic region-based coronary artery disease diagnostics using X-Ray angiography images. Sci. Data 11 (1). Note: doi: 10.1038/s41597-023-02871-z Cited by: §1. [32] O. Ronneberger, P. Fischer, and T. Brox (2015) U-Net: convolutional networks for biomedical image segmentation. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Munich, Germany, p. 234–241. Note: doi: 10.1007/978-3-319-24574-4_28 Cited by: §2.2. [33] S. Roy et al. (2023) MedNeXt: transformer-driven scaling of convnets for medical image segmentation. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Vancouver, BC, Canada, p. 405–415. Note: doi: 10.1007/978-3-031-43901-8_39 Cited by: §2.2. [34] S. Seo, I. K. Lee, H. Kim, J. Min, and C. Jung (2025-09) Diffusion-based user-guided data augmentation for coronary stenosis detection. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Daejon, Republic of Korea, p. 149–169. Cited by: §1. [35] X. Shi, T. Du, S. Chen, H. Zhang, C. Guan, and B. Xu (2020-07) UENet: a novel generative adversarial network for angiography image segmentation. In Proc. 42nd Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC), Montreal, QC, Canada, p. 1612–1615. Note: doi: 10.1109/EMBC44109.2020.9175334 Cited by: §1, §1. [36] S. Shit et al. (2021-06) clDice - a novel topology-preserving loss function for tubular structure segmentation. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 16555–16564. Note: doi: 10.1109/CVPR46437.2021.01629 Cited by: §2.4. [37] G. Sianos et al. (2005-08) The SYNTAX score: an angiographic tool grading the complexity of coronary artery disease. EuroIntervention 1 (2), p. 219–227. Cited by: §1, §2.1, §2.4. [38] A. A. Taha and A. Hanbury (2015-08) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med. Imaging 15 (1). Note: doi: 10.1186/s12880-015-0068-x Cited by: §2.4. [39] Z. Wang, J. Zheng, Y. Zhang, G. Cui, and L. Li (2024) Mamba-UNet: UNet-like pure visual mamba for medical image segmentation. arXiv preprint arXiv:2402.05079. Note: doi: 10.48550/ARXIV.2402.05079 Cited by: §2.2. [40] S. Woo et al. (2023-06) ConvNeXt V2: co-designing and scaling convnets with masked autoencoders. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Vancouver, BC, Canada, p. 16133–16142. Cited by: §2.2. [41] Z. Xian, X. Wang, S. Yan, D. Yang, J. Chen, and C. Peng (2020-10) Main coronary vessel segmentation using deep learning in smart medical. Math. Probl. Eng. 2020, p. 1–9. Note: doi: 10.1155/2020/8858344 Cited by: §1, §1. [42] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun (2018) Unified perceptual parsing for scene understanding. In Proc. Eur. Conf. Comput. Vis. (ECCV), Munich, Germany, p. 432–448. Note: doi: 10.1007/978-3-030-01228-1_26 Cited by: §2.2. [43] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo (2021) SegFormer: simple and efficient design for semantic segmentation with transformers. In Adv. Neural Inform. Process. Syst. (NeurIPS), p. 12077–12090. Cited by: §2.2. [44] Z. Xing, T. Ye, Y. Yang, G. Liu, and L. Zhu (2024) SegMamba: long-range sequential modeling mamba for 3D medical image segmentation. In Proc. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Marrakesh, Morocco, p. 578–588. Note: doi: 10.1007/978-3-031-72111-3_54 Cited by: §2.2. [45] H. Xu and Y. Wu (2024-08) G2ViT: graph neural network-guided vision transformer enhanced network for retinal vessel and coronary angiograph segmentation. Neural Netw. 176, p. 106356. Note: doi: 10.1016/j.neunet.2024.106356 Cited by: §1. [46] Q. Yang, H. Yi, L. Yi, M. Liu, and X. Chen (2026-02) Accurate segmentation and labeling of coronary artery segments in X-Ray angiography with an improved UNet-based cGAN architecture. Biomed. Signal Process. Control 112, p. 108812. Note: doi: 10.1016/j.bspc.2025.108812 Cited by: §1. [47] M. Zhai, T. Du, R. Yang, and H. Zhang (2019-07) Coronary artery vascular segmentation on limited data via pseudo-precise label. In Proc. 41st Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC), Berlin, Germany, p. 816–819. Note: doi: 10.1109/EMBC.2019.8856682 Cited by: §1, §1. [48] H. Zhang, Z. Gao, D. Zhang, W. K. Hau, and H. Zhang (2023-03) Progressive perception learning for main coronary segmentation in X-Ray angiography. IEEE Trans. Med. Imaging 42 (3), p. 864–879. Note: doi: 10.1109/TMI.2022.3219126 Cited by: §1, §1. [49] X. Zhang, P. Lu, Z. Zheng, and W. Li (2026-01) HR-UMamba++: a high-resolution multi-directional mamba framework for coronary artery segmentation in X-Ray coronary angiography. Fractal Fract. 10 (1), p. 43. Note: doi: 10.3390/fractalfract10010043 Cited by: §1, §1. [50] C. Zhao, M. Esposito, Z. Xu, and W. Zhou (2025-01) HAGMN-UQ: hyper association graph matching network with uncertainty quantification for coronary artery semantic labeling. Med. Image Anal. 99, p. 103374. Note: doi: 10.1016/j.media.2024.103374 Cited by: §1, §1, §1. [51] C. Zhao et al. (2023-11) AGMN: association graph-based graph matching network for coronary artery semantic labeling on invasive coronary angiograms. Pattern Recognit. 143, p. 109789. Note: doi: 10.1016/j.patcog.2023.109789 Cited by: §1, §1, §1. [52] C. Zhao, Z. Xu, P. Baral, M. Esposito, and W. Zhou (2026-01) Multi-graph graph matching for coronary artery semantic labeling in invasive coronary angiograms. Pattern Recognit. 169, p. 111943. Note: doi: 10.1016/j.patcog.2025.111943 Cited by: §1, §1, §1. [53] C. Zhao, Z. Xu, G. Hung, and W. Zhou (2023-11) EAGMN: coronary artery semantic labeling using edge attention graph matching network. Comput. Biol. Med. 166, p. 107469. Note: doi: 10.1016/j.compbiomed.2023.107469 Cited by: §1, §1, §1. [54] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia (2017-07) Pyramid scene parsing network. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, HI, USA, p. 6230–6239. Note: doi: 10.1109/CVPR.2017.660 Cited by: §2.2. [55] X. Zhu, Z. Cheng, S. Wang, X. Chen, and G. Lu (2021-03) Coronary angiography image segmentation based on PSPNet. Comput. Methods Programs Biomed. 200, p. 105897. Note: doi: 10.1016/j.cmpb.2020.105897 Cited by: §1, §1.