Paper deep dive
Dual-Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms
Olatunji Omisore, Ahmed Elazab, Ali Shahidinejad, Fariza Sabrina
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/7/2026, 5:44:23 AM
Summary
This paper introduces Dual-Part Multi-Lateral Branched Network (MLBNet), a deep learning architecture designed for simultaneous multi-class segmentation of guidewires, catheters, vessels, and background in cardiovascular catheterization angiograms. The model utilizes a dual-part design with lateral branching in both the encoder (for diverse feature extraction) and decoder (for class-specific learning) to handle class imbalance and structural overlap. It employs a hybrid loss function combining structured focal cross-entropy, Tversky loss, and specific constraints for tool containment and vessel boundaries to improve segmentation accuracy and stability.
Entities (9)
Relation Signals (8)
MLBNet → segments → Guidewire
confidence 95% · Results obtained showed the dual-part models could effectively separate guidewire... pixels to their classes
MLBNet → segments → Catheter
confidence 95% · Results obtained showed the dual-part models could effectively separate... catheter... pixels to their classes
MLBNet → segments → Vessels
confidence 95% · Results obtained showed the dual-part models could effectively separate... vessels... pixels to their classes
MLBNet → processes → Angiogram
confidence 94% · simultaneous multi-class segmentation in cardiovascular angiograms
MLBNet → appliedto → Cardiovascular Catheterization
confidence 93% · class-aware segmentation in cardiovascular catheterization scenes
MLBNet → usesbackbone → Xception
confidence 92% · The encoder part was implemented as a modified Xception-style backbone
MLBNet → employslossfunction → Tversky loss
confidence 90% · a branch-wise Tversky loss... is used improve overlap under imbalance
MLBNet → employslossfunction → Structured Focal Cross-Entropy Loss
confidence 90% · structured focal cross-entropy loss... is used to optimize each branch
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Catheterisation image processing requires segmentation models that are fast, accurate and explainable. While most of the existing studies usually focus on binary segmentation, there is a recent demand for simultaneous segmentation of multiple structures found in catheterization scenes. In this study, a dual-part MLBNet architecture is designed with multi-lateral encoder blocks and multi-head decoder branches for class-aware segmentation in cardiovascular catheterization scenes. Lateral branches in the encoder enables repeated feature extraction to learn diverse shared representations, while multiple decoder heads are used to introduce class-skewed branches that specialize in different structural properties in catheterization scenes. To analyze the performances of the dual-part MLBNet architecture, several multi-class segmentation angiogram data obtained during cardiovascular catheterization in phantom models, synthetic human-simulated aorta, and animal model are used for model training and evaluation. Results obtained showed the dual-part models could effectively separate guidewire, catheter, vessels and background pixels to their classes of memberships with high probability. The results demonstrate that all models were able to distinguish the dominant background class from foreground structures with high overall accuracy.
Tags
Links
- Source: https://arxiv.org/abs/2609.04590v1
- Canonical: https://arxiv.org/abs/2609.04590v1
Trouble viewing inline? Open PDF directly →
Full Text
41,684 characters extracted from source content.
Expand or collapse full text
DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 1 Abstract Catheterisation image processing requires segmentation models that are fast, accurate and explainable. While most of the existing studies usually focus on binary segmentation, there is a recent demand for simultaneous segmentation of multiple structures found in catheterization scenes. In this study, a dual-part MLBNet architecture is designed with multi-lateral encoder blocks and multi-head decoder branches for class-aware segmentation in cardiovascular catheterization scenes. Lateral branches in the encoder enables repeated feature extraction to learn diverse shared representations, while multiple decoder heads are used to introduce class-skewed branches that specialize in different structural properties in catheterization scenes. To analyze the performances of the dual-part MLBNet architecture, several multi-class segmentation angiogram data obtained during cardiovascular catheterization in phantom models, synthetic human-simulated aorta, and animal model are used for model training and evaluation. Results obtained showed the dual-part models could effectively separate guidewire, catheter, vessels and background pixels to their classes of memberships with high probability. The results demonstrate that all models were able to distinguish the dominant background class from foreground structures with high overall accuracy. Code is available on GitHub <URL> 1 1. Introduction Coronary artery disease, characterized by myocardial artery blockage, causes highest number of deaths amongst the cardiovascular diseases (CVDs) [1]. Traditional open- heart diagnosis and treatment require large incisions in patients’ cardiac region, causing patients to face numerous challenges. Alternatively, angiography, a less invasive procedure, that involves routine catheterization of patient’s vessels for diagnosis and treatment of vascular and cardiac conditions. Hence, patients experience reduced trauma and recovery time [2]. During angiography, X-ray imaging supports clinicians for intraoperative scene visualization, however they may be exposed to radiation depending on the 1 GitHub link is not added according to the conference review guideline. imaging quality and their experience in catheterization scene visualization and analytics during procedures [3]. Image segmentation techniques have been developed for processing angiographic images obtained during cardiac catheterization [4]. Cardiovascular image segmentation is challenging as angiography-based images are characterized by very low contrast, noise, motion artifacts, class overlaps in cluttered background, and severe class imbalance. The complexity of the tool and vessel structures combined with the imaging technology context pose additional challenges. Thus, fast and accurate delineation of the endovascular tool and vascular structures in interventional scenes remains a critical issue. A broad literature study on segmentation and tracking of blood vessels and endovascular tools in angiographic images has been published [5-7]. Domain overview shows segmentation of the different structures are done separately, and limited attention is found on unified multi-class segmentation of vessel, endovascular tools, and background for catheterisation scene parsing. Resolving this problem could enhance downstream tasks such as tool- vessel safety analytics in interventional cardiology. Use of classical segmentation approaches for automating visualization during catheterization focus on tracking the shape and motion of structures in angiograms. For instance, pixel-value thresholding, intensity-level analytics, or region clustering are commonly used for grayscale or RGB-based segmentation in cardiac X-ray angiograms [8]. An in-depth review of segmentation methods in this domain shows image sequences are pre-processed, and multi-scale filters are applied to segment vascular or tool structures in the angiograms [5]. To improve segmentation accuracy, some studies added image filters for preserving edges in the angiograms. Classical methods have been developed for binary class segmentation in cardiac angiograms [9]. While high segmentation performances were reported, classical techniques are commonly sensitive to blob-like non-target artifacts in the angiograms, and their performances are not generalizable [5-7, 9]. In some studies, target regions were manually defined to reduce the computational complexities involved [10]; however, this is not suitable when processing Dual Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms Olatunji Omisore 1,* , Ahmed Elazab 2 , Ali Shahidinejad 1 , Fariza Sabrina 1 1 Center for Machine Learning Networking and Education Technology, CQUniversity, Rockhampton, 4701, Australia. 2 Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China *Corresponding Author: o.omisore@cqu.edu.au DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 2 long or moving structures in catheterization angiograms. Deep learning (DL) techniques have been developed for segmentation of endovascular tools and vascular anatomies in catheterisation angiograms [11]. For instance, Wu et al. [12] addressed automatic guidewire-tip segmentation using convolutional neural networks (CNN) in 2D angiograms. Gherardini et al. improved catheter segmentation using synthetic data and transfer learning with lightweight U- Nets, highlighting the importance of data efficiency [13]. These studies provide strong approaches for cardiovascular image segmentation; however, they primarily focus on binary-class segmentation and single-tool extraction, typically involving guidewire or catheter. CNN-based DL architectures have provided state-of-the-art segmentation results when trained with fully supervised techniques. For model training, clinical experts are required to undergo manual largescale ground-truth annotation [10]. Segmentation of multiple endovascular tools, such as catheters and guidewires, in catheterization angiograms have also been attempted in previous studies [11-14]. While these studies showed the feasibility of automatic tool visualization through image segmentation, scope of the DL techniques were largely validated for binary segmentation of a single endovascular tool, typically either guidewire or catheter, and not simultaneous multi-class segmentation. Attention-based DL architectures have been explored to improve endovascular tool segmentation, visualisation, and tracking in fluoroscopic scenes. For instance, Zhou et al. [15] proposed an attention recurrent network for guidewire segmentation and tracking in intraoperative fluoroscopic videos, while Wang et al. [16] introduced a lightweight attention-based network for segmentation and localisation of tools during procedures. These methods incorporated attention mechanisms, recurrent modelling, and lightweight feature-fusion modules for tool delineation in low-contrast and class-imbalanced imaging conditions. The above studies focus either on background–vessel or background–tool binary segmentation. Thus, simultaneous segmentation of multiple structures in catheterization scenes remains underexplored. For multi-structure segmentation, Vlontzos et al. [17] extended binary class segmentation for multi-class learning. A UNet-based model was developed to produce background–catheter binary masks irrespective of the object categories in angiogram scenes. Meanwhile, catheter structure in the first frame gets transferred to later frames to delineate the blood vessel. Recently, Robertshaw et al. [18] implemented various DL models for three-class segmentation to aid multi-tool tip tracking in fluoroscopic images but without blood vessel segmentation. Bian [19] presented a multi-task learning model to segment vessels, catheters, and guidewire masks in fluoroscopic images . The model offered low segmentation performances while the guidewire class was invisible to the model. The research gaps identified motivate the development of a DL-based technique for simultaneous multi-class segmentation in cardiovascular angiograms. Built on the existing studies [9, 11], a dual-part multi-lateral branched network (MLBNet) is developed on previous MLBNet designs, and validated for segmentation of tool, vessel, and background structures in catheterisation angiogram scenes. A distinct step in this study is equipping the encoder and decoder sub-networks with multiple lateral branches for shared representation learning. The main contributions of this work are in threefold. First, a dual-part MLBNet with encoder–decoder lateral branching is proposed for simultaneous background–vessel–tool pixel-level segmentation. This addresses two main limitations of earlier works: i) sequential multi-head branching that could not ensure sufficient branch independence; and i) direct usage of shared decoder found suboptimal for handling geometrically similar structures coupled in angiograms. Second, a hybrid objective function with structure-specific loss terms in the decoder to prevent vulnerability to class dominance and imbalance issues in cardiovascular catheterization angiogram datasets. Third, comprehensive evaluation studies performed and reported to evaluate the dual-part MLBNet on multiple catheterization angiogram datasets in different anatomies. The remainder of this paper is organized as follows: the dual-part MLB-Net design is introduced in Section I, while the model implementation and training processes are presented in Section I, and experimental studies carried out for model evaluation are reported in Section IV. Lastly, conclusion of the study and future works are in Section V. 2. Multi-Class Segmentation Technique Segmentation of multiple angiogram scene structures is central to real-time procedural analytics in cardiovascular theranostics. This section presents the formulation of multi- class segmentation problem, while a new MLBNet design is proposed for pixel-level segmentation of background– vessel–tool structures in catheterization angiograms. 2.1. Problem Formulation Multi-class segmentation in catheterization angiograms is challenging since the scene structures are degraded by noise artifacts, low contrast, structural overlap, and severe class imbalance. Let 푥 ∈ℝ 퐻×푊×퐶 denote an angiogram with dimensions of 퐻 and 푊, and 퐶 is the number of input channels. The goal is to learn a mapping 푓 Θ :푥 →푌 � where 푌 � ∈[0,1] 퐻×푊×퐾 is a dense class-probability map over 퐾 procedure-specific scene structures with 퐾 classes, and 푌 ∈ 0,1 퐻×푊×퐾 is the corresponding pixel-level one-hot encoding ground-truth label. It is herein hypothesised that laterally-branched encoding and decoding neural blocks could support multi-class segmentation with improved and stable performances. This would enhance decoder learning independence for simultaneous multi-class segmentation. DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 3 Figure 1: Proposed Dual-Part Multi-Lateral Branched Network for Multi-Class Endovascular Scene Segmentation 2.2. Dual-Part MLBNet Design The dual-part MLBNet design is presented in Fig. 1, with the encoder–decoder lateral branching concept explained. 2.2.1 Encoder Sub-Network Lateral branching in the encoder involves using a multi- block sub-network built on an Xception backbone [11]. The input angiograms are initially processed through a shallow stem block, denoted 퐵 1 in Fig. 1, and the features obtained are passed through consecutive laterally branched segments in the encoder. The stem block executes a serial function: 퐹 1 =휙 stem (푥), where 휙 stem is composed of convolution, batch normalization and ReLU activation, to obtain low- level spatial features from an input angiogram scene. A. Lateral Encoder Branching Each lateral branch in the encoder is defined with three parallel convolutional paths to obtain diverse feature maps. The parallel branches are used for complementary feature learning and representations based on input feature maps from the images received. Depthwise convolutions are applied to preserve local edge and line responses crucial for thin elongated structures. The parallel branches have unique outputs (푈 ℓ ( 푖 ) ) defined in Eq. 1, and each branch is followed by batch normalization and ReLU activation. However, the middle branch is extended with separable convolution to improve the feature diversity, while max-pooling operation is also performed to reduce feature dimensionalities. Where 퐹 ℓ is an input feature map to the encoder block ℓ,휙 ℓ ( 푖=1 ) is a 1 ×1 projection branch, 휙 ℓ ( 푖=2 ) is the main separable- convolution branch, and 휙 ℓ ( 푖=3 ) is either a 3 ×3 convolution or an identity branch depending on the block. 푈 ℓ ( 푖 ) =휙 ℓ ( 푖 ) ( 퐹 ℓ ) (1) A fused output �퐹 � ℓ =ℱ ℓ �푈 ℓ ( 1 ) , 푈 ℓ ( 2 ) , 푈 ℓ ( 3 ) � is obtained through channel concatenation followed by projection or additive residual fusion, when dimensions align, and the residual update (Eq. 2) is performed, where 휓 ℓ (⋅)is an identity or projection mapping used to match dimensions. In addition, atrous separable convolutions with adaptive dilation rates are used to obtain contextual multi-scale features and reduce network parameters [11]. 퐹 ℓ+ 1 =퐹 � ℓ +휓 ℓ (퐹 ℓ ) (2) B. Multi-flow Structure The encoder blocks are organized into three connected fragments organized as entry flow, middle flow, and exit flow. The entry flow corresponds to blocks 퐵 2 –퐵 4 and it is used to capture low-level and intermediate contextual cues for descriptive feature representations. Dilation and field- of-view (FoV) expansion are applied to maintain consistent output dimensions across branches. The receptive-field growth per block is obtained with a scaling factor (훼). For progressive context aggregation, 훼>1 is used. The middle flow, specifically involving block 퐵 5 is used to encode class-specific features. It has similar operations to 퐵 6 , excluding max-pooling and stride, which are repeated 16 times to learn discriminative features and improve feature- map diversity during training. The exit flow, corresponding to 퐵 6 –퐵 7 , is applied to refine feature representation using pointwise and atrous convolutions. The residual features from 퐵 3 , 퐵 5 , and 퐵 7 are aggregated as 퐿 3 , 퐿 4 , and 퐿 5 , and bilinearly upsampled (in Eq. 3) to support multi-scale deep supervision in the decoder. This multi-scale fused feature map is then passed to the decoder sub-network. 퐹 agg =� 풰 푠 푠∈ 퐵3,퐵5,퐵7 ( 퐹 푠 ) , (3) where 푠 ∈ 퐹 퐵3 ,퐹 퐵5 ,퐹 퐵7 denote the skip features, and 풰 푠 (⋅) denotes bilinear upsampling and channel alignment. 2.2.2 Decoder Sub-network In cardiovascular catheterization, tool pixels are usually contained within or along a vessel structure, while vessel forms a continuous tree-like structure in the catheterization scenes. Hence, multi-class structure pixels overlap, and the use of single shared segmentation head may not provide learning stability for such class interactions. Rather than DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 4 relying on a single shared softmax head or sequential heads [9, 11, 19] in prior studies, the decoder sub-network in this study extends with class-specific lateral decoder branching. This design is motivated since target structure classes are geometrically coupled. The explicit decoder independence is aimed as a specialized class-aware learning mechanism. For context, the class-specific lateral branching works by having a multi-head architecture where one decoder branch is oriented towards tool class, another towards vessel class, and the last branch for scene background region external to foreground classes. This lateral decoder design preserves class-aware structure segmentation and prevent redundant heads in the earlier studies [9, 11, 19]. To learn meaningful scene representation, the class-specific head to each decoder outputs 푍 ( 푐 ) =휙 푑 (퐹 agg ) for class ( 푐 ) , where 휙 푑 is a scene- wide decoder for the K-class segmentation task. The final segmentation output is obtained through a late fusion of all decoder results, as given in Eq. 4, 휎(⋅) is a sigmoid function for the binary specialist branches, 푊 푓 ( ⋅ ) are learnable fusion weights and Π 푏 ,Π 푣 ,Π 푡 are projection operators that map the branch-specific outputs for background, vessel and tool pixels into a global class space. 푌 � =휎 � 푊 푓 ( 푡 ) Π 푔 (푍 ( 푡 ) ) +푊 푓 ( 푣 ) Π 푣 (푍 ( 푣 ) ) +푊 푓 ( 푏 ) Π 푏 (푍 ( 푏 ) ) � (4) This strategy allows the network to combine global scene context with class-specific structure refinements while also preserving the multi-lateral branching principles. 2.2.3 Class-Aware Scene Objective Function As standard cross-entropy function could be vulnerable to background dominance and class imbalance, we follow the idea of using hybrid loss function and propose a class- aware scene (CAS) objective function. Let the three lateral decoder branches produce class-specific probability maps 푃 ( 푏 ) , 푃 ( 푣 ) ,푃 ( 푡 ) ∈[0,1] 퐻×푊 for background, vessel and tool pixels in an angiogram scene, respectively. If the complete multiclass one-hot ground truth is denoted by 푌, the fused main decoder can be said to output a multi-class scene prediction 푌 � ∈[0, 1] 퐻×푊×3 where 푌 ( 푏 ) , 푌 ( 푣 ) , 푌 ( 푡 ) are the corresponding one-vs-rest binary targets. To preserve class discrimination, the CAS function transforms the MLBNet approach from generic segmentation network into a structured class-aware design. In addition to categorical cross-entropy, multi-class focal loss was optimized with connectivity-sensitive supervision. In angiographic scene analysis, endovascular tools, vessels, and background are both geometrically coupled rather than independent semantic classes. To enable class-specific learning in the MLBNet, CAS operates loss components for tool, vessel, and background classes in an angiogram scene. For branch 푐 ∈푡,푣,푏, structured focal cross-entropy loss, defined in Eq. 5, is used to optimize each branch as a one- vs-rest classifier with structured non-target supervision, thus all pixels constrain the branch, while non-target classes are adaptively weighted. This retains the discriminative strength of cross-entropy and reweighs the low frequency target classes against competing foreground and dominant background classes. Then a branch-wise Tversky loss, given in Eq. (6), is used improve overlap under imbalance. ℒ SFCE ( 푐 ) =− 1 푁 �훼 푐 푌 푖 ( 푐 ) (1−푃 푖 ( 푐 ) ) 훾 log푃 푖 ( 푐 ) 푖 +�훽 푐,푘 푘≠푐 푌 푖 ( 푘 ) (푃 푖 ( 푐 ) ) 훾 log(1−푃 푖 ( 푐 ) )� (5) ℒ Tv ( 푐 ) =1− 푇푃 푐 +휖 푇푃 푐 +훿 푐 퐹푁 푐 +(1−훿 푐 )퐹푃 푐 +휖 (6) where 훼 푐 controls the positive-class importance, and 훽 푐,푘 controls the penalty from each non-target class 푘. Also, 훾 is a focal focusing factor, and 훿 푐 controls the penalty trade-off between the false negatives and false positives for class 푐. A. Tool Branch Loss Term As the decoder design focuses on task-specific learning, one of the decoder branches is optimized to supervise tool related terms using tool Dice and skeleton-consistency functions. The tool branch learns intravascular plausibility of guidewire or catheter in an angiogram scene using Eq. 7. It defines skeleton-consistency terms �ℒ skel ( 푡 ) � in Eq. 8 that accounts for structure thinness, connectivity and centerline, and structural containment �ℒ in ( 푡 ) � in Eq. 9. The latter helps to account for catheterization scene parsing knowledge that tool prediction must lie within the vessel support. Thus, tool predictions outside the vessel support can be penalized. ℒ 푡 =ℒ SFCE ( 푡 ) +휆 푡1 ℒ Tv ( 푡 ) +휆 푡2 ℒ skel ( 푡 ) +휆 푡3 ℒ in ( 푡 ) (7) ℒ skel ( 푡 ) =1− 2�Skel( 푖 푃 푖 ( 푡 ) ) Skel(푌 푖 ( 푡 ) ) +휖 �Skel( 푖 푃 푖 ( 푡 ) ) +�Skel( 푖 푌 푖 ( 푡 ) ) +휖 (8) ℒ in ( 푡 ) = 1 푁 �max 푖 � 0, 푃 푖 ( 푣 ) −푌 푖 ( 푡 ) � (9) To ensure non-equivalent supervision, background can be treated as a stronger negative than vessel in this branch, that is, 훽 푡,푏 >훽 푡,푣 , because vessel pixels are anatomically related to the tool and should not be penalized in the same way as background. B. Vessel Branch Loss Term Similarly, a combined Dice-IoU loss term is applied with boundary-aware constraint on another decoder branch to understand vessel structures. The vessel branch is designed to emphasize anatomical region coherence, tool boundary and support continuity using the loss terms in Eq. 10. ℒ bd 푣 defined in Eq. 11, is a boundary-support term set to bound tool pixels within the vessel pixels, and ∇ denotes a spatial gradient operator. DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 5 ℒ 푣 =ℒ SFCE ( 푣 ) +휆 푣1 ℒ Tv ( 푣 ) +휆 푣2 ℒ bd ( 푣 ) (10) ℒ bd 푣 = 1 푁 �∥∇푃 푖 ( 푣 ) −∇푌 푖 ( 푣 ) ∥ 1 푖 (11) Again, background is the strongest negative, while tool acts as a structured internal competitor: 훽 푣,푏 >훽 푣,푔 . C. Background Branch Loss Term The background branch is designed to be discriminative without overwhelming the other branches, as given in Eq. 12. To reduce background dominance, its positive weight is intentionally moderated:훼 푏 <훼 푣 , 훼 푡 . Similarly, vessel is penalized more strongly than tool in the background branch since the vessel would occupy larger contiguous foreground support:훽 푣,푏 >훽 푡,푏 . This design ensures that each branch is fully constrained by all pixel classes, but target and non- target classes are not treated equivalently. ℒ 푏 =ℒ SFCE ( 푏 ) +휆 푏1 ℒ Tv ( 푏 ) (12) D. Fused Multiclass Scene Supervision For the fused scene head should remain the dominant segmentation objective, a multiclass scene supervision is used as a unified learning strategy where the scene branch is trained against a multi-class segmentation label map that jointly represents background, vessel, and tool classes. This allows the model to learn global scene context and inter- class relationships under one fused semantic objective. The unified supervision is optimized with class-balanced cross- entropy ( ℒ CBCE ) and Dice functions for multi-class supervision, as given in Eq. 14. The right-most term is used to ensure containment prior to preserve strong multi-class discrimination where standard categorical cross-entropy is vulnerable. ℒ 푚 =ℒ CBCE �푌,푌 � �+휆 푚1 ℒ Dice mc �푌,푌 � � + 휆 푚2 1 푁 �max(0, 푖 푌 � 푖,푔 −푌 � 푖,푣 ) (13) E. Total Loss The overall training loss function is therefore a weighted sum of the loss terms, as given in Eq. 14, where 휆 ⋅ are scalar hyperparameters. This function involves adaptive task balancing to reduce gradient interference across the four objectives ℒ 푡 ,ℒ 푣 ,ℒ 푏 ,ℒ 푚 using learnable uncertainty-based weights. 휎 푞 is a learnable scalar that allows the network to rebalance branch losses during training for task 푞. The optimal hyperparameter values can be determined through a grid search or ablation studies. ℒ total =� � 1 2휎 푞 2 ℒ 푞 +log휎 푞 � 푞∈푡,푣,푏,푚 (14) 3. Network Implementation and Training 3.1. Network Implementation The proposed dual-part MLBNet design is implemented with Tensorflow® Keras® and validated on catheterization datasets. The multi-lateral learning concept is applied to both the encoder and decoder sub-modules, as described in Section I. The encoder part was implemented as a modified Xception-style backbone with the repeated lateral branches fused for complementary representation learning. For decoder part, class-specific independent learning is approached through shared upper decoder trunk for class- aware segmentation. In a three-branch decoder design, the first branch could be used for tool segmentation, the second branch for vessel segmentation, and the third branch for the background. All branch heads receive shared feature maps from the encoder blocks. The latter branch heads are built on earlier decoder layers in addition to convolution and dropout layers. Thus, different feature maps are received by the three decoders, and this helps to prevent class-specific model overfitting. 3.2. Network Training Protocol The proposed MLBNet model was trained for pixel-level segmentation in three catheterization datasets. Each dataset was randomly partitioned for model training and testing. The images were resized to 256 × 256 pixels for network input. Adam optimizer with parameters: 훽 1 = 0.9, 훽 2 = 0.999, and 휀=10 −6 were applied during training. Based on preliminary studies, an initial learning rate of 10 −4 was used as it produced stable validation behaviour across the datasets. The learning rate dynamically adjusted using drop and decay factors of 1 and 0.95, respectively, to improve training convergence. Gradient clipping with clipnorm = 1.0 was explored for improved optimization. During model fitting, training and validation data splits were generated at 80% to 20%, respectively. These were processed through a batch-based angiogram loader with one-hot label encoding applied for all classes. The CAS loss terms designed in Section I was applied to address limitations of standard categorical cross-entropy under strong class imbalance and background dominance. Thus, branch-specific losses were assigned to the tool, vessel, and background branch heads. The network was trained multiple times with a batch size of 16 and 50 epochs on High-Performance Computing facility with NVIDIA A100 GPU cards and 48 GB memory. The training process involved active callbacks with checkpoint set on validation-loss for learning-rate reduction and early stopping. Optimally tuned multi-head weights: 0.15, 0.2, 1.0 were used for the three decoder heads, respectively. This reduced gradient interference and yielded the best results. While the weights only favoured Dataset 1-2, weights: 0.75, 1.25, 1.0 stabilized the performance in Dataset 3. Guided by F1-score, and mean Intersection over DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 6 Union (mIoU) metrics, model giving the best segmentation quality was selected for experimental and ablation studies. The best performing models obtained after training on each dataset were evaluated with the corresponding test sets. 3.3. Datasets and Preprocessing The model was validated with three angiogram datasets obtained during different catheterization procedures. These includes both in-vitro and in-vivo datasets described herein. The in-vitro datasets consist of proprietary camera-based 857 and 874 images of catheterization scenes saved during manual and robot-assisted tool navigation (Dataset 1a-b) in a silicone phantom, respectively. Each image frame has 640 × 480 pixels, and multi-class annotation was manually done with LabelMe 6.3.1. Secondly, a public dataset with frames extracted from fluoroscopy video (Dataset 2) was also used in this study. The dataset consists of 2,000 angiograms, each with 256 × 256 pixels, recorded in silicone-based aortic phantom [13]. The dataset only has tool masks in its original form; thus, extra step was taken to add vessel masks to the catheterization scenes. This was achieved with custom code written to propagate a self-created vessel mask done for first frame to the later frames. Lastly, CathAction dataset, largest public fluoroscopic dataset with angiograms recorded during catheterization in silicone-based phantom (Dataset 3a) and animal experiments (Dataset 3b) was also used in this study [20]. Unlike the previous datasets. The animal setup was made identical to human procedures with professional surgeons cannulating the left subclavian, left common carotid, and right common carotid vessels with commercial catheter and guidewire. The tool pixels were manually annotated separately in this dataset, thus could support multi-class segmentation tasks. The masks for first two datasets (1a, 1b, and 2) include background–vessel–tool pixels, thus multi-class annotation is achieved as background masks encoded by "0", vessel masks by “1”, and tool masks by “2”. For the CathAction dataset that rather has background–catheter–guidewire masks, background masks were encoded by "0", catheter masks by “1” and guidewire masks by “2”. The three datasets were pre-processed, used to train and validate the dual-part MLBNet model separately. In addition, data- agnostic tests were done by evaluating the performance of a model trained with one data on others without re-training. 4. Experimental Studies and Evaluations The dataset details and the segmentation results obtained with the dual-part MLBNet are presented. 4.1. Model Evaluation and Analysis To assess the model’s segmentation quality, both full- scene and foreground-sensitive metrics were selected to evaluate the model. These include accuracy, F-score, and mIoU. Background-excluded versions of the metrics were considered as overall performance could misinform due to the background pixel dominance in catheterization scenes. Fig. 2 shows the segmentation outputs from the dual-part MLBNet for a random frame in the test set of each dataset. As shown in third row of Fig. 2, the model could accurately delineate and track three-class structures in all the datasets, achieving high overall classification performance across all the test sets with higher macro F1 for Dataset 1 (96.94%), Dataset 2 (97.28%) and lower macro F1 (66.32%, 77.29%) for CathAction phantom and animal test sets in Datasets 3. Across all test sets, the models got a strong segmentation experience for the background class, with F1-score range of 99.67–99.90%. This shows a huge class imbalance skewing towards background, and bias to interpret the model using the accuracy values in Table I. However, the other metrics show the dual-part MLBNet architecture could also localize the foreground structures. The confusion matrices in Fig. 3 explains that the model achieved a lesser foreground performance in background–catheter–guidewire segmentation task compared to when the model is applied for the background–vessel–tool task. The foreground structure analyses show high error rates, as explained with precision (50.89%, 35.22%) and recall (64.58%, 46.45%) for Dataset 3, with Phantom and Animal angiograms, respectively. Interpreting this could mean the dual-part MLBNet approach struggles to correctly classify catheter and guidewire, tools with similar appearances, and confused them with one another or even the background. Figure 2: Segmentation results for selected frames in camera data Dataset 1 (a) Dataset 1(b) Dataset 2 Dataset 3(a) Dataset 3(b) Original Image Groundtruth Dual - part MLBNet MLBNet JSR - UNet DeepLabV3+ FAR - Net DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 7 TABLE I. MODELS’ PERFORMANCES ACROSS THE DATASETS (%) Test set Acc (%) F1-S (%) mIOU (%) for Case #1 #2 #3 #4 #5 Data 1a 99.30 96.94 94.21 91.15 89.78 94.21 39.26 Data 1b 99.27 96.84 94.03 89.55 89.83 94.03 86.69 Data 2 99.20 97.28 94.86 94.89 93.64 80.66 23.86 Data 3a 99.51 66.32 54.12 49.76 50.40 17.32 8.56 Data 3b 99.30 77.29 66.37 61.81 62.53 16.48 17.47 Figure 3: Performance analysis of models’ for four datasets 4.2. Analyses of Models’ Domain Adaptation The dual-part MLBNet architecture’s ability for domain adaptation was evaluated for models built on the different datasets. For this experiment, the model built on training of dataset was applied for multi-class segmentation on the test set of others. Cross-domain performances were analyzed using tool micro-average precision-recall curves (AUPRC) in results in Fig. 4. The curves show model transferability between visually similar domains, for instance the camera- based phantom Datasets 1, tool AUPRC = 0.863. However, the transferability degraded in cross-domain heterogeneous evaluation such as from animal and phantom sets, or vice- versa, in Dataset 3. Thus, dual-part MLBNet generalises well across visually similar domains but poorly in the presence of domain shifts caused by differences in vessel visibility, tool contrast, anatomical structure, and imaging. 4.3. Ablation Studies Studies were performed to analyze the decoders’ multi- head design and the learning functions. Figure 4: Segmentation results for selected frames in camera data A. Multi-head and Branch-Specific Loss Contributions The default multi-head setup and the branch specific loss terms in the Eq. 14 were used was taken as Case #1 with the mIOU values presented in Table I. Case #2 is a follow- up ablation conducted to determine how the proposed branch specific objective function contributes to the models’ segmentation performances. For this, the CAS objective function (in Eq. 14) was substituted with standard categorical cross-entropy. The results obtained showed that the branch specific three-decoder setup achieved better and more stable segmentation performances for four datasets, while its performance for Dataset 2 was also closer to when the standard categorical cross-entropy was used. Albeit the proposed branch specific objective function achieved the best and a more stable segmentation performance. We also checked if the models’ improved performance is majorly from the multi-decoder architecture. In an ablation, the multi-head architecture was set to use one decoder and the standard categorical cross-entropy. Compared with the outcomes for Cases #1 and Case #2, we found that the single-decoder configuration variant showed a less stable foreground segmentation performance. While the one- decoder configuration also achieved high performance on the Datasets 1-2, (i.e. the simpler silicone-based data), it showed a clear degradation in the foreground segmentation performances on the more challenging Dataset #3. These studies indicate that the proposed multi-head architecture and the branch specific loss terms complementary contributes to the models’ robust class-specific learning. B. Analyzing Segmentation Outputs from Different Heads While segmentation with multi-head branching could be more robust than single decoder architecture, differences in the decoder branches perturbations may cause performance variations across the decoder heads. To determine the best performing branch, the pixel-level predictions were done with the vessel-head and tool-head branches, as reported in Case #4 and Case #5 of Table I. respectively. Note that the branches in Datasets 3 were implemented to use catheter- head and guidewire-head or the respective legacy names (i.e., vessel-head and tool-head branches). Performing the pixel-level predictions with the scene-head branch gave the best performer across the datasets (Case #1), and this is followed by using the tool-head branch. The performances from Case #4 are similar to Case #1 while using the vessel- tool achieved the least stable performances. 4.4. Performances for Different DL Architectures Existing deep learning architectures used for binary tool (a) Homogenous Setting: Dataset 1a vs. Dataset 1b (c) Heterogenous Setting: Dataset 1b vs. Dataset 2 (c) Heterogenous Setting: Dataset 2 vs. Dataset 1a (b) Heterogenous Setting: Dataset 3a vs. Dataset 3b DICTA #**** Accepted for Presentation at the 2026 DICTA, Sydney Australia DICTA #**** 8 segmentation in previous study [11] were evaluated and their performances compared with that of the proposed dual-part MLBNet. The segmentation performances that were obtained for these models are presented in Table I. First, the models’ segmentation outputs displayed in Fig. 2 (Rows 3 – 7) show dual-part MLBNet models are robust for scene-level angiogram segmentation. As presented in Table I, the proposed architecture has best metrics in all architectures compared with. Tool (precision) error is the proportion of false tool prediction (catheter and guidewire are combined in Dataset 3) divided by the total predicted tool pixels in a test set. The dual-part MLBNet has the lowest error values, followed by JSR-UNet and the earlier MLBNet models. This informs the models’ ability for tool detection and in separating the foreground structures with respect to non-tool classes. TABLE I. COMPARISON OF DL ARCHITECTURES (METRICS IN %) Architecture Dataset 1a Dataset 2 Dataset 3b mIoU Tool Error mIoU Tool Error mIoU Tool Error Proposed 94.21 5.80 94.86 7.69 66.37 24.08 MLBNet 89.78 14.81 93.64 10.94 62.53 33.29 JSR-UNet 92.76 10.30 78.93 11.29 63.60 31.77 FAR-Net 91.90 10.91 90.40 12.83 60.71 34.66 DeepLabV3+ 80.68 23.23 93.27 11.05 62.25 33.25 5. Conclusion and Future Works In this study, a dual-part MLBNet architecture was proposed and validated for multi-structure segmentation in catheterization scenes. Models supporting different class- aware segmentation were evaluated on different datasets to aid intuitive scene parsing in cardiovascular catheterisation procedures. Compared with earlier methods, the proposed architecture integrates shared encoder with partially shared upper decoder trunk, and branch-specific decoder blocks. Evaluation results from both in-vitro and in-vivo datasets showed stable segmentation performances, and feasibility for cross-domain segmentation of multiple structure in the different catheterisation datasets was also demonstrated. Additional runs of the model would be useful to examine the average performances of the model, while integrating loss consistencies and alignment across decoder branches could offer more stable and generalized performances. In addition, there is need to consider catheterization scenes with other endovascular tools, such as stents and balloons, that are found in clinical practice. In addition, the limited transparency of deep learning systems remains a key concern in clinical and robot-assisted intervention setups. Thus, future AI-based segmentation models should not only be fast and accurate but also provide explainable outputs to support clinicians, offer procedural awareness, and enhance surgeon–robot collaborative interventions. Acknowledgment: AI-assistance was used to support model implementation, code debugging and drafting. These were reviewed, modified, and validated by the authors. References [1] G. Roth, et al. “Global, Regional, and National Burden of Cardiovascular Diseaes and Risk Factors in 204 Countries and Territories, 1990-2023”, JACC. 86(22):2167–243, 2025. [2] O. Omisore, et al., Towards Characterization and Adaptive Compensation of Backlash in a Novel Robotic Catheter System for Cardiovascular Interventions, IEEE Transactions on Biomedical Circuits and Systems, 12(4): p. 824-38, 2018. [3] D. Jesudason, et al. "Novel No-Lo imaging techniques to minimize intraoperative radiation exposure in vascular and endovascular surgery", J. Vasc Surg, 83(5 ):1423-1434, 2026. [4] M. Ghallab, et al. “Assessing and improving radiation safety in cardiac catheterization: a study from Cairo University Hospital”. Egypt Heart Journal, 76(17), 2024. [5] S. Moccia, et al. “Blood vessel segmentation algorithms - Review of methods, datasets and evaluation metrics”, Computer Methods & Programs Biomed, 158:71-91, 2018. [6] A. Ramadani, et al. “A survey of catheter tracking concepts and methodologies”, Medical Image Anal, 82:102584, 2022. [7] Y. Ning and J. Tong, “A Review on Vessel Segmentation of X-Ray Coronary Angiography Images Based on Deep Learning”, Int. Conf. Mach. Learn. & Mach. Intellig, 2021. [8] O. Omisore, et al. "Weakly-Supervised Learning via Multi- Lateral Decoder Branching for Tool Segmentation in Robot- Assisted Cardiovascular Catheterization", IEEE ICRA, 2025. [9] H. Fazlali, et al. “Vessel segmentation and catheter detection in X-ray angiograms using superpixels. Medical & Biology Engineering and Computing, 56:1515–1530, 2018. [10] O. Omisore, et al. “Automatic tool segmentation and tracking during robotic intravascular catheterization for cardiac interventions”, Quant. Imag.. Med. & Surg, 2021, 11(6). [11] O. Omisore et al. “Multi-lateral Branched Network for Endovascular Tool Segmentation during Robot-assisted Catheterization”, IEEE Trans Med Rob & Bion. 6(2), 2024. [12] Y. Wu et al., “Automatic Guidewire Tip Segmentation in 2D X-ray Fluoroscopy Using Convolution Neural Networks”, International Joint Conference on Neural Networks, 2018. [13] M. Gherardini, et al., “Catheter segmentation in X-ray fluoroscopy using synthetic data and transfer learning with light U-Nets”, Comp. Meth. & Prog. Biomed, 192, 2020. [14] P. Ambrosini et al. “Fully Automatic and Real-Time Catheter Segmentation in X-Ray Fluoroscopy”, MICCAI 2017. [15] Y. Zhou, et al., “Pyramid attention recurrent networks for real-time guidewire segmentation and tracking in intraopera- tive X-ray fluoroscopy”, Comp Med Imag & Grap, 83, 2020. [16] H. Wang, et al., “Lightweight attention network for guide- wire segmentation and localization in clinical fluoroscopic images of vascular interventional surgery”, QIMS, 15:5, 2025 [17] A. Vlontzos and K. Mikolajczyk, “Deep Segmentation and Registration in X-Ray Angiography Video,” BMVC, 2018. [18] H. Robertshaw, et al., “Toward real-time autonomous navigation: transformer-based catheter tip tracking in fluoroscopy”, Int. Journal of Comp Ass Rad & Surg, 2026. [19] Z. Bian, “Multitask segmentation of vessels, catheters, and guidewires in medical imaging using a YOLOP-UNet hybrid architecture”, CRAE, June 27-29, 2025. [20] H. Baoru, et al. “CathAction: A Benchmark for Endovascular Intervention Understanding”, arXiv:2408.13126, 2024.