Paper deep dive
Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures
Simone Garbin, Leonardo Venturoso, Marco Todescato
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/27/2026, 5:21:57 AM
Summary
This study evaluates automatic weld seam segmentation for industrial quality control, comparing RGB and polarimetric imaging across CNN and transformer architectures. It finds that while CNNs perform well in controlled environments, they fail under uncontrolled conditions and viewpoint shifts. Polarimetric imaging with geometric augmentation achieves high accuracy (mAP50 up to 0.93) on uncontrolled data. Transformers, particularly RF-DETR, demonstrate superior robustness to viewpoint changes compared to CNNs, making them preferable for variable deployment scenarios despite higher computational costs.
Entities (10)
Relation Signals (6)
RF-DETR-Seg → belongsto → Transformer
confidence 98% · transformer-based architectures (RF-DETR-Seg-Seg...)
YOLOv11 → belongsto → CNN
confidence 98% · Convolutional neural network (CNN) architectures (YOLOv8/YOLOv11 segmentation variants)
Fraunhofer Italia Research → authored → Automatic weld seam segmentation for industrial quality control
confidence 95% · Affiliation: Fraunhofer Italia Research... title: Automatic weld seam segmentation for industrial quality control
CNN → suffersfrom → Performance Drop
confidence 92% · drop to 0.22-0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component
RF-DETR-Seg → outperforms → CNN
confidence 90% · under a test-time viewpoint shift, the transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses.
Polarimetric Imaging → enables → Robust Segmentation
confidence 88% · Polarimetric imaging with alignment-preserving geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93... on uncontrolled RGB without requiring acquisition control.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Visual inspection of welded assemblies remains one of the least automated stages in many industrial production processes, still depending largely on the experience of human operators and thus subject to inter-operator variability; the manufacturing of special-purpose machinery cabins, the setting of this study, is one representative case. This work evaluates the feasibility of automatic weld seam segmentation from RGB and polarimetric imagery, comparing controlled laboratory acquisitions with images captured under real, uncontrolled conditions. Convolutional neural network (CNN) architectures and transformer-based architectures are benchmarked under a unified, threshold-independent protocol, training each CNN with three random seeds to separate genuine effects from seed noise. In controlled RGB conditions, CNN models reach a mean mask mAP50 of up to 0.87, but drop to 0.22-0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component of the inspection system. Polarimetric imaging with alignment-preserving geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93: on par with, rather than ahead of, the best controlled-RGB result, but reaching that accuracy on uncontrolled RGB without requiring acquisition control. The clearest architectural finding concerns viewpoint robustness. In-distribution, transformers and CNNs are broadly comparable; but under a test-time viewpoint shift, the transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses. The gap holds across three seeds and a resolution-matched control, pointing to architecture rather than training resolution. Within the CNN family, capacity brings no reliable in-distribution gain once seed variance is accounted for: small CNNs suffice for fixed viewpoints, transformers for variable ones.
Tags
Links
- Source: https://arxiv.org/abs/2608.25465v1
- Canonical: https://arxiv.org/abs/2608.25465v1
Trouble viewing inline? Open PDF directly →
Full Text
66,107 characters extracted from source content.
Expand or collapse full text
Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures Simone Garbin Leonardo Venturoso †thanks: Corresponding author: leonardo.venturoso@fraunhofer.it Marco Todescato Affiliation: Fraunhofer Italia Research, Via A. Volta 13/A, 39100 Bolzano, Italy Affiliation: simone.garbin, leonardo.venturoso, marco.todescato@fraunhofer.it Abstract Visual inspection of welded assemblies remains one of the least automated stages in many industrial production processes, still depending largely on the experience of human operators and thus subject to inter-operator variability; the manufacturing of special-purpose machinery cabins, the setting of this study, is one representative case. This work evaluates the feasibility of automatic weld seam segmentation from RGB and polarimetric imagery, comparing controlled laboratory acquisitions with images captured under real, uncontrolled conditions. Convolutional neural network (CNN) architectures (YOLOv8/YOLOv11 segmentation variants) and transformer-based architectures (RF-DETR-Seg, Mask2Former) are benchmarked under a unified, threshold-independent protocol, training each CNN with three random seeds to separate genuine effects from seed noise. In controlled RGB conditions, CNN models reach a mean mask mAP50 (averaged over three training seeds) of up to 0.87, but drop to 0.22–0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component of the inspection system. Polarimetric imaging with alignment-preserving geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93: on par with, rather than ahead of, the best controlled-RGB result, but reaching that accuracy on uncontrolled RGB without requiring acquisition control. The clearest architectural finding concerns robustness to viewpoint change. In-distribution, transformers and CNNs are broadly comparable; but when the camera viewpoint shifts at test time (zero-shot transfer to a close-range, robot-relevant viewpoint), the transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses. This gap holds across three training seeds and survives a resolution-matched control, indicating it reflects the architecture rather than training resolution. Within the CNN family, in contrast, model capacity gives no reliable in-distribution advantage once seed variance is accounted for. Small, fast CNNs are therefore a safe default when the deployment viewpoint matches training, whereas transformer-based models deserve consideration wherever that viewpoint may vary. 1 Introduction Quality control of welded assemblies remains one of the least automated stages in the manufacturing process [10]. Post-weld inspection is typically performed visually by specialized operators, a process that is time-consuming, inconsistent, and sensitive to environmental conditions [24, 1]. Inaccurate quality monitoring compromises product integrity, driving up production costs and reducing customer satisfaction [31]. Locating weld seams in workpiece imagery is the critical first step; automating this process is essential for downstream defect detection and automated robotic inspection [15]. From a computer vision standpoint, welded metallic surfaces are a challenging domain. Brushed and galvanized steel can produce strong specular reflections that complicate weld imaging; welds can also present irregular and variable geometry; and real shop-floor imagery adds background disturbance, reflections, shadows, blur, and viewpoint variation [35, 38]. These factors degrade the performance of deep learning segmentation models in industrial inspection, especially under small annotated datasets, since collecting and labeling industrial images is costly and time-consuming [32]. The goal of this work is to assess the feasibility of automatic weld seam instance segmentation across acquisition scenarios and imaging modalities, and to identify the factors that dominate performance in practice. Specifically, we compare: (i) RGB images acquired in a controlled laboratory setup versus handheld acquisition in an uncontrolled shop-floor environment; (i) conventional RGB imaging versus polarimetric imaging, from which six per-pixel maps (Intensity, AoLP, DoLP, ImaxI_ , IminI_ , Specular) are derived; (i) segmentation architectures spanning two families (single-stage CNNs and transformers) and more than an order of magnitude in parameter count; and (iv) training factors including input resolution, weight initialization, optimizer, and data augmentation. All experiments follow a unified protocol with data partitions defined at the level of the physically acquired weld sample. 2 Literature review Literature on automated weld inspection spans five core themes: seam segmentation, architecture selection, polarization imaging, small-data learning, and deployment efficiency. While lightweight vision pipelines dominate practical implementations, evidence for polarimetric weld segmentation remains sparse, drawing largely on general reflective-surface inspection [39, 6, 25]. 2.1 Vision-based weld inspection Although vision-based weld inspection has progressed from classical edge- and morphology-based pipelines toward learned segmentation, classical methods persist in robotic welding for their interpretability and ease of closed-loop integration [27]. Manual inspection remains common in robotic quality control due to subtle defect appearances, long training times, and variable inspection standards [25, 27]. Recent deep learning research focuses on pixel-level seam segmentation capable of handling arc light, spatter, and weak boundaries [39]. A prominent example is CvT-UNet, which uses a U-shaped convolution-transformer encoder–decoder to reach mean IoUs up to 93.75% across varied environments [39]. DSNet targets the real-time regime, reporting 78.01% IoU, 87.64% Dice (the overlap-based F1 score on the predicted masks) and 100 FPS [2]. 2.2 Segmentation architectures Instance segmentation has historically been dominated by detection-then-segmentation designs such as Mask R-CNN [8], while newer query-based methods enable direct end-to-end prediction. Mask2Former serves as a primary baseline, framing semantic, instance, and panoptic segmentation through a shared masked-attention decoder [4], though it relies on heavy pixel decoders and has not established clear superiority on real-time benchmarks [7]. FastInst mitigates this computational bottleneck with lighter decoders, reporting 32.5 FPS and 40.5 AP on COCO [7], while DN-DETR accelerates convergence in DETR-style networks via query denoising [17]. For weld inspection, however, YOLO variants remain far more prevalent than DETR-family models. Some segment weld seams directly, such as an improved YOLOv8s-Seg for seam tracking [41] and the boundary-enhanced SABE-YOLO [36]; others target detection and ROI localization, including S-YOLO [33], tailored architectures for seam identification [28, 37], and active–passive vision-fusion frameworks [9]. These implementations uniformly prioritize low latency, minimal footprint, and edge deployment. 2.3 Polarimetric imaging Polarimetric imaging measures the polarization state of light reflected by a surface, which depends on the surface’s geometry, roughness and material and is largely invisible to conventional intensity imaging. This makes it well suited to reflective metallic inspection: it captures surface and material cues that RGB misses, suppressing glare and improving uniformity [42, 40]. Recent sensor-level advances, such as chip-integrated full-Stokes imagers, enable single-shot acquisition in CMOS-compatible form factors, and division-of-focal-plane systems reconstruct Stokes parameters with DoLP and AoLP at full resolution [42, 3]. Polarimetric cues further support per-pixel material discrimination [16] and 3D reconstruction of specular surfaces when fused with deflectometry [34]. Applied industrial evidence is strongest in adjacent defect-inspection domains rather than in weld segmentation itself. A polarization system for highly reflective curved surfaces removed glare and improved image uniformity, with its YOLOv11 detector gaining 3.9% in precision over the best baseline [40], while a dual-stream polarization-plus-RGB network for rail defects reached 73.00% mIoU on low-contrast segmentation [26]. However, direct evaluation of polarimetric imaging for weld-seam instance segmentation remains largely unexplored. 2.4 Deep learning with small industrial datasets Small and imbalanced industrial datasets remain a primary constraint in applied deep learning, since collecting defective samples is expensive and limited diversity undermines robustness [20, 5]. Transfer learning from natural-image pretraining is the standard mitigation strategy. On injection-molding defects, model-based transfer learning with augmentation reached about 99% accuracy from only 200 images per class, against 88.7% for conventional CNNs [20]; on small fused-deposition-modeling (FDM), ImageNet fine-tuning with augmentation exceeded 90% accuracy on low-cost hardware [14]. Welding shares these constraints, with severe class imbalance motivating GAN-based augmentation and pretrained backbones [5]. Cross-domain evidence further warns that larger models do not automatically transfer better, and that random validation splits can overestimate deployment quality [6]. Consequently, the trade-off between model capacity and generalization is critical for small weld datasets, where nano-scale CNNs may match larger transformers that lack the data for stable optimization [39, 17]. 2.5 Real-time deployment For robotic and workstation-based inspection, the literature consistently prioritizes computational efficiency alongside segmentation accuracy rather than peak benchmark scores [22, 30]; DSGNet, for instance, reaches competitive surface-defect mIoU with only 0.49M parameters [22]. This requirement is particularly pronounced in weld inspection, where high headline scores are typically obtained under controlled acquisition and for task-specific formulations: an improved YOLOv8s-Seg seam-tracking robot reports a weld-recognition mAP50 of about 97.8% with a 4.88 MB model and roughly 54 ms inference on a Jetson Nano [41], while SABE-YOLO attains 127 FPS with 6.6M parameters [36]. Such figures, obtained in favorable and often single-task settings, leave open how segmentation behaves when acquisition is uncontrolled or the viewpoint changes at deployment time, precisely the regime this study targets. System-level studies reinforce that segmentation quality alone is insufficient: path planning, viewpoint placement and workflow integration all matter in practice [25, 30]. Few studies, however, evaluate weld segmentation on partner-owned industrial data under both controlled and uncontrolled acquisition [7, 25]. 2.6 Research gap and contribution While existing literature demonstrates the viability of deep weld segmentation, the benefits of polarization for reflective surface, and the need for lightweight models, it lacks a unified evaluation comparing RGB and polarimetric imaging under matched protocols [26, 40, 7]. Furthermore, current studies rarely isolate the impact of acquisition setup relative to architecture choice when operating on actual shop-floor data, despite evidence that real-world domain shift heavily influences model behaviour [6]. Finally, despite the strong performance of transformer-based architectures on standard benchmarks, there is little empirical evidence regarding their transferability to small industrial weld datasets, leaving it unclear whether higher-capacity models offer any practical benefit over compact CNN alternatives in this regime [4, 7, 20]. Building on these gaps, this study delivers three key findings. We first present a unified, threshold-independent evaluation comparing RGB and polarimetric imaging for weld seam instance segmentation, testing both convolutional and transformer architectures on identical dataset splits and evaluation protocols. Next, we measure the impact of acquisition conditions relative to model selection. The data demonstrates that transitioning from controlled to uncontrolled environments causes a performance degradation that outweighs the variances between model families, while multi-map polarimetric imaging effectively restores robustness on unmanaged surfaces without physical illumination controls. Finally, we analyze model behavior under deployment-time viewpoint shifts, observing that detection transformers generalize to unseen, close-range perspectives significantly better than the evaluated CNNs, an effect confirmed to be independent of input resolution and evaluation threshold parameters. Rather than searching for an optimal model architecture, this work serves as a controlled feasibility study. The goal is to isolate how acquisition modality, illumination control, and architectural choices interact under realistic industrial constraints. Accordingly, quantitative performance figures are presented to evaluate these trade-offs rather than to assert a single state-of-the-art result. 3 Materials and methods Figure 1 summarizes the end-to-end validation pipeline followed in this study, from data acquisition to the final metrics; the remainder of this section details each stage. Figure 1: End-to-end validation pipeline. Data partitions are split at the physical-weld level before any augmentation (Sec. 3.5); all six models are evaluated with the same pycocotools protocol at a fixed confidence threshold (Sec. 3.5), both in-distribution and, zero-shot, on the close-range set (Sec. 5). 3.1 Materials All images depict weld seams on metallic specimens provided by the industrial partner specialized in custom operator cabin manufacturing. Ground-truth annotations are polygonal instance masks of a single class (Weld), produced manually. Table 1 summarizes the datasets. Table 1: Datasets used in this study. “Train” counts physically distinct source acquisitions, with augmented training images in parentheses; augmented variants are generated from training sources only (Sec. 3.5). Acquisition conditions are detailed in the text. The close-range set is evaluation-only: by design it has no train or validation split (Sec. 3.1, “Close-range test set”), and all 62 images are used exclusively for zero-shot cross-evaluation of models trained on the other scenarios. Dataset Camera Resolution Train (aug.) Val Test RGB controlled OAK-D 1920×10801920×1080 51 (510) 15 12 RGB industrial OAK-D 1920×10801920×1080 35 (175) 11 8 PolarSens multi-map PolarSens 2448×20482448×2048 47 (1384) 15 5 PolarSens Intensity PolarSens 1224×10241224×1024 45 (450) 14 10 Close-range test OAK-D 1920×10801920×1080 — — 62 Controlled RGB dataset Images were acquired with an OAK-D RGB camera [23] fixed at 20 cm from the workpiece, with two auxiliary polarizer-equipped spotlights and a neutral black background, minimizing environmental variability. This setup provides an upper-bound scenario for model performance under consistent acquisition. Three metal specimens were imaged, for a total of 78 source images. Industrial RGB dataset A second set of 54 images was captured directly at the industrial production facility under unconstrained conditions. These acquisitions lack background or illumination control and include specular reflections, cluttered backgrounds, variable perspective angles, and non-uniform lighting to reflect operational deployment conditions. PolarSens datasets A division-of-focal-plane polarization camera (Baumer PolarSens, 0∘/45∘/90∘/135∘0 /45 /90 /135 micro-polarizer array) was used to acquire 67 physically distinct weld scenes under polarized illumination. From each acquisition, six per-pixel maps are derived, namely Intensity, angle of linear polarization (AoLP), degree of linear polarization (DoLP), ImaxI_ , IminI_ and Specular, each stored as a 3-channel image. These maps carry complementary information on surface structure, orientation and roughness, and allow assessing the contribution of polarization cues on reflective metallic surfaces. Because the polarization camera has a different viewpoint, this dataset is not pixel-aligned with the RGB ones. Figure 2 illustrates an example of the six polarization maps for a representative weld seam. Unlike the five magnitude-based maps (intensity, DoLP, ImaxI_ , IminI_ , and specular), which present similar grayscale appearances with distinct highlight distributions, the AoLP map exhibits a characteristic hue-like angular pattern. This spatial variance across representations provides the rationale for the multi-map strategy detailed in Sec. 3.4. Figure 2: The six PolarSens per-pixel maps derived from one acquisition (weld 20, test split), ground-truth weld outline in green. Close-range test set An additional evaluation-only dataset of 62 close-range images (∼10cm 10\,cm, single weld per image, manually captured) evaluates model generalization to the acquisition geometry expected in a robot-mounted camera setup. 3.2 Segmentation models We compare six instance segmentation models spanning two architectural families and more than an order of magnitude in parameter count (Table 2): four single-stage CNN detectors with mask heads from the YOLO family (YOLOv11-n/s/m-seg [12, 13], YOLOv8-s-seg [11]), and two transformer-based models: RF-DETR-Seg (Small) [29], a DINOv2-backbone detection transformer with a segmentation head, and Mask2Former (Swin-Small) with masked-attention segmentation [4]. This models selection balances inference throughput and capacity for industrial deployment. Transformer architectures are included to assess their transferability to data-scarce manufacturing settings and to anchor the upper bound of model capacity within our benchmark. All architectures are fine-tuned from COCO-pretrained weights. Table 2: Evaluated architectures. End-to-end latency (preprocessing, inference, postprocessing) per image on a single NVIDIA RTX 3090, measured on the polarimetric validation split at each model’s input resolution; transformer figures include amortized model loading. Model Family Params (M) Input res. (px) Latency (ms/img) YOLOv11-n-seg CNN 2.8 2464 24 YOLOv11-s-seg CNN 10.1 2464 49 YOLOv11-m-seg CNN 22.4 2464 111 YOLOv8-s-seg CNN 11.8 2464 42 RF-DETR-Seg-S Transformer 36.3 1120 132 Mask2Former (Swin-S) Transformer 69 1024† 454 †shorter side; transformer models are trained at reduced resolution due to GPU memory constraints, a caveat discussed in Sec. 5. 3.3 Training strategies All models are trained to convergence, with a maximum budget of 1000 epochs and early stopping. To train the CNN models, we use the Ultralytics pipeline [12] with its default hyperparameters: the SGD optimizer with an initial learning rate lr0=0.01lr_0=0.01 and a batch size of 2 at full resolution. The transformer models are trained with the AdamW optimizer [21] within their respective reference implementations: a learning rate of 5×10−55×10^-5 for Mask2Former, and the RF-DETR defaults with an effective batch size of 16 obtained through gradient accumulation. All experiments run on a single NVIDIA RTX 3090 (24 GB). Wall-clock training time scales with both model capacity and dataset size (largest for the augmented PolarSens multi-map set, smallest for industrial RGB): across scenarios and seeds, CNN training took on average 1.5 h (YOLOv11-n, range 0.2–7.6 h), 1.8 h (YOLOv11-s, range 0.3–6.2 h), 4.7 h (YOLOv11-m, range 0.8–15.6 h) and 1.6 h (YOLOv8-s, range 0.3–4.4 h); the transformers took 0.7–5.5 h (RF-DETR-Seg-S) and 0.9–9.6 h (Mask2Former), the same industrial-RGB-to-PolarSens range as the CNNs but consistently at the upper end of it, consistent with their larger parameter counts (Table 2). For the RGB scenarios, we additionally compare pretrained versus from-scratch initialization; native (1920×10801920×1080) versus halved (960×540960×540) input resolution; and, for the best configurations, a hyperparameter-optimization stage comparing the SGD and AdamW optimizers, using Ultralytics’ built-in genetic-algorithm tuner. Each “iteration” is one hyperparameter trial: it trains an independent model for 50 epochs with its own sampled hyperparameter values, and only the best-performing trial’s hyperparameters are carried forward to the final model. The search spends on the order of 100×50=5000100× 50=5000 epochs of training compute in total, split across 100 short, independent 50-epoch runs. Some datasets are augmented offline: augmented image variants are generated once from the training images and stored on disk, rather than being produced on the fly during training (the procedure is detailed in Sec. 3.5). For these pre-augmented datasets, the online augmentation of the training pipeline is disabled, so that all image-level variability comes from the stored offline augmentation and is identical across the models being compared. Seed repetition Reported metrics are mask mAP (mean average precision computed on the predicted segmentation masks, as opposed to bounding boxes). To separate genuine architectural effects from run-to-run randomness on our small test sets, each of the four CNN configurations is trained with three random seeds (varying weight initialization and data shuffling), and we report the mean and standard deviation across these seeds throughout Sec. 4. Training the transformer models is substantially more expensive, so their in-distribution results (Tables 3, 5, 7) are obtained with a single seed. The practical consequence is that small in-distribution gaps involving the transformers should be read with caution: for example, RF-DETR’s in-distribution lead over the best CNN on industrial RGB (Sec. 4.2) is only about three to four times the CNN seed standard deviation, so we cannot yet rule out that part of it is seed noise. Confirming these smaller in-distribution margins would require training the transformers with multiple seeds as well, which we leave to future work. The one comparison for which we train the transformers with three seeds is the domain-shift cross-evaluation to the close-range set (Sec. 5), where both transformers are retrained with three seeds for each source scenario (controlled and industrial RGB). Here the effect is an order of magnitude larger than the CNN seed variance, as the transformer-versus-CNN gap in mask mAP50 spans 0.6 to 0.8, compared to a maximum CNN seed standard deviation of 0.05. The multi-seed cost is therefore justified precisely where the claim is strongest. These same three industrial-RGB seeds are additionally re-evaluated in-distribution under the identical protocol, to check that RF-DETR’s out-of-distribution advantage reproduces across runs rather than resting on one lucky seed; this does not extend multi-seed training to the general in-distribution rankings of Sec. 4.2. 3.4 Polarimetric strategies We evaluate four strategies to leverage the polarimetric information (Figure 3): 1. Single-map (baseline): only the intensity map is used, as a standard 3-channel input, discarding the polarization cues. 2. Multi-map: all six maps are treated as independent 3-channel training samples that share the same ground-truth annotation, leaving the network architecture unmodified. In practice this multiplies the effective training-set size sixfold, since each map of a given weld is added to the dataset as a separate image with the same mask. 3. Single-backbone fusion: the six maps are stacked into an 18-channel input processed by a single backbone, evaluated with both a direct 18-channel first-layer convolution and an explicit channel-split layer. 4. Multi-backbone fusion: six parallel backbones, one per map, with feature-level fusion preceding the neck. For the two fusion strategies (3 and 4), data augmentation must strictly preserve spatial alignment across all six maps. Geometric transformations (horizontal/vertical flips, rotations within ±15∘± 15 , and shears within ±10∘± 10 ) are therefore sampled once per sample and applied identically to all six maps and to the polygon masks. Photometric augmentations are intentionally omitted, since they would corrupt the physical meaning of the polarimetric quantities. Results for strategies 3 and 4 are presented in Sec. 4.4. Figure 3: The four polarimetric strategies of Sec. 3.4. (a)–(b) use an unmodified YOLO11-seg architecture, single 3-channel input; they differ only in what is fed as training data. (c)–(d) use custom Split / ConvMultiInput layers to fuse all six maps inside the network, either through one shared backbone on the stacked 18-channel input (c) or through six parallel backbones concatenated at three feature scales before a shared neck (d). Verified against config/yolo11-seg-custom-polarsens-singlebackbone,multibackbone.yaml. 3.5 Experimental protocol Data partitions are strictly structured at the physical acquisition level: all original images and augmented variants of a given weld sample reside within the same partition to prevent data leakage. Augmentation is restricted exclusively to the training set, while validation and test sets consist solely of unaugmented original images. Within each experimental scenario, all models are trained and evaluated on identical splits. Evaluation metrics comprise COCO-style box and mask mAP50 and mAP50–95, alongside precision, recall, and F1 score at the selected operating point. To ensure a fair comparison across architectural paradigms, all CNN and transformer models are evaluated using a unified pycocotools pipeline [19] at a fixed low confidence threshold (0.010.01). This allows COCOeval [18] to sweep confidence thresholds across the full precision–recall spectrum. This threshold adjustment accounts for the lower calibrated confidence scores of query-based transformers. Applying higher operational thresholds (e.g., 0.50.5) truncates valid predictions and biases average precision against CNN baselines. The validation split plays a single, specific role: it is the signal used to pick the checkpoint that is later evaluated, for every architecture in this study. Concretely, the CNN training pipeline retains the checkpoint with the best validation fitness at each epoch [12], Mask2Former retains the checkpoint with the lowest validation loss, and RF-DETR retains the checkpoint with the best validation mAP50-95. The test split is never seen during training or checkpoint selection; it is touched only for the final, single evaluation reported in each table. Checkpoint selection optimizes a validation-set criterion, so validation scores could plausibly run systematically higher than test scores; empirically they don’t, in this study, in either direction. We verified that validation and test partitions never share a physical weld or an augmented variant of one, across every scenario, ruling out a split-leakage bug. Two factors explain the pattern instead. The validation and test splits are both small (8–15 and 5–12 images respectively, per scenario), so ranking noise from a handful of images is comparable in size to any genuine gap between splits. And the checkpoint-selection criterion (validation fitness, validation loss, or validation mAP50-95, depending on the framework) differs from the mask mAP50 column reported in the tables, so the checkpoint favored by the selection criterion can score differently on that specific metric. Results tables report both test and validation for every model so this can be inspected directly. Within each results table, models are listed in descending order of test-split mask mAP50, independently in every table; this produces a different row order in different tables and is not a stable per-model ranking carried over from one scenario to the next, which is the point being illustrated (Sec. 4.5). 4 Results Figure 4 previews the three acquisition scenarios qualitatively before the detailed quantitative evaluation: correct segmentation in controlled RGB and PolarSens maps, and a representative CNN failure mode under cluttered industrial RGB (false-positive background clutter and a missed weld), providing a concrete example of the limitations detailed in Sec. 4.2. Figure 4: Qualitative segmentation results (ground truth in solid green, prediction in dashed red) across the three main acquisition scenarios. Industrial RGB shows a representative CNN failure mode: a false-positive mask on cluttered background (bottom) alongside a missed weld (top right, green outline with no matching prediction). 4.1 Controlled RGB Table 3 reports model performance under controlled conditions using the uniform pycocotools evaluation protocol (Sec. 3.5) for seed 0. On the test split, RF-DETR-Seg achieves the highest performance (mask mAP50 of 0.9000.900), though it converges to the CNN cluster on the validation set (0.7550.755, compared to 0.72–0.810.72--0.81 for CNNs). Mask2Former lags across both splits (0.61–0.690.61--0.69). The four CNN architectures cluster tightly on both splits (test 0.79–0.82), with no consistent ranking by size. To quantify variance within the CNN family, Table 8 provides multi-seed statistics across three random runs, evaluated under the Ultralytics-native validation protocol rather than the uniform pycocotools protocol used in Table 3: the two protocols score the same checkpoint differently, so for a given architecture the seed-0 entry in Table 3 and the seed-0 sample underlying Table 8 are not the same number, only the same trained model. In the controlled RGB rows of Table 8, every CNN architecture yields a three-seed mean mask mAP50–95 between 0.460.46 and 0.550.55, with standard deviations ranging from 0.0050.005 to 0.0260.026. The YOLOv11 variants (nano, small, medium) fall within half a standard deviation of one another, with YOLOv8-s showing the largest deviation (∼1.6 1.6 pooled standard deviations on test). Given the small test partitions (8–15 images per split) and three-seed sampling, these minor variations remain within expected noise bounds, preventing a definitive ranking among CNN backbones. Earlier feasibility trials showed that dedicated hyperparameter optimization (100 SGD tuning iterations) boosted YOLOv11-s/m mask mAP50 to 0.89–0.900.89--0.90 (and up to 0.850.85 on the close-range set). However, achieving these gains over the default baselines requires significant tuning overhead. Direct zero-shot evaluation on the close-range set yields a distinct trend (Table 4). Averaged over three seeds, RF-DETR-Seg generalizes best to the new domain (mAP50 = 0.842, mAP50–95 = 0.537), outperforming CNN baselines (0.08–0.170.08--0.17) and Mask2Former (0.410.41) by a wide margin. In Sec. 5, we analyze these results and run a resolution-matched ablation to isolate architectural effects from image resolution. Table 3: Controlled RGB scenario, uniform pycocotools protocol, seed-0 checkpoints (test: 12 images; validation: 15 images). Test Validation Model mask mAP50 mAP50–95 mask mAP50 mAP50–95 RF-DETR-Seg-S 0.900 0.471 0.755 0.395 YOLOv11-n 0.819 0.452 0.792 0.436 YOLOv11-s 0.797 0.475 0.814 0.433 YOLOv11-m 0.795 0.437 0.783 0.433 YOLOv8-s 0.791 0.454 0.723 0.422 Mask2Former (Swin-S) 0.610 0.355 0.686 0.361 Table 4: Zero-shot cross-evaluation on the close-range test set (62 images) of the six models trained on controlled RGB (Table 3), uniform pycocotools protocol; no fine-tuning on close-range data. Mean ± population standard deviation over three training seeds for every model. Even the worst individual Mask2Former seed (0.357) exceeds the best individual CNN seed (YOLOv11-s, 0.215), so the transformer advantage is not an artifact of seed selection. Model mask mAP50 mAP50–95 RF-DETR-Seg-S 0.842±0.0150.842± 0.015 0.537±0.0270.537± 0.027 Mask2Former (Swin-S) 0.406±0.0500.406± 0.050 0.243±0.0420.243± 0.042 YOLOv11-s 0.168±0.0370.168± 0.037 0.091±0.0190.091± 0.019 YOLOv11-n 0.116±0.0050.116± 0.005 0.066±0.0040.066± 0.004 YOLOv11-m 0.102±0.0110.102± 0.011 0.048±0.0080.048± 0.008 YOLOv8-s 0.081±0.0050.081± 0.005 0.045±0.0040.045± 0.004 4.2 Industrial RGB Under uncontrolled shop-floor acquisition, performance drops sharply for the four CNN architectures (Table 5): on the seed-0 checkpoints, the best CNN reaches test mask mAP50 0.47 (YOLOv8-s, mAP50–95 0.24), against 0.79–0.82 for the same four models in controlled conditions (Table 3), a performance drop that far exceeds any difference among the CNN variants. Table 8 confirms this with three-seed statistics: CNN test means cluster at 0.22–0.48 mask mAP50, with standard deviations of 0.04–0.05; under this multi-seed average, YOLOv11-n emerges as the top-performing CNN (mean 0.477) rather than YOLOv8-s (mean 0.440), which led under seed 0 alone. This shift underscores the instability of single-seed rankings. Across runs, only YOLOv11-m consistently underperforms beyond seed variance, whereas the relative rankings among the nano, small, and YOLOv8-s variants remain statistically indistinguishable. RF-DETR is the best model overall on both splits (test 0.619/0.243, validation 0.643/0.311), clearly ahead of every CNN. Mask2Former is second on test (0.523/0.198) but drops into the CNN cluster on validation (0.435/0.212, below three of the four CNNs). This is the only in-distribution scenario in which a transformer outperforms every CNN outright. Zero-shot cross-evaluation on the close-range set (Table 6) provides the clearest evidence of this trend across all model comparisons. When both transformers are fine-tuned on the industrial RGB images and then evaluated, without any further training, on the close-range set, they outperform every CNN baseline by a large margin under three-seed statistics. RF-DETR reaches a three-seed mean mask mAP50 of 0.8100.810 (mAP50–95 0.5370.537) and Mask2Former 0.4610.461 (0.3000.300), whereas the four CNNs span only 0.0430.043–0.1890.189 (0.0180.018–0.0890.089). In other words, the weakest transformer result is still more than twice the strongest CNN result on this set. Two caveats temper the Mask2Former figure specifically. Its three-seed mean is inflated by a single unusually strong seed: the three runs score 0.3520.352, 0.3380.338 and 0.6930.693, giving a standard deviation (0.1640.164) far above the ≤0.05≤ 0.05 seen everywhere else in this study. Even so, its worst individual seed (0.3380.338) still exceeds the best individual CNN seed (0.2000.200), so the ranking does not depend on that lucky run; the high variance is a separate finding, revisited in Sec. 5. RF-DETR, by contrast, is stable across seeds (std 0.0070.007), so its advantage rests on firm ground. This close-range result mirrors the one obtained when the same models are instead fine-tuned on controlled RGB (Table 4): the transformers’ advantage on the shifted, close-range viewpoint therefore holds regardless of which RGB scenario they were trained on. Because this pattern is the paper’s central finding, it is analyzed in full, together with a resolution-matched ablation that separates an architectural explanation from a resolution artifact, in Sec. 5. On the CNN side, a typical failure mode on this set is background structure misclassified as weld in visually cluttered scenes.These findings illustrate the drop from lab conditions to field deployment, underpinning our acquisition recommendations in Sec. 5. In comparison, the preliminary feasibility study scored mask mAP50 of 0.43/0.36 for YOLOv11-s and 0.49/0.17 for YOLOv11-m (half vs. native resolution, default hyperparameters), consistent with our CNN baselines. Table 5: Industrial RGB scenario, uniform pycocotools protocol, seed-0 checkpoints (test: 8 images; validation: 11 images). Test Validation Model mask mAP50 mAP50–95 mask mAP50 mAP50–95 RF-DETR-Seg-S 0.619 0.243 0.643 0.311 Mask2Former (Swin-S) 0.523 0.198 0.435 0.212 YOLOv8-s 0.471 0.244 0.500 0.224 YOLOv11-s 0.414 0.167 0.477 0.209 YOLOv11-n 0.408 0.130 0.507 0.251 YOLOv11-m 0.247 0.088 0.432 0.219 Table 6: Zero-shot cross-evaluation on the close-range test set of the six models trained on industrial RGB (Table 5), uniform pycocotools protocol; no fine-tuning on close-range data. Mean ± population standard deviation over three training seeds for every model. Mask2Former’s variance here is exceptional (seeds 0.352, 0.338, 0.693): one seed nearly doubles the other two. Even so, its worst individual seed (0.338) exceeds the best individual CNN seed (YOLOv11-n, 0.200), so the ranking is unaffected; this high transformer variance is itself discussed in Sec. 5. Model mask mAP50 mAP50–95 RF-DETR-Seg-S 0.810±0.0070.810± 0.007 0.537±0.0140.537± 0.014 Mask2Former (Swin-S) 0.461±0.1640.461± 0.164 0.300±0.1240.300± 0.124 YOLOv11-n 0.189±0.0080.189± 0.008 0.089±0.0050.089± 0.005 YOLOv11-s 0.102±0.0370.102± 0.037 0.037±0.0110.037± 0.011 YOLOv8-s 0.061±0.0160.061± 0.016 0.025±0.0070.025± 0.007 YOLOv11-m 0.043±0.0150.043± 0.015 0.018±0.0100.018± 0.010 4.3 PolarSens Single map Using the Intensity map alone yields limited performance (best: mask mAP50 0.40, mAP50–95 0.23, YOLOv11-s pretrained; not tabulated, as this baseline is not the focus of the comparison), below the controlled RGB baseline: the intensity content of the polarimetric sensor, by itself, does not compensate for the reduced dataset size. Multi-map without augmentation Feeding all six maps as independent samples to an unmodified YOLOv11 improves results markedly (mask mAP50 up to 0.70, mAP50–95 0.36), indicating that the complementary polarization maps carry discriminative information about the weld region even when processed independently. Multi-map with geometric augmentation Adding the alignment-preserving geometric augmentation of Sec. 3.4 produces the strongest results of the study (Table 7). On welds never seen in training, all four CNN architectures localize welds reliably (test mask mAP50 0.74–0.91) and RF-DETR-Seg is essentially tied with the best CNN on this metric using the same seed-0 checkpoints as Table 7 (0.904 versus 0.914). Against the three-seed mean of the best CNN (0.926, YOLOv11-n, Table 8) the tie widens slightly, though still by less than the seed-to-seed variance discussed below. The stricter mAP50–95 suggests otherwise: every CNN baseline (0.38–0.500.38--0.50) outperforms RF-DETR (0.4210.421) and Mask2Former (0.3090.309), indicating that transformers achieve comparable coarse localization to CNNs, but with less precise boundaries. Table 8 quantifies CNN seed variance directly: three-seed test means span 0.537–0.570 mask mAP50–95 with standard deviations of 0.012–0.030, i.e. the four architectures perform comparably on this metric, though YOLOv11-n has the highest mean on both splits. The gap between the lenient and strict metrics for the CNNs themselves indicates that residual errors concentrate in boundary precision rather than in weld localization. Transformer baselines When evaluated under the same threshold-independent protocol as the CNNs, RF-DETR-Seg closes the gap on PolarSens significantly more than raw scores suggest: it matches the best CNN on mask mAP50 (Table 7) and trails only on the strict mAP50–95 metric, where all four CNNs maintain a distinct lead (0.38–0.50 versus 0.421). Mask2Former remains the weakest model on both splits and both metrics (0.52/0.31 test). Both transformers show early validation-loss saturation during training, indicative of over-parameterization relative to the ∼ 50 training welds; this training-curve evidence is independent of evaluation thresholding, offering the strongest support for how model capacity limits performance in low-data regimes. Their latency (132 and 454 ms/img respectively, versus 24 ms/img for YOLOv11-n) remains a practical deployment overhead regardless of accuracy. Part of the remaining performance gap likely stems from resolution constraints imposed by GPU memory limits (Tab. 2); disentangling resolution from architecture effects for this scenario specifically is left to future work (Sec. 5 reports a resolution ablation for the RGB scenarios). Table 7: PolarSens multi-map with geometric augmentation, uniform pycocotools protocol, seed-0 checkpoints: leakage-free sample-level splits (test: 5 unseen welds / 30 images; validation: 15 unseen welds / 90 images). Test Validation Model mask mAP50 mAP50–95 mask mAP50 mAP50–95 YOLOv11-n 0.914 0.501 0.756 0.350 RF-DETR-Seg-S 0.904 0.421 0.701 0.295 YOLOv8-s 0.862 0.500 0.700 0.336 YOLOv11-s 0.807 0.479 0.679 0.323 YOLOv11-m 0.743 0.382 0.644 0.289 Mask2Former (Swin-S) 0.519 0.309 0.595 0.278 Table 8: CNN seed variance across the three in-distribution scenarios: mean ± population standard deviation over 3 seeds, Ultralytics-native validation protocol. This protocol differs from the uniform pycocotools protocol used in Tables 3, 5 and 7, so the seed-0 entries here are different numbers from the seed-0 rows in those tables for the same checkpoints; the ranges reported per scenario in the text apply to that scenario’s rows only, not to the table as a whole. These figures characterize variance within the CNN family and are not comparable across families; see Tables 3, 5 and 7 for the protocol-matched comparison against the transformers, whose in-distribution results are single-seed (Sec. 3.3). Nothing is bolded, since no architecture is reliably distinguishable from the others at these standard deviations. Test Validation Model mask mAP50 mAP50–95 mask mAP50 mAP50–95 Controlled RGB YOLOv11-n 0.858±0.0090.858± 0.009 0.503±0.0200.503± 0.020 0.869±0.0130.869± 0.013 0.517±0.0250.517± 0.025 YOLOv11-s 0.860±0.0180.860± 0.018 0.504±0.0130.504± 0.013 0.849±0.0070.849± 0.007 0.521±0.0070.521± 0.007 YOLOv11-m 0.865±0.0120.865± 0.012 0.510±0.0140.510± 0.014 0.844±0.0140.844± 0.014 0.514±0.0050.514± 0.005 YOLOv8-s 0.779±0.0340.779± 0.034 0.463±0.0260.463± 0.026 0.855±0.0060.855± 0.006 0.549±0.0090.549± 0.009 Industrial RGB YOLOv11-n 0.477±0.0510.477± 0.051 0.201±0.0170.201± 0.017 0.521±0.0160.521± 0.016 0.268±0.0220.268± 0.022 YOLOv11-s 0.428±0.0390.428± 0.039 0.195±0.0190.195± 0.019 0.496±0.0320.496± 0.032 0.228±0.0100.228± 0.010 YOLOv11-m 0.221±0.0500.221± 0.050 0.081±0.0210.081± 0.021 0.372±0.0600.372± 0.060 0.193±0.0280.193± 0.028 YOLOv8-s 0.440±0.0500.440± 0.050 0.226±0.0360.226± 0.036 0.493±0.0250.493± 0.025 0.242±0.0050.242± 0.005 PolarSens multi-map YOLOv11-n 0.926±0.0210.926± 0.021 0.570±0.0120.570± 0.012 0.825±0.0280.825± 0.028 0.432±0.0090.432± 0.009 YOLOv11-s 0.890±0.0120.890± 0.012 0.545±0.0160.545± 0.016 0.838±0.0130.838± 0.013 0.442±0.0050.442± 0.005 YOLOv11-m 0.897±0.0280.897± 0.028 0.555±0.0300.555± 0.030 0.825±0.0120.825± 0.012 0.445±0.0070.445± 0.007 YOLOv8-s 0.889±0.0230.889± 0.023 0.537±0.0280.537± 0.028 0.833±0.0230.833± 0.023 0.445±0.0050.445± 0.005 4.4 Polarimetric channel fusion Table 9 reports the two fusion variants (nano scale, trained on the alignment-preserving augmented dataset of Sec. 3.4), evaluated with a dedicated multimodal validator that mirrors the training-time dataset construction. Both variants perform far below the multi-map baseline of Table 7 (test mask mAP50–95 0.03–0.04 versus a 0.54–0.57 three-seed mean, Table 8), and below the single-map baseline of Sec. 4.3 as well. Explicit channel fusion is therefore not competitive with treating the six polarization maps as independent training samples on this dataset. We attribute this to the combination of a small training set (∼ 47 welds) with architectures of substantially larger effective capacity (a channel-split layer plus, for the multi-backbone variant, six parallel backbones): both training curves plateau within the first 5–10 epochs, well before the 30–50-epoch budget that suffices for the multi-map CNNs, indicating an optimization regime dominated by insufficient data rather than by insufficient training time. The primary takeaway remains robust: even with a corrected architecture and a leakage-free split, both fusion variants perform well below the unmodified multi-map approach. In contrast, the relative ranking between the two fusion variants varies across metrics (multi-backbone leads only on test mAP50, while single-backbone wins on the remaining three), so we draw no firm conclusion on their ordering. Table 9: Polarimetric channel fusion (YOLOv11-n backbone, single class), alignment-preserving augmentation, leakage-free splits (test: 5 unseen welds / 30 images; validation: 15 unseen welds / 90 images). Multi-backbone leads only on test mask mAP50 and trails on every other column, including its own validation split; with only 5 test and 15 validation welds, single high-IoU-threshold hits or misses swing these metrics disproportionately, so the two variants should be read as comparably poor rather than one reliably beating the other (see the small-test-set caveat in Sec. 5). Test Validation Model mask mAP50 mAP50–95 mask mAP50 mAP50–95 Single-backbone (18ch, split layer) 0.117 0.040 0.178 0.068 Multi-backbone (6 parallel) 0.216 0.029 0.083 0.039 4.5 Architectural comparison Once evaluated under a unified protocol (Sec. 3.5) with quantified seed variance, the cross-model comparison is far less straightforward than single-run metrics imply, for two reasons. First, within the CNN family, model capacity shows no reliable in-distribution ranking once seed variance is accounted for (Table 8: three-seed means for the four architectures fall within roughly one to two standard deviations of each other in nearly every scenario and split). The one exception robust to seed noise is the medium model, which is the worst CNN by a margin exceeding its own variance on industrial RGB and is never the best elsewhere; beyond that, we do not consider a nano-versus-small-versus-YOLOv8-s ranking established. Second, the transformers do not follow a single pattern relative to the CNNs. In-distribution, RF-DETR is competitive to outright best in every scenario under the corrected protocol (best model overall on industrial RGB, best on controlled-RGB test, tied with the best CNN on PolarSens mask mAP50), while trailing the CNN cluster on the stricter mAP50–95 on PolarSens and, on validation splits, falling back into the CNN range rather than leading it; Mask2Former remains the weakest model on PolarSens and controlled RGB but is competitive on industrial RGB test. Latency still tracks capacity cleanly (Table 2): the nano CNN is the fastest model benchmarked (24 ms/img) and Mask2Former the slowest (454 ms/img, roughly 19× slower), independently of which model is most accurate in a given scenario. The strongest, most general pattern lies not in in-distribution capacity, but in behavior under domain shift. Across all close-range cross-evaluations, both transformers, particularly RF-DETR, outperform every CNN baseline by a wide margin. The next section analyzes this finding alongside a resolution-matched ablation to separate architectural effects from resolution artifacts. 5 Discussion and conclusions Acquisition quality is a system component The largest performance factor in this study is not architectural. Three-seed CNN test means span 0.78–0.87 mask mAP50 in controlled conditions and fall to 0.22–0.48 on shop-floor imagery (Table 8), a gap far wider than any difference between architectures within either scenario. The transformers degrade less steeply: on the seed-0 checkpoints Mask2Former drops from 0.610 to 0.523 and RF-DETR from 0.900 to 0.619, both smaller relative losses than the weakest CNNs, which anticipates the domain-shift behaviour discussed below. For industrial adoption, the acquisition setup (background, illumination, and camera distance) deserves at least as much engineering attention as the choice of network. Polarimetric imaging for reflective surfaces Polarization-derived maps are informative for weld segmentation on reflective metal. The multi-map strategy with alignment-preserving augmentation reaches a three-seed mean mask mAP50 of 0.93 (YOLOv11-n, Table 8) with an unmodified architecture. Two factors drive this: the complementary information across the six maps (0.40 to 0.70 mAP50 from single- to multi-map) and the geometric augmentation (0.70 to 0.93). Boundary precision remains the open problem, with mAP50–95 below 0.58. Explicit channel fusion does not help: both fusion variants fall well below the multi-map baseline (Sec. 4.4), so at this dataset size feeding independent maps to an unmodified network uses the polarimetric signal more efficiently than architectural fusion. Across scenarios, polarimetric multi-map (0.93) is on par with the best controlled-RGB result (RF-DETR, 0.900, Table 3), a difference within the seed variance seen elsewhere in this study (0.01–0.03 on comparable splits). We therefore do not claim polarimetric imaging is superior to RGB under controlled acquisition. Its practical advantage is over uncontrolled RGB, where it reaches 0.89–0.93 against 0.25–0.62 for RGB models (seed-0, Table 5): it matches controlled-RGB accuracy without the background, lighting, and distance control that RGB requires, at the cost of a dedicated camera and polarized illumination. Model capacity and small data Within the CNN family, model capacity gives no reliable in-distribution ranking once seed variance is accounted for. Across PolarSens and both RGB scenarios (Table 8), the four architectures’ three-seed means fall within one to two standard deviations of each other in almost every split, so nano, small, and YOLOv8-s cannot be separated with three seeds and 8–15-image test sets. The one ranking that survives is that the medium model is the worst CNN on industrial RGB by more than its own noise and is never best elsewhere. The defensible conclusion is narrow: capacity beyond the nano/small range brings no measurable gain at this data scale, and the choice among small variants should rest on latency and memory rather than an expected accuracy edge. For practice, embedded-friendly models are a safe default in-distribution, and effort is better spent on data collection and acquisition control than on model size. Transformers generalize better under domain shift The clearest exception to “smaller is enough” is transformer behaviour under distribution shift. In both zero-shot cross-evaluations to the close-range set, from controlled RGB (Table 4) and from industrial RGB (Table 6), RF-DETR leads by a wide margin regardless of training scenario (three-seed mean mask mAP50 0.842 and 0.810, against CNN means of 0.08–0.17 and 0.04–0.19 mask mAP50), and its worst seed still beats the best CNN seed in both cases. The effect does not depend on clean training data and is not a single-seed artifact. Figure 5 shows it on one image: the controlled-RGB CNN misses the weld entirely, while RF-DETR trained on the same data localizes it. Figure 5: Domain-shift comparison on a single close-range test image (ground truth in solid green, prediction in dashed red), both models trained on controlled RGB only and evaluated zero-shot: YOLOv11-n produces no prediction at all, while RF-DETR-Seg-S localizes the weld closely. Two alternative explanations were tested. The first is the evaluation threshold: query-based transformers emit low-confidence detections, and the fixed pycocotools protocol (Sec. 3.5) scores every model over the full precision–recall curve, removing the bias that a higher cutoff would place on the transformers. The second is input resolution (Table 2: CNNs at native resolution, transformers at 1120/1024 px). Because close-range welds are large in pixels, a lower resolution could in principle favour the transformers. The ablation in Table 10 rules this out: retraining YOLOv11-n from scratch at the transformer resolution (1120 px) recovers almost nothing on the close-range set (0.174 mask mAP50), far below RF-DETR at the same resolution (0.842 mask mAP50). The large recovery seen when a natively-trained CNN is merely run at 1120 px (0.611 mask mAP50) is a side effect of testing outside the trained resolution, not evidence that resolution explains the gap. Table 10: Resolution ablation: YOLOv11-n trained on controlled RGB, evaluated zero-shot on the close-range test set (62 images), uniform pycocotools protocol. RF-DETR-Seg-S is included as the transformer reference point at the same test resolution. The native-train/test and RF-DETR rows reproduce values already reported in Table 4. Configuration Train (px) Test (px) mask mAP50 mAP50–95 YOLOv11-n, native (baseline) 1920 1920 0.119 0.068 YOLOv11-n, inference-only 1920 1120 0.611 0.330 YOLOv11-n, retrained matched 1120 1120 0.174 0.081 RF-DETR-Seg-S (reference) 1120 1120 0.842 0.537 One feature of the industrial-trained RF-DETR needs care: its close-range score exceeds its own in-distribution score (0.810 versus a 0.612 three-seed mean mask mAP50, on every seed). A uniformly easier test set would raise all models, yet the CNNs collapse on the same images, so the close-range set is not simply easy; RF-DETR transfers to the new viewpoint well enough to exceed its noisier in-distribution result. Mask2Former does not show this. Its close-range mean (0.461 mask mAP50) is inflated by a single strong seed (0.693 against roughly 0.34 for the other two, std 0.164), and the pattern is not consistent across runs. We therefore attribute the domain-shift advantage to RF-DETR specifically, not to transformers as a class, and read Mask2Former’s spread as instability of its head when trained on only 35 industrial images. The mechanism most consistent with these results is that the attention-based, DINOv2- or Swin-pretrained backbones generalize to a large distance and viewpoint change more gracefully than the CNN backbones. Confirming this would require an ablation separating backbone pretraining from architecture family, and a CNN pretrained on a comparably large corpus, which we did not have. In practice, the result argues for testing candidate models at the deployment viewpoint rather than defaulting to a lightweight CNN, especially for a robot-mounted camera whose distance and angle change along the inspection path. Limitations This is an exploratory study. Test sets are small (5–15 welds per scenario), so single-weld outcomes can move per-split metrics even after averaging three seeds; per-weld variance reporting is planned. Three-seed repetition covers the domain-shift cross-evaluation for both families, but the in-distribution transformer results (Tables 3, 5, 7) are single-run, so the smaller transformer–CNN margins there, such as RF-DETR’s in-distribution lead on industrial RGB, warrant more caution than the domain-shift result. Mask2Former’s close-range variance (std up to 0.164), larger than any other result here, is itself a finding on head stability in this regime. The RGB datasets cover only three physical specimens; the resolution ablation covers one architecture and one scenario; transformer training is capped at reduced resolution by GPU memory; and the work addresses weld localization only, not defect classification, which is the natural next step. Finally, all latency figures are offline measurements on a desktop GPU (Table 2); online validation on live streams and embedded hardware, including the accuracy–throughput trade-off at the lower input resolutions such targets require, is left to future work. Industrial implications The results support a phased path. Lightweight CNNs on controlled-acquisition stations can already assist operators in weld localization, standardizing inspection and reducing its time. Polarimetric imaging is a worthwhile upgrade for reflective surfaces, mainly because it removes the acquisition-control burden rather than because it raises an already-competitive accuracy ceiling, at the cost of a dedicated camera and polarized illumination. The offline latency of the fastest model (24 ms/img on a desktop GPU) suggests embedded deployment in robot-guided cells is plausible but unverified on embedded hardware. In all cases the deployment viewpoint should be checked in advance, given the domain-shift results. Data availability Data will be made available on request. Acknowledgements This work was supported by internal institutional funding. References [1] L. Armesto, J. Tornero, A. Herraez, and J. Asensio (2011) Inspection system based on artificial vision for paint defects detection on cars bodies. In 2011 IEEE International Conference on Robotics and Automation, p. 1–4. Cited by: §1. [2] J. Chen, C. Wang, F. Shi, M. Kaaniche, M. Zhao, Y. Jing, and S. Chen (2024) DSNet: a dynamic squeeze network for real-time weld seam image segmentation. Engineering Applications of Artificial Intelligence 133, p. 108278. External Links: Document Cited by: §2.1. [3] Y. Chen, J. Wen, W. Shi, H. Gao, Y. Shao, L. Xu, S. Feng, X. Wang, Y. Shao, W. Shen, et al. (2025) Computational spectro-polarimetric imaging with resolution-preserving demosaicking. Optics Express 33 (8), p. 17990–18004. External Links: Document Cited by: §2.3. [4] B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar (2022) Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 1290–1299. External Links: Document Cited by: §2.2, §2.6, §3.2. [5] W. Dai, D. Li, D. Tang, H. Wang, and Y. Peng (2022) Deep learning approach for defective spot welds classification using small and class-imbalanced datasets. Neurocomputing 477, p. 46–60. External Links: Document Cited by: §2.4. [6] P. Goodarzi, A. Schütze, and T. Schneider (2022) Comparison of different ml methods concerning prediction quality, domain adaptation and robustness. tm-Technisches Messen 89 (4), p. 224–239. External Links: Document Cited by: §2.4, §2.6, §2. [7] J. He, P. Li, Y. Geng, and X. Xie (2023) Fastinst: a simple query-based model for real-time instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 23663–23672. External Links: Document Cited by: §2.2, §2.5, §2.6. [8] K. He, G. Gkioxari, P. Dollár, and R. Girshick (2017) Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, p. 2961–2969. External Links: Document Cited by: §2.2. [9] M. Hu, X. Hu, J. Zhao, and H. Zhan (2025) Weld seam roi detection and segmentation method based on active–passive vision fusion. Sensors 25 (24), p. 7530. External Links: Document Cited by: §2.2. [10] M. Imad, C. Hopkins, A. Hosseini, N. Z. Yussefian, and H. Kishawy (2022) Intelligent machining: a review of trends, achievements and current progress. International Journal of Computer Integrated Manufacturing 35 (4-5), p. 359–387. Cited by: §1. [11] G. Jocher, A. Chaurasia, and J. Qiu (2023) Ultralytics YOLOv8. Note: Software External Links: Link Cited by: §3.2. [12] G. Jocher and J. Qiu (2024) Ultralytics YOLO11. Note: Software External Links: Link Cited by: §3.2, §3.3, §3.5. [13] R. Khanam and M. Hussain (2024) Yolov11: an overview of the key architectural enhancements (2024). arXiv preprint arXiv:2410.17725. Cited by: §3.2. [14] H. Kim, H. Lee, and S. Ahn (2022) Systematic deep transfer learning method based on a small image dataset for spaghetti-shape defect monitoring of fused deposition modeling. Journal of Manufacturing Systems 65, p. 439–451. External Links: Document Cited by: §2.4. [15] D. D. Kumar, C. Fang, Y. Zheng, and Y. Gao (2023) Semi-supervised transfer learning-based automatic weld defect detection and visual inspection. Engineering Structures 292, p. 116580. Cited by: §1. [16] M. Kurachi, R. Kawahara, and T. Okabe (2025) One-shot polarization-based material classification with optimal illumination. Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2025 3, p. 738–745. External Links: Document Cited by: §2.3. [17] F. Li, H. Zhang, S. Liu, J. Guo, L. M. Ni, and L. Zhang (2022) Dn-detr: accelerate detr training by introducing query denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 13619–13627. External Links: Document Cited by: §2.2, §2.4. [18] T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick (2014) Microsoft coco: common objects in context. In European conference on computer vision, p. 740–755. Cited by: §3.5. [19] T. Lin et al. (2015) COCO API (pycocotools). Note: https://github.com/cocodataset/cocoapiAccessed: 2026-08-04 Cited by: §3.5. [20] J. Liu, F. Guo, H. Gao, M. Li, Y. Zhang, and H. Zhou (2021) Defect detection of injection molding products on small datasets using transfer learning. Journal of manufacturing processes 70, p. 400–413. External Links: Document Cited by: §2.4, §2.6. [21] I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. External Links: 1711.05101, Link Cited by: §3.3. [22] H. Lu, Y. Zhao, G. Yang, and S. Wu (2026) DSGNet: a lightweight network integrating depthwise separable and ghost convolutions for real-time surface defect segmentation. Expert Systems 43 (2), p. e70187. External Links: Document Cited by: §2.5. [23] Luxonis (2021) OAK-D. Note: https://shop.luxonis.com/products/oak-dAccessed: 2026-08-04 Cited by: §3.1. [24] J. Molina, J. E. Solanes, L. Arnal, and J. Tornero (2017) On the detection of defects on specular car body surfaces. Robotics and Computer-Integrated Manufacturing 48, p. 263–278. Cited by: §1. [25] A. Nandagopal, J. Beachy, C. Acton, and X. Chen (2025) A robotic surface inspection framework and machine-learning based optimal segmentation for aerospace and precision manufacturing. Journal of Manufacturing Processes 134, p. 146–157. External Links: Document Cited by: §2.1, §2.5, §2. [26] Y. Pan, J. Chen, P. Wu, H. Zhong, Z. Deng, and D. Sun (2025) Enhanced rail surface defect segmentation using polarization imaging and dual-stream feature fusion. Sensors 25 (11), p. 3546. External Links: Document Cited by: §2.3, §2.6. [27] A. Papavasileiou, G. Michalos, and S. Makris (2025) Quality control in manufacturing–review and challenges on robotic applications. International Journal of Computer Integrated Manufacturing 38 (1), p. 79–115. External Links: Document Cited by: §2.1. [28] H. Qu, Y. Zhou, J. Cheng, J. Cheng, H. Wang, X. Zheng, and L. Gao (2025) Research on real-time precise identification of weld seam types and roi localization based on lightweight improved yolov10n. In 2025 4th International Conference on Image Processing, Computer Vision and Machine Learning (ICICML), p. 1137–1142. External Links: Document Cited by: §2.2. [29] Roboflow (2025) RF-DETR: a real-time transformer-based object detection model. Note: https://github.com/roboflow/rf-detrSoftware repository Cited by: §3.2. [30] M. Soori, R. Dastres, B. Arezoo, and F. K. G. Jough (2024) Intelligent robotic systems in industry 4.0: a review. Journal of Advanced Manufacturing Science and Technology 4 (3), p. 2024007–2024007. External Links: Document Cited by: §2.5. [31] P. Stavropoulos, A. Papacharalampopoulos, and K. Sabatakakis (2023) Robust and secure quality monitoring for welding through platform-as-a-service: a resistance and submerged arc welding study. Machines 11 (2), p. 298. Cited by: §1. [32] D. Tabernik, S. Šela, J. Skvarč, and D. Skočaj (2020) Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31 (3), p. 759–776. Cited by: §1. [33] C. Wang, H. Hu, L. Xiao, and H. Zheng (2025) Weld seam detection and extraction method based on s-yolo. In 2025 4th International Symposium on Robotics, Artificial Intelligence and Information Engineering (RAIIE), p. 8–11. External Links: Document Cited by: §2.2. [34] J. Wang, O. Cossairt, and F. Willomitzer (2025) 3D imaging of complex specular surfaces by fusing polarimetric and deflectometric information. Optica 12 (4), p. 446–450. External Links: Document Cited by: §2.3. [35] J. Wang, X. Zhang, J. Liu, Y. Shi, and Y. Huang (2022) Line-structured light fillet weld positioning method to overcome weld instability due to high specular reflection. Machines 11 (1), p. 38. Cited by: §1. [36] R. Wen, W. Xie, Y. Fan, and L. Shen (2025) SABE-yolo: structure-aware and boundary-enhanced yolo for weld seam instance segmentation. Journal of Imaging 11 (8), p. 262. External Links: Document Cited by: §2.2, §2.5. [37] L. Xiao, C. Dong, S. Wu, C. Wang, H. Zheng, and H. Hu (2026) A weld seam recognition method based on improved yolo model and its feature point extraction method. Applied Sciences 16 (5), p. 2499. External Links: Document Cited by: §2.2. [38] T. Xin, Z. Yan, T. Xu, S. Li, R. Duan, and M. Peng (2025) Rapid visual detection of laser welding defects in bright stainless steel thin plates. Measurement Science and Technology 36 (3), p. 035203. Cited by: §1. [39] L. Yang, H. Wang, W. Meng, and H. Pan (2024) CvT-unet: a weld pool segmentation method integrating a cnn and a transformer. Heliyon 10 (15). External Links: Document Cited by: §2.1, §2.4, §2. [40] Z. Yu, D. Wang, and H. Wu (2025) Defect detection method for large-curvature and highly reflective surfaces based on polarization imaging and improved yolov11. In Photonics, Vol. 12, p. 368. External Links: Document Cited by: §2.3, §2.3, §2.6. [41] M. Zhao, X. Liu, K. Wang, Z. Liu, Q. Dong, P. Wang, and Y. Su (2024) Welding seam tracking and inspection robot based on improved yolov8s-seg model. Sensors 24 (14), p. 4690. Cited by: §2.2, §2.5. [42] J. Zuo, J. Bai, S. Choi, A. Basiri, X. Chen, C. Wang, and Y. Yao (2023) Chip-integrated metasurface full-stokes polarimetric imaging sensor. Light: Science & Applications 12 (1), p. 218. External Links: Document Cited by: §2.3.