Paper deep dive
The Impact of CutMix on Reliability and Robustness in Semantic Segmentation
Steven Landgraf, Markus Ulrich
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/20/2026, 4:59:35 AM
Summary
This study investigates the impact of the CutMix data augmentation strategy on the reliability and robustness of semantic segmentation models, specifically DeepLabV3+ and SegFormer. While CutMix has a minor effect on segmentation accuracy (mIoU), it consistently improves model reliability, particularly in uncertainty quality and calibration, under both in-domain and out-of-domain (foggy) conditions. The findings suggest that CutMix enhances the trustworthiness of confidence estimates rather than raw prediction accuracy, making it valuable for safety-critical applications.
Entities (10)
Relation Signals (8)
DeepLabV3+ → isevaluatedwith → CutMix
confidence 96% · We evaluate two representative architectures, the CNN-based DeepLabV3+ and the transformer-based SegFormer... analyzing the influence of CutMix
SegFormer → isevaluatedwith → CutMix
confidence 96% · We evaluate two representative architectures, the CNN-based DeepLabV3+ and the transformer-based SegFormer... analyzing the influence of CutMix
CutMix → improves → Reliability
confidence 95% · Our results show that CutMix has only a minor impact on segmentation accuracy but consistently improves the reliability, particularly under distribution shifts.
Cityscapes → isusedfortraining → DeepLabV3+
confidence 95% · we train all models on Cityscapes.
CutMix → hasminorimpacton → mIoU
confidence 94% · Our results show that CutMix has only a minor impact on segmentation accuracy
Foggy Cityscapes → isusedforoutofdomainevaluation → DeepLabV3+
confidence 94% · For out-of-domain analyses, we use the Foggy Cityscapes... using various backbone sizes for DeepLabV3+
CutMix → enhances → Uncertainty Quality
confidence 93% · These improvements indicate that CutMix primarily enhances the trustworthiness of the model's calibration and uncertainty
CutMix → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Ensuring not only high accuracy but also reliable and robust predictions is critical for the deployment of semantic segmentation models in safety-critical applications such as autonomous driving. Despite the widespread use of CutMix - a simple yet powerful data augmentation strategy - its effect on the reliability and robustness in dense predictions tasks remains unexplored. Motivated by recent findings that semi-supervised segmentation methods, where CutMix is a core component, can severely degrade reliability, this study isolates and systematically analyzes the influence of CutMix on segmentation accuracy, calibration, and uncertainty quality. We evaluate two representative architectures, the CNN-based DeepLabV3+ and the transformer-based SegFormer, across both in-domain and out-of-domain scenarios. Our results show that CutMix has only a minor impact on segmentation accuracy but consistently improves the reliability, particularly under distribution shifts. These improvements indicate that CutMix primarily enhances the trustworthiness of the model's calibration and uncertainty rather than the raw segmentation prediction itself. This distinction is crucial for safety-critical deployment, where reliable confidence estimates are as important as raw performance.
Tags
Links
- Source: https://arxiv.org/abs/2608.18715v1
- Canonical: https://arxiv.org/abs/2608.18715v1
Trouble viewing inline? Open PDF directly →
Full Text
34,873 characters extracted from source content.
Expand or collapse full text
The Impact of CutMix on Reliability and Robustness in Semantic Segmentation Steven Landgraf Thanks: Corresponding Authors Markus Ulrich Address: Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT), Germany, (steven.landgraf, markus.ulrich)@kit.edu Abstract Ensuring not only high accuracy but also reliable and robust predictions is critical for the deployment of semantic segmentation models in safety-critical applications such as autonomous driving. Despite the widespread use of CutMix – a simple yet powerful data augmentation strategy – its effect on the reliability and robustness in dense predictions tasks remains unexplored. Motivated by recent findings that semi-supervised segmentation methods, where CutMix is a core component, can severely degrade reliability, this study isolates and systematically analyzes the influence of CutMix on segmentation accuracy, calibration, and uncertainty quality. We evaluate two representative architectures, the CNN-based DeepLabV3+ and the transformer-based SegFormer, across both in-domain and out-of-domain scenarios. Our results show that CutMix has only a minor impact on segmentation accuracy but consistently improves the reliability, particularly under distribution shifts. These improvements indicate that CutMix primarily enhances the trustworthiness of the model’s calibration and uncertainty rather than the raw segmentation prediction itself. This distinction is crucial for safety-critical deployment, where reliable confidence estimates are as important as raw performance. keywordsCutMix, Reliability, Robustness, Semantic Segmentation 1 Introduction Deep neural networks achieve remarkable success in semantic segmentation (30), driving progress in a variety of domains ranging from autonomous driving, medical imaging, and remote sensing (31; 3; 25). However, as their deployment in safety-critical real-world applications increases, so does the need to ensure that their predictions are not only accurate but also reliable and robust. Reliability, in this context, refers to the degree to which model confidences reflect the true likelihood of correctness, while robustness describes the model’s ability to maintain performance under perturbations or distribution shifts. Due to the cost – and therefore scarcity – of manually labeled ground truth information for semantic segmentation tasks, recent work has embraced semi-supervised learning and strong data augmentations as core design elements. Among these, CutMix (53) has become a central component in recent semi-supervised learning frameworks (51; 52). By randomly pasting patches from one image into another and mixing their corresponding labels, CutMix promotes spatially localized feature learning and reduces overfitting. While this simple, yet effective strategy has been shown to improve classification reliability and robustness (36; 40; 53), its impact on dense prediction tasks such as semantic segmentation remain poorly understood. In addition, a recent study revealed a critical blind spot in this context: state-of-the-art semi-supervised segmentation methods, despite their strong performance in terms of accuracy, can severely deteriorate the reliability of neural networks (22). This observation raises an important research question, as CutMix is a key component of these methods and may contribute to this problem. Motivated by these findings, this work aims to analyze the impact of CutMix on reliability and robustness in semantic segmentation. We evaluate the effects of CutMix training in both in-domain and out-of-domain scenarios, considering accuracy, calibration, uncertainty quality, and their robustness. By disentangling the influence of CutMix from other components of semi-supervised learning frameworks, we aim to reveal its impact on the reliability and robustness in semantic segmentation – a research question that has yet to be answered. 2 Related Work (a) Input Image Label: Dog 1.0 (b) MixUp Label: Dog 0.5, Cat 0.5 (c) CutOut Label: Dog 1.0 (d) CutMix Label: Dog 0.6, Cat 0.4 Figure 1: A comparison between MixUp (54), CutOut (7), and CutMix (53). Images taken from 53. Reliability. A model’s reliability encompasses its calibration and uncertainty quality. The former describes how well the confidences of the predicted class reflect the true likelihood of correctness, most commonly measured by the Expected Calibration Error (ECE) (11). A model’s uncertainty quality describes its ability to align the entire softmax output with predictive ambiguities and errors (33). Despite impressive predictive capabilities, neural networks are known to suffer from poor calibration (11; 49; 48). In light of this, 11 have introduced temperature scaling as a straightforward, yet effective post-hoc calibration method. Because of its simplicity and non-invasiveness it is still widely used as a baseline for numerous alternative approaches (18; 35; 15; 8; 38). Besides calibration, a variety of techniques have been proposed to estimate high-quality uncertainties in deep learning models (29; 9; 19; 45; 46; 26; 34; 1; 23; 21). Unfortunately, however, all of these either introduce technical complexity or induce high computational cost, making them impractical for real-world applications like autonomous driving (32). Robustness. Whilst all of the previously mentioned methods offer an effective way of enhancing reliability in in-domain settings, there is no guarantee of the generalizability to out-of-domain scenes. In fact, multiple prior works have found that reliability deteriorates significantly under domain shifts (37; 6; 20). Consequently, model robustness, i.e., its ability to remain effective under perturbations, noise, or distribution shifts, is attracting increasing attention. Prior work discerns between robustness to perturbations (Gaussian noise, blur, occlusion), adversarial attacks (imperceptible changes to an image crafted to induce failure), and natural domain shifts (variations in weather, lighting, or geographic context) (12; 16; 17; 10; 13; 39; 41; 43; 14; 42; 47). Research Gap. By virtue of the importance of this topic, we are not the first to analyze reliability and robustness of deep learning models in the context of semantic segmentation (2; 16; 17; 55; 6; 27; 56). However, a recent study by 22 revealed a critical blind spot, showing that state-of-the-art semi-supervised segmentation methods severely deteriorate the reliability of neural networks. A core component in these methods is CutMix (53), which has been shown to have provable benefit for feature learning and improved reliability and robustness in classification tasks (36; 40; 53). At the same time, there is no clear answer as to how CutMix impacts reliability and robustness in dense prediction models, leaving a gap that this work aims to address. 3 Experimantal Setup The following describes the methodological background of this study, our training configurations, evaluation metrics, and datasets used to analyze the impact of CutMix on reliability and robustness in semantic segmentation. 3.1 CutMix CutMix (53) is highly inspired by both MixUp (54) and CutOut (7), as shown by Figure 1. The former blends a pair of images and labels based on convex combinations, while the latter randomly masks out regions of the input to regularize the model. CutMix combines both of these ideas by randomly cutting and pasting patches among training images while proportionally mixing the ground truth labels, as shown by Figure 2. Formally, a CutMix training sample (x~,y~)( x, y) can be defined as x~=M⊙xA+(1−M)⊙xB, x=M x_A+(1-M) x_B 5.0pt, (1) y~=λyA+(1−λ)yB, y=λ y_A+(1-λ)y_B 5.0pt, (2) where xAx_A and xBx_B denote two input images, and yAy_A and yBy_B their corresponding one-hot encoded labels. The binary mask M∈0,1HxWM∈\0,1\^HxW specifies a randomly sampled rectangular region within the image, determining which pixels are taken from image A and which from image B. The mixing ratio λ represents the proportion of pixels retained from xAx_A and is sampled from the beta distribution. With respect to the hyperparameters, we follow the original implementation (53). (a) Input Image (b) Ground Truth Figure 2: CutMix training example on the Cityscapes dataset (5). For visualization purposes, we have highlighted the inserted area with a red rectangle. 3.2 Training Configuration All experiments were conducted using two widely adopted semantic segmentation architectures: DeepLabV3+ (DLV3+) (4) and SegFormer (50). Both models were trained for 250 epochs without early stopping and with a batch size of 8 using a polynomial learning rate schedule. The initial learning rates and weight decay were selected following the recommended configurations for each architecture, as summarized by Table 1. All models were optimized using the standard pixel-wise cross-entropy loss and AdamW optimizer (28) with standard momentum parameters. These consistent hyperparameter and augmentation settings were chosen to ensure comparability between architectures and to isolate the specific effects of CutMix on reliability and robustness – both for Convolutional Neural Networks and Vision Transformers. Model Learning Rate (LR) Weight Decay Epochs Batch Size LR Schedule DeepLabV3+ 1×10−41× 10^-4 (×10 Decoder) 1×10−41× 10^-4 250 8 Polynomial SegFormer 5×10−65× 10^-6 (×10 Decoder) 1×10−21× 10^-2 Table 1: Training hyperparameters for DeepLabV3+ and SegFormer. 3.3 Metrics We evaluate model performance using a combination of accuracy, calibration, and uncertainty-based metrics. The mean Intersection over Union (mIoU) (24) serves as the primary measure of segmentation accuracy. Model calibration is assessed by the Expected Calibration Error (ECE) (11), which quantifies the discrepancy between predictive confidence and empirical accuracy. To capture the quality of uncertainty estimates, we first compute the pixel-wise predictive entropy (44): H(x)=−∑c=1Cp(y^c(x))logp(y^c(x)),H(x)=- _c=1^Cp( y_c(x)) p( y_c(x)) 5.0pt, (3) where p(y^c(x))p( y_c(x)) denotes the predicted softmax probability for class c given an input image x. These entropy values are then used to classify pixels as “certain” or “uncertain” based on the median uncertainty within each image, which was found to be the best default threshold (20). The conditional metrics p(acc||cer) and p(unc||inacc) proposed by 33 then quantify how often predictions are correct when marked certain, and incorrect when marked uncertain, respectively. The Reliable Segmentation Score (RSS) (20) integrates all previous complementary metrics into a single, holistic reliability measure using the harmonic mean: RSS=∑ωiω1mIoU+ω2(1−ECE)+ω3p(acc|cer)+ω3p(unc|inacc),RSS= Σ _i _1mIoU+ _2(1-ECE)+ _3p(acc$|$cer)+ _3p(unc$|$inacc) 5.0pt, (4) where the application-specific weights ωi _i are all set to 1.01.0 to avoid assumptions about the importance of any one metric. By leveraging the harmonic mean, RSS penalizes poor performance in any aspect, ensuring that a model achieves a high score only if it is accurate, well-calibrated, and uncertainty-aware. The combination of these metrics is justified by their largely orthogonal contributions (20). While the mIoU captures pixel-wise accuracy across classes, the ECE measures calibration, i.e., the alignment between predicted confidence and true likelihood of correctness, which can vary independently of mIoU. p(acc||cer) evaluates whether low-uncertainty predictions are indeed correct, highlighting the usefulness of certainty. p(unc||inacc) assesses whether false predictions are flagged as uncertain, enabling error mitigation. Besides, unlike mIoU and ECE, which rely solely on the maximum softmax probability and therefore capture information about the predicted class only, the conditional uncertainty metrics consider the full softmax distribution (via entropy, see Eq. 3), providing insights into prediction ambiguity. 3.4 Datasets To evaluate the impact of CutMix training for real-world scenarios, we train all models on Cityscapes (5). For out-of-domain analyses, we use the Foggy Cityscapes (42) validation sets without re-training the models. Foggy Cityscapes provides three versions defined by the attenuation coefficient β, where higher values correspond to denser fog. This setup enables a systematic assessment of model reliability and robustness under progressively challenging visual degradations. In terms of data augmentations, we employed random scaling, horizontal flipping, and random cropping as a baseline for all models. When evaluating the effect of CutMix, we applied it with a probability of 50%, following the original formulation (53). 4 Results In-Domain Evaluation. Table 2 summarizes the in-domain results on Cityscapes for different model backbones, comparing standard training and CutMix-augmented variants. As expected, larger backbones achieve higher segmentation accuracy (mIoU), while calibration (ECE) remains relatively stable across architectures. Interestingly, uncertainty quality – as measured by p(acc||cer) and p(unc||inacc) – appears largely independent of backbone size, with the CNN-based DeepLabV3+ slightly outperforming the Vision Transformer-based SegFormer models. Runtime scales predictably with model complexity, from ≈ 25 ms (40 FPS) for DeepLabV3+ (ResNet-34) to ≈ 315 ms (3 FPS) for SegFormer (B5). While the transformer-based SegFormer models offer competitive accuracy, they come at a considerably higher computational cost, emphasizing the continued efficiency advantage of CNN-based designs, particularly for images with high resolutions. Overall, CutMix has a minor effect on segmentation accuracy and calibration, improving mIoU in three out of six cases and leaving ECE nearly unchanged. However, it consistently enhances uncertainty quality, particularly in p(unc||inacc), where most models show notable gains. The only exception is the largest SegFormer (MiT-B5), whose reported checkpoint underperformed despite showing better averages across training epochs, clearly a case of suboptimal checkpoint selection. However, to ensure consistency, we retained this checkpoint rather than retraining or using early stopping. When aggregating evaluation results with the Reliable Segmentation Score (RSS), CutMix-trained models outperform their counterparts in all of the remaining models, indicating that CutMix substantially improves reliability. Surprisingly, the older CNN-based DeepLabV3+ architecture remains more reliable overall, surpassing the modern transformer-based models in terms of uncertainty quality. Encoder Params CutMix mIoU ↑ ECE ↓ p(acc|cer)p(acc|cer) ↑ p(unc|inacc)p(unc|inacc) ↑ RSS ↑ Inference Time [ms] ↓ FPS ↑ DeepLabV3+ RN34 ∼ 21M 0.743 0.033 0.912 0.731 0.826 24.81 ± 8.50 40.31 ✓ 0.743 0.027 0.943 0.838 0.864 RN101 ∼ 42M 0.774 0.034 0.931 0.797 0.859 49.72 ± 11.01 20.11 ✓ 0.754 0.033 0.947 0.848 0.870 RN152 ∼ 58M 0.762 0.034 0.927 0.777 0.849 57.31 ± 15.43 22.88 ✓ 0.774 0.034 0.939 0.817 0.867 SegFormer MiT-B0 ∼ 3.7M 0.658 0.034 0.899 0.713 0.789 43.71 ± 0.24 22.88 ✓ 0.688 0.034 0.930 0.807 0.833 MiT-B3 ∼ 45M 0.759 0.032 0.906 0.707 0.822 181.11 ± 0.71 5.52 ✓ 0.771 0.034 0.921 0.758 0.844 MiT-B5 ∼ 82M 0.788 0.027 0.921 0.766 0.853 315.50 ± 37.27 3.17 ✓ 0.773 0.033 0.909 0.721 0.831 Table 2: In-domain evaluation results on the Cityscapes validation dataset using DeepLabV3+ and SegFormer models with different backbone configurations. The reported inference times and frames per second (FPS) correspond to single-image forward passes performed at the native Cityscapes resolution (1024×2048) without any inference-time optimizations such as mixed precision or batching. All measurements were conducted on a single NVIDIA A100 GPU to ensure a consistent and comparable runtime evaluation across model architectures. Encoder CutMix mIoU ↑ ECE ↓ p(acc|cer)p(acc|cer) ↑ p(unc|inacc)p(unc|inacc) ↑ RSS ↑ Fog1 Fog2 Fog3 Fog1 Fog2 Fog3 Fog1 Fog2 Fog3 Fog1 Fog2 Fog3 Fog1 Fog2 Fog3 DeepLabV3+ RN34 0.702 0.651 0.568 0.041 0.067 0.092 0.907 0.901 0.886 0.741 0.753 0.753 0.813 0.793 0.752 ✓ 0.701 0.654 0.567 0.038 0.068 0.069 0.941 0.934 0.913 0.851 0.851 0.845 0.850 0.825 0.782 RN101 0.734 0.682 0.585 0.046 0.072 0.107 0.927 0.920 0.903 0.803 0.810 0.821 0.845 0.822 0.776 ✓ 0.726 0.675 0.565 0.038 0.076 0.081 0.941 0.932 0.909 0.845 0.842 0.836 0.858 0.829 0.776 RN152 0.726 0.679 0.573 0.044 0.068 0.120 0.921 0.908 0.890 0.782 0.773 0.798 0.835 0.810 0.761 ✓ 0.740 0.696 0.605 0.033 0.058 0.074 0.936 0.928 0.911 0.823 0.816 0.818 0.857 0.833 0.792 SegFormer MiT-B0 0.616 0.566 0.470 0.060 0.043 0.036 0.918 0.914 0.890 0.801 0.819 0.819 0.796 0.780 0.726 ✓ 0.640 0.578 0.467 0.048 0.031 0.022 0.936 0.927 0.890 0.852 0.857 0.836 0.824 0.799 0.729 MiT-B3 0.735 0.706 0.641 0.040 0.055 0.090 0.917 0.915 0.901 0.764 0.778 0.785 0.833 0.824 0.793 ✓ 0.748 0.716 0.642 0.035 0.052 0.083 0.923 0.919 0.906 0.785 0.794 0.800 0.846 0.833 0.800 MiT-B5 0.763 0.737 0.675 0.037 0.057 0.085 0.931 0.930 0.920 0.814 0.825 0.829 0.860 0.850 0.822 ✓ 0.760 0.730 0.667 0.030 0.046 0.062 0.913 0.912 0.903 0.755 0.770 0.781 0.839 0.831 0.808 Table 3: Out-of-domain evaluation on the Foggy Cityscapes validation sets (β=0.005β=0.005, β=0.01β=0.01, β=0.2β=0.2) using various backbone sizes for DeepLabV3+ and SegFormer. Models are trained on Cityscapes and tested without re-training, allowing assessment of reliability and robustness under increasing fog density. Out-of-Domain Evaluation. The results in Table 3 show the robustness of all models under increasing fog intensities on Foggy Cityscapes. As expected, segmentation performance (mIoU) consistently declines with stronger fog due to the growing domain gap. While CutMix has limited impact on segmentation robustness, it helps to maintain better calibration (ECE) and uncertainty quality (p(acc||cer) and p(unc||inacc)) across most configurations. Notably, SegFormer shows higher robustness than DeepLabV3+, particularly with larger backbones (MiT-B3 and MiT-B5), where both mIoU and calibration degrade less severely under adverse conditions. This trend suggests that transformer-based architectures generalize more gracefully across domain shifts compared to CNN-based ones. Overall, the RSS confirms that CutMix generally leads to more reliable calibration and uncertainty estimates across varying fog levels, even when segmentation accuracy itself remains mostly unchanged. These findings indicate that CutMix primarily enhances overall reliability and robustness rather than improving raw segmentation performance. Qualitative Evaluation. Figure 3 compares qualitative results for DeepLabV3+ (RN34) with and without CutMix on Cityscapes and for SegFormer (MiT-B3) on Foggy Cityscapes. Across both architectures and datasets, segmentation predictions remain visually similar, confirming that CutMix does not substantially change the model’s class assignments. However, the CutMix-augmented variants exhibit higher uncertainty in regions corresponding to erroneous or ambiguous predictions, as highlighted by the red rectangles in the uncertainty maps. Overall, these qualitative observations corroborate the quantitative findings: while CutMix has limited effect on segmentation accuracy, it consistently improves the reliability, even under adverse conditions, ultimately making models more robustness as well. Figure 3: Qualitative examples of DeepLabV3+ (RN34) with and without CutMix on Cityscapes, and SegFormer (MiT-B3) on Foggy Cityscapes. The accuracy maps highlight correct predictions in green, incorrect ones in red, and classes ignored during training in gray. 5 Conclusion This study systematically investigated the impact of CutMix on the accuracy, reliability, and robustness of semantic segmentation models. While CutMix is widely adopted, its effect on reliability and robustness in dense prediction tasks had not been considered yet. This is especially critical in light of recent findings by 22, which revealed that semi-supervised semantic segmentation frameworks – which use CutMix as a core component – severely deteriorate the reliability. By isolating its effects from other components, we evaluated the influence of CutMix across in-domain and out-of-domain scenarios on two representative architectures: the CNN-based DeepLabV3+ and the transformer-based SegFormer. Our results reveal that CutMix exerts only a minor influence on segmentation accuracy and calibration but consistently improves uncertainty quality. These improvements persist under domain shifts, where CutMix-trained models demonstrate not only more reliable uncertainty estimates but also better calibration despite similar segmentation performance. In other words, our findings show that CutMix primarily enhances how models express their uncertainty rather than what they predict – a crucial distinction for safety-critical applications. This suggests that the reliability deterioration observed in semi-supervised segmentation frameworks (22) cannot be attributed to CutMix itself, but rather to other components such as pseudo-labeling or consistency regularization. Future work should investigate these interactions and test whether CutMix offers a general mechanism for enhancing reliability and robustness across tasks and modalities by extending evaluations to other domains, such as medical imaging or remote sensing. References Amini et al. (2020) A. Amini, W. Schwarting, A. Soleimany, and D. Rus Deep evidential regression. Advances in neural information processing systems 33, p. 14927–14937. Cited by: §2. Arnab et al. (2018) A. Arnab, O. Miksik, and P. H. Torr On the robustness of semantic segmentation models to adversarial attacks. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 888–897. Cited by: §2. Azad et al. (2024) R. Azad, E. K. Aghdam, A. Rauland, Y. Jia, A. H. Avval, A. Bozorgpour, S. Karimijafarbigloo, J. P. Cohen, E. Adeli, and D. Merhof Medical image segmentation review: the success of u-net. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §1. Chen et al. (2018) L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Cited by: §3.2. Cordts et al. (2016) M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 3213–3223. Cited by: Figure 2, Figure 2, §3.4. De Jorge et al. (2023) P. De Jorge, R. Volpi, P. H. Torr, and G. Rogez Reliability in semantic segmentation: are we on the right track?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7173–7182. Cited by: §2, §2. DeVries and Taylor (2017) T. DeVries and G. W. Taylor Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552. Cited by: Figure 1, Figure 1, §3.1. Ding et al. (2021) Z. Ding, X. Han, P. Liu, and M. Niethammer Local temperature scaling for probability calibration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 6889–6899. Cited by: §2. Gal and Ghahramani (2016) Y. Gal and Z. Ghahramani Dropout as a bayesian approximation: representing model uncertainty in deep learning. In international conference on machine learning, p. 1050–1059. Cited by: §2. Goodfellow et al. (2014) I. J. Goodfellow, J. Shlens, and C. Szegedy Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. Cited by: §2. Guo et al. (2017) C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger On calibration of modern neural networks. In International Conference on Machine Learning, p. 1321–1330. Cited by: §2, §2, §3.3. Hendrycks and Dietterich (2019) D. Hendrycks and T. Dietterich Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261. Cited by: §2. Hendrycks et al. (2021) D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 15262–15271. Cited by: §2. Hu et al. (2019) X. Hu, C. Fu, L. Zhu, and P. Heng Depth-attentional features for single-image rain removal. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, p. 8022–8031. Cited by: §2. Ji et al. (2019) B. Ji, H. Jung, J. Yoon, K. Kim, et al. Bin-wise temperature scaling (bts): improvement in confidence calibration performance through simple scaling techniques. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), p. 4190–4196. Cited by: §2. Kamann and Rother (2020) C. Kamann and C. Rother Benchmarking the robustness of semantic segmentation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 8828–8838. Cited by: §2, §2. Kamann and Rother (2021) C. Kamann and C. Rother Benchmarking the robustness of semantic segmentation models with respect to common corruptions. International journal of computer vision 129 (2), p. 462–483. Cited by: §2, §2. Kull et al. (2019) M. Kull, M. Perello Nieto, M. Kängsepp, T. Silva Filho, H. Song, and P. Flach Beyond temperature scaling: obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in neural information processing systems 32. Cited by: §2. Lakshminarayanan et al. (2017) B. Lakshminarayanan, A. Pritzel, and C. Blundell Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30. Cited by: §2. Landgraf et al. (2025a) S. Landgraf, M. Hillemann, T. Kapler, and M. Ulrich A comparative study on multi-task uncertainty quantification in semantic segmentation and monocular depth estimation. tm-Technisches Messen. Cited by: §2, §3.3, §3.3, §3.3. Landgraf et al. (2026) S. Landgraf, M. Hillemann, T. Kapler, and M. Ulrich EMUFormer: efficient multi-task uncertainties for reliable joint semantic segmentation and monocular depth estimation. International Journal of Computer Vision 134 (4), p. 142. Cited by: §2. Landgraf et al. (2025b) S. Landgraf, M. Hillemann, and M. Ulrich Rethinking semi-supervised segmentation beyond accuracy: reliability and robustness. In DAGM German Conference on Pattern Recognition, p. 434–452. Cited by: §1, §2, §5, §5. Landgraf et al. (2024) S. Landgraf, K. Wursthorn, M. Hillemann, and M. Ulrich Dudes: deep uncertainty distillation using ensembles for semantic segmentation. PFG–Journal of Photogrammetry, Remote Sensing and Geoinformation Science 92 (2), p. 101–114. Cited by: §2. Lateef and Ruichek (2019) F. Lateef and Y. Ruichek Survey on semantic segmentation using deep learning techniques. Neurocomputing 338, p. 321–348. Cited by: §3.3. Li et al. (2024) J. Li, Y. Cai, Q. Li, M. Kou, and T. Zhang A review of remote sensing image segmentation by deep learning methods. International Journal of Digital Earth 17 (1), p. 2328827. Cited by: §1. Liu et al. (2020) J. Liu, Z. Lin, S. Padhy, D. Tran, T. Bedrax Weiss, and B. Lakshminarayanan Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in neural information processing systems 33, p. 7498–7512. Cited by: §2. Loiseau et al. (2024) T. Loiseau, T. Vu, M. Chen, P. Pérez, and M. Cord Reliability in semantic segmentation: can we use synthetic data?. In European Conference on Computer Vision, p. 442–459. Cited by: §2. Loshchilov and Hutter (2017) I. Loshchilov and F. Hutter Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §3.2. MacKay (1992) D. J. MacKay A practical bayesian framework for backpropagation networks. Neural computation 4 (3), p. 448–472. Cited by: §2. Minaee et al. (2021) S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos Image segmentation using deep learning: a survey. IEEE transactions on pattern analysis and machine intelligence 44 (7), p. 3523–3542. Cited by: §1. Muhammad et al. (2022) K. Muhammad, T. Hussain, H. Ullah, J. Del Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V. H. C. De Albuquerque Vision-based semantic segmentation in scene understanding for autonomous driving: recent achievements, challenges, and outlooks. IEEE Transactions on Intelligent Transportation Systems 23 (12), p. 22694–22715. Cited by: §1. Muhammad et al. (2020) K. Muhammad, A. Ullah, J. Lloret, J. Del Ser, and V. H. C. De Albuquerque Deep learning for safe autonomous driving: current challenges and future directions. IEEE Transactions on Intelligent Transportation Systems 22 (7), p. 4316–4336. Cited by: §2. Mukhoti and Gal (2018) J. Mukhoti and Y. Gal Evaluating bayesian deep learning methods for semantic segmentation. arXiv preprint arXiv:1811.12709. Cited by: §2, §3.3. Mukhoti et al. (2023) J. Mukhoti, A. Kirsch, J. Van Amersfoort, P. H. Torr, and Y. Gal Deep deterministic uncertainty: a new simple baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 24384–24394. Cited by: §2. Naeini et al. (2015) M. P. Naeini, G. Cooper, and M. Hauskrecht Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, Vol. 29. Cited by: §2. Oh and Yun (2024) J. Oh and C. Yun Provable benefit of cutout and cutmix for feature learning. Advances in Neural Information Processing Systems 37, p. 114656–114743. Cited by: §1, §2. Ovadia et al. (2019) Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and J. Snoek Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems 32. Cited by: §2. Patra et al. (2023) R. Patra, R. Hebbalaguppe, T. Dash, G. Shroff, and L. Vig Calibrating deep neural networks using explicit regularisation and dynamic data pruning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 1541–1549. Cited by: §2. Pedraza et al. (2022) A. Pedraza, O. Deniz, and G. Bueno Really natural adversarial examples. International Journal of Machine Learning and Cybernetics 13 (4), p. 1065–1077. Cited by: §2. Rao et al. (2023) A. Rao, J. Lee, and O. Aalami Studying the impact of augmentations on medical confidence calibration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 2462–2472. Cited by: §1, §2. Recht et al. (2019) B. Recht, R. Roelofs, L. Schmidt, and V. Shankar Do imagenet classifiers generalize to imagenet?. In International conference on machine learning, p. 5389–5400. Cited by: §2. Sakaridis et al. (2018) C. Sakaridis, D. Dai, and L. Van Gool Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision 126, p. 973–992. Cited by: §2, §3.4. Sakaridis et al. (2021) C. Sakaridis, D. Dai, and L. Van Gool ACDC: the adverse conditions dataset with correspondences for semantic driving scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, p. 10765–10775. Cited by: §2. Shannon (1948) C. E. Shannon A mathematical theory of communication. The Bell system technical journal 27 (3), p. 379–423. Cited by: §3.3. Valdenegro-Toro (2023) M. Valdenegro-Toro Sub-ensembles for fast uncertainty estimation in neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4119–4127. Cited by: §2. Van Amersfoort et al. (2020) J. Van Amersfoort, L. Smith, Y. W. Teh, and Y. Gal Uncertainty estimation using a single deep deterministic neural network. In International conference on machine learning, p. 9690–9700. Cited by: §2. Varma et al. (2019) G. Varma, A. Subramanian, A. Namboodiri, M. Chandraker, and C. Jawahar IDD: a dataset for exploring problems of autonomous navigation in unconstrained environments. In 2019 IEEE winter conference on applications of computer vision (WACV), p. 1743–1751. Cited by: §2. Wang et al. (2021) D. Wang, L. Feng, and M. Zhang Rethinking calibration of deep neural networks: do not be afraid of overconfidence. Advances in Neural Information Processing Systems 34, p. 11809–11820. Cited by: §2. Wilson and Izmailov (2020) A. G. Wilson and P. Izmailov Bayesian deep learning and a probabilistic perspective of generalization. Advances in Neural Information Processing Systems 33, p. 4697–4708. Cited by: §2. Xie et al. (2021) E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo SegFormer: simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems 34, p. 12077–12090. Cited by: §3.2. Yang et al. (2023) L. Yang, L. Qi, L. Feng, W. Zhang, and Y. Shi Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 7236–7246. Cited by: §1. Yang et al. (2025) L. Yang, Z. Zhao, and H. Zhao Unimatch v2: pushing the limit of semi-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §1. Yun et al. (2019) S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo Cutmix: regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, p. 6023–6032. Cited by: §1, Figure 1, Figure 1, §2, §3.1, §3.1, §3.4. Zhang et al. (2017) H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz Mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412. Cited by: Figure 1, Figure 1, §3.1. Zhou et al. (2022) D. Zhou, Z. Yu, E. Xie, C. Xiao, A. Anandkumar, J. Feng, and J. M. Alvarez Understanding the robustness in vision transformers. In International conference on machine learning, p. 27378–27394. Cited by: §2. Zhou et al. (2019) W. Zhou, J. S. Berrio, S. Worrall, and E. Nebot Automated evaluation of semantic segmentation robustness for autonomous driving. IEEE Transactions on Intelligent Transportation Systems 21 (5), p. 1951–1963. Cited by: §2.