Paper deep dive
Beyond scalar losses: calibrating segmentation models via gradient vector field surgery
Laurin Lux, Alexander H. Berger, Moritz Knolle, Daniel Rückert, Johannes C. Paetzold
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/18/2026, 2:15:43 AM
Summary
The paper addresses the miscalibration and overconfidence inherent in region-based loss functions like Dice loss for medical image segmentation. It proposes a novel gradient vector field surgery that scales the gradient's magnitude linearly with prediction error, effectively mitigating calibration issues while preserving high segmentation accuracy across 2D and 3D medical imaging tasks.
Entities (8)
Relation Signals (7)
Gradient vector field surgery → mitigates → model miscalibration
confidence 96% · We propose a "surgery" on the gradient vector field as a simple, yet effective intervention to mitigate calibration issues.
Dice Loss → causes → model miscalibration
confidence 95% · models trained using region-based loss functions are notoriously miscalibrated and typically yield over-confident predictions.
Gradient vector field surgery → scales → gradient magnitude
confidence 94% · This surgery adds a factor to the loss's partial derivative, scaling the gradient's magnitude linearly with the prediction error.
Proposed Method → improves → calibration accuracy
confidence 93% · we show that our proposed method improves model calibration while maintaining high segmentation performance.
experiments → evaluatedon → INbreast dataset
confidence 90% · We compare our custom vector field adaptations... on the INbreast dataset
experiments → evaluatedon → BraTS-METS dataset
confidence 90% · trained using each loss on the BraTS-METS dataset
experiments → evaluatedon → KiTS Dataset
confidence 90% · and the KiTS dataset
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Region-based loss functions, such as the Dice loss, have established themselves as the de facto standard for highly class- and region-imbalanced segmentation tasks. However, models trained using region-based loss functions are notoriously miscalibrated and typically yield over-confident predictions. In medical imaging applications, such as defining tumor resection margins, this miscalibration is hindering clinical adoption. In this work, we outline a novel gradient perspective on this overconfidence and show how it affects region-based loss functions. We propose a "surgery" on the gradient vector field as a simple, yet effective intervention to mitigate calibration issues. This surgery adds a factor to the loss's partial derivative, scaling the gradient's magnitude linearly with the prediction error. In empirical evaluations across 2D and 3D medical segmentation tasks, we demonstrate the effectiveness of this intervention while maintaining high prediction accuracy when used in conjunction with any region-based loss function.
Tags
Links
- Source: https://arxiv.org/abs/2607.14338v1
- Canonical: https://arxiv.org/abs/2607.14338v1
Trouble viewing inline? Open PDF directly →
Full Text
60,253 characters extracted from source content.
Expand or collapse full text
Proceedings of Machine Learning Research 315:3397–3423, 2026Medical Imaging with Deep Learning Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Laurin Lux 1,2,3 laurin.lux@tum.de Alexander H. Berger 1,3,4 a.berger@tum.de Moritz Knolle 3 moritz.knolle@tum.de Daniel R ̈uckert 1,2,4,6 daniel.rueckert@tum.de Johannes C. Paetzold 4,5 jpaetzold@med.cornell.edu 1 School of Computation, Information and Technology, TUM, Munich, Germany 2 Munich Center for Machine Learning, Munich, Germany 3 Department of Radiology, Weill Cornell Medicine, New York City, USA 4 School of Medicine and Health, TUM University Hospital, Munich, Germany 5 Cornell Tech, New York City, USA 6 Department of Computing, Imperial College London, London, UK Editors: Yuankai Huo, Mingchen Gao, Chang-Fu Kuo, Yueming Jin, Ruining Deng Abstract Region-based loss functions, such as the Dice loss, have established themselves as the de facto standard for highly class- and region-imbalanced segmentation tasks. However, models trained using region-based loss functions are notoriously miscalibrated and typically yield over-confident predictions. In medical imaging applications, such as defining tumor resection margins, this miscalibration is hindering clinical adoption. In this work, we outline a novel gradient perspective on this overconfidence and show how it affects region-based loss functions. We propose a ”surgery” on the gradient vector field as a simple, yet effective intervention to mitigate calibration issues. This surgery adds a factor to the loss’s partial derivative, scaling the gradient’s magnitude linearly with the prediction error. In empirical evaluations across 2D and 3D medical segmentation tasks, we demonstrate the effectiveness of this intervention while maintaining high prediction accuracy when used in conjunction with any region-based loss function. Keywords: Segmentation, Calibration, Optimization, Gradient Surgery, Metastases 1 1. Introduction The Dice Similarity Coefficient (DSC) has become a primary evaluation metric and loss function in medical image segmentation. Originally adapted for volumetric segmentation (Milletari et al. (2016)), the Dice loss and its derivatives (e.g. (Salehi et al., 2017; Taghanaki et al., 2019) excel in scenarios of extreme class imbalance—a ubiquitous challenge in medical imaging where foreground structures (e.g., lesions or vessel fragments) occupy negligible fractions of the image volume. By directly optimizing a continuous approximation of the 1. Published in Proceedings of the 9th International Conference on Medical Imaging with Deep Learning, PMLR 315:3397–3423, 2026. https://proceedings.mlr.press/v315/lux26a.html © 2026 C-BY 4.0, L. Lux, A.H. Berger, M. Knolle, D. R ̈uckert & J.C. Paetzold. arXiv:2607.14338v1 [cs.CV] 15 Jul 2026 Lux Berger Knolle R ̈ uckert Paetzold region overlap between predictions and ground truth, the Dice loss circumvents the local minima often encountered when training voxel-wise objectives on highly imbalanced data. However, this robustness to class imbalance comes at a cost. The Dice loss cannot inherently enforce probabilistic consistency with the underlying data-generation process, unlike e.g. the Cross-Entropy (CE) loss, which corresponds directly to a proper scoring rule (Gneiting and Raftery, 2007). Instead, models trained with Dice loss exhibit patho- logical overconfidence, pushing softmax probabilities toward 0 or 1 regardless of the actual epistemic uncertainty. This creates a significant dichotomy for clinical model development. In high-stakes workflows, such as defining tumor resection margins or radiotherapy tar- get volumes, a segmentation map is not merely a binary mask but a decision boundary. Well-calibrated predictions enable meaningful verification and the imperative possibility of adapting outputs to high-recall or high-sensitivity solutions (Sander et al., 2019; Jiang et al., 2012). In this work, we analyze partial derivatives w.r.t. the logits, influencing the gradient on the network’s weights, to identify the root cause of miscalibration inherent to all region- based segmentation losses. We show that the gradient dynamics of these losses effectively neglect the calibration of the predicted probabilities and only optimize for region overlap between predictions and ground truth. To mitigate this issue, we propose a gradient surgery, a simple yet effective intervention (surgery) on the gradient vector field of the model’s voxel- wise logit outputs. Given a network’s predicted probability p, this intervention rescales the loss’s partial derivative w.r.t. single pixel logits such that the error |y − p| has a linear influence. In extensive empirical experiments on 2D and 3D medical segmentation tasks, we show that our proposed method improves model calibration while maintaining high segmentation performance. 2. Related work Seminal works by Mehrtash et al. (2020); Bertels et al. (2019); Sander et al. (2019) demon- strate that segmentation models trained with Dice loss provide miscalibrated, overconfident predictions and thus questioned their clinical applicability. Initial mitigation strategies in- cluded model ensembles to improve the calibration of such region-based losses (Mehrtash et al., 2020). Other common strategies involve compound objectives, such as the Combo Loss (Taghanaki et al., 2019) or Unified Focal Loss (Yeung et al., 2022), which compute a weighted sum of Dice and CE (or Focal) terms. While these stabilize training, they often require extensive tuning of the weighting hyperparameter λ and represent a compromise rather than a theoretical fix for the miscalibration. The marginal L1 average calibration error (mL1-ACE) was recently proposed as an auxiliary loss that is specifically targeted at improving voxel-wise calibration (Barfoot et al., 2024). Another focus was the direct adaptation of region-based losses. The Tversky Loss (Salehi et al., 2017) generalizes the Dice coefficient to allow for individual weighting of false positives and false negatives, which impacts precision and recall but does not explicitly address probabilistic calibration. More recently, DSC++ (Yeung et al., 2023) introduced an exponent γ > 1 to the Dice formula- tion to selectively penalize overconfident, incorrect predictions. While the focal γ results in improved calibration, it can drastically change the gradient dynamics compared to the Dice loss through the down-weighting of samples with large fractions of false positives and false 3398 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery negatives (see Appendix E). An alternative to modifying the primary loss is post-hoc recal- ibration (Rousseau et al., 2021). Techniques such as temperature scaling (Guo et al., 2017), Platt scaling (Platt et al., 1999), and isotonic regression map (Zadrozny and Elkan, 2002) model outputs to calibrated probabilities after training. While effective on in-distribution data, these methods do not improve the quality of the learned feature representation and are known to degrade under the domain shifts common in medical deployment. Other works (Islam and Glocker, 2021; Murugesan et al., 2025, 2023a; Karani et al., 2023) have focused on specific solutions for the uncertainty specific to lesion boundaries that are inherent to the data annotation process. Spatially varying labels smoothing (SVLS) (Islam and Glocker, 2021) draws inspiration from label smoothing, specifically smoothing the voxels with varying neighbor annotations (i.e., boundary voxels). This method im- proves calibration for brain tumor, kidney tumor, long nodule, and prostate zone segmen- tation. Neighbor-Aware Calibration (NACL) (Murugesan et al., 2025) reformulates and extends SVLC by treating it as a neighborhood-aware penalty. Moreover, it applies a con- straint directly on the logits (Liu et al., 2022), effectively reducing their magnitude. The penalty formulation allows flexible weighting of the initial optimization objective with the neighborhood-aware logit distance constraint. Finally, boundary-weighted consistency regu- larization (BWCR) (Karani et al., 2023) forces logit consistency across corresponding pixels from different augmented versions of the same input. However, all of these methods are not specifically designed to address calibration issues in models trained with region-based losses. In contrast, our work specifically targets the overconfidence problem in region-based losses, aiming to improve performance when region-based losses are preferred over standard cross-entropy loss. 3. Gradient dynamics of region-based segmentation losses: analysis and intervention Below, we present a concise analysis of the Dice loss’s gradient dynamics alongside our proposed intervention that encourages calibrated predictions. We assume a binary seg- mentation problem using a final sigmoid activation function to turn logits into probability values. 3.1. Region-based losses converge to miscalibrated solutions The soft dice loss for a prediction/target mask pair P ∈ R N and Y ∈0, 1 N is defined as: DSC(P,Y ) = 1− 2 P N i=1 p i y i + ε P N i=1 p i + P N i=1 y i + ε = 1− 2I + ε P + Y + ε ,(1) where I is the intersection between P and Y . The partial derivative of the Dice loss w.r.t. the predicted probability p i for an individual input voxel i is: ∂L DSC ∂p i = 2y i (P + Y + ε)− (2I + ε) (P + Y + ε) 2 ,(2) which we will refer to as the “global” term G(i). Crucially, as with all region-based losses, G(i) can usually be approximated as a constant G because a single voxel has negligible 3399 Lux Berger Knolle R ̈ uckert Paetzold 0.25 0.50 0.75 δL δp i 0.0 0.1 0.2 δp δz i 0.0 0.1 Gradient dynamics push towards extremes δL δz i 0.00.51.0 p 0 20 40 0.00.51.0 p 0.0 0.1 0.2 0.00.51.0 p 0.0 0.5 1.0 Gradient dynamics are in equilibrium Absolute Value y= 1 y= 0 Figure 1: Partial derivatives of dice loss (top row) vs. cross entropy loss (bottom row) for single voxels. Sub-panels show the absolute value of: δL δp i , δp δz i , and δL δz i as a function of the predicted probability p for a foreground (y = 1, yellow) and a background (y = 0, purple) voxel. Red squares indicate intersection points where the magnitude of foreground and background derivatives is in equilibrium. For cross-entropy, the curves intersect at p = 0.5, encouraging uncertain predictions for indistinguishable voxel-representations with different labels. For the Dice loss, they intersect at p∈0, 1, effectively pushing all predicted probabilities to extreme values. influence on G as the image size N increases. Using sigmoid/softmax activation on the model’s output, the partial derivative of the loss w.r.t. a single voxel logit z i is: ∂L DSC ∂z i = ∂L DSC ∂p i · ∂p i ∂z i = G(i)· p i (1− p i ).(3) These partial derivatives have two undesirable properties, visualized in Figure 1. First, ∂L DSC /∂z i is maximized by uncertain voxels (p ≈ 0.5) while confident predictions (p ≈ 0 or p ≈ 1) have negligible influence on the gradient w.r.t. the network’s parameters. Importantly, this occurs independently of their correctness; i.e., confident but incorrect predictions do not contribute to the gradient w.r.t. the network weights, as shown in Figure 2. Second, the partial derivatives for foreground and background intersect only at the function’s boundaries, i.e., 0 and 1; see red squares in Figure 1, top row, which causes the network to converge to overconfident predictions. We make a simplified argument by considering a scenario where a network is trained to a point where it has exhausted its maximal discriminative capacities, i.e., there exist voxels a and b with different labels (y a = 1 and y b = 0) that are indistinguishable through the network’s latent representation l(x), i.e. l(a)≈ l(b). Therefore, the network is forced to output highly coupled probabilities for both voxels, i.e., p a ≈ p b . The described scenario naturally evolves when a network is trained 3400 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Dice Loss Fundus ImageProbability MapGT Overlay Ours Gradient w.r.t. to pixel logits Cross-Entropy Loss Figure 2: Visualization of the gradient w.r.t. the voxel-wise logits. Purple circles indicate confident errors, where gradients vanish through the activation function for Dice. towards convergence without reaching zero loss. In this scenario, a and b influence the gradient w.r.t. the network parameters in opposite directions through the opposing ground truth labels for these ”indistinguishable” voxels. For y a = 1, the network parameters are guided towards a higher probability, and for y b = 0 towards a lower probability. Therefore, the network’s weights converge to output the probability for l(a) and l(b) such that the influence of a and b on the gradient w.r.t. the network weights is in an equilibrium (red squares in Figure 1): ∂L ∂z a =− ∂L ∂z b (4) Formally, this equilibrium can be extended from a single pair of indistinguishable voxels v to sets of indistinguishable voxels S k := v | l(v) ≈ c k , where c k is the voxel set’s shared latent representation. As described above, the network predicts the same p k for all elements (voxels) in a set S k . In addition to p k , a set S k is characterized by its ratio between foreground and background labels r k = |S fg k | /|S k |, where S fg k := v ∈ S k |y(v) = 1 ⊂ S k is the subset of S k containing the voxels with ground truth label y = 1. These sets are naturally encountered when training on complete samples/batches consisting of a large number of voxels, whose influences accumulate, resulting in numerous different equilibrium probabilities that characterize the network’s calibration. 3401 Lux Berger Knolle R ̈ uckert Paetzold In the case of cross-entropy, the equilibrium for any set of indistinguishable voxels S k with foreground ratio r k is reached at p k = r k = |S 1 | /|S|. Note that this directly corresponds to perfect calibration, where the predicted probabilities correspond to the underlying data- generating distribution. (Guo et al., 2017). In the case of Dice, an equilibrium can only be reached for p a = p b = 0∨ p a = p b = 1, which is independent of the set’s label ratio and leads to overconfident predictions that are unrelated to the underlying data-generating distribution. 3.2. Combining calibration and region size imbalance awareness using gradient surgery We hypothesize that ideally, the partial derivatives w.r.t. the voxel logits respect (1) error magnitude to obtain equilibria resulting in calibrated probability outputs (similar to CE, see Figure 1), and (2) dynamic adaptation to drastic region size imbalance for overlap maximization. Suitable partial derivatives for foreground and background that fulfill these requirements are: ∂L ∂z fg i = (1− p i ) 2(P + Y + ε)− (2I + ε) (P + Y + ε) 2 = (1− p i ) G fg (i),(5) ∂L ∂z bg i =−p i (2I + ε) (P + Y + ε) 2 =−p i G bg (i),(6) respectively. Here, the magnitude scales linearly with the error while maintaining adap- tive, region-size-dependent foreground and background weighting. Notably, the global terms G fg and G bg are equal to the Dice formulation. Following the chain rule, where the partial derivative of the probability w.r.t. the logits is ∂p i /∂z i = (1−p i )∗p i , we would need a scalar loss that results in the following partial derivatives w.r.t. the single voxel probabilities: ∂L ∂p fg i = 1 p i 2(P + Y + ε)− (2I + ε) (P + Y + ε) 2 = 1 p i G fg (i),(7) ∂L ∂p bg i =− 1 (1− p i ) (2I + ε) (P + Y + ε) 2 =− 1 (1− p i ) G bg (i),(8) For these partial derivatives to form the gradient w.r.t. the logits ∇ z L for a scalar loss function L, we require symmetry of second derivatives, which is not guaranteed for all z i and z k as outlined in the proof in Appendix A. We identify this as the reason no loss with the desired partials was previously proposed. Instead of relying on a scalar loss, we define a vector field F (z) with F i (z) = p i (1− p i ) (y i p i + (1− y i )(1− p i )) 2y i (P + Y + ε)− (2I + ε) (P + Y + ε) 2 ,(9) that we use as an optimization objective for model training. Our gradient scale factor is p i when y i = 0 and 1− p i when y i = 1. This can be interpreted as either a ”region- imbalance” weighted cross-entropy gradient, or, as a linearly error-weighted dice gradient without the sigmoid derivative. Implementation details of the methods are described in Appendix D. 3402 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Empirically, we find that adding a relatively sharp decline near 0 and 1 results in higher performance. We add this sharp decline by multiplying by (1− (1−p) n ) and (1−p n ), where n regulates the steepness of the decline. For n → ∞ the function is essentially equivalent to |y− p| for 0 < p < 1, and 0 for p ∈ 0, 1. Effectively, for |y− p| values close to 0, this has similarity to label smoothing for cross-entropy loss (Szegedy et al., 2016; M ̈uller et al., 2019), by reducing the incentive of the model to push probabilities to maximal certainty. Symmetrically, for |y− p| close to 1, this can be interpreted as de-emphasizing extremely confident errors, potentially improving robustness against obvious cases of label noise. An ablation on the exponential n is displayed in section 4.2. Including the decline terms, the vector field is defined as: F i (z) = (1− (1− p) n )(1− p n ) p i (1− p i ) (y i p i + (1− y i )(1− p i )) 2y i (P + Y + ε)− (2I + ε) (P + Y + ε) 2 ,(10) 3.3. Vector field stability The non-existence of a scalar loss function yielding the desired partial derivatives implies that the logit vector field and, therefore, also the induced vector field on the network weights, is non-conservative. A non-zero curl of vector fields can result in difficulties during optimiza- tion that are well-studied, e.g., in the field of generative adversarial networks (Mescheder et al., 2017). However, the curl in our proposed vector field F (z) is negligible compared to the diagonal terms, which prevents the problematic ”orbiting” around solutions. A vi- sualization of F (z) compared to the gradient vector fields of other methods is displayed in Figure 4 in Appendix B. Moreover, F (z) provides favorable theoretical properties that make it suitable for model training: First, the proposed vector field F (z) is continuous and smooth on R ⋉ , since each component F i (z) is a smooth function, see Equation 10. Second, the vector field always points towards the ground truth solution g, since no sign flips occur for the componentsF i (z). These theoretical considerations, in conjunction with our empiri- cal evaluation in Section 4, showcase the proposed solution’s suitability for effective network training. Example training curves with different optimizers are displayed in Appendix C. 4. Experimentation and Results We compare our custom vector field adaptations for different region-based losses, including Dice, Tversky (Salehi et al., 2017), Combo loss (CE + Dice) (Taghanaki et al., 2019), m1L1- ACE (+Dice) loss, and Dice++ losses. Moreover, we include baselines employing spatially aware label smoothing (SVLS) (Islam and Glocker, 2021) and neighbor-aware calibration through penalty constraints (NACL (Murugesan et al., 2025)). Notably, these were not designed to work in conjunction with region-based losses (Murugesan et al., 2023b). Details on the hyperparameter settings for these losses are listed in Appendix C. We conduct a ran- dom hyperparameter search to find the optimal configuration for each setup with 25 and 10 runs for our 2D and 3D datasets, respectively. Implementation details for our optimization method are provided in the Appendix D. We use the UNet architecture (Ronneberger et al., 2015) with residual units (He et al., 2016) combined with heavy domain-specific augmenta- tions (Isensee et al., 2021). 3403 Lux Berger Knolle R ̈ uckert Paetzold Table 1: Average test set performance of the 5 best runs out of 25 (selected through valida- tion Dice scores) trained using each loss on the INbreast dataset (Moreira et al., 2012). Best results are displayed in bold, second-best results are underlined. Sig- nificantly better performance for standard losses vs. altered losses is highlighted through a ∗ using the 0.01 significance level. MethodNLL ↓ECE ↓MCE ↓Brier ↓DSC (%) ↑ INbreast Masses Cross Entropy0.0192±0.00350.0038±0.00150.1903±0.08720.0050±0.000866.21±3.57 NACL0.0208±0.00440.0038±0.00100.2390±0.10210.0050±0.000868.22±3.57 NACL + Dice0.0269±0.00750.0040±0.00070.3185±0.01210.0045±0.000772.44±2.62 SVLS0.0178±0.00270.0035±0.00100.2161±0.08160.0045±0.000668.35±2.90 SVLS + Dice0.0277±0.00490.0045±0.00080.2925±0.01380.0052±0.000868.76±2.70 Dice++0.0195±0.00240.0034±0.00090.1774±0.02440.0045±0.000767.51±3.17 CE + Dice0.0328±0.00670.0049±0.00070.3111±0.01300.0054±0.000671.76±2.10 CE + Dice - Surgery0.0153±0.00130.0023±0.00030.1568±0.00920.0039±0.0003 74.39±3.34 Dice0.0567±0.01600.0058±0.00060.3214±0.03240.0062±0.000564.93±3.31 Dice - Surgery0.0164 ∗ ±0.00420.0026 ∗ ±0.0010 0.1495 ∗ ±0.0112 0.0039 ∗ ±0.000974.34 ∗ ±2.77 Tversky0.0489±0.01140.0051±0.00120.3439±0.03720.0054±0.001267.63±2.90 Tversky - Surgery0.0187 ∗ ±0.00350.0030±0.00080.1650 ∗ ±0.01840.0047±0.000969.34±3.76 FIVES Cross Entropy0.0542±0.00110.0043±0.00010.0467±0.00150.0138±0.000287.54±0.22 NACL0.0536±0.00050.0043±0.00000.0478±0.00150.0138±0.000187.60±0.14 NACL + Dice0.0659±0.00230.0103±0.00050.1548±0.01050.0143±0.000287.94±0.11 Dice++0.0545±0.00060.0048±0.00020.0560±0.00310.0135±0.000187.97±0.07 Dice + m1L1-ACE0.0756±0.00200.0074±0.00030.0613±0.00540.0156±0.000485.96±0.45 CE + Dice0.0662±0.00130.0103±0.00020.1558±0.00470.0142±0.000287.99±0.16 CE + Dice - Surgery0.0533 ∗ ±0.00140.0045 ∗ ±0.00030.0486 ∗ ±0.00240.0136 ∗ ±0.000187.70±0.09 Dice0.3118±0.08640.0162±0.00060.3514±0.01440.0165±0.000588.03±0.25 Dice - Surgery0.0540 ∗ ±0.00130.0044 ∗ ±0.00030.0485 ∗ ±0.00320.0139 ∗ ±0.000187.60±0.12 Tversky0.2595±0.06800.0163±0.00010.3408±0.01270.0168±0.000287.77±0.21 Tversky - Surgery0.0532 ∗ ±0.0009 0.0044 ∗ ±0.00020.0488 ∗ ±0.00180.0136 ∗ ±0.000287.79±0.25 Metrics We evaluate model calibration using negative log-likelihood (NLL), expected cal- ibration error (ECE), maximum calibration error (MCE) (Naeini et al., 2015; Guo et al., 2017), and Brier score (Glenn et al., 1950). Calibration metrics are calculated on all voxels; a comparison to a calculation on the ”active” foreground region defined as the union of target and prediction foreground is displayed in Appendix H. Additionally, we report the Dice similarity coefficient (DSC) as an overlap-based metric. 3404 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Table 2: Best out of 10 runs (selected through validation Dice scores) trained using each loss on the BraTS-METS dataset (Maleki et al., 2025) (top block) and the KiTS dataset (Heller et al., 2019) (bottom block). Best results are displayed in bold, second-best results are underlined. Better performance for standard losses vs. altered losses is highlighted through italics. MethodNLL ↓ECE ↓MCE ↓Brier ↓DSC (%) ↑ BraTS Metastasis Dice++0.01870.00060.13590.001772.53 CE + Dice0.00940.00070.32190.001673.05 CE + Dice - Surgery 0.0051 0.00050.14510.001573.19 Dice0.02340.00090.41130.001969.53 Dice - Surgery0.00550.00050.1414 0.001471.02 Tversky0.03420.00100.42730.001972.39 Tversky - Surgery0.0265 0.00050.13790.001671.80 KiTS Metastasis CE0.01390.00190.18450.005864.79 SVLS0.01150.00200.14010.005970.68 SVLS + Dice0.02030.00240.26860.0056 75.82 NACL0.06140.05090.16400.011064.48 NACL + Dice0.03070.01760.20960.007574.56 Dice++0.01650.00240.14590.006675.57 CE + Dice0.02380.00290.28840.006475.77 CE + Dice - Surgery0.01250.00170.13890.005276.75 Dice0.05800.00330.35130.006876.62 Dice - Surgery0.01610.0022 0.13450.006374.01 Tversky0.07140.00410.36160.008373.10 Tversky - Surgery0.02290.00290.14040.007773.87 DatasetsWe perform experiments on datasets for 2D retinal vessel segmentation on the FIVES dataset (Jin et al., 2022), for 2D mass segmentation in mammography images (Moreira et al., 2012), for 3D metastasis segmentation on the BraTS-METS dataset (Maleki et al., 2025), and 3D tumor segmentation on the KiTS dataset (Heller et al., 2019). Detailed descriptions of the datasets and data splits are provided in Appendix C. 4.1. Results Tables 1 and 2 present our main results for the experiments on 2D and 3D datasets. Our proposed gradient vector field surgery, applied to the gradient of a region-based loss function, 3405 Lux Berger Knolle R ̈ uckert Paetzold Image A) OursDice B) Figure 3: Visualization of the predicted probability maps as heat maps on the BraTS dataset (best viewed zoomed in). Yellow and red indicate medium and high foreground probability, respectively. Blue arrows indicate regions of overconfi- dence of the Dice model, while our approach exhibits well-calibrated predictions. The Dice model overconfidently predicts background (A, both arrows and B, right arrow) and foreground (B, left arrow). improves calibration metrics compared to the respective baseline losses alone (ComboLoss, Dice, and Tversky) across all cases in all datasets. In some cases, our approach reduces NLL and ECE by factors of 4 to 6, with negligible (FIVES, BraTS, KiTS) or positive (INbreast) impact on binary prediction performance. Furthermore, our approach applied to varying baseline losses consistently yields the best (INBreast, KiTS, BraTS) or second-best (FIVES, BraTS) calibration scores in all metrics. Especially CE + Dice with gradient vector field surgery performs strongly on all datasets in terms of calibration and DSC. On the 2D datasets, the proposed surgery yields substantial calibration gains. For IN- breast, where standard losses often exhibit high instability due to the dataset’s inherent challenges (foreground size variability, low foreground-to-background differences, limited sample size), our method improves training, recovering DSC performance up to 74%. We furthermore observe strong calibration performance of the CE loss, stemming from its op- timal calibration properties as described in Section 3.1. However, this comes at the cost of low predictive performance, especially when facing highly imbalanced datasets, such as INbreast (DSC of 66%). On FIVES, where DSC scores are generally high (88.03%), our method maintains segmentation accuracy while drastically reducing ECE and MCE. 3406 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Table 3: Ablation study on the exponential parameter n for TunableGradSym on FIVES dataset (512×512 resolution). All other hyperparameters held constant. Bold indicates best achieved scores and underlineindicates second best results. Exponential nNLL ↓ECE ↓MCE ↓Brier ↓DSC (%) ↑ 10.05650.00530.04870.014387.26 20.05410.00480.04940.013787.62 50.05350.00460.04930.013787.76 200.05280.00430.04760.013687.84 400.05280.00420.0452 0.013687.84 600.05290.00420.04800.013687.80 800.05310.00410.04570.013787.76 1000.05380.00430.04740.013787.77 2000.05330.0040 0.04520.013887.75 10000.0529 0.00430.04850.013687.72 In the 3D domain, which presents challenges related to volumetric imbalance and la- bel noise, our approach consistently yields better-calibrated models without compromising segmentation accuracy. While all models trained with losses containing Dice components achieve comparable DSC on BraTS and KiTS, our gradient surgery majorly reduces the MCE and NLL compared to the baseline losses. The Dice++ is notably strong on our 3D datasets; however, it is overall still inferior to the proposed gradient surgery, particularly regarding NLL and ECE. Similarly to our 2D experiments, CE-based loss functions (CE, SVLS, NACL) yield well-calibrated models that show weaker Dice performances because of the datasets’ high region-imbalance. Combining these losses with a Dice component dras- tically improves DSC scores, while having an adverse effect on calibration. Notably, NACL shows poor calibration performance when evaluated on all pixels because it is underconfi- dent in background regions. When evaluating on active regions alone (Appendix H), NACL yields calibration comparable to our method. 4.2. Ablation on exponential decline factor To investigate the impact of the exponential n contained in the multiplicative terms ((1− (1 − p) n ) and (1 − p n )) we perform an ablation study on the fives dataset with n ∈ 1, 2, 5, 20, 40, 60, 80, 100, 200, 1000 and other hyperparameters fixed. The experiment shows that calibration and region overlap performance increase until n = 20 (see Table 3). For larger n, only minor differences in all metrics are observable, displaying robust performance across different values for the exponential n, above a certain threshold. 3407 Lux Berger Knolle R ̈ uckert Paetzold 5. Conclusion In this work, we theoretically analyze the partial derivatives of widespread region-based loss functions and show their formal connection to network calibration. We identify how the Dice/Tversky loss is incentivized to produce overconfident predictions and propose gradient surgery as a simple solution. This ”surgery” combines the benefits of gradients that scale with error magnitude with robustness to region imbalance. Instead of relying on a scalar loss, we directly define vector fields at the level of the logits as loss surrogates and prove how they cannot be formalized as scalar loss functions. While this comes at the expense of desirable theoretical guarantees due to the non-conservative nature of the vector fields, we theoretically and empirically demonstrate that our defined vector fields possess favorable properties for model training. Our method drastically improves calibration metrics across diverse medical segmentation datasets in 2D and 3D, including metastasis segmentation, where calibrated outputs provide valuable insights into borders and potential emergence of metastasis. Future work should focus on two directions: first, deriving theoretical bounds for the stability of such non-conservative gradient fields; second, exploring the utility of better-calibrated networks in clinical practice. Ultimately, this approach provides a gen- eralizable mechanism for training uncertainty-aware segmentation networks, a prerequisite for trustworthy clinical decision support. Acknowledgments This work was partially supported by the German Federal Ministry of Research, Technology, and Space (BMFTR) as part of the Software Campus 3.0 (TU M ̈unchen) under grant number 01IS23069. References Nabila Abraham and Naimul Mefraz Khan. A novel focal tversky loss function with im- proved attention u-net for lesion segmentation. In 2019 IEEE 16th international sympo- sium on biomedical imaging (ISBI 2019), pages 683–687. IEEE, 2019. Theodore Barfoot, Luis C Garcia Peraza Herrera, Ben Glocker, and Tom Vercauteren. Average calibration error: A differentiable loss for improved reliability in image segmen- tation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 139–149. Springer, 2024. Alexander H Berger, Laurin Lux, Alexander Weers, Martin J Menten, Daniel Rueckert, and Johannes C Paetzold. Pitfalls of topology-aware image segmentation. In International Conference on Information Processing in Medical Imaging, pages 297–312. Springer, 2025. Jeroen Bertels, David Robben, Dirk Vandermeulen, and Paul Suetens. Optimization with soft dice can lead to a volumetric bias. In International MICCAI Brainlesion Workshop, pages 89–97. Springer, 2019. W Brier Glenn et al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3, 1950. 3408 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359–378, 2007. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR, 2017. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. Nicholas Heller, Niranjan Sathianathen, Arveen Kalapara, Edward Walczak, Keenan Moore, Heather Kaluzniak, Joel Rosenberg, Paul Blake, Zachary Rengel, Makinna Oestreich, et al. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXiv preprint arXiv:1904.00445, 2019. Fabian Isensee, Paul F Jaeger, Simon A Kohl, Jens Petersen, and Klaus H Maier-Hein. nnu-net: a self-configuring method for deep learning-based biomedical image segmenta- tion. Nature methods, 18(2):203–211, 2021. Fabian Isensee, Tassilo Wald, Constantin Ulrich, Michael Baumgartner, Saikat Roy, Klaus Maier-Hein, and Paul F Jaeger. nnu-net revisited: A call for rigorous validation in 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 488–498. Springer, 2024. Mobarakol Islam and Ben Glocker. Spatially varying label smoothing: Capturing uncer- tainty from expert annotations. In international conference on information processing in medical imaging, pages 677–688. Springer, 2021. Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machado. Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association, 19(2):263–274, 2012. Kai Jin, Xingru Huang, Jingxing Zhou, Yunxiang Li, Yan Yan, Yibao Sun, Qianni Zhang, Yaqi Wang, and Juan Ye. Fives: A fundus image dataset for artificial intelligence based vessel segmentation. Scientific data, 9(1):475, 2022. Neerav Karani, Neel Dey, and Polina Golland. Boundary-weighted logit consistency im- proves calibration of segmentation networks. In International conference on medical im- age computing and computer-assisted intervention, pages 367–377. Springer, 2023. Bingyuan Liu, Ismail Ben Ayed, Adrian Galdran, and Jose Dolz. The devil is in the margin: Margin-based label smoothing for network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 80–88, 2022. Nazanin Maleki, Raisa Amiruddin, Ahmed W Moawad, Nikolay Yordanov, Athanasios Gkampenis, Pascal Fehringer, Fabian Umeh, Crystal Chukwurah, Fatima Memon, Bojan Petrovic, et al. Analysis of the miccai brain tumor segmentation–metastases (brats-mets) 2025 lighthouse challenge: Brain metastasis segmentation on pre-and post-treatment mri. arXiv preprint arXiv:2504.12527, 2025. 3409 Lux Berger Knolle R ̈ uckert Paetzold Alireza Mehrtash, William M Wells, Clare M Tempany, Purang Abolmaesumi, and Tina Kapur. Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE transactions on medical imaging, 39(12):3868–3878, 2020. Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans. Advances in neural information processing systems, 30, 2017. Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. Ieee, 2016. Inˆes C Moreira, Igor Amaral, Inˆes Domingues, Ant ́onio Cardoso, Maria Joao Cardoso, and Jaime S Cardoso. Inbreast: toward a full-field digital mammographic database. Academic radiology, 19(2):236–248, 2012. Rafael M ̈uller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural information processing systems, 32, 2019. Balamurali Murugesan, Sukesh Adiga Vasudeva, Bingyuan Liu, Herve Lombaert, Ismail Ben Ayed, and Jose Dolz. Trust your neighbours: Penalty-based constraints for model cal- ibration. In International conference on medical image computing and computer-assisted intervention, pages 572–581. Springer, 2023a. Balamurali Murugesan, Bingyuan Liu, Adrian Galdran, Ismail Ben Ayed, and Jose Dolz. Calibrating segmentation networks with margin-based label smoothing. Medical Image Analysis, 87:102826, 2023b. Balamurali Murugesan, Sukesh Adiga Vasudeva, Bingyuan Liu, Herve Lombaert, Ismail Ben Ayed, and Jose Dolz. Neighbor-aware calibration of segmentation networks with penalty- based constraints. Medical Image Analysis, 101:103501, 2025. Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015. John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61–74, 1999. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015. Axel-Jan Rousseau, Thijs Becker, Jeroen Bertels, Matthew B Blaschko, and Dirk Valken- borg. Post training uncertainty calibration of deep networks for medical image segmenta- tion. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 1052–1056. IEEE, 2021. Seyed Sadegh Mohseni Salehi, Deniz Erdogmus, and Ali Gholipour. Tversky loss function for image segmentation using 3d fully convolutional deep networks. In International workshop on machine learning in medical imaging, pages 379–387. Springer, 2017. 3410 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery J ̈org Sander, Bob D de Vos, Jelmer M Wolterink, and Ivana Iˇsgum. Towards increased trustworthiness of deep learning segmentation methods on cardiac mri. In Medical imaging 2019: image Processing, volume 10949, pages 324–330. SPIE, 2019. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016. Saeid Asgari Taghanaki, Yefeng Zheng, S Kevin Zhou, Bogdan Georgescu, Puneet Sharma, Daguang Xu, Dorin Comaniciu, and Ghassan Hamarneh. Combo loss: Handling input and output imbalance in multi-organ segmentation. Computerized Medical Imaging and Graphics, 75:24–33, 2019. Tassilo Wald, Saikat Roy, Fabian Isensee, Constantin Ulrich, Sebastian Ziegler, Dasha Trofimova, Raphael Stock, Michael Baumgartner, Gregor K ̈ohler, and Klaus Maier-Hein. Primus: Enforcing attention usage for 3d medical image segmentation. arXiv preprint arXiv:2503.01835, 2025. Michael Yeung, Evis Sala, Carola-Bibiane Sch ̈onlieb, and Leonardo Rundo. Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95:102026, 2022. Michael Yeung, Leonardo Rundo, Yang Nan, Evis Sala, Carola-Bibiane Sch ̈onlieb, and Guang Yang. Calibrating the dice loss to handle neural network overconfidence for biomedical image segmentation. Journal of Digital Imaging, 36(2):739–752, 2023. Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multi- class probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 694–699, 2002. 3411 Lux Berger Knolle R ̈ uckert Paetzold Appendix A. Proof of the non-existence of a corresponding scalar loss function A corresponding scalar function only exists for conservative vector fields. We show that the vector fieldF (z) is non-conservative and therefore no corresponding scalar loss function exists. For F (z) to be conservative, there must exist a potential function L (the desired loss function) such that: ∂ 2 L ∂z i ∂z k = ∂ 2 L ∂z k ∂z i With the desired partial derivatives, we have ∂L ∂z i = p i (1− p i ) (y i p i + (1− y i )(1− p i )) 2y i (P + Y + ε)− (2I + ε) (P + Y + ε) 2 , and ∂L ∂z k = p k (1− p k ) (y k p k + (1− y k )(1− p k )) 2y k (P + Y + ε)− (2I + ε) (P + Y + ε) 2 , with the second-order partial derivatives: ∂ 2 L ∂z k ∂z i = p i (1− p i ) (y i p i + (1− y i )(1− p i )) · −2y i (P + Y + ε) 2 − 2y k (P + Y + ε)− 2(2I + ε) (P + Y + ε) 3 ·p k (1−p k ) ∂ 2 L ∂z i ∂z k = p k (1− p k ) (y k p k + (1− y k )(1− p k )) · −2y k (P + Y + ε) 2 − 2y i (P + Y + ε)− 2(2I + ε) (P + Y + ε) 3 ·p i (1−p i ) taking e.g. y i = y k = 0 ∂ 2 L ∂z k ∂z i = p i −2(2I + ε) (P + Y + ε) 3 · p k (1− p k ) ∂ 2 L ∂z i ∂z k = p k −2(2I + ε) (P + Y + ε) 3 · p i (1− p i ) ∂ 2 L ∂z k ∂z i = ∂ 2 L ∂z i ∂z k only if 1− p k = 1− p i . This requires p k = p i and is obviously not true for arbitrary p k and p i . 3412 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Appendix B. Vector/gradient fields of different loss functions Figure 4 visualizes the gradient field w.r.t to the logits z for different loss functions, in addition to our proposed vector field. The fields are depicted in probability space for two variables p 1 and p 2 for better visualization, although the vector represents the gradient derivatives w.r.t. to the logits. For losses influenced by global statistics (all but CE), we assume an imbalanced example with 2 foreground voxels (y = 1) and 98 background voxels (y = 0). With assumed probability values of 0.8 for foreground voxels and 0.1 for background voxels. The displayed gradient/vector fields add 2 additional voxels y 1 = 1 and y 2 = 2, and show the gradient on their logits for different p 1 and p 2 values, while the probabilities/logits for the other 100 voxels stay unchanged. 3413 Lux Berger Knolle R ̈ uckert Paetzold 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 Dice Gradient Vector Field p 2 (GT: y 2 =0) 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 Dice++ Gradient Vector Field (γ = 4) 0.00.20.40.60.81.0 p 1 (GT: y 1 =1) 0.0 0.2 0.4 0.6 0.8 1.0 Dice + CE Gradient Vector Field (0.5 Dice + 0.5 CE) p 1 (GT: y 1 =1) p 2 (GT: y 2 =0) p 2 (GT: y 2 =0) 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 Cross-Entropy Gradient Vector Field p 2 (GT: y 2 =0) 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 Ours p 1 (GT: y 1 =1)p 1 (GT: y 1 =1) p 1 (GT: y 1 =1) p 1 (GT: y 1 =1) p 2 (GT: y 2 =0) Dice++ Gradient Vector Field (γ = 2) 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 p 2 (GT: y 2 =0) Figure 4: Gradient/vector fields w.r.t. logits of different loss functions and our proposed vector field. For better visualization, the axes are displayed as probabilities p instead of logits z. 3414 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Appendix C. Experimentation details Datasets The INbreast dataset (Moreira et al., 2012) contains 107 images with Masses. We separate 22 (∼ 20% of total) images for the test set. Validation metrics for model selection are calculated on 17 (= 20% of remaining) of the remaining images. We resize the images to a resolution of 512x× 512 for model training. The FIVES dataset (Jin et al., 2022) contains 800 fundus images with vessel annotations. We rescale images to a resolution of 1024× 1024, and train on the center and evaluate on the central patch of 512× 512 voxels. We separate 200 images (= 20% of total) for testing. Of the remaining data, we use 120 (= 20% of remaining) images as validation set. The original BraTS Metastasis dataset (Maleki et al., 2025) comprises a retrospective collection of 1296 pre- and post-treatment brain metastases, labeled in four classes: nonen- hancing tumor core, FLAIR hyperintensity, enhancing tumor, and resection cavity. Each sample has four input channels (T1, T1c, T2, FLAIR). We use a random, representative subset of 156 cases for training, 44 for validation, and 251 for testing. As input, we use only the T1c scan, disregarding the others. The images vary in size, orientation, and spacing. We preprocess each image in a nnUNet-style fashion (Isensee et al., 2021) with reorienta- tion to RAS+, resampling to an isotropic spacing of 1m, and z-score normalization. The resulting volumes have a median shape of [141× 175× 142]. We convert the labels to a binary format, where enhancing and non-enhancing tumor tissue is foreground and FLAIR hyperintensity, resection cavity, and healthy tissue is background. The lesions account for 0.17% of the total voxels with a standard deviation of 0.29% per sample. During training, we extract a random patch of size [80× 96× 80] with a foreground oversample ratio of 0.33. The KiTS dataset (Heller et al., 2019) comprises 489 abdominal CT scans where kid- neys, renal tumors, and renal cysts are labeled. The images vary in size, orientation, and spacing. We preprocess each image in a nnUNet-style fashion (Isensee et al., 2021) with reorientation to RAS+, resampling to an isotropic spacing of 1.5m, and intensity-clipping to the [0.5, 99.5] percentiles (i.e., [−58, 302] HU) followed by z-score normalization. We extract ROIs of varying sizes with a median of [218× 130× 160] around both kidneys. We convert the labels to a binary format, where the tumors are foreground and the kidneys, cysts, and the rest are background. The tumors account for 0.68% of the total voxels with a standard deviation of 1.18% per sample. We stratify the complete dataset into 192 training, 50 validation, and 247 test sets, which are balanced in terms of size and number of tumors. During training, we extract 8 random patches per sample, with a foreground oversample ratio of 0.45 and a fixed size of [96× 96× 80] for each patch. Loss hyperparameters For the Tversky loss, we set the α parameter (β = 1−α) as a hyperparameter. For Combo loss (Taghanaki et al., 2019), we fix the weighting to 0.5 (Isensee et al., 2021). For Dice++, we set the γ parameter to 2 (Yeung et al., 2023). For SVLS, we set the kernel size to 3 and use σ as a hyperparameter with possible settings of 1, 2, and 3 (Islam and Glocker, 2021). For NACL, we use the penalty formulation, set the balancing parameter to 0.1 (λ), the kernel size to 3, use a mean prior (τ ), and use an L1 penalty(Murugesan et al., 2025). 3415 Lux Berger Knolle R ̈ uckert Paetzold Finally, for the mL1-ACE loss, we use equal weighting with Dice loss and use 20 bins to discretize the probability space (Barfoot et al., 2024). Model and Training Procedure Our 3D experiments utilize a full-resolution 3D UNet with residual units, following a pipeline heavily influenced by nnUNet (Isensee et al., 2021), particularly in terms of network size, learning rate schedule, iterations, augmentations, and optimizer. We use SGD with Nesterov momentum as an optimizer. In our hyperparameter optimization, we further optimize for weight decay, initial learning rate, and momentum. At test time, we do sliding window inference on the complete volume with an overlap ratio of 0.5 and Gaussian weighting. Training curves stability Figure 5 shows training and validation curves with different optimizers. Training soft dice / dice loss Dice Loss + SGD Dice Loss + Adam Dice Loss - Surgery + SGD Dice Loss - Surgery + Adam Validation dice score Validation NLLValidation NLL Figure 5: Training curves with SGD+Nesterov Momentum and Adam optimizers for a dice loss training and a model with the same hyperparameters trained with our dice surgery training.objective 3416 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Appendix D. Implementation details for custom vector fields In practice, we implement our custom (”gradient”) vector field for the different region- based losses by overwriting the backward pass of the softmax activation. The forward pass through the softmax remains unchanged. We keep the partial derivatives of the Loss w.r.t. the probabilities and exchange the partial derivative of the probabilities w.r.t. the logits to reflect our desired vector field, e.g., by replacing p(1−p) with |y−p| and adding the sharp decline term close to p = 0 and p = 1, (1−(1−p) n ) and (1−p n ). The actual implementation is shown in the following listing. 1 class GradSurgeSoftmax(torch.autograd.Function): 2 @staticmethod 3 @custom_fwd(device_type="cuda", cast_inputs=torch.float32) 4 def forward(ctx , logits , targets , exponential_correction=None): 5 probs = torch.softmax(logits , dim=1) 6 7 error = torch.abs(probs - targets) 8 9 if exponential_correction is not None: 10 # Applying the correction term 11 error_weight = 0.25 * error * (1 - torch.pow(error , exponential_correction)) * (1 - torch.pow(1 - error , exponential_correction)) 12 else: 13 error_weight = 0.25 * error 14 15 ctx.save_for_backward(error_weight) 16 return probs 17 18 @staticmethod 19 @custom_bwd(device_type="cuda") 20 def backward(ctx , grad_output): 21 error_weight , = ctx.saved_tensors 22 23 grad_output = grad_output.to(error_weight.dtype) 24 25 # Assuming binary segmentation (background vs foreground) 26 weight = error_weight [:, 1:2] 27 grad_p_bg = grad_output [:, 0:1] 28 grad_p_fg = grad_output [:, 1:2] 29 30 coupling = (grad_p_fg - grad_p_bg) 31 32 grad_logits_bg = -weight * coupling 33 grad_logits_fg = weight * coupling 34 35 return torch.cat([ grad_logits_bg , grad_logits_fg], dim=1), None , None , None Listing 1: Implementation of GradSurgeSoftmax 3417 Lux Berger Knolle R ̈ uckert Paetzold Appendix E. Dice++ gradient The Dice ++ loss DSC + + = 1− 2 P N i=1 p i y i + ε 2 P N i=1 p i y i + P N i=1 (p i (1− y i )) γ + P N i=1 ((1− p i )y i ) γ + ε was proposed to resolve the calibration issues of the dice loss by introducing a focus γ on false positives and false negatives. ∂L DSC++ ∂p fg i = −2[γ(1− p i ) γ−1 I + FP γ + FN γ ] (2I + 2p i + (1− p i ) γ + FP γ −i + FN γ −i ) 2 ∂L DSC++ ∂p bg i = 2γp γ−1 i 2I (2I + p γ i + FP γ −i + FN γ −i ) 2 Exactly, for γ = 2, the partial derivative of the Dice++ loss w.r.t. the probabilities depends linearly on the error as for MSE loss on the probabilities ( ∂L DSC++ /∂p fb i = 2p i ) while the global term for y = 0 and y = 1 is the most similar to the original dice loss at γ = 2 compared to higher γ’s. Which we identify as the reason for the optimal performance in terms of Dice and calibration metrics for γ = 2. Besides this desired property, the γ parameter introduces a vast downscaling of the gradi- ent for samples with large proportions of false positives and false negatives. This can be problematic for cases where (1) the foreground regions have drastically different sizes and, in connection to that, drastically different values for false positives and false negatives, and (2) for cases where learning from ”hard” examples characterized through high false positive and false negative rates is crucial. On the contrary, previous works also showed that in some cases, focus on easy examples can be beneficial for segmentation performance (Abraham and Khan, 2019). Appendix F. Partial derivative function visualization p (p=σ(z)=1/(1+e -z )) p (p=σ(z)=1/(1+e -z )) z 6420246 0.0 0.2 0.4 0.6 0.8 1.0 Normalized dL/dz Dice: p 2 Cross-Entropy: p Dice + CE: 2p 2 Dice Surgery (n = 40): p(1p 40 )(1(1p) 40 ) 0.010.100.300.500.700.900.99 0.00.20.40.60.81.0 0.0 0.2 0.4 0.6 0.8 1.0 Normalized dL/dz Dice: p 2 Cross-Entropy: p Dice + CE: 2p 2 Dice Surgery (n = 40): p(1p 40 )(1(1p) 40 ) Figure 6: Visualization of the normalized partial derivatives derived from different loss func- tions. 3418 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Appendix G. Qualitative examples Probability Maps Dice Loss Ours (+Dice) Figure 7: Comparison of the probability maps for two samples from a Dice loss trained model and our adapted Dice vector field approach. The overconfidence phe- nomenon on the Dice probabilities is apparent, with probabilities almost appear- ing binarized. In comparison, our method has probabilities other than 0 and 1, which often reflect plausible vascular courses (e.g., purple ellipses) that are miss- ing in the probability maps of the Dice loss-trained model. 3419 Lux Berger Knolle R ̈ uckert Paetzold Appendix H. Active region evaluation Tables 4 and 5 show an evaluation of the calibration metrics on the ”active” foreground region defined as the union of target and (binarized) prediction foreground regions (Muruge- san et al., 2025). Evaluation only on the active region results in a stark difference for the absolute values on calibration metrics. However, we see that the trends in the improvement achieved through our method remain unchanged. Table 4: Average test set performance of the 5 best runs out of 25 (selected through valida- tion dice scores) on the INbreast and FIVES datasets with active region calibration result. The active region is defined as the union of the target foreground and the binarized prediction foreground. Best results are displayed in bold, second-best results are underlined . Significantly better performance is highlighted through a ∗ using the 0.01 significance level. MethodNLL ↓ECE ↓MCE ↓Brier ↓Dice (%) ↑ INbreast (Active) Cross Entropy1.1372±0.15710.3229±0.05020.5940±0.21300.3085±0.029066.20±3.57 NACL1.2444±0.23370.3167±0.04270.6731±0.27300.3064±0.034768.22±3.57 NACL + Dice2.0573±0.53110.3367±0.03320.5938±0.03800.3321±0.029772.07±2.57 SVLS1.1109±0.15550.3089±0.02950.6844±0.24950.2908±0.023368.35±2.90 SVLS + Dice1.7828±0.33160.3535±0.03670.5863±0.04280.3470±0.030468.76±2.70 Dice++1.2198±0.06280.3207±0.02510.5231±0.02930.3137±0.016667.51±3.17 CE + Dice2.1938±0.37760.3411±0.02700.5930±0.03530.3355±0.025671.76±2.10 CE + Dice - Surgery 0.8807 ∗ ±0.09260.2427 ∗ ±0.03220.4283 ∗ ±0.03880.2470 ∗ ±0.0250 74.39±3.34 Dice3.2670±1.30150.3996±0.04330.6189±0.06130.3940±0.040165.06±3.31 Dice - Surgery0.9670±0.11730.2397 ∗ ±0.0313 0.4228 ∗ ±0.05480.2496 ∗ ±0.025774.34 ∗ ±2.77 Tversky3.6811±0.63540.3981±0.02380.6365±0.02170.3912±0.021367.63±2.89 Tversky - Surgery1.0767 ∗ ±0.16260.2842 ∗ ±0.04050.4754 ∗ ±0.05270.2808 ∗ ±0.030769.34±3.77 FIVES (Active) Cross Entropy0.5448±0.01210.1265±0.00170.2856±0.00880.1465±0.001787.54±0.22 NACL0.5324±0.00710.1253±0.00210.2844±0.01090.1450±0.001287.60±0.14 NACL + Dice0.7641±0.03410.1597±0.00330.4003±0.01060.1634±0.002887.94±0.11 Dice++0.5365±0.00390.1343±0.00140.3286±0.01270.1452±0.001087.97±0.07 ACE0.8581±0.02740.1514±0.00440.2927±0.01160.1688±0.005285.96±0.45 CE + Dice0.7682±0.01600.1594±0.00250.3972±0.01060.1634±0.002087.99±0.16 CE + Dice - Surgery0.5340 ∗ ±0.02640.1249 ∗ ±0.0050 0.2815 ∗ ±0.01100.1447 ∗ ±0.002787.70±0.09 Dice2.7751±0.32240.1947±0.00500.5847±0.02640.1940±0.004988.03±0.25 Dice - Surgery0.5271 ∗ ±0.02470.1283 ∗ ±0.00500.2999 ∗ ±0.01310.1451 ∗ ±0.002487.60±0.12 Tversky2.5810±0.26710.1969±0.00180.5727±0.00810.1960±0.001987.77±0.21 Tversky - Surgery0.5189 ∗ ±0.01550.1258 ∗ ±0.00290.2953 ∗ ±0.01500.1432 ∗ ±0.002187.79±0.25 3420 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Table 5: Test-set performance on the KiTS dataset (Heller et al., 2019) with active re- gion calibration results. The active region is defined as the union of the target foreground and the binarized prediction foreground. Best results are displayed in bold, second-best results are underlined. Better performance for standard losses vs. altered losses is highlighted through italics. MethodNLL ↓ECE ↓MCE ↓Brier ↓DSC (%) ↑ KiTS Tumor CE1.66270.33060.51170.687064.79 SVLS0.92130.26150.48670.524970.68 SVLS + Dice2.51330.29940.54380.596975.82 NACL0.93980.30330.47300.626364.48 NACL + Dice1.37660.29810.55670.591874.56 Dice++6.37550.2580 0.44450.524475.57 CE + Dice8.90810.30420.56640.604175.77 CE + Dice - Surgery1.2973 0.2344 0.4132 0.483076.75 Dice5.53000.31480.63090.625376.62 Dice - Surgery1.29770.26110.43930.519374.01 Tversky7.34100.35060.61100.696173.10 Tversky - Surgery1.53390.26710.47230.525873.87 Appendix I. Effect on different foreground sizes Table 6 summarizes the detection performance and ECE for different tumor sizes on the Kits dataset. A tumor is considered detected if at least one voxel within its label area is correctly predicted as foreground. Very small foreground components (below 4cm 3 ) are not included in the comparison as they are assumed to constitute label noise (Berger et al., 2025). We observe consistent calibration improvements when applying our proposed gradi- ent field surgery across all tumor sizes. Furthermore, our approach improves the detection of small tumors slightly more than it does for large and very large tumors. We hypothesize that improved calibration is most beneficial for small tumors, where model uncertainty is naturally higher; in these borderline cases, accurate probability estimates are critical for successful detection because they help push the predicted probabilities of these difficult cases close to the detection threshold, thereby improving detection. ECE values are calculated on the lesion foreground pixels and averaged across lesions of each size category. 3421 Lux Berger Knolle R ̈ uckert Paetzold Table 6: Detection rate (Det) and Expected Calibration Error (ECE) across different lesion sizes and loss functions. Lesion Size MetricDice++ CE + Dice CE + Dice Surgery Dice Dice Surgery Tversky Tversky Surgery Small Det.0.92210.89610.94810.90910.90910.89610.9481 ECE0.22720.31120.21720.27640.21820.35860.1664 Large Det.0.98720.96150.93590.96150.97440.96150.9487 ECE0.09760.16070.12790.16110.09910.21280.1170 Very Large Det.1.0000.97440.98710.98720.98721.0000.9744 ECE0.0550.09490.06100.09490.05890.11340.0558 Appendix J. Experiments with transformer architecture In addition to the main experiments (n-unet style training), we perform experiments with the state-of-the-art Primus transformer architecture (Wald et al., 2025) for image segmen- tation. The results in 7 show that vector field surgery also works in combination with transformer architectures, yielding notably better calibration scores. However, the overall performance of the transformer approach was poor. In medical image segmentation, con- volutional approaches have proven more effective in numerous extensive evaluation studies (Isensee et al., 2021, 2024; Wald et al., 2025). Table 7: Results on the KiTS tumor segmentation dataset, using the PRIMUS (Wald et al., 2025) transformer architecture. MethodNLL ↓ECE ↓MCE ↓Brier ↓DSC (%) ↑ Dice + CE0.01880.00310.23080.007867.62 Dice + CE - Surgery 0.0156 0.0021 0.1173 0.007166.86 3422 Beyond scalar losses: calibrating segmentation models via gradient vector field surgery Appendix K. Effect on logit values Table 8 shows the average logit values in target foreground and background regions. Our method reduces the logit distance and the absolute value of the logits. Especially for the FIVES dataset, standard region-based losses show very large logit values, indicating over- confidence. Reduced logit magnitudes were found to result in improved calibration scores in earlier works on label smoothing and logit constraints (M ̈uller et al., 2019; Murugesan et al., 2025). The analysis provides direct evidence that the vector field intervention is effective in resolving the overconfidence problem of region-based losses. Table 8: Logit magnitude analysis on INbreast and FIVES datasets (average of top 5 runs ± standard deviation). Method Target FGTarget BG Dice (%) ↑ FG LogitBG LogitFG LogitBG Logit INbreast CE + Dice3.51±2.23 −1.07±1.27 −5.39±0.766.31±0.7171.76±2.10 CE + Dice - Surgery1.52±0.90 −1.33±0.89 −4.31±0.534.93±0.4774.39±3.34 Dice5.18±2.34 −1.42±3.43 −5.26±1.086.18±0.8165.06±3.31 Dice - Surgery1.87±1.89 −0.67±1.48 −4.41±0.405.53±0.6574.34±2.77 Tversky5.97±0.97 −1.79±2.55 −5.39±0.696.34±0.3867.63±2.89 Tversky - Surgery2.28±1.67 −0.33±1.76 −4.52±0.815.58±0.5469.34±3.77 FIVES CE + Dice3.71±1.13 −4.18±1.07 −4.08±0.064.57±0.1487.99±0.16 CE + Dice - Surgery2.71±1.03 −2.51±1.02 −3.19±0.243.71±0.2287.70±0.09 Dice26.42±13.03 −29.12±18.00 −7.80±1.187.74±0.5988.03±0.25 Dice - Surgery3.66±0.18 −1.91±0.45 −2.92±0.283.52±0.1987.60±0.12 Tversky18.79±11.26 −18.51±12.57 −6.82±1.127.03±0.7987.77±0.21 Tversky - Surgery3.40±0.99 −2.36±0.94 −2.96±0.123.49±0.1687.79±0.25 3423