Paper deep dive
On the Relaxation of Conditional Independence Assumption for Image Segmentation
Zixun Wang, Ben Dai
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation metrics without modifying model training. Despite its theoretical and empirical success, RankSEG relies on the restrictive Conditional Independence Assumption (CIA), which ignores crucial label correlations and therefore degrades performance in ambiguous or low-contrast scenarios. However, accounting for full label dependence is computationally prohibitive, requiring $\mathcal{O}(d^3)$ time. To address this, we replace the CIA with a Spatially Localized Dependence (SLD) structure that captures local label correlations while keeping the dependence model tractable. We further overcome the remaining computational bottleneck via a Reciprocal Moment Approximation coupled with a novel fixed-point optimization strategy that eliminates exhaustive search. The proposed algorithm achieves a highly practical $\mathcal{O}(d \log d)$ complexity and consistently outperforms conventional argmax and CIA-based RankSEG across diverse segmentation benchmarks. Improvements are significant in low-contrast or small-object scenarios, where label dependence offers valuable signals complementary to image information for accurate segmentation. The code of experiments is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.38930v1
- Canonical: https://arxiv.org/abs/2609.38930v1
Trouble viewing inline? Open PDF directly →
Full Text
77,413 characters extracted from source content.
Expand or collapse full text
On the Relaxation of Conditional Independence Assumption for Image Segmentation Zixun Wang Affiliation: Department of Statistics and Data Science Affiliation: The Chinese University of Hong Kong Email: 1155225012@link.cuhk.edu.hk Ben Dai Affiliation: Department of Statistics and Data Science Affiliation: The Chinese University of Hong Kong Email: bendai@cuhk.edu.hk Abstract In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation metrics without modifying model training. Despite its theoretical and empirical success, RankSEG relies on the restrictive Conditional Independence Assumption (CIA), which ignores crucial label correlations and therefore degrades performance in ambiguous or low-contrast scenarios. However, accounting for full label dependence is computationally prohibitive, requiring (d3)O(d^3) time. To address this, we replace the CIA with a Spatially Localized Dependence (SLD) structure that captures local label correlations while keeping the dependence model tractable. We further overcome the remaining computational bottleneck via a Reciprocal Moment Approximation coupled with a novel fixed-point optimization strategy that eliminates exhaustive search. The proposed algorithm achieves a highly practical (dlogd)O(d d) complexity and consistently outperforms conventional argmax and CIA-based RankSEG across diverse segmentation benchmarks. Improvements are significant in low-contrast or small-object scenarios, where label dependence offers valuable signals complementary to image information for accurate segmentation. The code of experiments is available at https://github.com/ZixunWang/RankSEG-DEP. 1 Introduction Semantic segmentation is a fundamental computer vision task that partitions an image into regions corresponding to different classes. It has a wide range of applications, including medical imaging [2], autonomous driving [21], and image editing [16]. Most existing approaches generate segmentation mask by applying thresholding or argmax operations to pixel-wise probabilities estimated by neural networks [20, 4, 30, 28]. However, they do not directly optimize the evaluation metrics commonly used in segmentation tasks, such as the Dice score or Intersection over Union (IoU), which are set-based metrics that evaluate the overall quality of the predicted mask. Although several surrogate loss functions have been proposed in an attempt to optimize these target metrics, such as soft-Dice/IoU loss [19, 10] and Lovász loss [1, 29], they still suffer from three main limitations: (1) their theoretical consistency [23] with the target metrics is either negative (Lovász hinge [12]) or unknown (soft-Dice/IoU); (2) unlike cross-entropy loss, they do not produce calibrated probabilities, which are crucial for uncertainty quantification in high-stakes settings such as medical imaging [17]; and (3) likely due to non-convexity, they are typically combined with cross-entropy loss through a weighted sum to stabilize training, introducing additional hyperparameters to tune. To address these limitations, a recent series of work [7, 25], known as RankSEG, have developed algorithms that directly optimize the Dice/IoU scores with theoretical guarantees. RankSEG takes calibrated probabilities as input and replaces the thresholding/argmax operation in the original inference step without modifying model training, making it easy to integrate into pre-trained models as a performance-enhancing post-processing module. Despite its theoretical and empirical success, RankSEG relies on the conditional independence assumption (CIA), i.e., Yj⟂Yj′|Y_j \!\!\! Y_j | X for j≠j′j≠ j , where X is the image and YjY_j and Yj′Y_j denote the labels of pixels j and j′j , respectively. This assumption ignores the correlation between labels of different pixels and thus loses valuable information; for example, if two pixels are similar in both intensity and position, they are very likely to share the same label. Such information is especially useful in challenging tasks where the image itself is too noisy to provide sufficient information for accurate segmentation. (a) Comparison of different methods in tumor segmentation. (b) Graphical model. Figure 1: (a) The intensity contrast, and consequently the foreground probabilities, are uniformly low, so traditional thresholding barely detects any tumor and RankSEG almost entirely misses the tumor in the upper right. In contrast, our proposed method more fully recovers the tumor by leveraging label dependencies within the tumor region. (b) When predicting Yj′Y_j , only information from X is used under CIA, whereas both X and the neighboring label YjY_j are utilized when the CIA is not assumed. Figure 1(a) illustrates such an example: the tumor region is hardly distinguishable due to low intensity contrast in the image, and both traditional thresholding and RankSEG fail to recover the tumor in the upper right. In contrast, our proposed method, which leverages the label dependencies within this region, successfully captures the tumor more completely. Using graphical models, Figure 1(b) depicts the information flow of methods with and without CIA: when predicting a particular label Yj′Y_j , both methods rely on the image information X, typically encoded as pixel-wise probabilities produced by deep neural networks. However, the graphical model without CIA explicitly accounts for the influence of other label YjY_j on the prediction of Yj′Y_j , and vice versa. Therefore, a natural question arises: can we develop an efficient method that directly optimizes the Dice/IoU scores yet without relying on CIA? This question is challenging primarily because the computational efficiency of RankSEG critically depends on the CIA; without this assumption, the associated optimization in RankSEG incurs a complexity of (d3)O(d^3), where d is the number of pixels in the image. Such complexity is impractical for real-world applications, since d typically reaches hundreds of thousands (e.g., d≈105d≈ 10^5 for a 384×384384× 384 image). To answer this question, we relax the CIA to a more realistic Spatially Localized Dependence (SLD) structure, which is strictly weaker than the CIA and well suited to image segmentation tasks. Under SLD, we develop an algorithm within the RankSEG framework that optimizes Dice/IoU in (dlogd)O(d d). Our main contributions are summarized as follows: • Relaxation of the CIA: We relax the CIA to SLD within the RankSEG framework, preserving valuable dependency information and improving predictions in low-contrast or ambiguous regions where independence assumptions fail. • An (dlogd)O(d d) algorithm: Building upon this relaxed assumption, we develop a novel and efficient algorithm grounded in the RankSEG framework that directly optimizes Dice/IoU scores without relying on the CIA, making it practical for real-world applications. • Empirical validation: We conduct extensive experiments on multiple image segmentation datasets, demonstrating the superior performance of our method compared to CIA-based methods (argmax/thresholding and RankSEG), especially in challenging scenarios with low image contrast. 2 Preliminaries We start with binary segmentation for ease of clarification. Let ∈ℝd,∈0,1d X ^d, Y∈\0,1\^d represent the random variables for an image and its corresponding segmentation mask, respectively. The segmentation function :ℝd→0,1d δ:R^d→\0,1\^d produces a predicted mask ()∈0,1d δ( x)∈\0,1\^d for a test image ∈ℝd x ^d. Our objective is to find the segmentation rule δ that maximizes the expected Dice score: ,(Dice())=,(2⊺()‖()‖1+‖1).E_ X, Y(Dice( δ))=E_ X, Y ( 2 δ ( X) Y\| δ( X)\|_1+\| Y\|_1 ). (1) We denote [d]=1,⋯,d[d]=\1,·s,d\ as the index set of pixels, and pj()=ℙ(Yj=1|)p_j( x)=P(Y_j=1| x) as the conditional probability of pixel j being a foreground pixel given the image x. In practice, the probability mask ()∈[0,1]d p( x)∈[0,1]^d is estimated by a deep neural network, and our focus is on how to derive the final segmentation mask from () p( x). For notational convenience, we henceforth suppress the conditioning on = X= x and write, e.g., pjp_j in place of pj()p_j( x). 2.1 Thresholding/Argmax over probability The traditional approach to segmentation applies the argmax operation to the probability mask for each pixel, which, in the binary case, is equivalent to thresholding at 0.5: δ~j=(pj≥0.5)for all j∈[d]. δ_j= 1(p_j≥ 0.5) all j∈[d]. (2) However, this method has been shown to be suboptimal with respect to Dice/IoU [7, 25]. Intuitively, this is because it makes pixel-wise decisions, whereas Dice/IoU assesses the overall overlap between the predicted and ground-truth masks. In contrast, as we show later, RankSEG accounts for this global evaluation criterion by ranking pixels according to their contributions to the expected score. 2.2 RankSEG Although the original RankSEG [7] is formulated under the CIA, we present the corresponding Theorem 1 without this assumption to better illustrate the associated computational challenges. Notably, this theorem partially coincides with the Bayes rule for optimizing F-measure in multi-label classification with label dependencies [8]. Theorem 1 ([8, 7]). The optimal segmentation rule ∗=argmax(Dice()) δ^*= argmaxE(Dice( δ)) is given by: δj∗=(j∈Topτ∗(1:d,τ∗))δ^*_j= 1(j _τ^*( S_1:d,τ^*)), where τ∗=argmaxτ∈[d]ωτ,ωτ=∑j∈Topτ(1:d,τ)Sj,τ τ^*= argmax_τ∈[d] _τ, _τ= _j _τ( S_1:d,τ)S_j,τ (3) and Sj,τ=pj(τ+Γj)−1=pj(∑k=1dℙ(Γj=k)τ+k). S_j,τ=p_jE(τ+ _j)^-1=p_j ( _k=1^d P( _j=k)τ+k ). (4) Γj _j distributed as ‖1|(Yj=1)\| Y\|_1 (Y_j=1), i.e., ℙ(Γj=k)=ℙ(‖1=k|Yj=1)P( _j=k)=P(\| Y\|_1=k|Y_j=1) for all k∈[d]k∈[d]. Intuition. ωτ _τ denotes the expected Dice score when predicting the best τ pixels as foreground, and Sj,τS_j,τ quantifies the contribution of pixel j to this score. Γj _j is interpreted as the expected volume conditioned on Yj=1Y_j=1, and τ as the volume we consider to predict. Γj _j and τ correspond to ‖1\| Y\|_1 and ‖()‖1\| δ( X)\|_1 in the denominator of (1), respectively, while pjp_j corresponds to the numerator; this explains why Sj,τS_j,τ represents the contribution of pixel j to the expected Dice score. Under the CIA, Dai and Li [7] further showed that Sj,τS_j,τ is monotonically increasing in pjp_j for any fixed τ, so ranking pixels by Sj,τS_j,τ reduces to ranking by pjp_j. This induces a global ranking invariant to τ: the optimal segmentation simply selects the τ∗τ^* pixels with the highest probabilities. Without the CIA, however, Sj,τS_j,τ depends on the full joint distribution of Y, and may no longer be monotone in pjp_j. Consequently, the optimal segmentation instead requires explicitly ranking pixels by Sj,τS_j,τ for each τ. 2.3 Challenges without CIA and main ideas Computational challenges. Theorem 1 naturally suggests a four-step procedure to obtain the optimal segmentation: (i) compute the contribution scores S according to (4); (i) for each candidate volume τ∈[d]τ∈[d], sort the column 1:d,τ∈[0,1]d S_1:d,τ∈[0,1]^d and compute the expected Dice score ωτ _τ by summing its top τ entries; (i) determine the optimal volume τ∗τ^* by maximizing ωτ _τ; and (iv) predict the top τ∗τ^* pixels in 1:d,τ∗ S_1:d,τ^* as foreground. This procedure is computationally prohibitive for high-resolution images. The inefficiency stems from three primary bottlenecks: (a) Modeling full dependence within Y Y. Although label dependence offers valuable predictive signals, modeling arbitrarily complex dependencies renders the distribution of Γj _j intractable. (b) Computing (τ+Γj)−1E(τ+ _j)^-1. Even given the distribution of Γj _j, performing the summation for all j,τ∈[d]j,τ∈[d] according to (4) incurs a prohibitive (d3)O(d^3) complexity. (c) Re-ranking across varying τ. Evaluating ωτ _τ for each candidate τ requires a fresh sorting over :, S_:,τ, which precludes reusing intermediate results from ωτ _τ when computing ωτ+1 _τ+1. Figure 2: Trade-off in dependence modeling. The two endpoints represent the extremes of CIA and Full dependence. By relaxing CIA to local dependence, SLD strikes a better trade-off. The graphs depict the label dependence (dep.) under each model: none under CIA, arbitrary under Full, and neighboring-only under SLD. Dependence Modeling. To overcome these challenges, we begin by reconsidering how to model the joint distribution of Y. Instead of attempting to model full correlations among all d labels, we propose to model Spatially Localized Dependence (SLD). The key intuition is that, in image segmentation, spatially neighboring pixels exhibit stronger label correlations, whereas distant pixels behave almost independently. By restricting the dependence to these local neighborhoods, it transforms an intractable global distribution into a structured and tractable form. Crucially, SLD provides a better trade-off between efficiency and generality, as illustrated in Figure 2. Algorithmic Contributions. Building on SLD, we introduce two key algorithmic contributions to address the remaining computational bottlenecks. First, we adopt the Reciprocal Moment Approximation [25] that replaces (τ+Γj)−1E(τ+ _j)^-1 with (τ+Γj)−1(τ+E _j)^-1, decoupling the expectation from τ and reducing the problem to merely computing ΓjE _j. SLD then enables to formulate this as a convolution, paired with an (dlogd)O(d d) solution via the Fast Fourier Transform. Second, we propose a fixed-point optimization strategy to avoid the costly re-ranking for each τ. Instead of sorting from scratch each time, we maintain a single global ranking and alternate between updating τ based on the current ranking and refining the ranking from the updated τ. This procedure converges quickly to an approximate τ∗τ^* without exhaustive enumeration. 3 Method In this section, we develop an efficient inference algorithm that incorporates SLD into the RankSEG framework. We first introduce the SLD assumption in Section 3.1, which relaxes the CIA to a local covariance structure tailored to image segmentation. These changes pose two challenges: evaluating the conditional reciprocal moment and avoiding repeated sorting across volumes. We address the first in Section 3.2 using a Reciprocal Moment Approximation, which reduces score computation to the conditional mean ΓjE _j and exploits the SLD structure to compute all such means efficiently by convolution and FFT. We address the second in Section 3.3 through a fixed-point optimization that iteratively refines the ranking and prediction volume without exhaustive search. 3.1 Spatially Localized Dependence Completely removing the CIA introduces significant theoretical and methodological challenges due to the vast dimensionality of the joint label distribution. However, in the context of image segmentation, assuming arbitrary dependencies between all pairs of pixels is unnecessarily complex. Visual structures naturally exhibit spatial locality: physically proximate pixels are highly likely to share the same semantic label, while the correlation between distant pixels drops off rapidly. We formalize this intuition through the SLD assumption in Assumption 1. Assumption 1 (Spatially Localized Dependence; SLD). Denote the covariance matrix of Y as , i.e., Σij=Cov(Yi,Yj) _ij=Cov(Y_i,Y_j). The covariance between any two labels satisfies: Σij=ΣiiΣjj⋅(r(i,j))with(r)=exp(−r2/2θ2), _ij= _i _j·K(r(i,j)) (r)= (-r^2/2θ^2), (5) where K is a Gaussian kernel, θ controls the decay rate, r(i,j)r(i,j) is the distance between i and j, and Σii=pi(1−pi) _i=p_i(1-p_i) is the variance of YiY_i. The SLD explicitly encodes a structural prior: the covariance between two labels is the geometric mean of their individual variances, weighted by an exponentially decaying kernel of their spatial distance. Notably, this formulation is strictly weaker than the CIA, as it permits local dependence and therefore enables the exploitation of such useful information for improved performance. Leveraging local dependence is a well-established principle in image segmentation literature, as exemplified by Conditional Random Fields (CRFs) [14, 11, 3]. However, these methods target Maximum A Posteriori (MAP) inference, seeking the single most probable joint label map, while do not directly optimize set-based metrics like the Dice score. In contrast, we incorporate spatial priors into RankSEG to directly optimize such metrics. See Appendix E for a more detailed discussion. 3.2 Reciprocal Moment Approximation under SLD The primary bottleneck in evaluating the pixel contribution score Sj,τS_j,τ lies in the reciprocal moment (τ+Γj)−1E(τ+ _j)^-1. Due to its nonlinearity in Γj _j, exact evaluation takes (d)O(d) terms according to (4). In contrast, evaluating the first moment, ΓjE _j, is straightforward, as it reduces to summing the individual means. To bridge this gap, we adopt the Reciprocal Moment Approximation (RMA) used in Wang and Dai [25] to replace (τ+Γj)−1E(τ+ _j)^-1 with (τ+Γj)−1(τ+E _j)^-1, which yields the following approximation: Mj,τ=pjτ+Γj≈pj(τ+Γj)−1=Sj,τ. M_j,τ= p_jτ+E _j≈ p_jE(τ+ _j)^-1=S_j,τ. (6) This approximation effectively interchanges the reciprocal and expectation, and can also be viewed as a first-order Taylor approximation to the reciprocal moment. Crucially, it depends only on the first moment μj:=Γj _j:=E _j. The following computation indicates that the SLD enables a more accurate estimate of μj _j by incorporating label correlations through a correction term beyond the CIA: μj=Γj=∑i=1d(Yi∣Yj=1)=∑i=1d(Yi)=∑i=1dpi,(CIA)∑i=1dpi+Σjjpj∑i=1dΣii(r(i,j)).(SLD) _j=E _j= _i=1^dE (Y_i Y_j=1 )= \ aligned & _i=1^dE(Y_i)= _i=1^dp_i, &(CIA)\\ & _i=1^dp_i+ _jp_j _i=1^d _iK(r(i,j)).&(SLD) aligned . (7) Under the CIA, label independence reduces (Yi|Yj=1)E(Y_i|Y_j=1) to the marginal (Yi)E(Y_i), so μj _j collapses to a sum of probabilities that is independent of j, discarding critical structural information. The SLD instead leverages the covariance structure, adding a correction term that captures local dependencies. While the SLD expression for μj _j appears complex, computing all j∈[d]j∈[d] simultaneously reveals a convolutional form. Let q=∑j=1dpjq= _j=1^dp_j and =diag()∈ℝd ν= diag( ) ^d, then =q⋅+⋅(⋆),where(⋆)j=∑i=1did−j+i=∑i=1dΣii(r(i,j)), μ=q· 1+ ν p·( ν κ), ( ν κ)_j= _i=1^d ν_i κ_d-j+i= _i=1^d _iK(r(i,j)), (8) and=((d−1),(d−2),⋯,(0),(1),⋯,(d−1))⊺∈ℝ2d−1. κ=(K(d-1),K(d-2),·s,K(0),K(1),·s,K(d-1)) ^2d-1. (9) Here we adopt a 1D notation for simplicity; the rigorous 2D form on the image grid is given in Appendix B. The convolution (⋆)( ν κ) can be efficiently evaluated via FFT in (dlogd)O(d d) time, with the remaining element-wise operations costing (d)O(d). Moreover, this is a one-time cost: once μ is precomputed, each RMA-based score Mj,τ=pj/(τ+μj)M_j,τ=p_j/(τ+ _j) is obtained in (1)O(1) cost. With the RMA reducing the reciprocal moment to the first moment μ, which admits efficient computation as shown above, the optimal volume follows as τ∗=argmaxτ∈[d]∑j∈Topτ(1:d,τ)Mj,τ. τ^*= argmax_τ∈[d] _j _τ( M_1:d,τ)M_j,τ. (10) However, unlike under CIA, the ranking of 1:d,τ M_1:d,τ depends on τ, which we resolve via a fixed-point iteration in the next section. 3.3 Fixed-point Optimization for Solving τ∗τ^* While RMA accelerates the evaluation of individual pixel scores, finding the optimal volume τ∗τ^* remains another bottleneck. Solving (10) naively requires re-ranking the score vector 1:d,τ M_1:d,τ for each candidate τ, resulting in (d2logd)O(d^2 d) complexity. In contrast, under the CIA-based RankSEG framework [7, 25], the ranking is invariant across τ and is solely decided by the pixel-wise probabilities p. Motivated by this observation, we isolate the global ranking from the volume selection by formulating the search for τ∗τ^* as a fixed-point iteration. We introduce a coupled operator T:[d]→[d]T:[d]→[d] that alternates between two steps: (i) computing the global ranking based on a fixed volume, and (i) selecting the optimal volume based on that ranking: T(τ)=argmaxτ′∈[d]πτ′((τ))=argmaxτ′∈[d]∑j=1τ′Moj(τ),τ′with(τ)=argsort(1:d,τ). T(τ)= argmax_τ ∈[d] _τ ( o(τ) )= argmax_τ ∈[d] _j=1^τ M_o_j(τ),τ o(τ)=argsort( M_1:d,τ). (11) The following lemma confirms that every optimal volume τ∗τ^* of Equation 10 is a fixed point of T. Lemma 1. Denote Fix(T)=τ∈[d]:T(τ)=τFix(T)=\τ∈[d]:T(τ)=τ\ as the set of fixed points of T, then τ∗∈Fix(T)τ^* (T). This leads to a natural fixed-point iteration for finding τ∗τ^*, τ(t+1)=T(τ(t))=argmaxτ′∈[d]πτ′((τ(t))),τ^(t+1)=T(τ^(t))= argmax_τ ∈[d] _τ ( o(τ^(t)) ), where (τ(0)):=argsort() o(τ^(0)):=argsort( p) is the initial global ranking induced by the pixel-wise probabilities. The procedure terminates when the ranking stabilizes. Lemma 2 (Monotone Ascent). Let τ(t)t≥0\τ^(t)\_t≥ 0 be the sequence generated by τ(t+1)=T(τ(t))τ^(t+1)=T(τ^(t)), and define Φ(τ)=πτ((τ)) (τ)= _τ( o(τ)). Then: (a) Φ(τ(t+1))≥Φ(τ(t)) (τ^(t+1))≥ (τ^(t)) for all t≥0t≥ 0. (b) If the maximizer in T is unique for every τ∈[d]τ∈[d], the iterates converge to a fixed point of T. The uniqueness condition in Lemma 2 (b) is exceedingly mild: non-uniqueness requires τ′↦πτ′((τ))τ _τ ( o(τ)) to attain identical values at two or more distinct integers, a degeneracy that defines an algebraic variety of measure zero in (,)( p, μ) space. In particular, when the probabilities p are produced by a neural network with continuous-valued outputs, the condition holds almost surely. Lemma 3 (One-Step Convergence). If the label distribution satisfies the following condition: pj≥pj′⟹μj≤μj′for all j≠j′, p_j≥ p_j _j≤ _j all j≠ j , then 1:d,τ M_1:d,τ shares the same ranking as p for all τ, and the iteration converges to τ∗τ^* in one step. Surprisingly, the iteration typically converges in a single step in practice. Lemma 3 offers a partial explanation by establishing a sufficient condition under which the initial ranking induced by p already matches with the true global ranking. More broadly, we conjecture that the ranking induced by p remains close to that of 1:d,τ∗ M_1:d,\,τ^* even when the condition in Lemma 3 does not hold exactly. The empirical prevalence of single-step convergence suggests that typical label distributions in segmentation yield a well-behaved landscape for T. With the approximate cumulative sums pre-computed as described in Appendix C, each iteration of (11) costs (dlogd)O(d d), dominated by the sorting operation. Given the one-step convergence observed in practice, we conclude that the overall complexity remains (dlogd)O(d d). 4 Error Analysis of RMA Theorem 2 provides bounds on the reciprocal moment in terms of the first and second moments. Theorem 2 (Reciprocal Moment Approximation [26]). Let Γ be a sum of dependent Bernoulli random variables with mean μ and variance σ2σ^2. For any τ>0τ>0, it satisfies: (μ+τ)−1≤(τ+Γ)−1≤μ+σ2/τμ(μ+τ)+σ2. (μ+τ)^-1 (τ+ )^-1≤ μ+σ^2/τμ(μ+τ)+σ^2. (12) The approximation error, defined as the difference between the upper and lower bounds, satisfies: ℰ≤σ2τ[μ(μ+τ)+σ2]. ≤ σ^2τ[μ(μ+τ)+σ^2]. (13) This theorem generalizes the result of Wang and Dai [25], which applies only to sums of independent Bernoulli random variables, to the setting where CIA does not hold. Moreover, under SLD, the covariance between labels decays exponentially, so the variance of the sum does not fluctuate significantly, growing only at the rate (d)O(d). Consequently, as shown in Theorem 3, the error decays at the rate (d−1)O(d^-1), coinciding with that of Wang and Dai [25], which is desirable for image segmentation tasks where d is typically large. The additional regularity assumptions are mild and naturally satisfied in practice: the condition μj/d=Θ(1) _j/d= (1) simply requires that the target object occupies a non-vanishing fraction of the image; while the strict positivity of the softmax function followed from the neural network inherently ensures pjp_j is bounded away from zero. Theorem 3 (RMA error under SLD). For Γj=∑i=1dYi|Yj=1 _j= _i=1^dY_i Y_j=1, denote σj2=Var(Γj) _j^2=Var( _j). Then: (a) Under Assumption 1 and pj≥c>0p_j≥ c>0 for some constant, σj2=(d) _j^2=O(d) for all j∈[d]j∈[d]; (b) Further assuming μj/d=Θ(1) _j/d= (1), then |Mj,τ−Sj,τ|=(d−1)|M_j,τ-S_j,τ|=O(d^-1) for all j,τ∈[d]j,τ∈[d]. In words, the SLD structure prevents accumulated variance from growing faster than (d)O(d). Consequently, when the target occupies a non-vanishing fraction of the image and pixel probabilities are nondegenerate, the RMA score approximation becomes increasingly accurate as d grows. 5 Experiments 5.1 Settings Datasets. We evaluate our method on five publicly available datasets spanning diverse applications: LiTS [2] and KiTS [13] for medical imaging, DeepGlobe Land [9] for remote sensing, and ADE20K [31] and Cityscapes [6] for natural image segmentation. Networks. Since our method is model-agnostic and applies as a post-processing step on the predicted probability map, we evaluate it across diverse segmentation networks, including UNet [20], DeepLabV3+ [5], PSPNet [30], UPerNet [27], and SegFormer [28]. The first four are CNN-based models, while SegFormer is a transformer-based model. See Appendix G for more training details. Compared Methods. To demonstrate the benefits of relaxing CIA to SLD and to evaluate our method as a training-free post-processing module, we compare against approaches using the same predicted probability map without modifying training: (i) Argmax-prob, which assigns each pixel the highest-probability category. This is the most widely used inference rule prior to the RankSEG; (i) CIA-RankSEG [25], which optimizes Dice/IoU under the CIA assumption. We also evaluated CRF [14], but since it consistently underperformed Argmax-prob, we defer its results to Appendix E. Evaluation Metrics. We report the image-wise mDice and mIoU for all datasets, following the computation of mDiceI^I and mIoUI^I defined in Wang et al. [24]. Algorithmic Details. We uniformly set θ=300θ=300 for the Gaussian kernel in SLD across all datasets and networks. For multi-class segmentation, we adopt the same strategy as Wang and Dai [25]: first apply binary RankSEG to each class independently, and then resolve conflicts by selecting the class with the highest score increment; See details in Appendix F. 5.2 Main Results We evaluate our method against the traditional Argmax-prob baseline and state-of-the-art CIA-RankSEG. As summarized in Tables 1 and 2, our method yields consistent improvements by relaxing the CIA to SLD and capturing label dependence. We highlight the following observations: Model- and Dataset-Agnostic Improvements. Gains are robust across both architectures and data domains. They hold uniformly across CNN- and transformer-based backbones, showing insensitivity to the model design, and transfer seamlessly from medical to natural segmentation. This dual robustness confirms that our method serves as a versatile post-processing module. Pronounced Gains on Low-Contrast Datasets. The improvements are most pronounced on low-contrast medical images with ambiguous boundaries (LiTS, KiTS), where label dependence provides a crucial signal for accurate segmentation. For example, on LiTS and KiTS with DeepLabV3+, our method gains +2.49+2.49 and +2.91+2.91 Dice over Argmax-prob, which are +0.37+0.37 and +0.51+0.51 over CIA-RankSEG, respectively. This corroborates our motivation that capturing label dependence is especially valuable when predictions are inherently uncertain. Table 1: Comparison of performance (Dice and IoU %) in LiTS and KiTS. Model Prediction KiTS LiTS IoU Dice IoU Dice UNet Argmax-prob 51.00 57.36 38.45 47.58 CIA-RankSEG 53.54 60.07 40.70 50.07 Ours 53.92 60.48 40.77 50.16 DeepLabV3+ Argmax-prob 54.19 61.16 38.34 47.38 CIA-RankSEG 56.22 63.56 40.09 49.50 Ours 56.70 64.07 40.49 49.88 Table 2: Comparison of performance (mDice and mIoU %) in ADE20K, Cityscapes, and DeepGlobe. Model Prediction ADE20K Cityscapes DeepGlobe mIoU mDice mIoU mDice mIoU mDice PSPNet Argmax-prob 51.32 58.66 73.07 80.45 57.24 66.23 CIA-RankSEG 51.57 59.17 73.72 81.14 58.20 67.82 Ours 51.92 59.49 73.84 81.27 58.47 68.14 DeepLabV3+ Argmax-prob 52.53 59.57 73.37 80.59 57.75 66.73 CIA-RankSEG 52.64 59.95 73.92 81.24 58.38 67.94 Ours 53.02 60.27 74.08 81.42 58.69 68.27 SegFormer Argmax-prob 54.09 61.03 73.32 80.53 59.87 68.74 CIA-RankSEG 54.72 61.92 74.10 81.38 60.87 70.14 Ours 55.07 62.23 74.23 81.51 61.12 70.42 UPerNet Argmax-prob 56.94 63.98 75.66 82.61 59.49 68.60 CIA-RankSEG 57.67 64.92 76.17 83.21 60.19 69.57 Ours 57.94 65.19 76.25 83.31 60.53 69.91 5.3 Fined-grained Improvements To investigate the source of our improvements, we conduct a fine-grained analysis along two dimensions: (i) easy vs. hard cases, where difficulty is measured by Argmax-prob performance; and (i) small vs. large categories, where difficulty is measured by the average pixel area of each category. In both analyses, we report relative improvements over baselines to account for varying difficulty levels. Table 3: Relative Dice improvements over baselines on KiTS subsets, formed by the lower quantiles of Argmax-prob performance. Quantile 0.2 0.4 0.6 Argmax-prob +14.37 % +7.46 % +3.97 % CIA-RankSEG +2.51 % +1.04 % +0.48 % Easy vs. Hard cases: We split KiTS into subsets by the lower quantile of Argmax-prob performance, where lower values correspond to harder cases. As shown in Table 3, our method yields larger gains on harder subsets, while improvements on easier subsets become modest. This confirms that modeling label dependence is most valuable when segmentation is challenging. Table 4: Relative Dice improvements over baselines on ADE20K across category groups, defined by average pixel area. Group small medium large Argmax-prob +5.47 % +2.92 % +2.01 % CIA-RankSEG +1.19 % +0.82 % +0.42 % Small vs. Large categories: We divide ADE20K categories into small, medium, and large groups of equal size based on their average pixel area. As shown in Table 4, we observe a similar trend: improvements are more pronounced on smaller categories, which are typically harder to segment and benefit more from modeling label dependence. 5.4 Statistical Significance of Improvements Figure 3: Our method consistently outperforms CIA-RankSEG across 10 retraining runs on (a) ADE20K, (b) Cityscapes, and (c) DeepGlobe, with paired t-test p-values ≪0.01 0.01 confirming the statistical significance. We conduct 10 independent retraining runs with different random seeds on ADE20K, Cityscapes, and DeepGlobe using UPerNet, and report per-run performance comparison between CIA-RankSEG and our method in Figure 3. Each plot also reports the p-value of a paired t-test, which is well below 0.01, confirming the statistical significance of the improvements. Furthermore, our method consistently outperforms CIA-RankSEG across all runs, reinforcing its role as an effective training-free inference module that squeezes additional performance from pre-trained networks. 5.5 Runtime Comparison To assess practicality, we compare the runtime of our method with CIA-RankSEG, using DeepLabV3+ on LiTS/KiTS and UPerNet on the remaining datasets. All experiments run on a single NVIDIA A100 GPU, with runtime measured over the entire inference process on the test set. As shown in Figure 5, the model forward pass (bottom bar) dominates the overall inference time, and the additional cost introduced by our method is marginal relative to CIA-RankSEG. This efficiency stems from a careful algorithmic design that reduces the complexity to (dlogd)O(d d), matching CIA-RankSEG up to a slightly larger constant factor arising from the additional convolution steps. These results confirm that our method serves as a practical module that can be readily integrated into existing segmentation pipelines without incurring significant computational overhead. 5.6 Parameter Robustness in SLD Our method introduces a single hyperparameter, the SLD kernel bandwidth θ, which controls the decay rate of label covariance. We perform a sensitivity analysis by sweeping θ from 33 to 600600 on KiTS and LiTS with DeepLabV3+. As shown in Figure 5, the curve exhibits two distinct regimes. (a) Robust plateau (100≤θ≤600100≤θ≤ 600): performance is nearly invariant across this range, indicating that our method requires no careful tuning and delivers consistent gains out of the box. (b) Baseline convergence (θ<100θ<100): as θ→0θ→ 0, performance degrades toward the CIA-RankSEG baseline. This behavior directly follows from (5): as θ→0θ→ 0, the covariance vanishes for all distinct label pairs, reducing SLD to CIA. This further validates our theoretical motivation of our method, and serves as an implicit ablation, underscoring the importance of incorporating label dependence. Figure 4: Runtime comparison between CIA-RankSEG and our method during the inference. The additional cost of our method is minor compared to the overall inference time. Figure 5: Robustness to θ in SLD. The performance is stable and saturates as θ≥100θ≥ 100. 6 Conclusion In this work, we addressed a critical limitation in image segmentation by relaxing the CIA to SLD modeling. We incorporated local label correlations into RankSEG while developing an efficient algorithm for optimizing Dice/IoU with practical (dlogd)O(d d) complexity. Extensive experiments across multiple datasets demonstrate that our method consistently outperforms CIA-based methods, especially on challenging cases where image information is noisy and label dependence is critical. One limitation of our work is that the proposed SLD captures label dependence purely through spatial structure, without exploiting appearance cues such as similarities between XiX_i and XjX_j or between pip_i and pjp_j. Incorporating such cues is highly non-trivial, as it requires careful redesign to effectively capture the complex interplay between appearance and spatial structure, while preserving computational efficiency. We leave this as a promising direction for future work. References [1] M. Berman, A. R. Triki, and M. B. Blaschko (2018) The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 4413–4421. Cited by: §1. [2] P. Bilic, P. Christ, H. B. Li, E. Vorontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. H. Mamani, G. Chartrand, et al. (2023) The liver tumor segmentation benchmark (lits). Medical Image Analysis 84, p. 102680. Cited by: §1, §5.1. [3] L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille (2014) Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv Preprint arXiv:1412.7062. Cited by: §E.2, Appendix E, §3.1. [4] L. Chen, G. Papandreou, F. Schroff, and H. Adam (2017) Rethinking atrous convolution for semantic image segmentation. arXiv Preprint arXiv:1706.05587. Cited by: §E.2, §1. [5] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), p. 801–818. Cited by: §E.2, §5.1. [6] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele (2016) The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 3213–3223. Cited by: §5.1. [7] B. Dai and C. Li (2023) RankSEG: a consistent ranking-based framework for segmentation. Journal of Machine Learning Research 24 (224), p. 1–50. Cited by: §A.5, §1, §2.1, §2.2, §2.2, §3.3, Theorem 1. [8] K. Dembczynski, A. Jachnik, W. Kotlowski, W. Waegeman, and E. Hüllermeier (2013) Optimizing the f-measure in multi-label classification: plug-in rule approach versus structured loss minimization. In International conference on machine learning, p. 1130–1138. Cited by: §A.5, §2.2, Theorem 1. [9] I. Demir, K. Koperski, D. Lindenbaum, G. Pang, J. Huang, S. Basu, F. Hughes, D. Tuia, and R. Raskar (2018) Deepglobe 2018: a challenge to parse the earth through satellite images. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, p. 172–181. Cited by: §5.1. [10] T. Eelbode, J. Bertels, M. Berman, D. Vandermeulen, F. Maes, R. Bisschops, and M. B. Blaschko (2020) Optimization for medical image segmentation: theory and practice when evaluating with dice score or jaccard index. IEEE Transactions on Medical Imaging 39 (11), p. 3679–3690. Cited by: §1. [11] C. Farabet, C. Couprie, L. Najman, and Y. LeCun (2012) Learning hierarchical features for scene labeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (8), p. 1915–1929. Cited by: §3.1. [12] J. J. Finocchiaro, R. Frongillo, and E. B. Nueve (2022) The structured abstain problem and the lovász hinge. In Conference on Learning Theory, p. 3718–3740. Cited by: §1. [13] N. Heller, F. Isensee, K. H. Maier-Hein, X. Hou, C. Xie, F. Li, Y. Nan, G. Mu, Z. Lin, M. Han, et al. (2021) The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: results of the kits19 challenge. Medical Image Analysis 67, p. 101821. Cited by: §5.1. [14] P. Krähenbühl and V. Koltun (2011) Efficient inference in fully connected crfs with gaussian edge potentials. Advances in Neural Information Processing Systems 24. Cited by: §E.1, Appendix E, §3.1, §5.1. [15] J. Lafferty, A. McCallum, and F. C. Pereira (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. Cited by: Appendix E. [16] H. Ling, K. Kreis, D. Li, S. W. Kim, A. Torralba, and S. Fidler (2021) Editgan: high-precision semantic image editing. Advances in Neural Information Processing Systems 34, p. 16331–16345. Cited by: §1. [17] A. Mehrtash, W. M. Wells, C. M. Tempany, P. Abolmaesumi, and T. Kapur (2020) Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE Transactions on Medical Imaging 39 (12), p. 3868–3878. Cited by: §1. [18] J. M. Ortega and W. C. Rheinboldt (2000) Iterative solution of nonlinear equations in several variables. SIAM. Cited by: §D.2. [19] M. A. Rahman and Y. Wang (2016) Optimizing intersection-over-union in deep neural networks for image segmentation. In International Symposium on Visual Computing, p. 234–244. Cited by: §1. [20] O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, p. 234–241. Cited by: §1, §5.1. [21] M. Siam, M. Gamal, M. Abdel-Razek, S. Yogamani, M. Jagersand, and H. Zhang (2018) A comparative study of real-time semantic segmentation for autonomous driving. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, p. 587–597. Cited by: §1. [22] A. Tarski (1955) A lattice-theoretical fixpoint theorem and its applications.. Cited by: §D.2. [23] A. Tewari and P. L. Bartlett (2007) On the consistency of multiclass classification methods.. Journal of Machine Learning Research 8 (5). Cited by: §1. [24] Z. Wang, M. Berman, A. Rannen-Triki, P. Torr, D. Tuia, T. Tuytelaars, L. V. Gool, J. Yu, and M. Blaschko (2023) Revisiting evaluation metrics for semantic segmentation: optimization and evaluation of fine-grained intersection over union. Advances in Neural Information Processing Systems 36, p. 60144–60225. Cited by: Appendix G, §5.1. [25] Z. Wang and B. Dai (2025) RankSEG-rma: an efficient segmentation algorithm via reciprocal moment approximation. arXiv Preprint arXiv:2510.15362. Cited by: Appendix F, Appendix G, §1, §2.1, §2.3, §3.2, §3.3, §4, §5.1, §5.1. [26] D. A. Wooff (1985) Bounds on reciprocal moments with applications and developments in stein estimation and post-stratification. Journal of the Royal Statistical Society: Series B (Methodological) 47 (2), p. 362–371. Cited by: §A.5, Theorem 2. [27] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun (2018) Unified perceptual parsing for scene understanding. In Proceedings of the European conference on computer vision (ECCV), p. 418–434. Cited by: §5.1. [28] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo (2021) SegFormer: simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34, p. 12077–12090. Cited by: §1, §5.1. [29] J. Yu and M. B. Blaschko (2018) The lovász hinge: a novel convex surrogate for submodular losses. IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (3), p. 735–748. Cited by: §1. [30] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia (2017) Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 2881–2890. Cited by: §1, §5.1. [31] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba (2017) Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 633–641. Cited by: §5.1. Appendix A Proofs A.1 Proof of Lemma 1 Proof. From (10), τ∗τ^* can be expressed as: τ∗=argmaxτ∈[d]max∑j=1τ∈ρ[d]Mlj,τ, τ^*= argmax_τ∈[d] _ l∈ _[d] _j=1^τM_l_j,τ, where ρ[d] _[d] denotes the set of all permutations of [d][d]. Since (τ∗)=argsort(1:d,τ∗) o(τ^*)=argsort( M_1:d,τ^*) already maximizes the inner sum for τ∗τ^*, so we have ∑j=1τ∗Moj(τ∗),τ∗=maxτ∈[d]max∑j=1τ∈ρ[d]Mlj,τ≥max∑j=1τ∈[d]Moj(τ∗),τ _j=1^τ^*M_o_j(τ^*),τ^*= _τ∈[d] _ l∈ _[d] _j=1^τM_l_j,τ≥ _τ∈[d] _j=1^τM_o_j(τ^*),τ The inequality is due to replacing the inner maximizer with a specific choice (τ∗) o(τ^*). The right-hand side is precisely what is maximized in T, and thus T(τ∗)=τ∗T(τ^*)=τ^*. ∎ A.2 Proof of Lemma 2 Proof. For any τ∈[d]τ∈[d], we can establish the following chain of inequalities: Φ(T(τ)) (T(τ)) =πT(τ)((T(τ))) = _T(τ) ( o(T(τ)) ) (=∑j=1T(τ)Moj(T(τ)),T(τ)) (= _j=1 [rgb]0,0.5,0T(τ)M_o_j( [rgb]0,0.5,0T(τ)), [rgb]0,0.5,0T(τ) ) ≥πT(τ)((τ)) ≥ _T(τ) ( o(τ) ) (=∑j=1T(τ)Moj(τ),T(τ)) (= _j=1 [rgb]0,0.5,0T(τ)M_o_j( [rgb]1,0,0τ), [rgb]0,0.5,0T(τ) ) ≥πτ((τ)) ≥ _τ ( o(τ) ) (=∑j=1τMoj(τ),τ) (= _j=1 [rgb]1,0,0τM_o_j( [rgb]1,0,0τ), [rgb]1,0,0τ ) =Φ(τ). = (τ). The first inequality holds because (T(τ))=argsort(1:d,T(τ)) o(T(τ))=argsort( M_1:d,T(τ)) maximizes the top-T(τ)T(τ) partial sum over all permutations, and the second follows from T(τ)=argmaxτ′πτ′((τ))T(τ)= argmax_τ _τ ( o(τ)). This establishes claim (a). For claim (b), if the maximizer in T is always unique and T(τ)≠τT(τ)≠τ, the second inequality above is strict, giving Φ(T(τ))>Φ(τ) (T(τ))> (τ). Applied along the iterates, this means Φ(τ(t+1))>Φ(τ(t)) (τ^(t+1))> (τ^(t)) whenever T(τ(t))≠τ(t)T(τ^(t))≠τ^(t). If this strict inequality were to hold for all t∈0,1,…,dt∈\0,1,…,d\, we would obtain d+1d+1 pairwise distinct values Φ(τ(0))<Φ(τ(1))<⋯<Φ(τ(d)) (τ^(0))< (τ^(1))<·s< (τ^(d)), contradicting the fact that Φ takes at most d values on the finite set [d][d]. Hence there exists a smallest index t⋆≤dt ≤ d with T(τ(t⋆))=τ(t⋆)T(τ^(t ))=τ^(t ). Setting τ⋆:=τ(t⋆)τ :=τ^(t ), a trivial induction gives τ(t⋆+s)=τ⋆τ^(t +s)=τ for all s≥0s≥ 0, so the iterates reach the fixed point τ⋆τ of T in at most d steps. ∎ A.3 Proof of Lemma 3 Proof. Recall the definition Mj,τ=pj/(τ+μj)M_j,τ=p_j/(τ+ _j). For any j≠j′j≠ j such that pj≥pj′p_j≥ p_j , we have μj≤μj′ _j≤ _j by assumption, and thus Mj,τ=pjτ+μj≥pj′τ+μj′=Mj′,τfor all τ. M_j,τ= p_jτ+ _j≥ p_j τ+ _j =M_j ,τ all τ. Hence, the ranking of 1:d,τ M_1:d,τ is exactly identical to that of p for all τ. In particular, (τ)=argsort(1:d,τ)=argsort() o(τ)=argsort( M_1:d,τ)=argsort( p) for all τ. Since we initialize with (τ(0))=argsort() o(τ^(0))=argsort( p), the iteration converges in one step. ∎ A.4 Proof of Theorem 3 Proof. Without loss of generality, let the image be an L×L× L grid where d=L2d=L^2. Each pixel index i∈[d]i∈[d] maps to a 2D coordinate (ui,vi)∈[L]2(u_i,v_i)∈[L]^2 given by: ui=⌊(i−1)/L⌋+1,vi=((i−1) mod L)+1. u_i= (i-1)/L +1, v_i=((i-1) mod L)+1. The squared Euclidean distance between pixels i and k is then given by: r(i,k)2=(ui−uk)2+(vi−vk)2. r(i,k)^2=(u_i-u_k)^2+(v_i-v_k)^2. Step 1: Bound the unconditional variance. Denote Γ=∑i=1dYi = _i=1^dY_i. Its variance under SLD is: Var(Γ)=∑i=1d∑k=1dΣik,whereΣik=ΣiiΣkkexp(−r(i,k)22θ2). ( )= _i=1^d _k=1^d _ik, _ik= _i _k (- r(i,k)^22θ^2 ). Note Σii=pi(1−pi)≤1/4 _i=p_i(1-p_i)≤ 1/4 for all i. It follows that Σik≤0.25exp(−(ui−uk)22θ2)exp(−(vi−vk)22θ2). _ik≤ 0.25 (- (u_i-u_k)^22θ^2 ) (- (v_i-v_k)^22θ^2 ). For a fixed pixel i, we can extend the summation boundaries from the finite L×L× L grid to the infinite ℤ2Z^2 grid, and obtain ∑k=1dΣik≤0.25(∑Δu=−∞exp(−Δu22θ2))(∑Δv=−∞exp(−Δv22θ2)). _k=1^d _ik≤ 0.25 ( _ u=-∞^∞ (- u^22θ^2 ) ) ( _ v=-∞^∞ (- v^22θ^2 ) ). Consider ∑k=−∞exp(−k2/2θ2) _k=-∞^∞ (-k^2/2θ^2), since the terms decay exponentially, the series converges to a finite constant that depends only on θ, denoted as CθC_θ. Hence ∑k=1dΣik≤0.25Cθ2=:Kθ _k=1^d _ik≤ 0.25C_θ^2=:K_θ for all i, and therefore Var(Γ)=∑i=1d∑k=1dΣik≤∑i=1dKθ=Kθd=(d). ( )= _i=1^d _k=1^d _ik≤ _i=1^dK_θ=K_θd=O(d). Step 2: Bound the conditional variance. We decompose Var(Γ)Var( ) conditioning on YjY_j by the law of total variance: Var(Γ)=Yj[Var(Γ∣Yj)]+VarYj[(Γ∣Yj)]. ( )=E_Y_j [Var( Y_j) ]+Var_Y_j [E( Y_j) ]. Dropping the second term gives pjVar(Γ∣Yj=1)+(1−pj)Var(Γ∣Yj=0)=Yj[Var(Γ∣Yj)]≤Var(Γ)=(d). p_jVar( Y_j=1)+(1-p_j)Var( Y_j=0)=E_Y_j [Var( Y_j) ] ( )=O(d). Since pj≥c>0p_j≥ c>0 for some constant c, we have Var(Γ∣Yj=1)=(d)Var( Y_j=1)=O(d). This proves claim (a). Step 3: Bound the RMA error. From Theorem 2, the approximation error of RMA is bounded by ℰ≤σj2/[τ(μj(μj+τ)+σj2)]E≤ _j^2/[τ( _j( _j+τ)+ _j^2)]. Note that |Mj,τ−Sj,τ|≤pjℰ|M_j,τ-S_j,τ|≤ p_jE, and pj≤1p_j≤ 1. Under the additional assumption μj=Θ(d) _j= (d), we have: |Mj,τ−Sj,τ|≤ℰ≤σj2τ[μj(μj+τ)+σj2]=(d)τ[Θ(d)(Θ(d)+τ)+(d)]≤(d)Θ(d2)=(d−1). |M_j,τ-S_j,τ| ≤ _j^2τ[ _j( _j+τ)+ _j^2]= O(d)τ[ (d)( (d)+τ)+O(d)]≤ O(d) (d^2)=O(d^-1). This proves claim (b). ∎ A.5 Proof of Theorem 1 and Theorem 2 Theorem 1 combines results from Dembczynski et al. [8] and Dai and Li [7], and Theorem 2 is a direct application of Wooff [26, Theorem 1]. For completeness, we provide the proof of Theorem 1 here, while deferring the proof of Theorem 2 to the original paper. Proof. Recall that the expected Dice score for a segmentation mask ∈0,1d δ∈\0,1\^d is defined as: [Dice()]=[2⊺‖1+‖1]=[2∑j=1dδjYj‖1+‖1]. [Dice( δ)]=E [ 2 δ Y\| δ\|_1+\| Y\|_1 ]=E [ 2 _j=1^d _jY_j\| δ\|_1+\| Y\|_1 ]. Let τ=‖1τ=\| δ\|_1 denote the predicted volume. The expected Dice score can then be rewritten as: [Dice()]=2∑j=1dδj[Yjτ+Γj]=2∑j=1dδjpj[1τ+Γj∣Yj=1]=2∑j=1dδjpj(τ+Γj)−1. [Dice( δ)]=2 _j=1^d _jE [ Y_jτ+ _j ]=2 _j=1^d _jp_jE [ 1τ+ _j Y_j=1 ]=2 _j=1^d _jp_jE(τ+ _j)^-1. Denote the contribution of pixel j for a fixed volume τ as Sj,τ=pj(τ+Γj)−1S_j,τ=p_jE(τ+ _j)^-1. We decompose the maximization into bi-level optimization: maxτ∈[d]max∈0,1d:‖1=τ2∑j=1dδjSj,τ. _τ∈[d] _\ δ∈\0,1\^d:\| δ\|_1=τ\2 _j=1^d _jS_j,τ. For the inner maximization with τ fixed, the optimal strategy is to set δj=1 _j=1 for the top-τ pixels with the largest Sj,τS_j,τ, yielding ωτ=∑j∈Topτ(1:d,τ)Sj,τ _τ= _j _τ( S_1:d,τ)S_j,τ. For the outer maximization, we evaluate ωτ _τ for all τ and select the best one. This completes the proof of Theorem 1. ∎ Appendix B Two-Dimensional Convolutional Form of μ In Section 3.2, we express μ through a one-dimensional convolution over the vectorized labels for notational simplicity, where the kernel entry d−j+i=(|i−j|) κ_d-j+i=K(|i-j|) is indexed by the offset between the 1D indices i and j. For images, however, the distance r(i,j)r(i,j) in Assumption 1 is the spatial distance between two pixels on the 2D grid, which is not a function of |i−j||i-j| after vectorization. In this section, we present the rigorous two-dimensional counterpart, which is what our implementation computes. Setup. Consider an image of height H and width W, so that d=HWd=HW. Following the notation in the proof of Theorem 3, each pixel index j∈[d]j∈[d] maps to a 2D coordinate (uj,vj)∈[H]×[W](u_j,v_j)∈[H]×[W] under the row-major ordering: uj=⌊(j−1)/W⌋+1,vj=((j−1) mod W)+1, u_j= (j-1)/W +1, v_j=((j-1) mod W)+1, and the distance in Assumption 1 is the Euclidean distance on the grid, i.e., r(i,j)=(ui−uj)2+(vi−vj)2r(i,j)= (u_i-u_j)^2+(v_i-v_j)^2. Accordingly, we reshape p, =diag() ν= diag( ), and μ into matrices ,,∈ℝH×W P, V, U ^H× W, with Puj,vj=pjP_u_j,v_j=p_j, Vuj,vj=ΣjjV_u_j,v_j= _j, and Uuj,vj=μjU_u_j,v_j= _j. Throughout this section, (u,v)∈[H]×[W](u,v)∈[H]×[W] denotes a generic 2D coordinate, while i,j∈[d]i,j∈[d] are reserved for 1D pixel indices. 2D convolutional form. Define the 2D kernel ∈ℝ(2H−1)×(2W−1) K ^(2H-1)×(2W-1) that stores the kernel values at all possible displacements (Δu,Δv)( u, v): H+Δu,W+Δv=(Δu2+Δv2),|Δu|≤H−1,|Δv|≤W−1, K_H+ u,\,W+ v=K ( u^2+ v^2 ), | u|≤ H-1,\ | v|≤ W-1, so that the center entry H,W=(0)=1 K_H,W=K(0)=1 corresponds to zero displacement. Then the SLD correction term in μj _j can be written as a 2D convolution (cross-correlation) of V with K: ∑i=1dΣii(r(i,j)) _i=1^d _i\,K(r(i,j)) =∑u=1H∑v=1WVu,v((u−uj)2+(v−vj)2) = _u=1^H _v=1^WV_u,v\,K ( (u-u_j)^2+(v-v_j)^2 ) =∑u=1H∑v=1WVu,vH+u−uj,W+v−vj=:(⋆)uj,vj. = _u=1^H _v=1^WV_u,v\, K_H+u-u_j,\,W+v-v_j=:( V K)_u_j,v_j. Therefore, computing all j∈[d]j∈[d] simultaneously yields the matrix form =q⋅H×W+⊙(⋆), U=q· 1_H× W+ V P ( V K), (14) where q=∑j=1dpjq= _j=1^dp_j, H×W 1_H× W is the all-ones matrix, and both the division and ⊙ are element-wise. (14) is the exact 2D analogue of the 1D expression in Section 3.2: the vector ∈ℝ2d−1 κ ^2d-1 is replaced by the matrix ∈ℝ(2H−1)×(2W−1) K ^(2H-1)×(2W-1), and the 1D offset |i−j||i-j| is replaced by the 2D displacement (ui−uj,vi−vj)(u_i-u_j,v_i-v_j). Computational complexity. The 2D convolution ⋆ V K can be evaluated via the 2D FFT: after zero-padding V and K to a common size of (H)×(W)O(H)×O(W) to avoid circular wrap-around, the cost is (HWlog(HW))=(dlogd)O(HW (HW))=O(d d), matching the complexity stated in Section 3.2. The remaining element-wise operations in (14) cost (d)O(d). Moreover, the Gaussian kernel is separable: since (Δu2+Δv2)=exp(−Δu2/2θ2)exp(−Δv2/2θ2)=(|Δu|)(|Δv|)K( u^2+ v^2)= (- u^2/2θ^2) (- v^2/2θ^2)=K(| u|)K(| v|), the 2D kernel is rank-one, =HW⊺ K= κ_H κ_W , where H=((H−1),⋯,(0),⋯,(H−1))⊺∈ℝ2H−1, κ_H=(K(H-1),·s,K(0),·s,K(H-1)) ^2H-1, W=((W−1),⋯,(0),⋯,(W−1))⊺∈ℝ2W−1. κ_W=(K(W-1),·s,K(0),·s,K(W-1)) ^2W-1. Consequently, the 2D convolution decomposes into two 1D convolutions of the same form as in Section 3.2: first convolving each column of V with H κ_H, and then convolving each row of the result with W κ_W, i.e., (⋆)uj,vj=∑v=1W(|v−vj|)∑u=1H(|u−uj|)Vu,v. ( V K)_u_j,v_j= _v=1^WK(|v-v_j|) _u=1^HK(|u-u_j|)\,V_u,v. Using the 1D FFT for each row and column, the total cost is (HW(logH+logW))=(dlogd)O(HW( H+ W))=O(d d). This also clarifies the role of the 1D notation in Section 3.2: it is exactly the building block of the separable 2D convolution. The same argument extends directly to 3D volumes (e.g., H×W×DH× W× D voxels) with a separable 3D Gaussian kernel, again with (dlogd)O(d d) complexity. Appendix C Efficient Cumulative Sums in Fixed-Point Iteration via Taylor Approximation We turn to the efficient implementation of the fixed-point iteration described in (11), focusing on the computation after (τ) o(τ) is obtained. Without loss of generality, we regard the ranking (τ) o(τ) as the natural ordering and simplify the notation to πτ=∑j=1τpj/(τ+μj) _τ= _j=1^τp_j/(τ+ _j). A direct implementation requires (d)O(d) time for each τ, resulting in (d2)O(d^2) total complexity. The core inefficiency arises because the denominator tightly couples τ and j, preventing the reuse of intermediate quantities. To decouple τ from j, we consider the Taylor expansion of x↦(τ+x)−1x (τ+x)^-1 around μ¯τ=1τ∑j=1τμj μ_τ= 1τ _j=1^τ _j: 1τ+μj=1τ+μ¯τ−μj−μ¯τ(τ+μ¯τ)2+(μj−μ¯τ)2(τ+μ¯τ)3+((μj−μ¯τ)3(τ+μ¯τ)4). 1τ+ _j= 1τ+ μ_τ- _j- μ_τ(τ+ μ_τ)^2+ ( _j- μ_τ)^2(τ+ μ_τ)^3+O ( ( _j- μ_τ)^3(τ+ μ_τ)^4 ). Truncating at the second-order term is an empirical trade-off between approximation accuracy and numerical stability. This yields our final approximation πτ=∑j=1τpjτ+μj≈π~τ _τ= _j=1^τ p_jτ+ _j≈ π_τ =∑j=1τpj(1τ+μ¯τ−μj−μ¯τ(τ+μ¯τ)2+(μj−μ¯τ)2(τ+μ¯τ)3) = _j=1^τp_j ( 1τ+ μ_τ- _j- μ_τ(τ+ μ_τ)^2+ ( _j- μ_τ)^2(τ+ μ_τ)^3 ) =Zτ0τ+μ¯τ−Zτ1−μ¯τZτ0(τ+μ¯τ)2+Zτ2−2μ¯τZτ1+μ¯τ2Zτ0(τ+μ¯τ)3, = Z_τ^0τ+ μ_τ- Z_τ^1- μ_τZ_τ^0(τ+ μ_τ)^2+ Z_τ^2-2 μ_τZ_τ^1+ μ_τ^2Z_τ^0(τ+ μ_τ)^3, where Zτ0=∑j=1τpj,Zτ1=∑j=1τpjμj,Zτ2=∑j=1τpjμj2. Z_τ^0= _j=1^τp_j, Z_τ^1= _j=1^τp_j _j, Z_τ^2= _j=1^τp_j _j^2. 1:d0 Z_1:d^0, 1:d1 Z_1:d^1, and 1:d2 Z_1:d^2 can be computed in (d)O(d) via cumulative sums, and ~1:d π_1:d is obtained in (d)O(d). Appendix D Convergence of the Fixed-Point Optimization In Section 3.3, we established two theoretical guarantees for the fixed-point iteration τ(t+1)=T(τ(t))τ^(t+1)=T(τ^(t)): under the uniqueness condition for the maximizer of T, the iterates monotonically increase the objective Φ and converge to a fixed point (Lemma 2); moreover, when the label distribution satisfies certain conditions, convergence to the global optimum τ∗τ^* is achieved in a single step (Lemma 3). Although these conditions are difficult to verify analytically on real data, the empirical results in this section show that the iteration behaves in close accordance with these guarantees: it converges within a few steps and reaches a point that is essentially near the global optimum. Specifically, we address two questions: (Q1) How many iterations are required for convergence in practice? (Q2) How close is the limit point to the global optimum? The findings below provide strong support for the practical effectiveness of the algorithm and offer empirical validation of Lemmas 2 and 3. D.1 Convergence within Three Steps Table 5 reports the distribution of the number of iterations required for convergence across all test samples. For multi-class datasets, we average the number of steps over classes for each sample. Across all five benchmarks, the iteration terminates within 2.52.5 steps for over 99.8%99.8\% of samples, and the fraction of samples requiring more than 2.52.5 steps never exceeds 0.22%0.22\%. Table 5: Distribution of the number of iterations required for convergence across all test samples. On every dataset, the iteration terminates within 2.52.5 steps for the vast majority of samples, with only a negligible fraction requiring more. Dataset LiTS KiTS ADE20K Cityscapes DeepGlobe #Steps <1.5<1.5 63.36% 88.52% 9.30% 2.80% 40.49% 1.5≤1.5≤ #Steps <2.5<2.5 36.42% 11.41% 90.55% 97.20% 50.51% 2.5≤2.5≤ #Steps 0.22% 0.07% 0.15% 0.00% 0.00% These results indicate that the fixed-point optimization converges substantially faster than the worst-case bound, allowing us to safely conclude that the proposed algorithm runs in (dlogd)O(d d) time in practice. We attribute this fast convergence to two factors: (i) the initialization τ(0)τ^(0) induced by the probabilities p is already close to the optimal τ∗τ^*, and (i) the landscape of T is well-behaved, with a large basin of attraction around τ∗τ^*. D.2 Convergent Point is Near-Optimal To assess whether the iteration locates the global optimum, we conduct a brute-force search on LiTS and KiTS, for which exhaustive enumeration is computationally affordable, and compare the performance between the global optimum and the convergent point of the fixed-point optimization. Table 6: Comparison between the convergent point obtained by the fixed-point optimization and the global optimum obtained by brute-force search. The performance gap is at most 0.030.03 on both metrics, indicating that the iteration consistently recovers a near-optimal fixed point. Dataset KiTS LiTS Metric IoU Dice IoU Dice Fixed-point Optimization 56.70 64.07 40.48 49.88 Brute-force Search 56.73 64.10 40.48 49.88 As shown in Table 6, the gap between the fixed-point solution and the global optimum is at most 0.030.03 across both datasets and both metrics, and is essentially zero on LiTS. These results suggest that the fixed-point optimization not only converges rapidly but also reliably identifies a solution that is very close to the best achievable performance. This near-optimality can again be attributed to the favorable initialization and the well-behaved landscape of T [22, 18]. Appendix E Discussion on SLD and CRF In Section 3.1, we introduced the SLD model as a more realistic alternative to the CIA for modeling label distributions in image segmentation. A natural question is how SLD relates to the widely used Conditional Random Field (CRF) model [15, 14], which also captures local label correlations and acts as a post-processing step following the network outputs [3]. In this section, we clarify the distinction between these two models and explain why SLD is more suitable for our purpose. E.1 The Conditional Random Field (CRF) Model CRFs have been a cornerstone of semantic segmentation, typically used to refine pixel-wise predictions by modeling the posterior distribution ℙ(∣)P( Y X). A CRF represents the conditional distribution as a Gibbs distribution: ℙ(∣)=1Z()exp(−E(,)), ( Y X)= 1Z( X) (-E( Y, X)), where E(,)E( Y, X) is the energy function (we omit the dependence on X hereafter), usually decomposed into unary and pairwise potentials [14]: E()=∑i=1dψu(Yi)+∑i<jψp(Yi,Yj). E( Y)= _i=1^d _u(Y_i)+ _i<j _p(Y_i,Y_j). The unary potential ψu _u is typically the negative log-probability from the network output, while the pairwise potential ψp _p is designed as: ψp(Yi,Yj)=μ(Yi,Yj)(w(1)exp(−|ui−uj|22θα2−|Xi−Xj|22θβ2)⏟appearance kernel+w(2)exp(−|ui−uj|22θγ2)⏟smoothness kernel), _p(Y_i,Y_j)=μ(Y_i,Y_j) (w^(1) (- |u_i-u_j|^22 _α^2- |X_i-X_j|^22 _β^2 )_appearance kernel+w^(2) (- |u_i-u_j|^22 _γ^2 )_smoothness kernel ), where μ(Yi,Yj)=(Yi≠Yj)μ(Y_i,Y_j)= 1(Y_i≠ Y_j) measures the label compatibility, and uiu_i and XiX_i denote the spatial coordinate and RGB value of pixel i, respectively. The appearance kernel encourages nearby pixels with similar colors to share the same label, while the smoothness kernel encourages nearby pixels to share the same label regardless of their colors. After modeling the conditional distribution, CRF-based methods typically perform Maximum a Posteriori (MAP) inference, through certain approximate inference techniques, to obtain the final segmentation mask: MAP=argmax∈0,1dℙ(=∣)=argmin∈0,1dE(). δ_MAP= argmax_ δ∈\0,1\^dP( Y= δ X)= argmin_ δ∈\0,1\^dE( δ). E.2 Key Distinction between SLD and CRF Although both SLD and CRF leverage Gaussian kernels to capture spatial locality, they differ fundamentally in their mathematical formulation and intent: • Covariance vs. Energy Modeling: SLD explicitly specifies the covariance structure of the label distribution. This allows us to efficiently compute the moment of Γj _j, which is the core requirement for optimizing the Dice/IoU expectation. In contrast, CRFs specify the joint energy. The resulting covariance matrix of a CRF is implicitly defined and generally lacks the closed-form structure needed for efficient RankSEG integration. • MAP Inference vs. Metric Optimization: CRF-based methods typically perform MAP inference to find the single most probable joint configuration. However, the MAP solution is not necessarily optimal for set-based metrics like Dice or IoU. SLD is designed specifically to facilitate direct metric optimization by max[Dice()] [Dice( δ)]. • Parameter Complexity: SLD requires only a single decay parameter θ to control spatial dependence. CRFs typically rely on a suite of hyperparameters (θα _α, θβ _β, θγ _γ, w(1)w^(1), and w(2)w^(2)) that often require exhaustive cross-validation to tune for specific datasets. The diminishing returns of CRF post-processing in modern pipelines are illustrated by the evolution of the DeepLab series. While DeepLabv2 [3] utilized CRFs to sharpen boundaries, subsequent versions [4, 5] removed them, finding that deep architectures capture sufficient context internally. Table 7: Performance comparison of CRF, Argmax-prob, CIA-RankSEG, and our method using UPerNet. Method CRF Argmax-prob CIA-RankSEG Ours mIoU mDice mIoU mDice mIoU mDice mIoU mDice DeepGlobe 58.61 67.40 59.49 68.60 60.19 69.57 60.52 69.91 Cityscapes 73.26 80.35 75.66 82.61 76.17 83.21 76.25 83.31 ADE20K 56.05 62.61 56.94 63.98 57.67 64.92 57.94 65.19 Our empirical results in Table 7 corroborate this: CRFs can even degrade performance compared to simple Argmax-prob. While the MAP prior was useful for refining probability masks in early vision tasks, it may actually hinder performance when applied to the highly accurate, calibrated probabilities produced by modern networks. Appendix F Extension to Multi-class Segmentation In non-overlapping multi-class segmentation, each pixel is assigned to exactly one class; that is, we seek ϕ∈[C]d φ∈[C]^d, where C denotes the number of classes. To extend our method to this setting, we adopt the incremental score strategy of Wang and Dai [25]. Specifically, we first apply the RankSEG algorithm independently to each class, yielding C binary masks ∈0,1C×d δ∈\0,1\^C× d. Based on these masks, we define the following three index sets: Pixels predicted as class c: predicted as class c: ℐc+ _c^+ =j:δc,j=1, =\j: _c,j=1\, Overlapping pixels: ℐoverlap ^overlap =∪c≠c′(ℐc+∩ℐc′+), = _c≠ c (I_c^+ _c ^+), Non-overlapping pixels of class c: -overlapping pixels of class c: ℐc _c =ℐc+∖ℐoverlap. =I_c^+ ^overlap. Pixels in ℐcI_c are safely assigned to class c. For each overlapping pixel j∈ℐoverlapj ^overlap, we assign it to the class that yields the largest incremental score: ϕj=argmaxc∈[C]Δc,j,∀j∈ℐoverlap, _j= argmax_c∈[C] _c,j, ∀ j ^overlap, where Δc,j=(∑i∈ℐcpc,i|ℐc|+1+μc,i+pc,j|ℐc|+1+μc,j)⏟score after adding pixel j to class c−∑i∈ℐcpc,i|ℐc|+μc,i⏟score before adding pixel j to class c. _c,j= ( _i _c p_c,i|I_c|+1+ _c,i+ p_c,j|I_c|+1+ _c,j )_score [rgb]1,0,0after adding pixel j to class c- _i _c p_c,i|I_c|+ _c,i_score [rgb]1,0,0before adding pixel j to class c. Here, pc,ip_c,i and μc,i _c,i denote the probability and conditional mean of pixel i for class c, respectively, analogous to the binary case. Since Δc,j _c,j quantifies the score gain from assigning pixel j to class c, it is natural to assign j to the class that maximizes this gain. This procedure is guaranteed to produce a valid non-overlapping segmentation mask. Appendix G Training Details The training settings follow Wang et al. [24], Wang and Dai [25], and we provide the details here for completeness. All models are trained with the standard cross-entropy loss to estimate calibrated probability masks. For DeepGlobe Land, Cityscapes, and ADE20K, we adopt the AdamW optimizer with a weight decay of 0.01. The learning rate starts from 1e-61e-6 and is linearly warmed up during the first 1%1\% of iterations to the initial learning rate of 6e-56e-5. Subsequently, the learning rate is decayed under a “poly” policy with an exponent of 1. The number of warm-up iterations is 400 for Cityscapes, and 800 for ADE20K. The total number of training iterations is 20,000 for DeepGlobe Land, 40,000 for Cityscapes, and 80,000 for ADE20K. Data augmentation consists of (i) random scaling within the range [0.5,2.0][0.5,2.0] and (i) random horizontal flipping with a probability of 0.5. Since ADE20K and Cityscapes provide designated validation sets, we report performance on the validation set for these two datasets. For DeepGlobe Land, we manually split the original training set into a training subset (80%80\%) and a validation subset (20%20\%), and report performance on the validation subset. For LiTS and KiTS, we train the models using SGD with an initial learning rate of 0.010.01, momentum of 0.90.9, and weight decay of 0.00050.0005. The learning rate is decayed under a “poly” policy with an exponent of 0.9. The batch size is 88, and the number of epochs is 6060. Although these two datasets are originally multi-class segmentation tasks, we convert them into binary segmentation problems by treating only the tumor as the foreground. This conversion is necessary because we compare our method with RankDice-BA, which is applicable only to binary segmentation. Moreover, since LiTS and KiTS do not include designated test sets, we employ 5-fold cross-validation to evaluate performance. Appendix H Qualitative Results Figure 6: Qualitative comparison of segmentation masks produced by different methods on LiTS. Figure 7: Qualitative comparison of segmentation masks produced by different methods on KiTS. Figure 8: Qualitative comparison on ADE20K. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region. Figure 9: Qualitative comparison on Cityscapes. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region. Figure 10: Qualitative comparison on DeepGlobe Land. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region. We complement the quantitative results in the main paper with qualitative visualizations. Figures 6 and 7 present representative examples from LiTS and KiTS, comparing the segmentation masks produced by different methods. In these medical datasets, the foreground objects (tumors) are often small and exhibit low contrast relative to the background. Such challenging conditions frequently lead to incomplete predictions by the baselines and require exploiting label dependence to recover the full object shape. In contrast, our method consistently produces more complete objects and more accurate shapes than the baselines, in line with the observed improvements in Dice and IoU scores. Figures 8, 9 and 10 present examples from ADE20K, Cityscapes, and DeepGlobe Land. On these natural-image datasets, our method primarily improves the segmentation of thin and distant structures whose pixel footprint is too small for the baselines to resolve. For instance, in both examples in Figure 9, our method successfully identifies or recovers more complete shapes of the distant human figures. Although these pedestrians occupy only a small number of pixels, accurately localizing them is critical for downstream tasks such as autonomous driving.