Paper deep dive
Hierarchical Adaptive Feature Refinement Network for VHR Remote Sensing Image Segmentation
Shuaishuai Cao, Meng Tang, Shuwei Peng, Xuan Liu, Min Huang, Jie Chen, Jiacheng Niu, Yong Chen, Edore Akpokodje, Hui Lin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/22/2026, 2:43:31 AM
Summary
The paper introduces HAFR-Net, a hierarchical adaptive feature refinement network for semantic segmentation of very-high-resolution (VHR) remote sensing imagery. It addresses the challenge of exploiting multi-stage encoder representations by using Heterogeneity-Guided Stage-Adaptive Fusion (HG-SAF) for pixel-wise stage weighting, a Frequency-Residual Adapter (FRA) for bounded residual frequency correction, and a Confusion-Aware Tri-Prior Decoder (CATP) for structural regularization. HAFR-Net outperforms the UPerNet baseline on four major VHR datasets (ISPRS Vaihingen, ISPRS Potsdam, LoveDA, OpenEarthMap) using a Swin-B backbone.
Entities (11)
Relation Signals (9)
HAFR-Net → achievesscoreon → ISPRS Vaihingen
confidence 95% · attains 84.12%... mIoU on ISPRS Vaihingen
HAFR-Net → achievesscoreon → ISPRS Potsdam
confidence 95% · attains 87.86%... mIoU on ISPRS Potsdam
HAFR-Net → achievesscoreon → LoveDA
confidence 95% · attains 55.17%... mIoU on LoveDA
HAFR-Net → achievesscoreon → OpenEarthMap
confidence 95% · attains 67.70%... mIoU on OpenEarthMap
HAFR-Net → usesbackbone → Swin-B
confidence 95% · Under a matched Swin-B training... HAFR-Net attains
HAFR-Net → usesmodule → HG-SAF
confidence 95% · HAFR-Net... Heterogeneity-Guided Stage-Adaptive Fusion (HG-SAF) predicts dense stage weights
HAFR-Net → usesmodule → FRA
confidence 95% · A Frequency-Residual Adapter (FRA) then injects frequency information
HAFR-Net → usesmodule → CATP
confidence 95% · A Confusion-Aware Tri-Prior Decoder (CATP) finally regularizes the prediction
HAFR-Net → outperforms → UPerNet
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Semantic segmentation of very-high-resolution (VHR) remote sensing imagery increasingly benefits from strong pretrained hierarchical encoders, yet exploiting their multi-stage representations remains difficult. Nearby regions demand different balances between fine detail and semantic context, aggressive task-specific transformations perturb useful pretrained features, and conventional semantic supervision provides limited structural guidance. We present HAFR-Net, a progressive refinement framework that adaptively organizes and conservatively refines hierarchical representations instead of replacing them with a monolithic decoder transformation. Heterogeneity-Guided Stage-Adaptive Fusion (HG-SAF) predicts dense stage weights conditioned on local feature variation. A Frequency-Residual Adapter (FRA) then injects frequency information through a bounded, zero-initialized residual branch that keeps the fused representation as its reference. A Confusion-Aware Tri-Prior Decoder (CATP) finally regularizes the prediction with boundary, objectness, and training-derived class-relation cues. Under a matched Swin-B training and single-scale inference protocol, HAFR-Net attains 84.12%, 87.86%, 55.17%, and 67.70% mIoU on ISPRS Vaihingen, ISPRS Potsdam, LoveDA, and OpenEarthMap, improving the matched UPerNet baseline by 0.55, 0.95, 1.55, and 1.84 percentage points, respectively. Controlled analyses further show consistent spatial reweighting beyond content-only routing, improved boundary and thin-structure accuracy over matched spatial and spectral alternatives, and reduced confusion on pre-declared class pairs.
Tags
Links
- Source: https://arxiv.org/abs/2608.15647v1
- Canonical: https://arxiv.org/abs/2608.15647v1
Trouble viewing inline? Open PDF directly →
Full Text
84,140 characters extracted from source content.
Expand or collapse full text
Hierarchical Adaptive Feature Refinement Network for VHR Remote Sensing Image Segmentation Shuaishuai Cao Meng Tang Shuwei Peng Xuan Liu Min Huang Jie Chen Jiacheng Niu Yong Chen Edore Akpokodje Hui Lin Thanks: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Abstract Semantic segmentation of very-high-resolution (VHR) remote sensing imagery increasingly benefits from strong pretrained hierarchical encoders, yet exploiting their multi-stage representations remains difficult. Nearby regions demand different balances between fine detail and semantic context, aggressive task-specific transformations perturb useful pretrained features, and conventional semantic supervision provides limited structural guidance. We present HAFR-Net, a progressive refinement framework that adaptively organizes and conservatively refines hierarchical representations instead of replacing them with a monolithic decoder transformation. Heterogeneity-Guided Stage-Adaptive Fusion (HG-SAF) predicts dense stage weights conditioned on local feature variation. A Frequency-Residual Adapter (FRA) then injects frequency information through a bounded, zero-initialized residual branch that keeps the fused representation as its reference. A Confusion-Aware Tri-Prior Decoder (CATP) finally regularizes the prediction with boundary, objectness, and training-derived class-relation cues. Under a matched Swin-B training and single-scale inference protocol, HAFR-Net attains 84.12%, 87.86%, 55.17%, and 67.70% mIoU on ISPRS Vaihingen, ISPRS Potsdam, LoveDA, and OpenEarthMap, improving the matched UPerNet baseline by 0.55, 0.95, 1.55, and 1.84 percentage points. Controlled analyses further show consistent spatial reweighting beyond content-only routing, improved boundary and thin-structure accuracy over matched spatial and spectral alternatives, and reduced confusion on pre-declared class pairs. The trained LoveDA model is released, with the source code to follow, at https://github.com/anticipate218/HAFRNet. Index Terms: Frequency-domain refinement, hierarchical feature fusion, semantic segmentation, confusion-aware decoding, very-high-resolution (VHR) remote sensing. I Introduction Very-high-resolution (VHR) remote sensing imagery provides detailed observations of buildings, roads, vehicles, vegetation, and other land-cover elements at fine spatial scales [62, 15, 11]. Modern segmentation systems based on CNNs, Transformers, and state-space models have benefited substantially from large pretrained encoders and hierarchical feature representations [31, 41, 64, 2, 30, 58, 10, 47]. As encoder capacity continues to improve, however, a growing part of the segmentation problem shifts from representation extraction to representation utilization: the decoder must determine how to combine spatially detailed shallow features with semantically strong deep features, and how to refine them without unnecessarily destroying information already encoded by pretraining. Fig. 1: Positioning of HAFR-Net. (a)–(c) CNN, Transformer and state-space decoders combine the encoder hierarchy through a fixed aggregation rule. (d) HAFR-Net keeps the pretrained hierarchy and refines it progressively, through stage-specialized preparation, heterogeneity-guided stage-adaptive fusion, bounded residual frequency adaptation, and confusion-aware tri-prior decoding. A key difficulty in VHR imagery is that the preferred balance between spatial detail and semantic abstraction varies strongly within the same image (Fig. 1). Small vehicles, narrow roads, roof edges, and object boundaries benefit from high-resolution stages, whereas large roofs, agricultural parcels, and homogeneous regions often require deeper contextual representations. Nevertheless, many commonly used decoders aggregate multi-stage features through fixed top-down connections, predefined summation, or image-level weighting [27, 57, 50, 48]. Such strategies are effective in general, but they provide limited flexibility when the desired stage contribution changes from pixel to pixel within a highly heterogeneous scene. A second consideration concerns how additional decoder capacity is introduced. Frequency-domain and global-context operators can enhance detail or long-range interactions [39, 5, 61, 24, 8], but unrestricted transformations may require the optimizer to simultaneously preserve pretrained representations and learn a new task-specific mapping. For VHR datasets, which are often considerably smaller than generic pretraining corpora, a more conservative strategy is attractive: rather than replacing the fused representation, the decoder should introduce bounded residual corrections that remain close to a known reference behavior at the beginning of fine-tuning. Finally, accurate VHR segmentation is not determined by region classification alone. Errors frequently occur around object boundaries, small foreground regions, and semantically similar categories [42, 19, 52, 65]. These patterns motivate the use of structural supervision and class-relation constraints during decoding. Rather than treating such auxiliary signals as independent tasks, they can be used as lightweight regularizers that shape the final feature space while leaving inference dominated by a single semantic prediction branch. When class-relation pairs are used, they are identified exclusively from a training-only pilot split and are fixed before final evaluation. Based on these observations, we propose HAFR-Net, a Hierarchical Adaptive Feature Refinement Network for VHR remote sensing image segmentation. The framework follows a progressive refinement philosophy. A pretrained Swin-B encoder first provides four hierarchical feature stages. Heterogeneity-Guided Stage-Adaptive Fusion (HG-SAF) adaptively aggregates these features with pixel-wise weights conditioned on local heterogeneity. Frequency-Residual Adapter (FRA) then performs a bounded residual correction in the frequency domain instead of replacing the fused representation with a full spectral transform. Finally, the Confusion-Aware Tri-Prior Decoder (CATP) regularizes semantic decoding using structural and class-relation cues. The resulting decoder explicitly separates adaptive hierarchical fusion, conservative feature refinement, and final tri-prior decoding. The main contributions are as follows. • We formulate VHR decoding as hierarchical adaptive refinement rather than generic feature aggregation. HG-SAF predicts pixel-wise stage weights from local feature heterogeneity, enabling the contribution of shallow spatial detail and deep semantic context to vary across a scene. • We introduce a bounded residual refinement strategy for fused hierarchical representations. FRA performs channel-bottlenecked spectral modulation with an explicitly bounded gate and residual initialization, refining detail while starting from the pretrained spatial representation rather than an unconstrained frequency transform. • We combine adaptive fusion and conservative refinement with confusion-aware tri-prior decoding and validate each design through matched controls. Experiments on four VHR benchmarks include controlled decoder comparisons, direct spatial and frequency alternatives, stage-weight analysis, boundary metrics, class-confusion analysis, statistical uncertainty, and efficiency measurements. I Related Work I-A Hierarchical Feature Fusion for Remote Sensing Segmentation Encoder–decoder models from FCN [31] and U-Net [41] to PSPNet [64], DeepLabv3+ [2], HRNet [45] and UPerNet [57] established multi-level aggregation as a standard recipe for dense prediction. The same idea has been specialized for remote sensing through CNN and Transformer decoders such as ABCNet [23], BANet [49], MANet [22], A2-FPN [21], UNetFormer [50], the FT-UNetFormer implementation in GeoSeg [51] and DCSwin [48]. These methods differ mainly in how encoder stages are combined: some use fixed top-down laterals, others add global pooling or channel attention [14, 54, 27], and a smaller subset predicts spatially varying weights. Existing hierarchical decoders therefore already demonstrate the value of multi-level aggregation, while different designs vary in how strongly they adapt fusion to local content. HG-SAF focuses specifically on dense spatial routing conditioned by a lightweight local heterogeneity statistic. It is evaluated against mean fusion, globally pooled gates, and content-adaptive pixel-wise alternatives under the same backbone, so that the contribution of the heterogeneity cue can be isolated from the mere use of a pixel-wise gate. I-B Spatial-Frequency Feature Refinement Generic spectral mixers including GFNet [39], FNet [20] and FFC [5], together with Fourier operator learning [26], show that Fourier-domain modulation can complement spatial convolution. Recent remote sensing models go further. SFFNet [61] fuses wavelet and spatial features, FDNet [24] decouples frequency bands, FGNet [63] inserts FFT-guided filters into a Swin backbone, AFENet [8] modulates bands according to image content, FreDNet [25] performs frequency-guided denoising, and WgANet [53] uses wavelet-guided attention. These works establish frequency modeling as a competitive tool for VHR segmentation rather than as a new primitive. The remaining design question is how a spectral branch should be attached to an already strong fused representation. HAFR-Net does not use frequency modeling to replace the fused spatial features. FRA restricts frequency modeling to a bounded residual correction inside a channel bottleneck and is directly compared with standard spatial residuals and spectral alternatives at the same insertion point. I-C Structure-Aware Semantic Decoding Boundary supervision [42, 29, 19], objectness or imbalance losses [28, 37, 1], and pixel-level contrast or prototype learning [52, 65] are established tools for sharpening contours and separating similar classes. Individual boundary, objectness, and prototype mechanisms are not claimed as novel primitives. CATP uses them as a compact final decoder that follows hierarchical adaptive refinement, and matched controls test whether the combination improves on conventional auxiliary losses. Recent Transformer and Mamba backbones [30, 58, 4, 34, 47, 66] strengthen long-range context, and a parallel line keeps a pretrained model fixed and adapts it with lightweight modules, as in the adapter and LoRA fine-tuning of SAM for multimodal remote sensing segmentation [33]. Neither line by itself specifies how the hierarchical features of such an encoder should be organized and refined for VHR scenes. HAFR-Net keeps Swin-B as the representation backbone and places the emphasis on progressive decoding. I Method I-A Overview HAFR-Net is designed around a single principle: progressively refine pretrained hierarchical representations rather than replace them with an increasingly complex task-specific decoder. Given an input image I, a pretrained Swin-B encoder [30] produces four stage features Fss=14\F_s\_s=1^4. Each stage first passes a fixed preparation block TsT_s, which prepares the stage feature and is kept identical across the module studies of Section VI (Section I-B). HG-SAF then projects the stages to a common space and adaptively aggregates them into FfuseF_fuse. FRA computes a bounded residual correction to obtain FrefF_ref. CATP finally maps FrefF_ref to semantic logits, while auxiliary structural signals are used only during training. Fig. 2: Overall architecture of HAFR-Net. A pretrained Swin-B encoder produces four hierarchical stages, which pass fixed preparation blocks (SSDB at the two high-resolution stages, BCS-Mamba at the two low-resolution stages). HG-SAF then performs pixel-wise adaptive fusion, FRA applies a bounded residual frequency correction, SOE acts on the refined feature, and CATP produces the semantic prediction. Auxiliary heads are used only in training. The encoder is Swin-B pretrained on ImageNet-22k and fine-tuned on ImageNet-1k [7], whose four stages have widths (128,256,512,1024)(128,256,512,1024); a 1×11×1 projection maps them to (96,192,384,768)(96,192,384,768) before the preparation blocks. Each stage is then aligned to width Ce=384C_e=384 and spatial size (H1,W1)=(H/4,W/4)(H_1,W_1)=(H/4,W/4) before fusion. As illustrated in Fig. 2, the inference path is Psem=CATP(SOE(FRA(HG−SAF(Ts(Fs)s=14)))).P_sem=CATP (SOE (FRA (HG-SAF(\T_s(F_s)\_s=1^4) ) ) ). (1) The decomposition in (1) separates three functions that are commonly entangled in a decoder: selecting hierarchical information, refining the aggregated representation, and imposing the tri-prior decoding constraints. The remainder of this section first records the fixed preparation blocks, then describes the three refinement stages in that order, and closes with their initialization and the training objective. I-B Stage-Specialized Preparation Three blocks are inherited from our implementation and are not claimed as contributions: SSDB and BCS-Mamba prepare the encoder stages, and SOE acts on the refined feature just before decoding. They are held identical in every module-level comparison of Section VI, so they cannot account for a difference between the rows of those studies, and Section VI-E measures what they contribute to the margin of Table I. The two stage-preparation blocks are shown in Fig. 3. The two high-resolution stages use a spectral–spatial decoupled block (SSDB, Fig. 3(a)). A normalized stage feature is processed in parallel by a learnable per-channel complex filter applied to its orthonormal two-dimensional real FFT and by a depthwise 3×33×3 convolution followed by a pointwise convolution; a gate conditioned on both branches mixes them as a residual update, and a convolutional feed-forward tail closes the block. The two low-resolution stages use a bi-directional cross-scan Mamba block (BCS-Mamba, Fig. 3(b)): rows and columns are scanned as independent sequences, which shortens each scan from H1W1H_1W_1 to max(H1,W1) (H_1,W_1) steps, two orthogonal depthwise bridges (3×13×1 and 1×31×3) re-inject the cross-axis context that axial scanning removes, and a direction-adaptive weighting merges the horizontal and vertical outputs. The third fixed block, small-object enhancement (SOE), combines parallel dilated depthwise convolutions at rates 22, 44 and 88 with channel attention and a residual scale initialized to 0.10.1; it is not a stage-preparation block but acts on FrefF_ref immediately before decoding, identically in every reported model. Section VI-E measures each of the three on the reference decoder; all three effects are below +0.2+0.2 p, so the gains analyzed in this paper are not produced by these fixed blocks. Fig. 3: Fixed stage preparation blocks, shown with the shared sub-blocks on the right. (a) SSDB decouples a normalized stage feature into a learnable complex spectral branch and a depthwise spatial branch, and merges them by a gated residual update. (b) BCS-Mamba scans rows and columns as independent selective state-space sequences, restores cross-axis context through two orthogonal depthwise bridges, and merges the two directions by content-adaptive weights. Both blocks are held identical across the module studies of Section VI; their measured effect is reported in Section VI-E. I-C Heterogeneity-Guided Stage-Adaptive Fusion Hierarchical encoders expose features with progressively lower spatial resolution and stronger semantic abstraction. In VHR imagery, the relative usefulness of these stages can vary considerably within one tile. HG-SAF therefore performs dense rather than globally shared stage weighting. Each stage is projected to width CeC_e and upsampled to (H1,W1)(H_1,W_1), giving F~s F_s. Local feature heterogeneity is a lightweight statistic of variation in the deepest projected representation, ℋ()=1Ce∑c=1CeVark×k(F~4(c))()+ε,H( x)= 1C_e _c=1^C_e Var_k× k\! ( F_4^(c) )( x)+ , (2) with a k×k×k box filter (k=5k=5) and ε=10−5 =10^-5. A convolutional gate gθg_θ, a 3×33×3 convolution to Ce/4C_e/4 channels followed by normalization, a GELU and a 1×11×1 convolution to the four stage logits, predicts pixel-wise stage weights =(w1,…,w4)⊤ w=(w_1,…,w_4) , normalized over the stage axis, from the mean stage context B=14∑sF~sB= 14 _s F_s and ℋH (Fig. 4(a)), () w( x) =softmaxs(gθ(cat[B,ℋ])())∈Δ3⊂ℝ4, =softmax_s\! (g_θ\! (cat[B,H] )( x) )∈ ^3\!⊂\!R^4, (3) Ffuse() F_fuse( x) =∑s=14ws()F~s(). = _s=1^4w_s( x)\, F_s( x). (4) The final projection of gθg_θ is zero-initialized, so (3) returns uniform stage weights at the first forward pass and (4) reduces to the mean of the projected stages. HG-SAF therefore starts from uniform mean fusion of the projected stages, a non-adaptive reference, and learns spatially varying routing during fine-tuning; the stage projections themselves are trained from scratch. Matched controls keep the gate architecture and capacity of gθg_θ and drop only ℋH from its input in (3), so that pixel-wise routing and the heterogeneity statistic are not confounded. Fig. 4: The two refinement blocks after the encoder pyramid. (a) HG-SAF predicts per-pixel stage weights from B and ℋH with a zero-initialized gate. (b) FRA adds a channel-bottlenecked residual rFFT branch with a tanh-bounded gate and γ=0γ=0. I-D Frequency-Residual Adapter After hierarchical fusion, the representation already contains substantial spatial and semantic information. We therefore avoid replacing it with an unrestricted spectral transformation. Instead, FRA uses frequency modeling only to construct a bounded residual correction (Fig. 4(b)). With a channel bottleneck of width r=max(Ce/4,16)r= (C_e/4,16), Z Z =rFFT(Conv×1Ce→r(Ffuse)), =rFFT\! (Conv_1\!×\!1^C_e→ r(F_fuse) ), (5) Z~ Z =Z^⊙g,g=1+αtanh(gϕ(G)), = Z g, g=1+α \! (g_φ(G) ), (6) Fref F_ref =Ffuse+γConv×1r→Ce(irFFT(Z~,H1,W1)), =F_fuse+γ\,Conv_1\!×\!1^r→ C_e\! (irFFT( Z;H_1,W_1) ), (7) Both transforms are orthonormal and act on the half spectrum, so the output of (7) is real by construction. Here G concatenates the log-amplitude, the sine and cosine of the phase, and a radial prior over the half-spectrum grid, α=0.5α=0.5, and γ is initialized to 00. Three constraints follow from (5)–(7). The bottleneck of width r in (5) caps both the cost of the spectral branch and the channel rank of its residual mapping. The multiplier g of (6) obeys |g−1|<α|g-1|<α, so the relative rescaling of any spectral coefficient stays inside (1−α,1+α)(1-α,1+α) and no coefficient can be arbitrarily amplified; the overall correction is additionally scaled by γ. The residual scale of (7) makes the first forward pass an exact identity: the branch starts from zero contribution and is learned from data during fine-tuning. At inference, FRA remains a single residual correction on FfuseF_fuse; it does not introduce an auxiliary prediction head. I-E Confusion-Aware Tri-Prior Decoder Even after hierarchical fusion and feature refinement, a single semantic head provides only region-level supervision. CATP therefore uses lightweight auxiliary signals to regularize boundaries, foreground structure, and inter-class relations during training. Fig. 5: Confusion-aware tri-prior decoder. (a) The small-object enhancement block (SOE), a fixed block on the fused path rather than a claimed contribution. (b) Semantic, boundary, objectness and prototype-feature maps on a Vaihingen tile. The objectness head predicts a coarse foreground prior, which regularizes F¯ F without selecting objects by size. A two-layer fuse block produces F¯ F. A boundary branch of two 3×33×3 convolutions at width Ce/4C_e/4 followed by a 1×11×1 projection predicts single-channel boundary logits PedgeP_edge, which are fed back as F¯′=F¯+βeenhance(maxpool3×3(σ(Pedge))⊙F¯), F \;=\; F+ _e\,enhance\! (maxpool_3× 3\! (σ(P_edge) ) F ), (8) where σ is the logistic sigmoid, broadcast over channels, enhanceenhance is a 3×33×3 convolution with normalization and GELU, and βe _e is initialized to 0.050.05. Three 1×11×1 heads applied to F¯′ F output semantic logits PsemP_sem, objectness logits PobjP_obj and a prototype embedding Q∈ℝ64×H1×W1Q ^64× H_1× W_1 (Fig. 5). The objectness target TobjT_obj marks the valid pixels outside the two background-like classes of each legend (Impervious surface and Clutter on ISPRS, Background and Barren on LoveDA, Developed and Bareland on OpenEarthMap), so the head learns a coarse foreground prior; small-component IoU is reserved for evaluation. The edge target is the 44-neighbour class-difference mask. Both ℒedgeL_edge and ℒobjL_obj are binary CE in which positive pixels carry the weight clip(N−/N+,1,20)clip(N_-/N_+,1,20), where N+N_+ and N−N_- count positive and negative pixels of the mini-batch. Some class pairs exhibit substantially larger cross-class errors than others. We therefore define a small relation set C from a pilot model using only an internal split of the training partition; the official evaluation split is never used for pair selection. Per-class prototype c p_c is the mini-batch mean of Q over the valid pixels of class c, re-normalized to unit ℓ2 _2 norm so that ⟨a,b⟩ p_a, p_b is a cosine similarity; classes absent from a batch are skipped, and ℒproto=1||∑(a,b)∈max(0,m+⟨a,b⟩),L_proto= 1|C|\! _(a,b) \! \! (0,\;m+ p_a, p_b ), (9) with m=0.2m=0.2, so a pair stops contributing once its prototypes reach ⟨a,b⟩≤−m p_a, p_b ≤-m. On the six-class ISPRS label set, C contains four pairs. The pair lists for all four datasets are given in the supplement. At inference, only PsemP_sem is used. I-F Conservative Initialization of Refinement Branches HAFR-Net uses conservative initialization to avoid introducing large task-specific perturbations at the beginning of fine-tuning [40, 13, 44]. The three components do not share an identical mathematical behavior. As summarized in Table I, HG-SAF starts from baseline-equivalent uniform stage fusion, FRA is an exact residual identity when γ=0γ=0 in (7), and the structural feedback path in CATP starts from the small residual coefficient βe _e of (8). We distinguish these cases rather than treating all components as exact identities. TABLE I: Conservative Initialization of the Three Refinement Branches Branch Init. First forward Role HG-SAF last gate =0=0 uniform 1/41/4 fusion adaptive fusion FRA γ=0γ=0 exact identity residual refine. CATP fb. βe=0.05 _e=0.05 near-ref. residual struct. feedback The three branches do not share one mathematical identity: HG-SAF starts from mean fusion, FRA is an exact residual identity, and the CATP feedback path is a small residual. These three reference behaviors are recorded so that later ablations can isolate initialization from architecture. The next subsection defines the training objective on the semantic head and the auxiliary structural signals. I-G Training Objective Let Ω=i:yi≠255 =\i:y_i≠ 255\ be the valid pixels, so that ignore index 255255 keeps unlabeled and eroded-boundary pixels out of ℒCEL_CE and ℒFocalDiceL_FocalDice. Let DSCc=(2∑i∈Ωpi,cyi,c+εd)/(∑i∈Ωpi,c+∑i∈Ωyi,c+εd)DSC_c=(2 _i∈ p_i,cy_i,c+ _d)/( _i∈ p_i,c+ _i∈ y_i,c+ _d) with εd=1 _d=1, one-hot targets yi,cy_i,c and p=softmaxc(Psem)p=softmax_c(P_sem). The semantic loss is ℒsem=ℒCE+λdℒFocalDiceL_sem=L_CE+ _dL_FocalDice, where ℒFocalDice=1Ncls∑c=1Ncls(1−DSCc)κ,L_FocalDice= 1N_cls _c=1^N_cls (1-DSC_c )^κ, (10) NclsN_cls is the number of scored classes, κ=2.5κ=2.5, and ℒCEL_CE uses class-balanced weights (Car 3×3×, Clutter 5×5× on ISPRS). The full objective is ℒ=ℒsem+λeℒedge+λoℒobj+λpℒproto+ℒaux,L=L_sem+ _eL_edge+ _oL_obj+ _pL_proto+L_aux, (11) with λd=1.0 _d=1.0, λe=0.4 _e=0.4, λo=0.2 _o=0.2 and λp=0.1 _p=0.1. The auxiliary term ℒauxL_aux attaches CE and FocalDice heads to the prepared stages Ts(Fs)s=14\T_s(F_s)\_s=1^4 with weights (0.4,0.3,0.2,0.1)(0.4,0.3,0.2,0.1). Every term of (11) is evaluated at the label resolution: the auxiliary and structural logits are bilinearly upsampled to the input size, so the discrete label map is never interpolated and keeps its ignore index. Only the class map of ℒprotoL_proto is resampled, to (H1,W1)(H_1,W_1) and by nearest neighbour. All auxiliary heads are disabled at inference. The next section specifies the datasets, matched protocol, and metrics used to evaluate this design. IV Experimental Setup IV-A Datasets and Exact Splits ISPRS Vaihingen [18] contains 3333 NIR-RG tiles at 99 cm GSD. The 1616 official training tiles are 1, 3, 5, 7, 11, 13, 15, 17, 21, 23, 26, 28, 30, 32, 34 and 37; the remaining 1717 tiles form the test set, stored as 113113 non-overlapping 1024×10241024×1024 patches, which are the evaluation and bootstrap unit. Five foreground classes are scored; Clutter is excluded; official 33 px boundary erosion and ignore index 255255 are used. ISPRS Potsdam [17] contains 3838 RGB tiles at 55 cm GSD and the same six-class legend. Train tiles: 2_102\_10–2_122\_12, 3_103\_10–3_123\_12, 4_104\_10–4_124\_12, 5_105\_10–5_125\_12, 6_76\_7–6_126\_12, 7_77\_7–7_127\_12. Test tiles: 2_132\_13, 2_142\_14, 3_133\_13, 3_143\_14, 4_134\_13–4_154\_15, 5_135\_13–5_155\_15, 6_136\_13–6_156\_15, 7_137\_13. The same five-class eroded protocol is used. LoveDA [46] contains 59875987 RGB images at 3030 cm GSD and seven classes. We report the official validation split. Several published Transformer and Mamba numbers were originally submitted to the online test server; they appear only as contextual literature values. OpenEarthMap [56] contains 50005000 images at 2525–5050 cm GSD. mIoU averages the eight official land-cover classes; unlabeled pixels are ignored. There is no separate Background class. Every image enters the network at 512×512512×512: the ISPRS tiles are held as the non-overlapping 1024×10241024×1024 patches listed above and each patch is resampled to that size, as are the LoveDA and OpenEarthMap images, which the two benchmarks already distribute as single tiles. Labels are resampled by nearest neighbour so that class indices stay valid, and the ignore index is preserved. Normalization uses ImageNet mean and standard deviation. No CRF, multi-crop merging, or other post-processing is applied. IV-B Controlled Baselines and Published Comparisons Conclusions are drawn from a controlled Swin-B comparison: UPerNet [57], FPN [27], DeepLabv3+ [2], SegFormer MLP [58], a UNetFormer-style decoder [50], SE-gated fusion [14] and an FFT mixer [39], all retrained with the single recipe of Section IV-C. Published CNN, Transformer and Mamba numbers are provided for contextual comparison and are not treated as strictly controlled comparisons, because training, backbone, metric, and inference configurations differ across sources. FT-UNetFormer is cited as the GeoSeg implementation [51]. A provenance table is given in the supplement. IV-C Implementation Details Training uses PyTorch 2.1 [38] and one RTX 4090. Every controlled model, including HAFR-Net, shares one recipe: 512×512512×512 inputs, at most 8080 epochs with batch size 88, AdamW [32] with head learning rate 1.2×10−41.2×10^-4 and backbone learning rate 4×10−54×10^-5, a cosine schedule with a 55-epoch warmup [9], weight decay 10−410^-4, gradient clipping 1.01.0 and BF16 mixed precision. Augmentation is horizontal or vertical flip and discrete 90∘90 rotation, each with probability 0.50.5. The auxiliary deep supervision of (11) belongs to this shared recipe: every controlled model carries CE and FocalDice heads on the corresponding encoder stages with the weights (0.4,0.3,0.2,0.1)(0.4,0.3,0.2,0.1), so Table I compares decoding paths and not the amount of training signal. Seeds are 4242, 4343 and 4444, and the reported checkpoint is the best-mIoU epoch under one monitoring rule that is identical for every controlled model. Controlled inference is single-scale without test-time augmentation (TTA). For the two ISPRS sets we additionally report flip and 90∘,180∘,270∘\90 ,180 ,270 \ TTA so that TTA gain can be separated from method gain. LoveDA is the only dataset with a stronger photometric recipe: color jitter 0.40.4, Gaussian blur 0.30.3, EMA with decay 0.9990.999 [43] and early stopping with patience 1212; ISPRS and OpenEarthMap use none of these. This difference is recorded rather than hidden, and no controlled baseline on LoveDA is trained under a weaker recipe than HAFR-Net. IV-D Evaluation Metrics We report mIoU and per-class IoU. HG-SAF is further examined through heterogeneity- and size-stratified IoU and stage-weight statistics; the strata are computed once from the reference model and reused for every variant, so all rows are scored on identical pixel sets. FRA is examined through Boundary IoU [3], boundary F-score [6], thin-structure IoU (local width below 88 px) and small-component IoU (area quartile Q1). CATP is examined through pairwise confusion mass Ea,b=Mab+MbaE_a,b=M_ab+M_ba on the pre-declared relation set, where M is the confusion matrix normalized by the number of valid pixels. Uncertainty is a paired bootstrap (1 0001\,000 resamples) of the mIoU difference versus UPerNet over the non-overlapping evaluation patches, so no pixel enters a replicate twice. Efficiency uses input 512×512512×512, batch size 11, BF16, 5050 warm-up iterations and 500500 timed iterations with torch.cuda.synchronize(); the timed loop excludes host-device copy and post-processing. V Experimental Results V-A Controlled Comparison Table I reports the controlled comparison under a matched training and inference protocol, and the conclusions of this paper are drawn from it. It holds the encoder, the schedule and the inference setting fixed, so that a difference between two rows is a difference in the decoding path rather than in training or inference. HAFR-Net carries the fixed blocks of Section I-B in addition to its three refinement stages, and Section VI-E measures their separate share of the margin. TABLE I: Controlled Comparison of Decoders Under a Matched Protocol Method (Swin-B) Vaih. Pots. LoveDA OEM Avg. UPerNet [57] 83.57 86.91 53.62 65.86 72.49 FPN [27] 83.21 86.54 53.18 65.41 72.09 DeepLabv3+ [2] 83.38 86.72 53.35 65.58 72.26 SegFormer MLP [58] 83.44 86.80 53.48 65.71 72.36 UNetFormer dec. [50] 83.79 87.14 54.02 66.18 72.78 SE-gated fusion [14] 83.68 87.05 53.88 66.02 72.66 FFT spectral mixer [39] 83.62 86.98 53.71 65.94 72.56 HAFR-Net (no TTA) 84.12 87.86 55.17 67.70 73.71 TTA (flip++rotation), reported only for the two ISPRS sets UPerNet† 83.89 87.24 – – – HAFR-Net† 84.48 88.20 – – – All models share the split, crop, optimizer, schedule, augmentation, and checkpoint rule of Section IV-C, and use single-scale inference without TTA unless marked †. Entries are the mean mIoU (%) over seeds 4242, 4343, and 4444. The paired bootstrap of Section IV-D gives 95%95\% confidence intervals of [0.31,0.79][0.31,0.79], [0.66,1.22][0.66,1.22], [1.18,1.91][1.18,1.91] and [1.47,2.19][1.47,2.19] p for the no-TTA margin of HAFR-Net over UPerNet on the four datasets, so every margin excludes zero. The best and second-best entries in each column are bold red and underlined blue, respectively. The no-TTA and TTA blocks are ranked separately. Under the no-TTA protocol, HAFR-Net improves Swin-B++UPerNet by +0.55+0.55 p on Vaihingen (95%95\% CI [0.31,0.79][0.31,0.79]), +0.95+0.95 p on Potsdam ([0.66,1.22][0.66,1.22]), +1.55+1.55 p on LoveDA ([1.18,1.91][1.18,1.91]), and +1.84+1.84 p on OpenEarthMap ([1.47,2.19][1.47,2.19]); all four intervals exclude zero, and the ordering is preserved across the three seeds. The gains are larger on the more heterogeneous LoveDA and OpenEarthMap sets than on the saturated ISPRS urban tiles. Among the controlled decoders, the UNetFormer-style decoder is the strongest alternative, and HAFR-Net exceeds it by +0.33+0.33 to +1.52+1.52 p across the four sets. TTA adds a further +0.36+0.36 / +0.34+0.34 p on the two ISPRS sets. The improvement holds on all four datasets under a single protocol, so it follows from the decoding design rather than from a dataset-specific trick. V-B Contextual Comparison with Published Results For broader context, we additionally report representative published results from CNN-, Transformer-, and Mamba-based remote sensing models. Because these values are obtained under different backbones, training recipes, and inference settings, they are treated as contextual references rather than strict controlled comparisons. Under this reading, HAFR-Net attains the best or the second-best reported accuracy on each of the four benchmarks. TABLE I: Contextual Comparison With Published Results on ISPRS Vaihingen and Potsdam ISPRS Vaihingen ISPRS Potsdam Method Imp.S. Build. L.Veg. Tree Car mIoU Imp.S. Build. L.Veg. Tree Car mIoU CNN-based UNet [41] 90.1 88.0 69.1 78.4 64.5 78.02 82.8 89.9 73.8 75.6 83.2 81.06 PSPNet [64] 91.0 88.7 69.5 79.1 66.8 79.02 84.2 91.0 74.8 76.5 85.2 82.34 DenseASPP [60] 90.3 87.6 68.4 77.9 62.5 77.34 83.3 90.3 73.5 75.3 83.8 81.24 MANet [22] 91.2 89.2 70.1 79.4 68.9 79.76 84.8 91.8 75.4 77.1 86.2 83.06 HRNet [45] 91.6 89.6 70.5 79.9 69.9 80.30 83.8 90.7 74.2 76.1 84.5 81.86 ABCNet [23] 90.7 88.5 69.3 78.9 67.6 79.00 84.3 91.2 75.0 76.7 85.5 82.54 BANet [49] 89.8 87.3 68.1 77.8 65.0 77.60 83.5 90.5 73.9 75.7 84.2 81.56 A2-FPN [21] 91.4 89.4 70.4 79.5 69.5 80.04 85.2 92.1 75.9 77.5 86.6 83.46 CGNet [55] 90.5 88.1 68.9 78.6 67.2 78.66 84.0 90.8 74.5 76.4 85.0 82.14 Transformer-based FT-UNetFormer† [51] 91.9 89.9 71.1 80.3 73.3 81.30 86.2 92.9 76.8 78.3 88.0 84.44 DCSwin† [48] 92.7 90.5 71.7 80.8 75.1 82.16 88.0 94.2 78.6 80.8 91.9 86.70 Mask2Former† [4] 92.9 90.7 72.1 81.2 75.4 82.46 82.8 94.0 79.2 82.8 73.2 82.40 LOGCAN++ [35] 92.1 90.3 72.0 81.2 77.1 82.54 87.6 93.8 77.9 80.0 89.9 85.84 CG-Swin† [36] 93.1 90.8 72.4 81.3 77.6 83.04 88.7 94.9 79.4 81.3 93.4 87.54 UNetFormer† [50] 93.4 91.2 72.6 81.8 78.1 83.42 86.9 93.4 77.5 79.2 88.7 85.14 MMT† [59] 93.4 91.4 72.9 81.8 78.7 83.64 89.6 93.8 78.9 81.2 92.7 87.24 Mamba-based RS3Mamba† [34] 92.5 89.9 72.1 81.0 76.2 82.34 87.3 93.6 77.8 79.5 90.5 85.74 UNetMamba† [66] 92.9 90.8 72.7 81.2 76.8 82.88 88.3 94.4 78.7 80.7 92.1 86.84 D2LS† [67] 93.6 91.6 73.3 82.0 79.1 83.92 89.2 94.9 79.2 81.7 93.1 87.62 PyramidMamba† [47] 94.2 92.5 74.7 82.3 80.5 84.84 89.9 94.6 79.5 81.7 93.4 87.82 HAFR-Net† (Ours) 93.8 92.1 75.0 82.0 79.5 84.48± 0.11 90.7 95.1 80.2 82.1 92.9 88.20± 0.09 Per-class IoU and mIoU (%) on the five scored foreground classes (Imp.S.: impervious surface; Build.: building; L.Veg.: low vegetation); Clutter is excluded and the official 33-px boundary erosion is applied. Methods marked † use TTA as reported by their sources. Because backbones, splits, and inference settings differ across papers, these values are literature context rather than a matched benchmark; the matched Swin-B comparison is Table I. The best and second-best entries in each column are bold red and underlined blue, respectively. Best and second best are ranked separately within each dataset. The ± value is the three-seed standard deviation of HAFR-Net. Table I summarizes the two ISPRS urban benchmarks. With TTA, HAFR-Net reaches 84.48%84.48\% mIoU on Vaihingen and 88.20%88.20\% on Potsdam. The per-class pattern is consistent with the later analyses: on Vaihingen HAFR-Net leads only on Low Vegetation, and on Potsdam its largest margins are on Impervious surface (+0.8+0.8 p) and Low Vegetation (+0.7+0.7 p). HAFR-Net holds the highest Potsdam mIoU of the table; on Vaihingen only PyramidMamba reports a higher mean, by 0.360.36 p, with a different backbone and TTA setting. TABLE IV: Contextual Comparison With Published Results on the LoveDA Validation Split Method Backgr. Building Road Water Barren Forest Agricult. mIoU CNN-based UNet [41] 50.24 59.77 52.79 57.02 31.08 41.73 51.81 49.20 PSPNet [64] 52.50 59.46 51.15 61.10 27.46 37.48 50.32 48.49 DenseASPP [60] 50.77 55.67 53.08 55.49 23.99 35.24 43.54 45.40 MANet [22] 49.82 58.08 52.98 54.30 25.21 37.65 51.87 47.13 HRNet [45] 49.90 57.33 55.94 52.34 31.64 41.94 48.50 48.23 A2-FPN [21] 47.73 59.19 52.20 64.08 32.09 37.96 53.00 49.46 Transformer-based LOGCAN++ [35] 47.37 58.38 56.46 80.05 18.44 47.91 64.80 53.35 AerialFormer [12] 47.80 60.70 59.30 81.50 17.90 47.90 64.00 54.10 SFA-Net [16] 48.40 60.30 59.10 81.90 24.10 46.20 64.00 54.90 Mamba-based RS3Mamba [34] 41.60 58.23 54.03 77.34 17.97 43.81 61.37 50.62 UNetMamba [66] 47.08 59.16 56.74 81.37 18.15 46.61 64.31 53.35 PyramidMamba [47] 51.07 56.39 52.44 66.81 28.95 35.13 45.84 48.09 D2LS [67] 47.60 61.20 59.10 81.60 23.80 48.80 64.80 55.30 HAFR-Net (Ours) 52.32 60.16 58.80 70.38 39.80 42.67 62.05 55.17± 0.12 Per-class IoU and mIoU (%) on the seven LoveDA classes. HAFR-Net is evaluated on the official validation split with single-scale inference and no TTA. Several published Transformer and Mamba entries were originally submitted to the online test server and are therefore not strictly comparable; they are shown only as literature context, and the matched Swin-B comparison is Table I. The best and second-best entries in each column are bold red and underlined blue, respectively. The ± value is the three-seed standard deviation of HAFR-Net; per-class standard deviations do not exceed 0.180.18 and are omitted for readability. Table IV reports LoveDA on the official validation split without TTA. HAFR-Net attains 55.17%55.17\% mIoU. The largest per-class margin is on Barren, the class that the training-only relation set C of (9) pairs with Background on this dataset. Only D2LS reports a higher mean, by 0.130.13 p, and part of the table was produced on the online test server rather than on the validation split, as recorded in the supplement. TABLE V: Contextual Comparison With Published Results on OpenEarthMap Method Bareland Range. Develop. Road Tree Water Agricult. Building mIoU CNN-based UNet [41] 33.6 49.2 49.8 56.1 64.7 71.4 70.9 72.3 58.50 PSPNet [64] 35.2 50.1 50.6 57.4 65.8 72.6 71.8 73.0 59.56 DenseASPP [60] 32.9 48.4 49.1 55.6 63.9 70.7 70.1 71.6 57.79 MANet [22] 36.0 50.8 51.2 58.0 66.5 73.4 72.5 73.8 60.27 HRNet [45] 35.6 50.5 50.9 57.7 66.1 73.0 72.1 73.4 59.91 ABCNet [23] 34.4 49.7 50.2 56.9 65.3 72.1 71.4 72.7 59.09 BANet [49] 33.0 48.6 49.4 55.8 64.2 70.9 70.4 71.9 58.03 A2-FPN [21] 36.4 51.0 51.5 58.3 66.8 73.7 72.8 74.1 60.57 CGNet [55] 34.9 49.9 50.4 57.1 65.6 72.4 71.6 72.9 59.35 Transformer-based SegFormer† [58] 41.0 56.4 53.2 58.7 70.9 77.7 76.7 76.2 63.85 FT-UNetFormer† [51] 41.8 55.0 52.6 57.6 69.4 77.2 75.5 75.1 63.02 DCSwin† [48] 42.5 55.6 53.2 58.1 69.9 77.8 76.0 75.7 63.60 Mask2Former† [4] 42.6 56.0 53.9 59.8 70.0 77.5 76.3 76.4 64.06 UNetFormer† [50] 42.8 56.2 53.5 60.9 70.2 77.4 76.6 76.9 64.31 LOGCAN++ [35] 43.6 57.8 55.4 62.1 70.8 78.9 77.6 77.0 65.40 Mamba-based RS3Mamba† [34] 39.9 51.0 48.7 56.9 66.8 74.4 75.0 71.4 60.51 UNetMamba† [66] 43.2 57.1 54.8 61.4 70.5 78.3 77.1 77.2 64.95 PyramidMamba† [47] 45.0 59.4 57.9 64.9 72.1 81.3 79.4 79.6 67.45 HAFR-Net (Ours) 45.6 59.8 58.4 65.3 71.9 80.8 79.7 80.1 67.70± 0.10 Per-class IoU and mIoU (%) over the eight official land-cover classes; unlabeled pixels are ignored and are not counted as a Background class. Methods marked † use TTA as reported by their sources, whereas HAFR-Net uses single-scale inference. These literature numbers are not a matched protocol; see Table I for the controlled comparison. The best and second-best entries in each column are bold red and underlined blue, respectively. The ± value is the three-seed standard deviation of HAFR-Net; per-class standard deviations do not exceed 0.130.13 and are omitted for readability. Table V reports OpenEarthMap under the eight-class official protocol. The no-TTA model reaches 67.70%67.70\% mIoU. Among the pre-declared relation pairs, the largest reduction of confusion mass is on Rangeland ↔ Agriculture (Table VIII, Panel B). Across the three tables HAFR-Net therefore holds the best published mean on Potsdam and OpenEarthMap and trails the best entry by 0.360.36 and 0.130.13 p on Vaihingen and LoveDA, and Table I is the matched evidence behind that standing. VI Analysis and Ablation The previous section reports overall accuracy. This section asks a narrower question: whether each refinement stage behaves in the way it was designed to behave. We keep the Swin-B encoder fixed and swap only the component under test, and we close with visual examples of the strata that the analyses identify. Unless a table note states otherwise, ISPRS scores in this section follow the flip-and-rotation protocol of Table I, so the full model is the 84.48%84.48\% entry of Table I. VI-A Stage-Adaptive Fusion Table VI isolates the contribution of HG-SAF. Every row of Panel A replaces only the fusion stage, so the ladder decomposes the +0.20+0.20 p that HG-SAF gains over mean fusion: a globally pooled gate recovers +0.04+0.04, a pixel-wise gate that sees only the mean context B recovers +0.06+0.06, and the heterogeneity statistic of (2) supplies the remaining +0.14+0.14. The cue is therefore the larger part of the effect, and it is the statistic rather than the gate that carries it: with the same gate and capacity, the input-gradient and prediction-entropy proxies recover only +0.06+0.06 and +0.03+0.03 of that +0.14+0.14. TABLE VI: Analysis of Heterogeneity-Guided Stage-Adaptive Fusion on ISPRS Vaihingen Setting Pixel-wise Extra cue mIoU Δ Panel A: fusion-gate controls H1 Mean fusion – – 84.28 ↓ 0.20 H2 Global-pool gate – GAP 84.32 ↓ 0.16 H3 Pixel gate, B only ✓ – 84.34 ↓ 0.14 H4 Pixel gate ++ input gradient ✓ |∇I| ∇ I 84.40 ↓ 0.08 H5 Pixel gate ++ entropy ✓ ℍ(p)H(p) 84.37 ↓ 0.11 H6 HG-SAF (B+ℋB+H) ✓ ℋH 84.48 – Stratum UPerNet ++HG-SAF Δ Panel B: stratified IoU Heterogeneity Q1 (lowest) 86.12 86.28 +0.16+0.16 Heterogeneity Q2 84.05 84.41 +0.36+0.36 Heterogeneity Q3 81.74 82.38 +0.64+0.64 Heterogeneity Q4 (highest) 76.84 78.41 +1.57+1.57 Small objects (area Q1) 74.52 76.04 +1.52+1.52 Medium objects 83.56 83.94 +0.38+0.38 Large objects 87.12 87.31 +0.19+0.19 Region Shallow Deep Panel C: mean stage weights Boundary 0.370.37 0.630.63 Small object 0.380.38 0.620.62 High ℋH 0.370.37 0.630.63 Low ℋH 0.360.36 0.640.64 Large interior 0.350.35 0.650.65 Panel A isolates the heterogeneity statistic ℋH from pixel-wise routing. Every row replaces only the fusion stage and keeps the preparation blocks, FRA and CATP, so H1 coincides with the fusion-off row of Table IX; H3–H6 additionally share the gate capacity of (3) and differ only in the extra cue. Panel B reports IoU by heterogeneity quartile and by object size. Panel C lists mean stage weights (shallow =w1+w2=w_1+w_2, deep =w3+w4=w_3+w_4), with Spearman ρ(ℋ,w1+w2)=0.10ρ(H,w_1+w_2)=0.10 over the 113113 test patches; Fig. 6 shows the underlying maps and the pixel-wise distribution behind these two panels. The deep pair holds the larger share in every stratum, so the effect is a shift in share rather than a change of the dominant stage. As in Table I, Vaihingen scores include flip and rotation TTA, so the full model corresponds to 84.48%84.48\%. The best and second-best entries in each column are bold red and underlined blue, respectively. Ranking applies to the mIoU column of Panel A, where Δ is the drop with respect to H6; Panels B and C are paired measurements rather than a competition. Panel B of Table VI shows that stratified IoU increases most in the highest quartile of (2) (+1.57+1.57 p) and on small objects (+1.52+1.52 p), whereas large objects change by only +0.19+0.19 p. Panel C reports the corresponding routing statistics, and Fig. 6 shows them directly on the predicted maps. The heterogeneity statistic tracks object outlines and small structures (Fig. 6(c)), and the predicted weight field varies spatially at a comparable scale (Fig. 6(d)). The shift is consistent in sign across strata: shallow stages take a higher share on boundaries, small objects and high-ℋH pixels than in large interiors, the deep stages keep the larger share of the total weight everywhere, and the pixel-wise trend of w1+w2w_1+w_2 against ℋH is positive (Fig. 6(e),(f)). HG-SAF therefore redistributes stage emphasis toward detail exactly in the strata where Panel B reports the largest gains, while the deep semantic context of the pretrained pyramid remains dominant. Fig. 7 provides a qualitative counterpart on eight tiles. Fig. 6: Measured HG-SAF routing on ISPRS Vaihingen. (a) Input tile and (b) ground truth. (c) The heterogeneity statistic of (2) responds to object outlines and small structures. (d) Predicted shallow-stage weight w1+w2w_1+w_2 of (3), with the colour range set to the 11–9999 percentile of the tile so that the actual spatial variation is visible. (e) Pixel density of w1+w2w_1+w_2 against ℋH over the 113113 test patches, with the binned median; the rank correlation over all pixels is ρ=0.10ρ=0.10. (f) Mean stage weights per region, i.e. the quantities in Panel C of Table VI. The routing shift is consistent in sign across strata while deep stages keep the larger share everywhere. Fig. 7: Illustrative HG-SAF fusion on eight tiles (Input, shallow visual cues, deep semantic context, fused map). Quantitative routing statistics are in Table VI. The fused maps in Fig. 7 retain shallow edge responses in heterogeneous regions and deeper homogeneous responses in large interiors. This visualization is the qualitative counterpart of the routing statistics in Panel C. The next subsection asks whether a subsequent residual correction can further improve structural detail. VI-B Frequency-Residual Adaptation Table VII compares FRA with matched spatial and spectral alternatives at the same fused-feature location. The goal is to separate three possible sources of gain: the use of a residual branch, the use of a frequency operator, and the bounded low-rank parameterization. TABLE VII: Matched Frequency-Module Alternatives on ISPRS Vaihingen Variant Par. mIoU BIoU BF Thin Small None (spatial only) – 84.31 71.42 74.86 68.26 76.61 Spatial residual (matched) 1.1 84.22 71.78 75.14 68.58 76.84 Standard FFT filter 1.2 84.19 72.06 75.41 68.94 76.72 FFC-like branch [5] 3.4 84.08 71.64 75.02 68.41 76.38 GFNet mixer [39] 4.8 83.96 71.28 74.71 68.05 76.12 GFNet ++ zero-init res. 4.8 84.28 72.54 75.88 69.42 77.05 Full-rank spectral mixer 5.6 83.65 70.84 74.18 67.52 75.66 Unbounded gate 1.2 83.88 71.16 74.52 67.88 75.94 FRA 1.2 84.48 73.58 76.91 70.84 77.86 Every variant replaces FRA at the same fused-feature location, and Vaihingen scores include flip and rotation TTA as in Table I, so the first row coincides with the frequency-off row of Table IX. Par.: added parameters (M); BIoU: Boundary IoU; BF: boundary F-score; Thin: thin-structure IoU; Small: small-component IoU. The best and second-best entries in each column are bold red and underlined blue, respectively. The parameter column reports cost and is not ranked. A parameter-matched spatial residual and a standard FFT filter change mIoU only slightly, whereas unrestricted full-rank mixing is the weakest variant. A GFNet mixer with the same zero-init residual parameterization already recovers part of the gain and is the second-best variant on four of the five accuracy metrics, which supports residual parameterization itself; the additional bounded low-rank design of FRA further improves Boundary IoU and thin-structure IoU. Inspecting the learned gate confirms that it operates well inside the envelope that (6) imposes: averaged over the 113113 Vaihingen test patches and over all coefficients of the half spectrum, its distance from identity is |g−1|=0.28|g-1|=0.28, that is, 56%56\% of the α=0.5α=0.5 bound, so the branch is neither idle nor saturated at its limit. The constraints of (5)–(7) apply to every coefficient alike, so the margin in Table VII follows from the bounded low-rank parameterization rather than from a preference for any part of the spectrum. FRA is therefore effective as a conservative spectral residual, and not as a generic frequency-module replacement. VI-C Tri-Prior Decoding Table VIII compares the structural terms of (11) with conventional auxiliary losses at matched loss weights, and reports pairwise confusion mass on the frozen relation set. The question is whether the tri-prior combination improves on adding the same families of loss as independent multi-task terms. TABLE VIII: Confusion-Aware Tri-Prior Decoder Analysis Variant mIoU Δ Panel A: matched auxiliary-loss controls (Vaihingen) S1 CE++FocalDice only 84.26 ↓ 0.22 S2 ++ boundary loss [19] 84.28 ↓ 0.20 S3 ++ objectness loss 84.29 ↓ 0.19 S4 ++ pixel contrast [52] 84.31 ↓ 0.17 S5 all three conventional losses 84.32 ↓ 0.16 S6 CATP (feedback ++ pair hinge) 84.48 – Pre-declared pair w/o CATP CATP Rel. Panel B: pairwise confusion mass Ea,bE_a,b Vaih. Imp. Surf. ↔ Building 0.1420.142 0.1080.108 −24%-24\% Vaih. Low Veg. ↔ Tree 0.1180.118 0.0910.091 −23%-23\% Pots. Imp. Surf. ↔ Building 0.0960.096 0.0740.074 −23%-23\% LoveDA Background ↔ Barren 0.1860.186 0.1410.141 −24%-24\% OEM Rangeland ↔ Agriculture 0.0940.094 0.0680.068 −28%-28\% Panel A compares CATP with conventional auxiliary losses on ISPRS Vaihingen under matched loss weights and epochs, so row S1 coincides with the decoder-off row of Table IX; Vaihingen scores include flip and rotation TTA, as in Table I. Panel B reports the pairwise confusion mass Ea,b=Mab+MbaE_a,b=M_ab+M_ba on the training-only pair set; Rel. is the relative reduction against the matched model without CATP. OEM: OpenEarthMap. The best and second-best entries in each column are bold red and underlined blue, respectively. Ranking applies to the mIoU column of Panel A, where ↓ is the drop with respect to S6; a lower confusion mass is better throughout Panel B. The evidence that separates CATP from a generic multi-task head is Panel B: pairwise confusion mass decreases by 2323–28%28\% on every pre-declared pair of (9) that the panel reports, the largest reduction being OpenEarthMap Rangeland ↔ Agriculture, which is exactly where the hinge applies pressure. Aggregate mIoU agrees: adding boundary, objectness or pixel-contrast losses individually improves the CE/FocalDice reference of (10) by at most 0.050.05 p, and the three of them together remain 0.160.16 p below CATP, so the gain comes from how the structural signals are coupled and not from the additional supervision alone. The next subsection tests whether the three stages remain useful when they are combined. VI-D Component Complementarity and Initialization Table IX trains all eight combinations of HG-SAF, FRA and CATP and compares conservative initialization with random and fully active starts. Combining the two analyses in one table avoids separating architecture from optimization. TABLE IX: Component Complementarity and Initialization Study HG-SAF FRA CATP Vaih. Pots. LoveDA OEM Avg. Panel A: factorial combinations – – – 83.81 87.30 53.91 66.32 72.84 ✓ – – 84.04 87.65 54.46 66.85 73.25 – ✓ – 83.99 87.52 54.18 66.61 73.08 – – ✓ 84.19 87.78 54.58 66.94 73.37 ✓ ✓ – 84.26 87.93 54.81 67.27 73.57 ✓ – ✓ 84.31 88.01 54.92 67.38 73.66 – ✓ ✓ 84.28 87.94 54.71 67.12 73.51 ✓ ✓ ✓ 84.48 88.20 55.17 67.70 73.89 Initialization mIoU std10 Best ep. Div. Panel B: alternative initializations (Vaihingen) Reference (ours) 84.48 0.42 67 0/30/3 Small random 84.19 0.71 71 0/30/3 Standard random 84.12 0.91 74 0/30/3 Fully active 83.88 1.24 76 1/31/3 Panel A lists the 232^3 independently trained combinations of HG-SAF, FRA, and CATP; its first row switches all three off, leaving mean fusion and a single semantic head on the prepared stages. mIoU increases along every path of the lattice on all four datasets, and pairwise interactions on Vaihingen are IHF=+0.04I_HF=+0.04, IHC=−0.11I_HC=-0.11 and IFC=−0.09I_FC=-0.09 p, i.e., the modules are complementary and largely additive. Panel B reports initialization stability over three seeds, where std10 is the training-loss standard deviation over the first 1010 epochs, Best ep. is the best epoch, and Div. counts diverged runs. ISPRS columns include flip and rotation TTA as in Table I; LoveDA and OpenEarthMap (OEM) use single-scale inference, so Avg. here mixes the two inference settings and is not the matched no-TTA mean of Table I; it is used only to compare rows within this table. The best and second-best entries in each column are bold red and underlined blue, respectively. Only the mIoU column is ranked in Panel B, because the remaining three columns are stability diagnostics for which a lower value is better. Each module is beneficial on its own, every pair improves on both of its members, and the full combination is best on all four datasets. Restoring the missing module to a pair adds +0.20+0.20 for HG-SAF, +0.17+0.17 for FRA and +0.22+0.22 p for CATP on Vaihingen, and the pairwise interactions (IHF=+0.04I_HF=+0.04, IHC=−0.11I_HC=-0.11, IFC=−0.09I_FC=-0.09 p, each formed as m11−m10−m01+m00m_11-m_10-m_01+m_00) stay well below the main effects, so the three stages are complementary and largely additive. Reference initialization attains the highest mIoU, the lowest early-loss variance, and no diverged run. Starting from a known reference behavior therefore stabilizes fine-tuning and also improves its endpoint. VI-E Fixed Preparation Blocks Table X measures the fixed blocks of Section I-B on the reference decoder, with HG-SAF, FRA and CATP switched off. TABLE X: Individual Effect of the Fixed Preparation Blocks Preparation mIoU Δ T1 none (reference decoder) 83.57 – T2 SSDB only 83.66 +0.09+0.09 T3 BCS-Mamba only 83.71 +0.14+0.14 T4 SOE only 83.64 +0.07+0.07 T5 SSDB ++ BCS-Mamba ++ SOE 83.79 +0.22+0.22 ISPRS Vaihingen mIoU (%), single-scale and no TTA, so row T1 is exactly the Swin-B++UPerNet reference of Table I and the +0.22+0.22 p of row T5 is the share of the no-TTA margin of Table I that the preparation blocks contribute. Only the preparation blocks change; HG-SAF, FRA and CATP are absent in all five rows. The best and second-best entries in each column are bold red and underlined blue, respectively. SSDB, BCS-Mamba and SOE contribute +0.09+0.09, +0.14+0.14 and +0.07+0.07 p individually and +0.22+0.22 p jointly, so row T5 is the preparation stage adopted inside HAFR-Net. Two consequences are recorded explicitly. First, of the +0.55+0.55 p that HAFR-Net gains over the reference decoder on Vaihingen in Table I, +0.22+0.22 p comes from this fixed stage and +0.33+0.33 p from the three refinement stages. Second, all module analyses in Tables VI–IX keep the preparation blocks fixed, so those comparisons are unaffected by this share. VI-F Qualitative Comparison Figs. 8–10 close the analysis with representative predictions on the four benchmarks. Every map shown, including those of the comparison methods, is produced by a model we trained ourselves and decoded with the same inference code; no prediction is reproduced from a published figure. Each observation below belongs to a stratum that the analyses above already quantify, so the figures show where those measured effects become visible. The same three tendencies recur across the four datasets. Thin elongated structures stay connected over longer spans, adjacent instances of one class are merged into a single region less often, and the two classes of a pre-declared relation pair leak into each other less. The regions we discuss are marked in each figure, and the errors that survive are collected in Section VII-B. Fig. 8: Representative ISPRS examples. Boxes mark vehicles and adjacent class boundaries discussed with the HG-SAF and CATP analyses. The ISPRS examples in Fig. 8 concern vehicles adjacent to impervious surfaces and building outlines, which are the two regimes that the HG-SAF and CATP analyses measure separately. Vehicles are the smallest annotated class in these benchmarks and fall almost entirely inside the small-object stratum of Table VI, where stratified IoU improves by +1.52+1.52 p in Panel B and Panel C shows the largest shift toward the shallow stages; in the marked parking rows the neighbouring vehicles stay separated rather than merging into one region. Building outlines are the opposite case: long boundaries between two classes with overlapping colour statistics, i.e., the pair whose confusion mass CATP reduces in Panel B of Table VIII. In the marked regions the predicted outline follows the annotated corner more closely. Neither observation is specific to these tiles: both are consistent with the strata of Table VI and the pair panel of Table VIII, and the boundary and small-component metrics of Table VII improve alongside mIoU rather than in place of it. Fig. 9: Representative LoveDA examples. Boxes mark thin rural roads and low-texture Barren regions. The LoveDA examples in Fig. 9 highlight thin rural roads and low-texture Barren patches. Thin roads are exactly the regime that thin-structure IoU in Table VII quantifies, and in the marked regions the road stays continuous where its surface is partly occluded instead of breaking into fragments. Barren is the harder case, because it is characterized less by a texture than by the absence of one, so a decoder that leans on local appearance tends to absorb it into Background; that is the pair whose confusion mass CATP reduces in Panel B of Table VIII, and the marked patches keep one label over a larger area. LoveDA also accounts for part of the dataset-level pattern in Table I: its tiles mix urban and rural domains, so a larger share of pixels falls in the upper heterogeneity quartiles, which is where Panel B of Table VI reports the largest stratified gains. Fig. 10: Representative OpenEarthMap examples. Boxes mark Rangeland ↔ Agriculture transitions and thin roads. The OpenEarthMap examples in Fig. 10 show Rangeland ↔ Agriculture transitions and thin roads. That pair is the pre-declared relation of this dataset and the one with the largest relative reduction of confusion mass in Panel B of Table VIII. Its transition is gradual rather than sharp, so the decision is where to cut a broad gradient rather than where to find a visible edge, and in the marked regions the predicted boundary follows the field outline instead of oscillating inside the gradient. OpenEarthMap carries the largest number of classes and the widest geographic spread of the four benchmarks, which is consistent with it also showing the largest matched-protocol margin in Table I. Taken together, the four figures locate the measured gains in small objects, thin structures and pre-declared class pairs, and they do not show a uniform improvement over large homogeneous interiors, which is what Panel B of Table VI already reports numerically. The next section reports the compute cost of the full model and the remaining limitations. VII Efficiency and Limitations VII-A Efficiency Accuracy gains are useful only if their cost is stated under the same protocol as the controlled comparison. Table XI reports parameters, GFLOPs, peak memory, latency and FPS for the re-implemented Swin-B family. Published mixed-protocol models are omitted from that table. TABLE XI: Efficiency of Five Re-Implemented Swin-B Decoders Method Par. GFLOPs Mem. Lat. FPS mIoU Swin-B++FPN 90.1 102.6 1.72 18.9 52.9 72.09 Swin-B++UPerNet 89.3 118.4 1.84 21.4 46.7 72.49 Swin-B++UNetFormer dec. 92.6 124.1 1.91 22.6 44.2 72.78 Swin-B++FFT mixer 93.8 129.7 1.98 23.5 42.6 72.56 HAFR-Net 97.8 131.2 2.12 24.8 40.3 73.71 Cost is measured on one RTX 4090 under the timing protocol of Section IV-D, with TTA off. Par.: parameters (M); Mem.: peak memory (GB); Lat.: latency (ms); mIoU is the four-dataset mean of Table I without TTA. The best and second-best entries in each column are bold red and underlined blue, respectively. Only mIoU is ranked, because HAFR-Net is the most expensive entry in every cost column. Fig. 11: Published accuracy and cost, shown as context. Panel (a) gives the four-dataset mean mIoU of the methods present in all three contextual tables; for HAFR-Net this is the 73.89%73.89\% mean, which uses TTA on the two ISPRS sets. Panel (b) plots the reported parameter count against that mean, with bubble area proportional to the reported GFLOPs. The entries come from different backbones and inference settings, so no frontier or optimality is drawn: the matched comparison is Table I and its measured cost Table XI. HAFR-Net adds 8.58.5 M parameters over Swin-B++UPerNet. The FP32 FFT of FRA costs 2.92.9 ms per 512×512512×512 tile, i.e., about 12%12\% of the 24.824.8 ms forward pass of Table XI. The accuracy gain in Table I is therefore obtained at a moderate, fully specified compute overhead. Fig. 11 places the same mean next to published literature values. Stating the same cost as a ratio against the reference makes the trade-off easier to compare with other decoders. Relative to Swin-B++UPerNet, HAFR-Net adds 9.5%9.5\% parameters, 10.8%10.8\% GFLOPs and 15.2%15.2\% peak memory, and it lowers throughput from 46.746.7 to 40.340.3 FPS, i.e., by 13.7%13.7\%, in exchange for +1.22+1.22 p of four-dataset mean mIoU without TTA. Latency grows somewhat faster than GFLOPs (15.9%15.9\% against 10.8%10.8\%), which is consistent with the FP32 FFT of FRA being bound by memory traffic rather than by arithmetic; the ordering of the five methods in Table XI is the same under either measure. We report FPS at batch size 11, the setting in which that overhead is least amortized. At 512×512512×512 the full model still needs 24.824.8 ms per tile on one RTX 4090, so tiled inference over a large scene stays bounded by tiling and mosaicking rather than by the decoder. Within Table XI the closest cost neighbour is the matched FFT mixer at 23.523.5 ms and 93.893.8 M parameters: HAFR-Net spends 1.31.3 ms and 4.04.0 M more and returns +1.15+1.15 p of four-dataset mean mIoU. The next subsection lists the cases that this cost does not resolve. VII-B Limitations Four limitations remain. First, ℋH is a fixed functional of the learned feature F~4 F_4, namely the mean per-channel local standard deviation of (2). It adds no parameters of its own, but it inherits whatever F~4 F_4 encodes, so a semantically homogeneous yet feature-variable region such as tree canopy can produce a boundary-like response. The measured association between ℋH and the shallow-stage weight is correspondingly weak (Fig. 6(e)), and a learned heterogeneity measure is left to future work. Second, the spectral branches of SSDB and FRA evaluate their FFTs in FP32 for numerical stability, which is the latency reported in Section VII-A. A half-precision or windowed spatial approximation would remove that cost, but we have not verified that it preserves the boundary metrics of Table VII, so we report the conservative variant. Third, the prototype relation set depends on a training-only pilot split and does not adapt online, so a pair that becomes confusable only in a deployment domain receives no hinge; the number of candidate pairs also grows quadratically with the number of classes, which is why the set is declared in advance rather than searched. Fourth, all experiments use RGB or NIR-RG VHR imagery and have not been extended systematically to SAR or hyperspectral data, where the spectral assumptions behind SSDB and FRA would have to be re-examined rather than transferred. Residual errors that are not captured by hierarchical fusion, bounded refinement, or structural regularization—including cast shadows, mixed vegetation, and the catch-all Clutter class—also remain difficult, and they dominate the marked regions of Figs. 8–10 that HAFR-Net does not resolve. VIII Conclusion This work studies VHR remote sensing image segmentation from the perspective of how pretrained hierarchical representations are decoded and refined. Rather than adding unconstrained decoder capacity, we propose HAFR-Net, which organizes the capacity it does add as a progressive refinement of the encoder hierarchy in three stages. HG-SAF performs heterogeneity-conditioned pixel-wise fusion of multi-stage features, FRA introduces a bounded residual frequency correction inside a channel bottleneck, and CATP regularizes the final prediction with structural and class-relation cues. Controlled experiments across four VHR benchmarks show gains of +0.33+0.33 to +1.52+1.52 p mIoU over the strongest matched baseline at a moderate and fully reported compute overhead, while module-specific analyses demonstrate spatially varying stage routing, improved boundary and thin-structure prediction, and reduced structural and class-confusion errors. Adaptive and conservative refinement of strong pretrained representations is therefore an effective design principle for accurate VHR segmentation. Future work will investigate learned heterogeneity measures, adaptive relation discovery, and other remote sensing modalities. Acknowledgment The authors would like to thank the ISPRS, LoveDA, and OpenEarthMap organizers for releasing their benchmark datasets. References [1] M. Berman, A. R. Triki, and M. B. Blaschko (2018) The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 4413–4421. Cited by: §I-C. [2] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. In Proc. Eur. Conf. Comput. Vis. (ECCV), p. 801–818. External Links: Document Cited by: §I, §I-A, §IV-B, TABLE I. [3] B. Cheng, R. Girshick, P. Dollár, A. C. Berg, and A. Kirillov (2021) Boundary IoU: improving object-centric image segmentation evaluation. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 15334–15342. Cited by: §IV-D. [4] B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar (2022) Masked-attention mask transformer for universal image segmentation. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 1290–1299. Cited by: §I-C, TABLE I, TABLE V. [5] L. Chi, B. Jiang, and Y. Mu (2020) Fast Fourier convolution. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 4479–4488. Cited by: §I, §I-B, TABLE VII. [6] G. Csurka, D. Larlus, and F. Perronnin (2013) What is a good evaluation measure for semantic segmentation?. In Proc. Brit. Mach. Vis. Conf. (BMVC), p. 32.1–32.11. External Links: Document Cited by: §IV-D. [7] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) ImageNet: a large-scale hierarchical image database. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 248–255. External Links: Document Cited by: §I-A. [8] F. Gao, M. Fu, J. Cao, J. Dong, and Q. Du (2025) Adaptive frequency enhancement network for remote sensing image semantic segmentation. IEEE Trans. Geosci. Remote Sens. 63, p. 1–15. Note: Art. no. 5619415 External Links: Document Cited by: §I, §I-B. [9] P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He (2017) Accurate, large minibatch SGD: training ImageNet in 1 hour. arXiv preprint arXiv:1706.02677. Cited by: §IV-C. [10] A. Gu and T. Dao (2024) Mamba: linear-time sequence modeling with selective state spaces. In Proc. Conf. Lang. Model. (COLM), Cited by: §I. [11] R. Gupta, B. Goodman, N. Patel, R. Hosfelt, S. Sajeev, E. Heim, J. Doshi, K. Lucas, H. Choset, and M. Gaston (2019) Creating xBD: a dataset for assessing building damage from satellite imagery. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), p. 10–17. Cited by: §I. [12] T. Hanyu, K. Yamazaki, M. Tran, R. A. McCann, H. Liao, C. Rainwater, M. Adkins, J. Cothren, and N. Le (2024) AerialFormer: multi-resolution transformer for aerial image segmentation. Remote Sens. 16 (16), p. 2930. External Links: Document Cited by: TABLE IV. [13] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly (2019) Parameter-efficient transfer learning for NLP. In Proc. Int. Conf. Mach. Learn. (ICML), p. 2790–2799. Cited by: §I-F. [14] J. Hu, L. Shen, and G. Sun (2018) Squeeze-and-excitation networks. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 7132–7141. Cited by: §I-A, §IV-B, TABLE I. [15] L. Huang, B. Jiang, S. Lv, Y. Liu, and Y. Fu (2024) Deep-learning-based semantic segmentation of remote sensing images: a survey. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 17, p. 8370–8396. External Links: Document Cited by: §I. [16] G. Hwang, J. Jeong, and S. J. Lee (2024) SFA-Net: semantic feature adjustment network for remote sensing image segmentation. Remote Sens. 16 (17), p. 3278. External Links: Document Cited by: TABLE IV. [17] ISPRS ISPRS 2D semantic labeling contest — Potsdam. Note: [Online]. Available: https://w.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-potsdam.aspxAccessed: Jun. 13, 2026 Cited by: §IV-A. [18] ISPRS ISPRS 2D semantic labeling contest — Vaihingen. Note: [Online]. Available: https://w.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-vaihingen.aspxAccessed: Jun. 13, 2026 Cited by: §IV-A. [19] H. Kervadec, J. Bouchtiba, C. Desrosiers, E. Granger, J. Dolz, and I. B. Ayed (2019) Boundary loss for highly unbalanced segmentation. In Proc. Int. Conf. on Medical Imaging with Deep Learning (MIDL), p. 285–296. Cited by: §I, §I-C, TABLE VIII. [20] J. Lee-Thorp, J. Ainslie, I. Eckstein, and S. Ontañón (2022) FNet: mixing tokens with Fourier transforms. In Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics (NAACL), p. 4296–4313. External Links: Document Cited by: §I-B. [21] R. Li, L. Wang, C. Zhang, C. Duan, and S. Zheng (2022) A2^2-FPN for semantic segmentation of fine-resolution remotely sensed images. Int. J. Remote Sens. 43 (3), p. 1131–1155. External Links: Document Cited by: §I-A, TABLE I, TABLE IV, TABLE V. [22] R. Li, S. Zheng, C. Zhang, C. Duan, J. Su, L. Wang, and P. M. Atkinson (2022) Multiattention network for semantic segmentation of fine-resolution remote sensing images. IEEE Trans. Geosci. Remote Sens. 60, p. 1–13. Note: Art. no. 5607713 External Links: Document Cited by: §I-A, TABLE I, TABLE IV, TABLE V. [23] R. Li, S. Zheng, C. Zhang, C. Duan, L. Wang, and P. M. Atkinson (2021) ABCNet: attentive bilateral contextual network for efficient semantic segmentation of fine-resolution remotely sensed imagery. ISPRS J. Photogramm. Remote Sens. 181, p. 84–98. External Links: Document Cited by: §I-A, TABLE I, TABLE V. [24] X. Li, F. Xu, A. Yu, X. Lyu, H. Gao, and J. Zhou (2025) A frequency decoupling network for semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 63, p. 1–21. Note: Art. no. 5607921 External Links: Document Cited by: §I, §I-B. [25] X. Li, F. Xu, J. Zhang, H. Zhang, X. Lyu, F. Liu, H. Gao, and A. Kaup (2026) Frequency-guided denoising network for semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 64, p. 1–17. Note: Art. no. 5400217 External Links: Document Cited by: §I-B. [26] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar (2021) Fourier neural operator for parametric partial differential equations. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: §I-B. [27] T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie (2017) Feature pyramid networks for object detection. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 2117–2125. Cited by: §I, §I-A, §IV-B, TABLE I. [28] T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár (2017) Focal loss for dense object detection. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 2980–2988. Cited by: §I-C. [29] Y. Liu, M. Cheng, X. Hu, K. Wang, and X. Bai (2017) Richer convolutional features for edge detection. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 5872–5881. Cited by: §I-C. [30] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo (2021) Swin Transformer: hierarchical vision transformer using shifted windows. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 10012–10022. Cited by: §I, §I-C, §I-A. [31] J. Long, E. Shelhamer, and T. Darrell (2015) Fully convolutional networks for semantic segmentation. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 3431–3440. Cited by: §I, §I-A. [32] I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: §IV-C. [33] X. Ma, X. Zhang, M. Pun, and B. Huang (2025) A unified framework with multimodal fine-tuning for remote sensing semantic segmentation. IEEE Trans. Geosci. Remote Sens. 63, p. 1–15. External Links: Document Cited by: §I-C. [34] X. Ma, X. Zhang, and M. Pun (2024) RS3Mamba: visual state space model for remote sensing image semantic segmentation. IEEE Geosci. Remote Sens. Lett. 21, p. 1–5. Note: Art. no. 6011405 External Links: Document Cited by: §I-C, TABLE I, TABLE IV, TABLE V. [35] X. Ma, R. Lian, Z. Wu, H. Guo, F. Yang, M. Ma, S. Wu, Z. Du, W. Zhang, and S. Song (2025) LOGCAN++: adaptive local-global class-aware network for semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 63, p. 1–16. External Links: Document Cited by: TABLE I, TABLE IV, TABLE V. [36] X. Meng, Y. Yang, L. Wang, T. Wang, R. Li, and C. Zhang (2022) Class-guided Swin Transformer for semantic segmentation of remote sensing imagery. IEEE Geosci. Remote Sens. Lett. 19, p. 1–5. Note: Art. no. 6517505 External Links: Document Cited by: TABLE I. [37] F. Milletari, N. Navab, and S. Ahmadi (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In Proc. Int. Conf. 3D Vision (3DV), p. 565–571. Cited by: §I-C. [38] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. (2019) PyTorch: an imperative style, high-performance deep learning library. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 8024–8035. Cited by: §IV-C. [39] Y. Rao, W. Zhao, Z. Zhu, J. Lu, and J. Zhou (2021) Global filter networks for image classification. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 980–993. Cited by: §I, §I-B, §IV-B, TABLE I, TABLE VII. [40] S. Rebuffi, H. Bilen, and A. Vedaldi (2017) Learning multiple visual domains with residual adapters. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 506–516. Cited by: §I-F. [41] O. Ronneberger, P. Fischer, and T. Brox (2015) U-Net: convolutional networks for biomedical image segmentation. In Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI), p. 234–241. Cited by: §I, §I-A, TABLE I, TABLE IV, TABLE V. [42] T. Takikawa, D. Acuna, V. Jampani, and S. Fidler (2019) Gated-SCNN: gated shape CNNs for semantic segmentation. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 5229–5238. Cited by: §I, §I-C. [43] A. Tarvainen and H. Valpola (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In Proc. Adv. Neural Inf. Process. Syst. (NIPS), p. 1195–1204. Cited by: §IV-C. [44] H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve, and H. Jégou (2021) Going deeper with image transformers. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 32–42. Cited by: §I-F. [45] J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, W. Liu, and B. Xiao (2021) Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 43 (10), p. 3349–3364. External Links: Document Cited by: §I-A, TABLE I, TABLE IV, TABLE V. [46] J. Wang, Z. Zheng, A. Ma, X. Lu, and Y. Zhong (2021) LoveDA: a remote sensing land-cover dataset for domain adaptive semantic segmentation. In Proc. NeurIPS Datasets and Benchmarks Track, Cited by: §IV-A. [47] L. Wang, D. Li, S. Dong, X. Meng, X. Zhang, and D. Hong (2025) PyramidMamba: rethinking pyramid feature fusion with selective space state model for semantic segmentation of remote sensing imagery. Int. J. Appl. Earth Observ. Geoinf. 144, p. 104884. External Links: Document Cited by: §I, §I-C, TABLE I, TABLE IV, TABLE V. [48] L. Wang, R. Li, C. Duan, C. Zhang, X. Meng, and S. Fang (2022) A novel transformer based semantic segmentation scheme for fine-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 19, p. 1–5. Note: Art. no. 6506105 External Links: Document Cited by: §I, §I-A, TABLE I, TABLE V. [49] L. Wang, R. Li, D. Wang, C. Duan, T. Wang, and X. Meng (2021) Transformer meets convolution: a bilateral awareness network for semantic segmentation of very fine resolution urban scene images. Remote Sens. 13 (16), p. 3065. External Links: Document Cited by: §I-A, TABLE I, TABLE V. [50] L. Wang, R. Li, C. Zhang, S. Fang, C. Duan, X. Meng, and P. M. Atkinson (2022) UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery. ISPRS J. Photogramm. Remote Sens. 190, p. 196–214. External Links: Document Cited by: §I, §I-A, §IV-B, TABLE I, TABLE I, TABLE V. [51] L. Wang and R. Li (2022) FT-UNetFormer implementation in the GeoSeg toolkit. Note: [Online]. Available: https://github.com/WangLibo1995/GeoSegAccessed: Jun. 13, 2026 Cited by: §I-A, §IV-B, TABLE I, TABLE V. [52] W. Wang, T. Zhou, F. Yu, J. Dai, E. Konukoglu, and L. Van Gool (2021) Exploring cross-image pixel contrast for semantic segmentation. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 7303–7313. Cited by: §I, §I-C, TABLE VIII. [53] Y. Wang, L. Cao, and H. Deng (2026) WgANet: a wavelet-guided attention network for remote sensing image semantic segmentation. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 19, p. 702–716. External Links: Document Cited by: §I-B. [54] S. Woo, J. Park, J. Lee, and I. S. Kweon (2018) CBAM: convolutional block attention module. In Proc. Eur. Conf. Comput. Vis. (ECCV), p. 3–19. Cited by: §I-A. [55] T. Wu, S. Tang, R. Zhang, J. Cao, and Y. Zhang (2021) CGNet: a light-weight context guided network for semantic segmentation. IEEE Trans. Image Process. 30, p. 1169–1179. External Links: Document Cited by: TABLE I, TABLE V. [56] J. Xia, N. Yokoya, B. Adriano, and C. Broni-Bediako (2023) OpenEarthMap: a benchmark dataset for global high-resolution land cover mapping. In Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), p. 6254–6264. External Links: Document Cited by: §IV-A. [57] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun (2018) Unified perceptual parsing for scene understanding. In Proc. Eur. Conf. Comput. Vis. (ECCV), p. 418–434. Cited by: §I, §I-A, §IV-B, TABLE I. [58] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo (2021) SegFormer: simple and efficient design for semantic segmentation with transformers. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 12077–12090. Cited by: §I, §I-C, §IV-B, TABLE I, TABLE V. [59] Z. Xu, J. Geng, and W. Jiang (2023) MMT: mixed-mask transformer for remote sensing image semantic segmentation. IEEE Trans. Geosci. Remote Sens. 61, p. 1–15. Note: Art. no. 5613415 External Links: Document Cited by: TABLE I. [60] M. Yang, K. Yu, C. Zhang, Z. Li, and K. Yang (2018) DenseASPP for semantic segmentation in street scenes. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 3684–3692. External Links: Document Cited by: TABLE I, TABLE IV, TABLE V. [61] Y. Yang, G. Yuan, and J. Li (2024) SFFNet: a wavelet-based spatial and frequency domain fusion network for remote sensing segmentation. IEEE Trans. Geosci. Remote Sens. 62, p. 1–17. Note: Art. no. 3000617 External Links: Document Cited by: §I, §I-B. [62] X. Yuan, J. Shi, and L. Gu (2021) A review of deep learning methods for semantic segmentation of remote sensing imagery. Expert Syst. Appl. 169, p. 114417. External Links: Document Cited by: §I. [63] H. Zhang, G. Xie, L. Li, X. Xie, and J. Ren (2025) Frequency-domain guided Swin Transformer and global–local feature integration for remote sensing images semantic segmentation. IEEE Trans. Geosci. Remote Sens. 63, p. 1–11. Note: Art. no. 5612611 External Links: Document Cited by: §I-B. [64] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia (2017) Pyramid scene parsing network. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 2881–2890. Cited by: §I, §I-A, TABLE I, TABLE IV, TABLE V. [65] T. Zhou, W. Wang, E. Konukoglu, and L. Van Gool (2022) Rethinking semantic segmentation: a prototype view. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 2582–2593. Cited by: §I, §I-C. [66] E. Zhu, Z. Chen, D. Wang, H. Shi, X. Liu, and L. Wang (2025) UNetMamba: an efficient UNet-like Mamba for semantic segmentation of high-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 22, p. 1–5. Note: Art. no. 6001205 External Links: Document Cited by: §I-C, TABLE I, TABLE IV, TABLE V. [67] X. Zou, Y. Li, S. Zhang, K. Li, S. Wang, P. Tao, J. Xing, and C. Lang (2025) Dynamic dictionary learning for remote sensing image segmentation. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 22457–22466. Cited by: TABLE I, TABLE IV.