Paper deep dive
Pixel-Level Transformers in Remote Sensing: A Canopy Height Case Study
Sven Ligensa, Jan Pauls, Karsten Schrรถdter, Ibrahim Fayad, Fabian Gieseke
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Predicting canopy height from medium-resolution satellite imagery is a common and scalable approach for assessing the condition of the world's forests, which play a crucial role in climate change mitigation. While Transformer-based architectures have shown strong performance in many domains, their straightforward application to dense (i.e., pixel-level) regression tasks often yields suboptimal results. In particular, the patch size has a crucial impact on the model performance. In this work, we consider pixel-level attention schemes and show that the resulting models generally outperform those relying on larger patch sizes. However, pixel-level attention can be a prohibitively resource-intensive operation. For this reason, we conduct an extensive experimental study using efficient attention variants to identify favorable trade-offs between prediction quality and resource requirements, facilitating the practical deployment of the proposed models. In addition, we perform a comprehensive comparison with several well-established models in the field and show that, with suitable hyperparameter choices, Transformer-based architectures can outperform competing approaches. Our findings provide practical guidance for designing models for pixel-level regression tasks on medium-resolution satellite imagery, including canopy height and biomass estimation, soil moisture mapping, and yield forecasting.
Tags
Links
- Source: https://arxiv.org/abs/2609.37809v1
- Canonical: https://arxiv.org/abs/2609.37809v1
Trouble viewing inline? Open PDF directly โ
Full Text
71,953 characters extracted from source content.
Expand or collapse full text
Pixel-Level Transformers in Remote Sensing: A Canopy Height Case Study Sven Ligensa sven.ligensa@uni-muenster.de University of Mรผnster Mรผnster, Germany Jan Pauls jan.pauls@uni-muenster.de University of Mรผnster Mรผnster, Germany Karsten Schrรถdter karsten.schroedter@uni-muenster.de University of Mรผnster Mรผnster, Germany Ibrahim Fayad ibrahim.fayad@lsce.ipsl.fr Laboratoire des Sciences du Climat et de lโEnvironnement Paris, France Fabian Gieseke fabian.gieseke@uni-muenster.de University of Mรผnster Mรผnster, Germany Abstract Predicting canopy height from medium-resolution satellite imagery is a common and scalable approach for assessing the condition of the worldโs forests, which play a crucial role in climate change mit- igation. While Transformer-based architectures have shown strong performance in many domains, their straightforward application to dense (i.e., pixel-level) regression tasks often yields suboptimal results. In particular, the patch size has a crucial impact on the model performance. In this work, we consider pixel-level attention schemes and show that the resulting models generally outperform those relying on larger patch sizes. However, pixel-level attention can be a prohibitively resource-intensive operation. For this rea- son, we conduct an extensive experimental study using efficient attention variants to identify favorable trade-offs between predic- tion quality and resource requirements, facilitating the practical deployment of the proposed models. In addition, we perform a comprehensive comparison with several well-established models in the field and show that, with suitable hyperparameter choices, Transformer-based architectures can outperform competing ap- proaches. Our findings provide practical guidance for designing models for pixel-level regression tasks on medium-resolution satel- lite imagery, including canopy height and biomass estimation, soil moisture mapping, and yield forecasting. CCS Concepts โข Computing methodologiesโComputer vision problems; Neu- ral networks; Supervised learning by regression;โข Applied com- putingโEnvironmental sciences;โข General and referenceโ Experimentation; Performance. Keywords Vision Transformers, Efficient Deep Learning, Remote Sensing, Canopy Height Prediction This work is licensed under a Creative Commons Attribution 4.0 International License. SIGSPATIAL โ26, Riverside, CA, USA ยฉ 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2950-8/2026/11 https://doi.org/10.1145/3841645.3842978 ACM Reference Format: Sven Ligensa, Jan Pauls, Karsten Schrรถdter, Ibrahim Fayad, and Fabian Gieseke. 2026. Pixel-Level Transformers in Remote Sensing: A Canopy Height Case Study. In The 34th ACM International Conference on Advances in Geographic Information Systems (SIGSPATIAL โ26), November 03โ06, 2026, Riverside, CA, USA. ACM, New York, NY, USA, 14 pages. https://doi.org/10. 1145/3841645.3842978 1 Introduction Forests play a key role in the carbon cycle and in regulating climate change [1]. One important component of forest carbon stocks is above-ground biomass (AGB), which refers to the mass of living vegetation located above the soil surface, including stems, branches, bark, foliage, and shrubs [18]. While collecting field estimates of AGB is time-consuming and expensive, satellite imagery is widely available and permits the estimation of, e.g., tree height [11], which can in turn be used to estimate AGB via the approximate power-law relationship between canopy height and AGB [37]. For that reason, machine learning and in particular deep learning approaches have been employed extensively in the past to generate global-scale canopy height maps [23, 29, 30, 33, 41]. Generating a canopy height map of global scale is a significant en- gineering challenge, requiring a considerable amount of resources for the inference [30]. The goal is to train a model, which achieves high predictive performance, while being efficient in terms of hav- ing a short training and inference time. While for high-resolution images, the semantic usually does not change significantly among a few neighbored pixels, it is different for dense prediction on medium-resolution satellite imagery, like Sentinel-2 data, where a single pixel typically corresponds to an area of 10ร10 meters [11]. Recently, the importance of pixel-level attention was stressed for general Vision Transformer (ViT) models [28,45]. In this work, we investigate the importance and effectiveness of pixel-level ViT- based architectures for dense prediction tasks in remote sensing, using canopy height estimation as a representative large-scale re- gression task. The patch sizeํ โNis a critical hyperparameter for ViT-based architectures, as it affects their memory and runtime complexity as well as the prediction quality [45]. Figure 1 shows exemplary pixel-level tree-height predictions on satellite images for models with varying patch size. Choosing a smallerํleads to a larger number of tokensํ= ํปํ ํ 2 to be processed by ViT-based arXiv:2609.37809v1 [cs.CV] 29 Sep 2026 SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. Sentinel-2 RGBALS Label ํ= 1, ํ ํ = 96ํ= 2, ํ ํ = 192ํ= 4, ํ ํ = 192ํ= 8, ํ ํ = 192 30m 0m Figure 1: Qualitative comparison of pixel-level regression by Transformer-based models with varying patch size (ํ) and model dimension (ํ ํ ). The first column shows the input image (only RGB channels shown, while the models are trained on twelve channels), the second column shows Airborne Laser Scanning (ALS) canopy height labels, and the following columns show the predictions of models with increasingly larger patch sizes. Especially for areas with fine details, models with smaller patch sizes produce predictions of higher perceptual quality. Further details are provided in Section 5 and additional examples are given in the appendix. models, whereํ,ํป โNare the imageโs width and height. With vanilla attention, memory and runtime complexity increase quadrat- ically with the sequence length [10,42]. For the ViT introduced by Dosovitskiy et al. [10], selecting a comparably large patch size of ํ=16 was necessary to be able to process reasonably-sized images (e.g., 224ร224 pixels). They also observed that smaller patches sizes generally improved performance; however the quadratic scaling behavior of the attention mechanism with regard to the sequence length made a smaller patch size prohibitive in practice [10]. With the introduction of more efficient attention mechanisms, such as shifted-window attention, the patch size was reduced toํ=4, which made the ViT-architectures more suitable for dense predic- tion tasks [25, 50]. To investigate the role of pixel-level ViTs for dense prediction in remote sensing, we conduct an extensive empirical study on canopy height estimation. 1 Our contributions are as follows: (1) We give practical guidance on choosing crucial hyperparam- eters of ViTs (likeํor the model dimensionํ ํ ) for dense regression tasks on medium-resolution satellite images. (2)We unveil shortcomings of traditional quantitative evalua- tion of canopy height predictions, based on noisy labels. (3)We propose a new model configuration that outperforms other well-known models on canopy height prediction. 2 Background In this section, we first give an overview of related work on ViTs, recent work on pixel-level ViTs, and proposed variants for reducing the attention mechanismโs complexity. Then, specifics of the canopy height prediction task are revised. 1 To facilitate reproducibility and future research, the code of our experimental evalua- tion is publicly available at https://github.com/SvenLigensa/pixel-level-transformers. 2.1 Vision Transformers The original ViT [10] adapted the Transformer architecture [42] to the task of image classification via minimal changes, giving rise to a whole new family of Vision Transformers. Hierarchical Transformers improved on the ViTโs two main weaknesses (1) by introducing hierarchical processing to account for visual elements of varying scale and (2) by reducingํfrom 16 to 4 pixels, which was facilitated by improving the attention computation. The two most well-known, concurrently developed, architec- tures are the so-called Swin Transformer [25] and the Pyramid Vision Transformer (PVT) [47]. Both architectures comprise four stagesโeach consisting of multiple Transformer layersโoperating on increasingly coarser spatial resolution. They differ, however, with regard to the attention mechanism used, and the way adjacent tokens are combined in between stages. The Swin Transformer uses shifted-window attention, which restricts attention to local windows applied with two alternating positions [25], while the PVT reduces the number of keys and values and thereby the number of computations of the attention by a constant factor [47]. To reduce the resolution between stages, the Swin Transformer merges the four patches inside a 2ร2 window by concatenating them along the embedding dimension, and then linearly projecting them to twice the previous embedding dimension [25]. The PVT reduces the resolution by applying an overlapping patch embedding layer with a patch size of two at the beginning of every stage [47]. For Transformers performing dense prediction tasks like seman- tic segmentation (pixel-level classification) and canopy height pre- diction (pixel-level regression), the output needs to have the same spatial dimension as the input. To achieve that, the architectures generally follow the encoder-decoder paradigm, where the decoder constitutes the counterpart of the encoder, and increases the spatial dimensionality [2,50,54]. Note that as most models choose a patch Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA size ofํ=4 up toํ=16 pixels, the decoderโs output resolution is still smaller than the input resolution. To obtain a prediction for every pixel of the input image, it is common practice to per- form non-parametric upsampling (e.g. bilinear interpolation) [54] or parametric upsampling (e.g. transposed convolution or patch expand) [2]. The architectures mainly differ in how they combine the hierarchical feature maps produced by the encoder. Popular Seg- mentation Transformers are the SETR [54], SegFormer [50], and the Swin-Unet [2]. A recently proposed variant is the U-MixFormer [51], fusing encoder and decoder features by using mix-attention, which allows the encoder feature to query the decoder feature hierarchy. 2.2 Pixel-Level Vision Transformers Recent work targeting pixel-level ViTs include Nguyen et al. [28] and Wang et al. [45]. Nguyen et al. [28]first demonstrated the effec- tiveness of treating each pixel as a token. Due to the computational complexity, they focused on small-scale datasets like CIFAR-100. They primarily analyzed the results from the perspective of re- moving the locality inductive bias [28]. Wang et al. [45]performed additional experiments, thoroughly testing the impact of the patch size hyperparameter across various vision tasks, input scales, and ar- chitectures. The emerging patterns are called patchification scaling laws, stating that decreasing patch sizes generally result in lower test loss. 2 In ViT-based architectures, a patch is the โbasic unit of operation after the first projection layerโ [28], i.e., all pixels of the same patch are considered jointly throughout the whole model. The patchification step can be viewed as a compression step, resulting in possibly irreversible information loss; operating on a pixel-level can avoid this bottleneck [45]. This compression is stronger the larger the patch size and the smaller the model dimension. Still, training ViT-based models on tens to hundreds of thou- sands of tokens per image 3 is challenging, as also noted by [28]. The following section gives a short overview of attention variants, which reduce the memory and runtime complexity of the operation, making pixel-level ViTs more efficient in practice. 2.3 Attention Mechanism Variants Dozens of attention mechanism variants have been proposed to alle- viate the memory (and runtime) complexity of the vanilla attention by Vaswani et al. [42], which is quadratic in the number of tokensํ. Sparse attention reduces the number of tokens that can interact with each other. The pattern of interacting tokens can either be fixed or dynamic. Examples for fixed sparse attention mechanisms are local window attention [52], shifted window attention [25], and attention along axes [9,17]. Dynamic (i.e., data-dependent) patterns include clustering the tokens via locality-sensitive hash- ing [22,43] orํ-means [3,36,43], and only calculating attention among tokens of the same cluster. Low-rank approximations are based on the empirical observation that the self-attention matrix can often be approximated via a low-rank matrix, meaning that it can be efficiently represented as the product of smaller low- dimensional matrices [46]. Examples are the Linformer [46], Linear Transformer [21], Performer [5], random feature attention [32], and 2 Similar to Kaplan et al. [20], who showed that the test loss scales according to a power-law w.r.t. both model size and dataset size. 3 Images of size 256ร256 (512ร512) pixels already give rise to 65 536 (262 144) tokens. efficient attention [40]. A more detailed description of the attention mechanisms considered in this work is given in Section 3. 2.4 Canopy Height Prediction To analyze pixel-level ViTs for dense prediction in remote sensing, we consider canopy height prediction, a key task in Earth obser- vation applications, where the goal is to predict the height of the (usually) tallest tree for each pixel of a remote sensing image, e.g., a medium-resolution satellite image. Creating such canopy height maps at a large scale relies on satellites providing spatially and temporally continuous and consistent images. Two common data sources are the Sentinel-2 mission operated by the ESA [6,11] and the Landsat mission operated by the NASA [13]. The corresponding satellites provide, among other things, optical, i.e. passive, multi- spectral images at a resolution of up to 10 m and 30 m, respectively. The Sentinel-2 mission comprises two active satellites in the same sun-synchronous orbit, phased half an orbit apart [11]. The revisit time is 5 days at the equator and resolutions range from 10 m to 60 m depending on the band. The imaging instrument ofSentinel-2 captures 13 bands between 433nmand 2280nm, including RGB, near infrared and others [11]. Commonly used canopy height labels are either derived from space-borne instruments, such as Global Ecosystem Dynamics In- vestigation (GEDI) [12] or ICESat [27], or measurements from Air- borne Laser Scanning (ALS) campaigns [15]. While the latter ones exhibit a much higher vertical accuracy and spatial resolution, both their spatial and temporal coverage is heavily limited. For large- scale applications, models are typically trained on data from space- borne instruments, such as the GEDI mission, as it provides data from 2019 onwards and exhibits a global coverage between 51.6 โฆ ํ and 51.6 โฆ ํ. The GEDI system [12] is a set of three full-waveform LiDAR sensors mounted on the ISS that capture eight parallel tracks (two power beams measuring on two tracks each and one cover- age beam measuring on four tracks) with a 600 m inter-track and 60 m intra-track spacing. Each measurement has an approximate diameter of 25 m (slightly varying depending on the ISS height) and starting with the second release of the GEDI product, the geoloca- tion accuracy has met its target of being unbiased (i.e., having a mean of zero) and having a standard deviation of less than 10 m. To convert the waveform of each measurement into a scalar label, two commonly used metrics are the relative height (rh) at which 95% (rh95) and 98% (rh98) of the cumulative returned energy was received. ALS data on the other hand does not suffer from low geolocation accuracy or low vertical measurement quality. Typically captured by LiDAR sensors mounted on airplanes, they exhibit a resolution of 1 m up to 10cm. Also, in contrast to GEDIโs sparse measurements, LiDAR sensors generally provide dense measurements. 4 Estimating canopy height from remote sensing data has been es- tablished as a reliable source for large-scale forest monitoring, as it can be applied across vast areas more easily than field-based inven- tories. Such canopy height maps have been created at regional [38], national [14,19,37], continental [24] and global [23,29,30] scale 4 For the task of canopy height estimation, OpenCanopy [15] provides a dataset of combined optical satellite data and unified ALS campaign over France, which will be used for evaluation in our experiments. SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. ํฟ ํ ร ํ ํ ํธ ํ ร ํ ํ ํฒ ํ ร ํ ํ ํฝ ํ ร ํ ํฃ ยทํพ ํ ํ ํ ร ํ ํ ยทํพ ํพ ํ ํ ร ํ ํ ยทํพ ํ ํ ํ ร ํ ํฃ ํจ ํ ร ํ Softmax โ ํธยทํฒ โค โ ํ ํ ํถ ํ ร ํ ํฃ ํจํฝ Figure 2: Computations of a vanilla attention head [42]. and resort to either classical machine learning methods [19,33], convolutional neural networks [30,37,38,44] or attention-based architectures [14,29]. Although not perfectly matching ALS mea- surements, these maps have been shown to be reliable estimators for various downstream tasks [26]. 3 Attention Mechanisms In this section, we present and compare popular attention variants with the vanilla attention mechanism of the Transformer architec- ture [42]. To simplify notation, we focus on a single attention head. In practice, Transformers use Multi-Head Attention (MHA) layers, which include multiple attention heads performing attention inde- pendently and in parallel. We refer to Vaswani et al. [42]for further details. 3.1 Notation Letํฟ โR ํรํ ํ be an input sequence ofํ โNtoken embeddings, each of dimensionํ ํ โN. Letํ ํ โNandํ ํฃ โNdenote the embedding dimensions for the keys and queries, and the values, respectively. The query-, key- and value-transformation matrices are denoted byํพ ํ โR ํ ํ รํ ํ ,ํพ ํพ โR ํ ํ รํ ํ , andํพ ํ โR ํ ํ รํ ํฃ . Accordingly, the query, key and value features are defined byํธโ ํฟ ยท ํพ ํ โR ํรํ ํ ,ํฒโ ํฟ ยท ํพ ํพ โR ํรํ ํ andํฝโ ํฟ ยท ํพ ํ โR ํรํ ํฃ . The row-wise softmax functionSoftmax โ :R ํรํ โR ํรํ normalizes the element-wise exponential of the matrix in a row- wise manner, i.e.,Softmax โ (ํบ) ํ,ํ = exp(ํบ ํ,ํ )/ ร ํ ํ=1 exp(ํบ ํ,ํ )for a score matrixํบ โR ํรํ and indices 1โค ํ, ํ โค ํ. Similarly, the column-wise softmaxSoftmax โ :R ํรํ โR ํรํ normalizes the exponential column-wise. 3.2 Vanilla Attention We shortly revisit the original formulation of scaled-dot product attention as introduced by Vaswani et al. [42], which we will refer to as vanilla attention in this work. In vanilla attention, the query featuresํธ โR ํรํ ํ and key featuresํฒ โR ํรํ ํ are used to calculate the attention matrix ํจโ Softmax โ ํธ ยท ํฒ โค โ ํ ํ โR ํรํ .(1) The outputํถof the attention head is then calculated by multiplying the attention matrix with the value features, i.e. Attention vanilla (ํธ, ํฒ, ํฝ)โ ํถโ ํจยท ํฝ โR ํรํ ํฃ .(2) The computation flow of vanilla attention is visualized in Fig- ure 2. Vanilla attention has a runtime complexity ofO(ํ 2 ํ ํ )and ํฟ ํ ร ํ ํ ํธ ํ ร ํ ํ ํฒ ํ ร ํ ํ ํฝ ํ ร ํ ํฃ ยทํพ ํ ํ ํ ร ํ ํ ยทํพ ํพ ํ ํ ร ํ ํ ยทํพ ํ ํ ํ ร ํ ํฃ ํธ ํ ร ํ ํ Softmax โ (ํธ) ํฒ โค ํ ํ ร ํ Softmax โ (ํฒ) โค ํช ํ ํ ร ํ ํฃ ํฒ โค ํฝ ํถ ํ ร ํ ํฃ ํธํช Figure 3: Computations of an efficient attention head [40]. a memory complexity ofO(ํ 2 ), both of which are dominated by computing/storing the attention matrix. 5 The underlying intuition stems from the observation that inside ํธยทํฒ โค each of theํquery vectors is compared to allํkey vectors via a dot-product and, after scaling, is normalized using the softmax operation. Therefore, the rows of the attention matrixํจโR ํรํ can be interpreted as indicating how much each key contributes to solving the rowโs query. The corresponding attention weights are then used to obtain, for each query, a weighted sum of the corresponding values. 3.3 Efficient Attention Efficient attention [40] reduces the runtime complexity toO(ํํ 2 ํ ) and the memory complexity toO(ํํ ํ + ํ 2 ํ ) by slightly rear- ranging the computations and at the cost of only providing an approximation of vanilla attention. 6 The underlying observation is that matrix multiplication is associative and that, up to the softmax operation, vanilla attention is just a scaled multiplication ofํธ,ํฒ โค andํฝ, where the computational bottleneck lies in the materializa- tion ofํธ ยท ํฒ โค โR ํรํ . The basic idea is to change the order of computation by first calculatingํฒ โค ยท ํฝ โR ํ ํ รํ ํฃ . On the way, the normalization via the softmax function needs to be adjusted. More precisely, efficient attention first calculates a column-wise softmax of the key features ํฒ and then computes ํชโ Softmax โ (ํฒ) โค ยท ํฝ โR ํ ํ รํ ํฃ ,(3) which is multiplied with a row-wise softmax-normalized version of the query features, so that the output ํถ is given by Attention eff (ํธ, ํฒ, ํฝ)โ ํถโ Softmax โ (ํธ)ยท ํช โR ํรํ ํฃ .(4) Figure 3 illustrates the underlying computations in form of a graph. Shen et al. [40]provide the following intuition for the modified attention computations: The modified keys, i.e.,Softmax โ (ํฒ) โ R ํรํ ํ , can be seen asํ ํ feature-maps, which indicate how a given feature is distributed across the image. They guide where to aggre- gate information from. The learned features could be of lower level (e.g., edge orientation) or higher level (e.g., human-made structures), depending on their location in the network. The values,ํฝ โR ํรํ ํฃ , containing the content of the patches, are passed on through the network (after being weighted). The product of the keys and values 5 We assume thatํ> ํ ํ , the typical case in practice. Then, the memory required for the projections is dominated by that of the attention matrix. 6 This small yet crucial change results in an immense practical performance gain, as usually ํ โซ ํ ํ . Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA (a) Local window [52](b) Shifted window [25] Figure 4: Windowed attention with windows of size 8ร8 pixels. Only pixels inside the same window can attend to each other. An example window is highlighted in cyan. is the โglobal contextโํช โR ํ ํ รํ ํฃ . It aggregates the content (val- ues) of the patches by the different features (key dimensions), thus producing a value (column) vectorํช ํ,ยท โR ํ ํฃ for each feature map 1โค ํ โค ํ ํ . For the query, every row ofSoftmax โ (ํธ) โR ํรํ ํ defines what information is important for the corresponding patch, i.e., how much each feature map contributes to this patch. Finally, the output for a patch is given by the corresponding weighted sum of the values per feature map from the global context ํช . 3.4 Windowed Attention Window-based attention mechanisms reduce the number of to- kens a token attends to by splitting the input into parts. The sim- plest variant is the local window attention (see Figure 4a), where attention is performed within non-overlapping windows of size ํ ร ํtokens [52],ํ โN. In the context of ViTs, the input ํฟ โR ํรํ ํ originates from an image of heightํป โฒ and width ํ โฒ of embedding vectors with dimensionํ ํ , i.e.ํฟ โR ํป โฒ รํ โฒ รํ ํ , whereํ= ํป โฒ ยทํ โฒ .ํฟis thenโafter zero-padding if necessaryโ divided into windows of sizeํร ํ, formally by reshapingํฟto ํฟ reshape โR (ํป โฒ ยทํ โฒ )/ํ 2 รํ 2 รํ ํ . Viewing the input as a batch of se- quences of lengthํ 2 , attention is performed on each window indi- vidually. This approach achieves linear memory and computational complexity in the number of patchesํ. Note that in windowed attention each pixel can only interact with pixels in the same win- dow. Liu et al. [25]address this by using shifted windows, giving the Swin Transformer its name. The windows alternate between a regular partitioning (see Figure 4a) and one shifted by half the window size (see Figure 4b), ensuring that information can flow across window boundaries. 3.5 Flash Attention Efficient attention and window-based attention achieve a runtime linear in the sequence length by approximating the attention mecha- nism. In contrast, FlashAttention [8], optimizes the exact calculation of attention on the hardware-side by tiling, which reduces the need of copying data between GPU high bandwidth memory and GPU on-chip SRAM. In an extension, FlashAttention-2 [7], the work par- titioning between different thread blocks and warps on the GPU is identified as a remaining bottleneck, which achieves a 2รspeedup compared to FlashAttention. Later extensions improve performance specifically for Hopper GPUs [39] and Blackwell GPUs [53]. 4 Model Architecture Our model architecture follows the encoder-decoder design, using an adapted version of the Mix Transformer (MiT) [50] as encoder and U-MixFormer [51] as decoder. Figure 5 shows the adapted architecture. We first revisit the original architectures and then detail our adaptations and the reasoning behind them. 4.1 Mix Transformer The MiT was introduced as the encoder of the SegFormer [50], where it was combined with a lightweight MLP decoder, and ach- ieved state-of-the-art performance for semantic segmentation in terms of efficiency and accuracy. The MiTโs design is inspired by the Pyramid Vision Transformer (PVT) [47], and thus also has four stages, which create multi-scale image features [50]. It utilizes the PVTโs spatial reduction attention [47], and performs a patch embedding at the beginning of every stage [50]. The MiT architecture is based on two main innovations: the Mix-FFNand an overlapped patch embedding (PatchEmbed) [50]. TheMix-FFNinserts a depth-wise convolution (DW-Conv) layer with a kernel size of 3 and padding and stride of 1 in between the linear layers of the FFN, which can be formalized as: ํ Mix-FFN = Linear(GELU(DW-Conv(Linear(ํฟ Mix-FFN ))))+ํฟ Mix-FFN While this adaptation goes against the initial idea of theFFNto post- process every token individually, it has the merit thatโas the tokens interact with their surrounding tokensโno positional encoding is required [50]. ThePatchEmbedlayer has an increased kernel size to include information from the surrounding patches and enhance local continuity. For the initial patch embedding, aConvlayer with a kernel size 7, stride 4, and padding 3 is used, which divides the image into patches of size 4ร4. For the subsequent ones, aConv layer with a kernel size 3, stride 2, and padding 1 is used, halving the image size with every recurring embedding. 4.2 U-MixFormer The U-MixFormer decoder also consists of four stages, each of which comprises a single Transformer block [51]. Instead of adding or concatenating the skipped encoder features and decoder features, they introduce the so-called mix-attention module to fuse them, replacing the self-attention mechanism [51]. Figure 6 shows the structure of a Transformer block with mix-attention. Self-attention projects the queries, keys and values from a single input, while cross-attention projects the queries from one input and the keys and values from another one [42]. For mix-attention, a single-scale feature is the input for the queries, while a second, multi-scale feature map is the input for the keys and values [51]. Note that the distinction between self-, cross-, and mix-attention is independent of the attention mechanism used. The query for the decoder block with index 4โ ํ +1 with ํ โ 1, . . .,4is obtained by linearly projecting the output of the corresponding encoder blockํ, i.e.,ํธ dec 4โํ+1 = Linear(ํ enc ํ ). The keys and values for the same decoder block are derived from the hi- erarchical feature map constructed from all encoder outputs, where the encoder featureํ enc ํ is replaced by its decoder feature of same dimensionsํ dec 4โํ+1 , if it is available. Thus, for the first decoder block,ํฒ dec 1 andํฝ dec 1 are derived from all the encoder features, SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. Decoder Encoder Block 4 (dec 4 ) ํ enc 1 ,ํ dec 3 ,ํ dec 2 ,ํ dec 1 ํ dec 4 Block 3 (dec 3 ) ํ enc 1 ,ํ enc 2 ,ํ dec 2 ,ํ dec 1 ํ dec 3 Up 2 รโ Block 2 (dec 2 ) ํ enc 1 ,ํ enc 2 ,ํ enc 3 ,ํ dec 1 ํ dec 2 Up 4 รโ Block 1 (dec 1 ) ํ enc 1 ,ํ enc 2 ,ํ enc 3 ,ํ enc 4 ํ dec 1 Up 8 รโ Concat FFN Stage 1 (enc 1 ) Stage 2 (enc 2 ) Stage 3 (enc 3 ) Stage 4 (enc 3 ) ํ enc 1 ํ enc 2 ํ enc 3 ํ enc 4 PatchEmbed LN MHA (eff) + LN Mix-FFN + 2ร Stage ํ Figure 5: Model architecture: The encoder consists of four stages, each of which starts with aPatchEmbedlayer followed by two Transformer blocks, which use efficient attention andMix-FFN. The decoder consists of four blocks, which perform efficient mix-attention. The encoder and decoder featuresโ shapes areํ enc ํ = ํ dec 4โํ+1 = ํป ํยท2 ํโ1 ร ํ ํยท2 ํโ1 ร2 ํโ1 ํ ํ (e.g., forํ=1 after encoder stage 1 and decoder block 4 we haveํปรํรํ ํ ). After the decoder featuresโ spatial dimension is aligned via bilinear upsampling, the features are concatenated and post-processed via a simple FFN to generate the prediction. Block 4 MHA ( eff ) + LN Mix-FFN + ํ dec 4 LN LN ํป 8 ร ํ 8 ร ร 4 ํ = 1 2 ํ โ 1 ํ ํ ConcatLinear 4 ยท ํป 8 ร ํ 8 ร 2 ํ โ 1 ํ ํ AvgPool ํ enc 1 ํป ร ํ ร ํ ํ Linear ํ enc 1 ,ํ dec 3 ,ํ dec 2 ,ํ dec 1 ํธ ํฒ ํฝ Figure 6: Architecture of the decoderโs block 4 of a model with patch sizeํ=1. The other blocks have a similar struc- ture, but operate on tensors of different shapes. The encoder feature is the query, while the encoder-decoder feature pyra- mid provides the keys and values. ํ enc 1 , . . ., ํ enc 4 , as no decoder features are available yet, while for the last decoder block,dec 4 , the three lower-dimensional encoder feature maps have already been created andํฒ dec 4 andํฝ dec 4 are derived fromํ enc 1 , ํ dec 3 , ํ dec 2 , ํ dec 1 . The transformation of the four features (of different resolution) into the keys and values works as follows: AnAvgPoollayer unifies the spatial dimensions by pooling all feature maps to the resolution of the lowest encoder block ( ํป 32 ร ํ 32 for the original setting ofํ=4 and ํป 8 ร ํ 8 when settingํ=1). Then aLinearlayer postprocesses the channel dimension and aConcatlayer concatenates all feature maps. Subsequently, the spatial dimensions are flattened, and the channel dimension split into two (for the individual projections for keys and values) [51]. After computing theMHA, the tokens are postprocessed via aMix-FFN, as introduced by the SegFormerโs MiT (see Section 4.1) [50, 51]. 4.3 Architectural Adaptations We adapt the original implementations in two minor yet crucial ways. In our experiments, we vary the patch sizeํ โ 1,2,4,8to study its impact. To achieve this, we implement the first overlapped patch embedding via aConvlayer with a kernel size of 2ํโ1, stride ofํ, and padding ofํ โ1 (subsequent patch embedding layers stay unchanged). A bigger patch size would result in an output with a proportionally smaller spatial resolution. To counteract this effect, we restore the original resolution by addingPatchExpand layers in front of the final linear layer, which creates the predic- tion. ThePatchExpandlayer was introduced by Cao et al. [2]to increase a tensorโs spatial resolution. It first expands the embedding dimension by a factor of two, then reshapes the tensor to twice the spatial dimension and half the original embedding dimension. This approach of parametric upsampling is common practice, and superior to the non-parametric bilinear upsampling of the predic- tions [2]. We further substitute all attention blocks across the model with efficient attention [40] by default and experiment with other attention variants described in Section 5.3.2. 5 Experiments Followingly, we empirically study pixel-level ViTs for canopy height prediction w.r.t. prediction quality and computational efficiency. We first describe two datasets used for training and evaluation, and then analyze the impact of patch size, model dimension, and Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA attention mechanism. Finally, we benchmark the resulting model against established dense prediction architectures. 5.1 Datasets We consider two datasets for experimental evaluation: TheEurope dataset is based on Pauls et al. [31]and theFrancedataset con- tains labels from the OpenCanopy dataset by Fogel et al. [15]. Both datasets have Sentinel-2 images as input, which consist of twelve channels and have a shape of 256ร256 pixels. We take median composites across multiple months to reduce the prevalence of clouds. All channels are clipped to a per-channel reflectance range before being min-max normalized to a range of zero to one [31]. TheEuropedataset consists of about 395 000 patches, which were split into a training (95%) and validation set (5%). It has a temporal coverage from 2019 to 2023. The GEDI labels follow a right-skewed distribution with the 5% quantile at 2.4 m, 7 the median at 3.7 m, and the 95% quantile at 28.2 m. This dataset does not have high-resolution ALS labels. TheFrancedataset combines the ALS labels provided by the OpenCanopy dataset [15] with the available GEDI labels from the same year and region. The ALS labels are available on a resolution of 1.5 m, which we resampled to 10 m by taking the maximum to match the resolution of our Sentinel-2 inputs. Using the large tiles from OpenCanopy, we use a sliding window approach to create non-overlapping cutouts of size 256ร256 pixels (to match the reso- lution of theEuropedataset). The GEDI labels have the 5% quantile at 2.5 m, the median at 10.4 m and the 95% quantile at 35.7 m, while the quantiles of the ALS labels are at 0.1 m, 10.9 m, and 30.2 m. We train and evaluate our models on theEuropedataset. For the final evaluation, we consider theFrancedataset, ensuring that none of its patches overlap with those used during training. The geographi- cal extent of theEuropedatasetโs training and validation patches, as well as theFrancedatasetโs evaluation patches, is shown in Figure 13 in the appendix. 5.2 Evaluation Metrics Apart from perceptual (i.e. qualitative) evaluation, we quantitatively evaluate the models on validation datasets along two dimensions: predictive accuracy and computational cost. We obtain robust esti- mates by training each model configuration five times with different seeds. To assess predictive accuracy, we use two metrics. First, we compute the mean absolute error over all pixels with a label greater than 5 m (ํํดํธ >5m โ [0,โ), lower is better). This threshold miti- gates the influence of grassland pixels, which dominate the dataset, focusing on pixels containing trees. Second, we report the coef- ficient of determination (ํ 2 โ (โโ,1], higher is better), which quantifies the variance reduction achieved by the model relative to a constant mean prediction. We also indicate the label type (GEDI or ALS) on which the metric is computed. Computational cost is assessed by measuring the runtime of a single training step on a mini-batch containing one image (forward and backward pass), along with its peak VRAM usage. For perceptual evaluation, we show predictions of the model with the median ํํดํธ >5m (ALS). 7 Due to the measurement process and preprocessing to obtain the labels, GEDI labels indicate a height of approximately 2.5 m for grassland. 12244896192384 4.5 5 5.5 6 ํ ํ ํํดํธ > 5 m (GEDI) (m) 12244896192384 0.55 0.6 0.65 ํ ํ ํ 2 (GEDI) ํ=1ํ=2ํ=4ํ=8 Figure 7: Performance comparison of models with varying patch size (ํ) and model dimension (ํ ํ ) on the validation subset of theEuropedataset (GEDI labels). Shown are the mean and standard deviation of five training runs per config. 12244896192384 0 25 50 75 100 125 150 175 ํ ํ Training step (ms) 12244896192384 1/32 1/16 1/8 1/4 1/2 1 2 4 8 16 ํ ํ Peak VRAM (GiB) ํ=1ํ=2ํ=4ํ=8 Figure 8: Efficiency comparison of models with varying patch size (ํ) and model dimension (ํ ํ ). Shown is the mean of 1 000 training steps on single-image batches across five runs. The standard deviation is imperceptible, due to its small scale. 5.3 Results We first investigate the impact of varying patch sizes and model dimensions, fixing the attention mechanism to efficient attention. Then, we explore the impact of different attention mechanisms, fixing the patch size and model dimension to the optimal values determined by the first experiment. Finally, we compare the best- performing configuration with established baseline architectures commonly used for canopy height prediction, depth estimation, and semantic segmentation more broadly. All models, except for the baselines, follow the architecture described in Section 4. The number of attention heads doubles in every stage from 1 to 8. The decoder dimension is fixed to 96. 5.3.1Impact of Patch Size & Model Dimension. We independently vary the patch sizeํ โ 1,2,4,8and model dimensionํ ํ โ 12,24,48,96,192,384 8 , resulting in 24 model configurations. 9 All 8 We start withํ ํ equal to the number of input channels and double it in each subsequent configuration. Models withํ ํ = 768 went out-of-memory regardless of the patch size. 9 Training for the model with ํ= 1 andํ ํ = 384 took too long. SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. 12244896192384 6 6.5 7 7.5 ํ ํ ํํดํธ > 5 m (GEDI) (m) 12244896192384 4 4.5 5 5.5 ํ ํ ํํดํธ > 5 m (ALS) (m) 12244896192384 25 30 35 ํ ํ Relative Improvement (%) ํ=1 ํ=2 ํ=4 ํ=8 Figure 9: Comparison ofํํดํธ >5m computed on GEDI labels (left) and ALS labels (center) of theFrancedataset across models with varying patch size (ํ) and model dimension (ํ ํ ). (Right:) Relative improvement when computing theํํดํธ >5m on ALS labels instead of GEDI labels. models were trained for six epochs, which was enough for the mod- els to converge (note that one epoch is based on 395 000 instances), while not requiring too many resources. No form of regularization was used, as no overfitting occurred for any of the models, presum- ably due to the sparsity of the supervision signal and the size and diversity of the dataset. We employed gradient checkpointing after every stage to reduce the VRAM requirements of our models [4,16]. Models with larger patch sizes usePatchExpandlayers to increase the resolution to the original input resolution. 10 Quantitative Evaluation: Figure 7 shows the two regression met- rics,ํํดํธ >5m andํ 2 , in dependence of the model dimension and patch size, averaged over five training runs. Both metrics follow the same trend: For a fixed model dimension, a smaller patch size results in stronger performance, which is in line with the patchifica- tion scaling laws [45]. The larger the model dimension, the weaker the effect. At the same time, for a fixed patch size, a larger model dimension generally improves performance, but with diminishing returns. Qualitative Evaluation: Figures 1 and 18 (in the appendix) show predictions of models with varying patch size on exemplary vali- dation patches of theFrancedataset. In general, the differences in perceptual quality are most pronounced for image regions with fine details, like narrow streams or paths through a forest. For models with a patch sizeํ>1, unpleasant grid-like artifacts arise, with the size of a grid โcellโ equal to the patch size. To quantify the gridsโ intensity, inspired by Wu and Yuen[49]and Wang et al. [48], we compute the differences of two adjacent pixelsโ residuals (based on the ALS labels) within a cell of lengthํ(referred to asํด ํ ) and across a cellโs borders (ํต ํ ). Their ratio (ํต ํ /ํด ํ ) is shown in Fig- ure 15 in the appendix, indicating that larger patch sizes lead to stronger grid artifacts. This is plausible, as more upsampling steps need to be performed in the decoder. Runtime and Memory Analysis: However, the performance gains come at the price of reduced efficiency, as shown in Figure 8. Trans- formers are resource-intensiveโeven when using an attention vari- ant with linear complexity, like efficient attention. Training time for a single instance and required memory scale roughly by a factor of 1/ํ 2 , i.e., halving the patch size roughly quadruples the training 10 This approach produced better results across the board than using bilinear upsam- pling in the prediction space. time. The training time as well as the modelsโ number of parameters grow quadratically in the model dimension, due to the nature of fully connected linear layers. The number of parameters is inde- pendent of the patch size. GEDI vs. ALS Ground Truth: As the GEDI labels are inherently noisy, we also evaluated our models on the ALS labels given in theFrancedataset. The results of this evaluation are shown in Figure 9. Note that here theํํดํธ >5m computed on the GEDI la- bels is substantially worse than the one computed on theEurope dataset, as the model has not seen any inputs close to theFrance dataset, which was only used for testing, not for training. Across all models, theํํดํธ >5m is significantly better when computed on ALS labels instead of GEDI labels. The relative improvement ranges from 25.1% to 37.4%. Interestingly, the models which performed better initially (i.e., models with a smaller patch size or larger model dimension) have an even bigger relative improvement when evalu- ated on the higher-quality ALS labels. This implies that models with larger patch sizes get evaluated favorably by GEDI data, misrepre- senting the actual performance difference. An explanation is that due to the label noise present with GEDI labels, models producing smoother predictions have an advantage. It can also be observed that for smaller patch sizes, models with comparably smaller model dimensions can perform better. The best models have the configu- rations (ํ=1,ํ ํ =96), (ํ=2,ํ ํ =192), (ํ=4,ํ ํ =192), and (ํ=8,ํ ํ =192). That the optimal model dimension is smaller, when choosing a smaller patch size, can be explained by viewing patchification as a compression step [45]: Smaller patch sizes mean less information has to be compressed into a single token, so the dimensionality of each token can be reduced. Analyzing the error for various bins of label heights (see Figure 14 in the appendix), we observe that smaller patch sizes result in a lower variance of errors on ALS labels for smaller trees, and a smaller absolute error for taller trees. Figure 10 shows the joint distribution of the labels (one with GEDI, the other with ALS labels) and the model predictions as heatmaps, and the difference between the two heatmaps. The dif- ference for labels smaller than 10 m arises due to a particularity of GEDI labels: Structures shorter than 2.5 m are generally not reliably detected, resulting in labels of around 2.5 m, even when there is no vegetation. For labels greater than 10 m, it can be clearly seen that the density around the perfect-fit line (prediction = label) is Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA ํํดํธ >5m = 6.06 m ํ 2 = 0.426 01020304050 0 10 20 30 40 50 GEDI Label (m) Prediction (m) ํํดํธ >5m = 3.84 m ํ 2 = 0.751 01020304050 ALS Label (m) 10 0 10 1 10 2 10 3 10 4 Number of pixels 01020304050 0 10 20 30 40 50 ALS Label (m) Prediction (m) โค โ200 โ150 โ100 โ50 0 50 100 150 โฅ 200 Difference: ALS โ GEDI Figure 10: (Left:) Reference canopy height (GEDI/ALS label) vs. predictions by the model withํ=1,ํ ํ =96. Both heatmaps are computed only for pixels where both labels exist. (Right:) Difference of the heatmaps, where red color indicates a higher density of the ALS heatmap and blue color a higher density of the GEDI heatmap. 708090170180 3.8 3.9 4 4.1 4.2 4.3 4.4 4.5 4.6 Training step (ms) ํํดํธ > 5 m (ALS) (m) Encoder effflash swin Decoder effflash swin Figure 11: Trade-off between performance (ํํดํธ >5m com- puted on ALS labels) and efficiency (training time for a single- image batch). Compared are nine models with varying atten- tion mechanisms in the encoder (indicated by marker color) and decoder (indicated by marker shape): efficient (eff), flash (flash), and shifted window (swin). Shown are the mean and standard deviation of five training runs per configuration. consistently higher for the ALS-vs-prediction heatmap. Further away from the perfect-fit line, the GEDI-vs-prediction heatmap has a greater density. This shows that the modelโs predictions match the ALS labels better than the GEDI labels (which it was trained on) across almost all tree heights. The observed effect is less pro- nounced for models with larger patch sizes (see Figure 17 in the appendix). Figure 16 in the appendix shows that GEDI and ALS labels have a surprisingly small correlation. 5.3.2 Impact of Attention Mechanism. Next, we fix the hyperpa- rameters toํ=1 andํ ํ =96 to investigate the impact of three well-known attention mechanisms in isolation: efficient attention (eff), flash attention (flash), and shifted window attention (swin), which we vary individually for the encoder and decoder. The win- dow size ofswinattention is set to 7ร7 tokens. The other two variants do not require setting any hyperparameters other than the ones shared by all attention mechanisms, like the number of attention heads and attention dropout. Figure 11 shows the performance-efficiency trade-off for mod- els with varying attention mechanisms, where better models are positioned towards the bottom left corner. We observe significant performance gains when using global attention (efforflash) in- stead of local attention (swin) in the decoder and/or encoder. The model with the best performance, while still being comparatively efficient, uses efficient attention in the encoder and decoderโas was also used in the previous experimentโand has an averageํํดํธ >5m (computed on ALS labels) of 3.84 m and a runtime of about 78 ms. The training time of models withswinandeffattention is very similar, whileflashtakes more than twice as long. In general, the impact of the encoderโs attention mechanism is significantly bigger than the decoderโs, as the keys and values are pooled to the lowest resolution in the decoder block (see Figure 6). All architectures have a similar peak VRAM: Training the model with a batch size of 1 requires about 2.66GiBfor the architectures usingeffandflash attention and about 2.81GiBfor the architecture usingswinatten- tion throughout the model. In contrast, vanilla attention already produces an out-of-memory error for inputs of this size. 11 For a qualitative evaluation, predictions on an exemplary valida- tion patch can be seen in Figure 19 in the appendix. 5.3.3 Comparison with Baseline Models. We compare the perfor- mance of the best configuration of the previous experiments 12 with several models commonly used for dense prediction tasks. The U- Net architecture [35] consists of a symmetric convolutional encoder and decoder and has been used to generate a global canopy height map [30] and predict canopy height and above-ground biomass across France [37]. The Residual U-Net (ResUnet) has been used as a baseline by Tolan et al. [41], and their final model is based on the Dense Prediction Transformer, which has a default patch size of ํ=16 [34]. The SegFormer consists of a Mix Transformer encoder 11 Training was performed on NVIDIAโs RTX 4090 GPUs, which have 24GiBof VRAM. 12 I.e., when setting patch size ํ= 1 and model dimensionํ ํ = 96. SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. Sentinel-2 RGBALS Label Ours (ํ= 1,ํ ํ = 96) ResUnetU-NetDPTSwinUnetSegFormer 30m 0m Figure 12: Qualitative comparison of predictions made by various models. The UNet and ResUnet are strong CNN baselines, while ViT-based architectures with a high patch size produce predictions which are blurry or show a grid pattern. Table 1: Comparison of well-known dense prediction base- lines using three predictive metrics, as well as the runtime per training step (ts) in ms and peak VRAM usage in MiB. The modelOursusesํ=1,ํ ํ =96 and is averaged over its five seed repeats; the baselines were trained once. Modelํํดํธ >5m โ ํ 2 โ ํํดํธ >5m โtsโVRAMโ (GEDI)(GEDI)(ALS) SegFormer5.020.6414.6252.2525 SwinUnet5.170.6294.6645.3551 DPT5.350.6155.0418.12069 U-Net4.590.6724.148.2851 ResUnet4.560.6744.197.4517 Ours4.310.6873.8477.72722 with patch sizeํ=4 and a lightweight Multilayer Perceptron decoder [50]. Based on the Swin Transformer [25], the Swin-Unet is a symmetric encoder-decoder architecture, and was adapted by Pauls et al. [29]to produce a multi-year global canopy height map. Table 1 shows the resulting metrics and Figure 12 the predic- tions on an example patch. While our pixel-level Transformer model achieves the strongest predictive performance (measured byํํดํธ >5m andํ 2 ), the results also show that U-Net-based archi- tectures remain highly competitive: They offer substantially faster and more resource-efficient training while achieving performance superior to Transformer models with larger patch sizes. Meanwhile, our experiments also demonstrate that carefully configured Trans- former architectures can surpass other models, suggesting that the additional computational cost can translate into meaningful gains in prediction quality. 5.3.4Discussion. Our results should be interpreted as relative com- parisons within a controlled experimental setting. As our focus lies on comparing architectural choices rather than maximizing canopy height estimation performance, the reported metrics should not be directly compared with those of existing canopy height products. The architectural choices considered in this work only represent a small subset of the design space of Transformer-based models. While we focused on patch size, model dimension, and the attention mechanism, other factors such as model depth and decoder design may further improve performance. In particular, a stronger decoder might be able to reconstruct the spatial dimensions lost in the patchification step. If the training is highly resource-constrained and a small perfor- mance loss is acceptable, Transformer-based models with a patch size of ํ= 2 or CNN-based models remain strong alternatives. 6 Conclusion We conducted experiments on adapting important hyperparameters of Vision Transformer-based models: patch size, model dimension and the attention mechanism used. The results indicate that, while pixel-level Transformers are remarkably resource-hungry, choosing an appropriate model dimension and using an efficient attention approximation gives rise to models with a very strong performance. Architectures operating at a pixel level have the further benefit of being intuitive (as the image is not split into arbitrary chunks) and simpler (as the model predictions inherently have the same resolution as the input). For the studied task of canopy height prediction, even though the models are trained on noisy labels derived from GEDI data, they indeed learn fine details, which can be evaluated using higher-quality ALS labels that are only available for selected regions. Only evaluating the models on GEDI labels paints an incomplete picture of model performance. Note that, while the benchmark comparison is based on the task of canopy height prediction, we believe that similar observations can be made for other remote sensing tasks such as biomass estimation, soil moisture mapping, or crop yield forecasting. Through our experiments, we hope to motivate other researchers to explore efficient pixel-level Transformers for dense prediction tasks, especially when given medium- or low-resolution satellite imagery as input. Acknowledgments This work was supported via the AI4Forest project, which is funded by the German Federal Ministry of Research, Technology and Space (BMFTR; grant number 01IS23025A) and the French National Re- search Agency (ANR; grant number ANR-22-FAI1-0002). Calcu- lations for this publication were performed on the HPC cluster PALMA I of the University of Mรผnster, subsidized by the DFG (INST 211/667-1). Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA References [1]Gordon B. Bonan. 2008. Forests and Climate Change: Forcings, Feedbacks, and the Climate Benefits of Forests. Science 320, 5882 (2008), 1444โ1449. doi:10.1126/ science.1155121 [2]Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. 2022. Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In Computer Vision โ ECCV 2022 Workshops. Springer-Verlag, Berlin, Heidelberg, 205โ218. doi:10.1007/978-3-031-25066-8_9 [3]Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. 2020. Generative Pretraining From Pixels. In International Con- ference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). PMLR, 1691โ1703. https://proceedings.mlr.press/v119/chen20s.html [4]Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training Deep Nets with Sublinear Memory Cost. arXiv:1604.06174 https://arxiv.org/abs/1604. 06174 [5]Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J Colwell, and Adrian Weller. 2021. Rethinking Attention with Performers. In International Conference on Learning Representations. https://openreview.net/forum?id=Ua6zuk0WRH [6]Copernicus Sentinel-2 (processed by ESA). 2021. MSI Level-2A BOA Reflectance Product. Collection 1. doi:10.5270/S2_-znk9xsj [7]Tri Dao. 2024. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. In International Conference on Learning Representations. https: //openreview.net/forum?id=mZn2Xyh9Ec [8]Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Rรฉ. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. In Advances in Neural Information Processing Systems, Vol. 35. Curran Associates, Inc., 16344โ16359. doi:10.52202/068431-1189 [9] Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. 2022. CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPRโ22). 12114โ12124. doi:10.1109/ CVPR52688.2022.01181 [10]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In Interna- tional Conference on Learning Representations. https://openreview.net/forum?id= YicbFdNTTy [11] Matthias Drusch, Umberto del Bello, Stefane Carlier, Olivier Colin, Valรฉrie Fer- nandez, Ferran Gascรณn, Bianca Hoersch, Claudia Isola, Paolo Laberinti, Philippe Martimort, Aimรฉ Meygret, Franรงois Spoto, Omar Sy, Franco Marchese, and Pier G. Bargellini. 2012. Sentinel-2: ESAโs Optical High-Resolution Mission for GMES Operational Services. Remote Sensing of Environment 120 (2012), 25โ36. doi:10.1016/j.rse.2011.11.026 [12]Ralph Dubayah, James Bryan Blair, Scott Goetz, Lola Fatoyinbo, Matthew Hansen, Sean Healey, Michelle Hofton, George Hurtt, James Kellner, Scott Luthcke, John Armston, Hao Tang, Laura Duncanson, Steven Hancock, Patrick Jantz, Suzanne Marselis, Paul L. Patterson, Wenlu Qi, and Carlos Silva. 2020. The Global Ecosystem Dynamics Investigation: High-resolution laser ranging of the Earthโs forests and topography. Science of Remote Sensing 1 (2020), 100002. doi:10.1016/j.srs.2020.100002 [13] Earth Resources Observation and Science (EROS) Center. 2020. Landsat 8-9 Operational Land Imager / Thermal Infrared Sensor Level-2, Collection 2. doi:10. 5066/P9OGBGM6 [14]Ibrahim Fayad, Philippe Ciais, Martin Schwartz, Jean-Pierre Wigneron, Nicolas Baghdadi, Aurรฉlien de Truchis, Alexandre dโAspremont, Frederic Frappart, Sassan Saatchi, Ewan Sean, Agnes Pellissier-Tanon, and Hassan Bazzi. 2024. Hy-TeC: a hybrid vision transformer model for high-resolution and large-scale mapping of canopy height. Remote Sensing of Environment 302 (2024), 113945. doi:10.1016/j. rse.2023.113945 [15] Fajwel Fogel, Yohann Perron, Nikola Besic, Laurent Saint-Andrรฉ, Agnรจs Pellissier-Tanon, Martin Schwartz, Thomas Boudras, Ibrahim Fayad, Alexandre dโAspremont, Loic Landrieu, and Philippe Ciais. 2025. Open-Canopy: Towards Very High Resolution Forest Monitoring. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPRโ25). 1395โ1406. doi:10.1109/CVPR52734. 2025.00138 [16]Andreas Griewank and Andrea Walther. 2000. Algorithm 799: Revolve: An Implementation of Checkpointing for the Reverse or Adjoint Mode of Com- putational Differentiation. ACM Trans. Math. Software 26, 1 (2000), 19โ45. doi:10.1145/347837.347846 [17]Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans. 2019. Axial Attention in Multidimensional Transformers. arXiv:1912.12180 https://arxiv.org/ abs/1912.12180 [18]Intergovernmental Panel on Climate Change. 2006. Agriculture, Forestry and Other Land Use. In 2006 IPCC Guidelines for National Greenhouse Gas Inventories, Harald Aalde, Patrick Gonzalez, Michael Gytarsky, Thelma Krug, Werner A. Kurz, Stephen Ogle, John Raison, Dieter Schoene, and N. H. Ravindranath (Eds.). Vol. 4. Institute for Global Environmental Strategies. https://w.ipcc-nggip.iges.or. jp/public/2006gl/vol4.html [19]Patrick Kacic, Frank Thonfeld, Ursula Gessner, and Claudia Kuenzer. 2023. Forest Structure Characterization in Germany: Novel Products and Analysis Based on GEDI, Sentinel-1 and Sentinel-2 Data. Remote Sensing 15, 8, Article 1969 (2023). doi:10.3390/rs15081969 [20] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models. arXiv:2001.08361 https://arxiv.org/ abs/2001.08361 [21] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franรงois Fleuret. 2020. Transformers are RNNs: Fast Autoregressive Transformers with Linear At- tention. In International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). PMLR, 5156โ5165. https://proceedings.mlr.press/ v119/katharopoulos20a.html [22] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The Efficient Transformer. In International Conference on Learning Representations. https: //openreview.net/forum?id=rkgNKkHtvB [23]Nico Lang, Walter Jetz, Konrad Schindler, and Jan Dirk Wegner. 2023. A High- Resolution Canopy Height Model of the Earth. Nature Ecology & Evolution 7, 11 (2023), 1778โ1789. doi:10.1038/s41559-023-02206-6 [24]Siyu Liu, Martin Brandt, Thomas Nord-Larsen, Jerome Chave, Florian Reiner, Nico Lang, Xiaoye Tong, Philippe Ciais, Christian Igel, Adrian Pascual, Juan Guerra- Hernandez, Sizhuo Li, Maurice Mugabowindekwe, Sassan Saatchi, Yuemin Yue, Zhengchao Chen, and Rasmus Fensholt. 2023. The overlooked contribution of trees outside forests to tree cover and woody biomass across Europe. Science Advances 9, 37 (2023), eadh4097. doi:10.1126/sciadv.adh4097 [25] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In IEEE/CVF International Conference on Computer Vision (ICCVโ21). 9992โ10002. doi:10.1109/ICCV48922.2021.00986 [26]Petr Lukeลก, Viktor Myroniuk, Andrii Shamrai, Viktor Melnichenko, Martin Schwartz, and Jan Pauls. 2026. Integrating Global Canopy Height Models with Satellite Data for Improved Forest Inventory in Ukraine. Agricultural and Forest Meteorology 388 (2026), 111265. doi:10.1016/j.agrformet.2026.111265 [27]Thorsten Markus, Tom Neumann, Anthony Martino, Waleed Abdalati, Kelly Brunt, Beata Csatho, Sinead Farrell, Helen Fricker, Alex Gardner, David Harding, Michael Jasinski, Ron Kwok, Lori Magruder, Dan Lubin, Scott Luthcke, James Morison, Ross Nelson, Amy Neuenschwander, Stephen Palm, Sorin Popescu, CK Shum, Bob E. Schutz, Benjamin Smith, Yuekui Yang, and Jay Zwally. 2017. The Ice, Cloud, and land Elevation Satellite-2 (ICESat-2): Science requirements, concept, and implementation. Remote Sensing of Environment 190 (2017), 260โ273. doi:10.1016/j.rse.2016.12.029 [28] Duy Kien Nguyen, Mido Assran, Unnat Jain, Martin R. Oswald, Cees G. M. Snoek, and Xinlei Chen. 2025. An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels. In International Conference on Learning Representations. https://openreview.net/forum?id=tjNf0L8QjR [29]Jan Pauls, Karsten Schrรถdter, Sven Ligensa, Martin Schwartz, Berkant Turan, Max Zimmer, Sassan Saatchi, Sebastian Pokutta, Philippe Ciais, and Fabian Gieseke. 2026. ECHOSAT: Estimating Canopy Height Over Space And Time. arXiv:2602.21421 https://arxiv.org/abs/2602.21421 [30] Jan Pauls, Max Zimmer, Una M. Kelly, Martin Schwartz, Sassan Saatchi, Philippe Ciais, Sebastian Pokutta, Martin Brandt, and Fabian Gieseke. 2024. Estimat- ing Canopy Height at Scale. In International Conference on Machine Learn- ing (Proceedings of Machine Learning Research, Vol. 235). PMLR, 39972โ39988. https://proceedings.mlr.press/v235/pauls24a.html [31]Jan Pauls, Max Zimmer, Berkant Turan, Sassan Saatchi, Philippe Ciais, Sebastian Pokutta, and Fabian Gieseke. 2025. Capturing Temporal Dynamics in Large- Scale Canopy Tree Height Estimation. In International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267). PMLR, 48422โ48438. https://proceedings.mlr.press/v267/pauls25a.html [32]Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah Smith, and Lingpeng Kong. 2021. Random Feature Attention. In International Conference on Learning Representations. https://openreview.net/forum?id=QtTKTdVrFBB [33]Peter Potapov, Xinyuan Li, Andres Hernandez-Serna, Alexandra Tyukavina, Matthew C. Hansen, Anil Kommareddy, Amy Pickens, Svetlana Turubanova, Hao Tang, Carlos Edibaldo Silva, John Armston, Ralph Dubayah, J. Bryan Blair, and Michelle Hofton. 2021. Mapping Global Forest Canopy Height through Integration of GEDI and Landsat Data. Remote Sensing of Environment 253 (2021), 112165. doi:10.1016/j.rse.2020.112165 [34] Renรฉ Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. 2021. Vision Transformers for Dense Prediction. In IEEE/CVF International Conference on Computer Vision (ICCVโ21). 12159โ12168. doi:10.1109/ICCV48922.2021.01196 [35]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer International Publishing, SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. Cham, 234โ241. doi:10.1007/978-3-319-24574-4_28 [36]Aurko Roy, Mohammad Saffar, Ashish Vaswani, and David Grangier. 2021. Effi- cient Content-Based Sparse Attention with Routing Transformers. Transactions of the Association for Computational Linguistics 9 (2021), 53โ68. doi:10.1162/tacl_ a_00353 [37]Martin Schwartz, Philippe Ciais, Aurรฉlien de Truchis, Jรฉrรดme Chave, Catherine Ottlรฉ, Cedric Vega, Jean-Pierre Wigneron, Manuel Nicolas, Sami Jouaber, Siyu Liu, Martin Brandt, and Ibrahim Fayad. 2023. FORMS: Forest Multiple Source height, wood volume, and biomass maps in France at 10 to 30m resolution based on Sentinel-1, Sentinel-2, and Global Ecosystem Dynamics Investigation (GEDI) data with a deep learning approach. Earth System Science Data 15, 11 (2023), 4927โ4945. doi:10.5194/essd-15-4927-2023 [38] Martin Schwartz, Philippe Ciais, Catherine Ottlรฉ, Aurelien De Truchis, Cedric Vega, Ibrahim Fayad, Martin Brandt, Rasmus Fensholt, Nicolas Baghdadi, Franรงois Morneau, David Morin, Dominique Guyon, Sylvia Dayau, and Jean-Pierre Wigneron. 2024. High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach. International Journal of Applied Earth Observation and Geoinformation 128 (2024), 103711. doi:10.1016/j.jag.2024.103711 [39]Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. 2024. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. In Advances in Neural Information Processing Systems, Vol. 37. Curran Associates, Inc., 68658โ68685. doi:10.52202/079017-2193 [40]Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. 2021. Efficient Attention: Attention With Linear Complexities. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACVโ21). 3531โ3539. doi:10.1109/WACV48630.2021.00357 [41] Jamie Tolan, Hung-I Yang, Benjamin Nosarzewski, Guillaume Couairon, Huy V. Vo, John Brandt, Justine Spore, Sayantan Majumdar, Daniel Haziza, Janaki Va- maraju, Theo Moutakanni, Piotr Bojanowski, Tracy Johns, Brian White, Tobias Tiecke, and Camille Couprie. 2024. Very High Resolution Canopy Height Maps from RGB Imagery Using Self-Supervised Vision Transformer and Convolutional Decoder Trained on Aerial Lidar. Remote Sensing of Environment 300 (2024), 113888. doi:10.1016/j.rse.2023.113888 [42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc. https://proceedings.neurips.c/paper_files/paper/2017/hash/ 3f5e243547dee91fbd053c1c4a845a-Abstract.html [43]Apoorv Vyas, Angelos Katharopoulos, and Franรงois Fleuret. 2020. Fast Trans- formers with Clustered Attention. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 21665โ21674.https://proceedings. neurips.c/paper/2020/hash/f6a8d1c954c8506aadc764c32b895e-Abstract.html [44] Fabien H Wagner, Ricardo Dalagnol, Griffin Carter, Mayumi CM Hirye, Shiv- raj Gill, Le Bienfaiteur Sagang Takougoum, Samuel Favrichon, Michael Keller, Jean PHB Ometto, Lorena Alves, Cynthia Creze, Stephanie P George-Chacon, Shuang Li, Zhihua Liu, Adugna Mullissa, Yan Yang, Erone G Santos, Sarah R Worden, Martin Brandt, Philippe Ciais, Stephen C Hagen, and Sassan Saatchi. 2025. High Resolution Tree Height Mapping of the Amazon Forest using Planet NICFI Images and LiDAR-Informed U-Net Model. arXiv:2501.10600 https://arxiv. org/abs/2501.10600 [45]Feng Wang, Yaodong Yu, Wei Shao, Yuyin Zhou, Alan Yuille, and Cihang Xie. 2025. Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More. In International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267). PMLR, 65278โ65290. https://proceedings.mlr.press/ v267/wang25ed.html [46]Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-Attention with Linear Complexity. arXiv:2006.04768 https://arxiv. org/abs/2006.04768 [47]Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions. In IEEE/CVF International Conference on Computer Vision (ICCVโ21). 548โ558. doi:10.1109/ICCV48922.2021. 00061 [48]Zhou Wang, Hamid R. Sheikh, and Alan C. Bovik. 2002. No-Reference Percep- tual Quality Assessment of JPEG Compressed Images. In Proceedings. Interna- tional Conference on Image Processing, Vol. 1. Iโ477โIโ480. doi:10.1109/ICIP.2002. 1038064 [49] Hong Ren Wu and Michael Yuen. 1997. A Generalized Block-Edge Impairment Metric for Video Coding. IEEE Signal Processing Letters 4, 11 (1997), 317โ320. doi:10.1109/97.641398 [50]Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo. 2021. SegFormer: Simple and Efficient Design for Semantic Segmenta- tion with Transformers. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 12077โ12090. https://proceedings.neurips.c/ paper_files/paper/2021/hash/64f1f27bf1b4ec22924fd0acb550c235-Abstract.html [51]Seul-Ki Yeom and Julian Von Klitzing. 2025. U-MixFormer: UNet-Like Trans- former with Mix-Attention for Efficient Semantic Segmentation. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACVโ25). 1โ10. doi:10. 1109/WACV61041.2025.00750 [52]Yuhui Yuan, Rao Fu, Lang Huang, Weihong Lin, Chao Zhang, Xilin Chen, and Jingdong Wang. 2021. HRFormer: High-Resolution Vision Transformer for Dense Prediction. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 7281โ7293. https://proceedings.neurips.c/paper_files/paper/ 2021/hash/3bbfdde8842a5c44a0323518eec97cbe-Abstract.html [53]Ted Zadouri, Markus Hoehnerbach, Jay Shah, Vijay Thakkar, and Tri Dao. 2026. FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmet- ric Hardware Scaling. In Proceedings of Machine Learning and Systems (ML- Sysโ26). 912โ926. https://proceedings.mlsys.org/paper_files/paper/2026/hash/ ae8b0b5838ba510daff1198474e7b984-Abstract-Conference.html [54] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip H.S. Torr, and Li Zhang. 2021. Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPRโ21). 6877โ6886. doi:10.1109/CVPR46437.2021.00681 Pixel-Level TransformersSIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USA A Supporting Figures Figure 13: Geographical distribution of training patches (light grey) and validation patches (dark grey) of theEuropedataset, and validation patches of the France dataset (black). ํ= 1,ํ ํ = 96ํ= 2,ํ ํ = 192ํ= 4,ํ ํ = 192ํ= 8,ํ ํ = 192 5-10m10-15m15-20m20-25m25-30m30-35m35-40m40-45m45-50m โ30 โ20 โ10 0 10 GEDI Label Bins Error (m) 5-10m10-15m15-20m20-25m25-30m30-35m35-40m40-45m45-50m โ30 โ20 โ10 0 10 ALS Label Bins Figure 14: Comparison of errors across labels of different height for GEDI labels on theEuropedataset (left) and ALS labels on the France dataset (right). All models underestimate taller treesโ height, but models with smaller patch sizes less severely. 12244896192384 1.00 1.05 1.10 1.15 ํ ํ Ratio ํต ํ ํด ํ ํ=1 ํ=2 ํ=4 ํ=8 Figure 15: Ratio ํต ํ ํด ํ of how much larger the differences of residuals (computed on the Francedatasetโs ALS labels) are across borders of a grid cell of sizeํ(ํต ํ ) compared to within a grid cell (ํด ํ ). Deviating from this, we compute ํต 2 ํด 2 for the model withํ=1. ํ 2 = 0.436 01020304050 0 10 20 30 40 50 ALS Label (m) GEDI Label (m) 10 0 10 1 10 2 10 3 10 4 Number of pixels Figure 16: Correlation of ALS and GEDI labels on theFrancedataset (computed where both labels are available). Table 2: Heightํป(equal to the widthํ) of the feature maps for various patch sizes and stages, given byํป/(ํยท2 ํโ1 )รํ/(ํยท 2 ํโ1 ) , for an input of ํป=ํ= 256. The channels double after every stage. Patch size ํ Stage ํ 12 4 8 1256 128 64 32 2128 64 32 16 364 32 16 8 432 16 8 4 SIGSPATIAL โ26, November 03โ06, 2026, Riverside, CA, USALigensa et al. ํํดํธ >5m = 6.75 m ํ 2 = 0.354 01020304050 0 10 20 30 40 50 GEDI Label (m) Prediction (m) ํํดํธ >5m = 4.84 m ํ 2 = 0.663 01020304050 ALS Label (m) 10 0 10 1 10 2 10 3 10 4 Number of pixels 01020304050 0 10 20 30 40 50 ALS Label (m) Prediction (m) โค โ200 โ150 โ100 โ50 0 50 100 150 โฅ 200 Difference: ALS โ GEDI Figure 17: Predicted vs. reference canopy height for theํ=8,ํ ํ =192 model, over pixels where both labels exist. Evidently, the model has not (yet) learned to predict trees taller than 30 m. Sentinel-2 RGBALS Label ํ= 1, ํ ํ = 96ํ= 2, ํ ํ = 192ํ= 4, ํ ํ = 192ํ= 8, ํ ํ = 192 30m 0m Figure 18: Qualitative comparison of models with different patch sizes (ํ) and model dimensions (ํ ํ ). Models with smaller patch sizes handle the vertical line of missing input data better and produce more accurate predictions. Sentinel-2 RGBALS Label effflash swin 30m 0m Figure 19: Qualitative comparison of models with different attention mechanisms (same in encoder and decoder).