Paper deep dive
RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance
Xianbao Hou, Yonghao He, Zeyd Boukhers, John See, Hu Su, Wei Sui, Cong Yang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/22/2026, 5:23:11 AM
Summary
RSGen is a plug-and-play framework designed to enhance layout-driven remote sensing (RS) image generation by providing fine-grained, pixel-level control. It utilizes an Edge2Edge module to generate diverse edge priors and an L2I FGControl module that employs frequency-gated structural decoupling and spatial gating to ensure generated instances strictly adhere to bounding box constraints, significantly improving downstream object detection performance.
Entities (6)
Relation Signals (4)
RSGen → comprises → Edge2Edge
confidence 100% · RSGen consists of two modules: Edge2Edge and L2I FGControl
RSGen → comprises → L2I FGControl
confidence 100% · RSGen consists of two modules: Edge2Edge and L2I FGControl
RSGen → enhances → CC-Diff
confidence 95% · Extensive experiments across three baseline models demonstrate that RSGen significantly boosts the capabilities of existing L2I models.
RSGen → improvesperformanceon → DOTA
confidence 95% · The improvements are even more significant on the DOTA-v1.0 [26] dataset
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Diffusion models have significantly mitigated the impact of annotated data scarcity in remote sensing (RS). Although recent approaches have successfully harnessed these models to enable diverse and controllable Layout-to-Image (L2I) synthesis, they still suffer from limited fine-grained control and fail to strictly adhere to bounding box constraints. To address these limitations, we propose RSGen, a plug-and-play framework that leverages diverse edge guidance to enhance layout-driven RS image generation. Specifically, RSGen employs a progressive enhancement strategy: 1) it first enriches the diversity of edge maps composited from retrieved training instances via Image-to-Image generation; and 2) subsequently utilizes these diverse edge maps as conditioning for existing L2I models to enforce pixel-level control within bounding boxes, ensuring the generated instances strictly adhere to the layout. Extensive experiments across three baseline models demonstrate that RSGen significantly boosts the capabilities of existing L2I models. For instance, with CC-Diff on the DOTA dataset for oriented object detection, we achieve remarkable gains of +9.8/+12.0 in YOLOScore mAP50/mAP50-95 and +1.6 in mAP on the downstream detection task. Our code will be publicly available: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2603.15484v2
- Canonical: https://arxiv.org/abs/2603.15484v2
Trouble viewing inline? Open PDF directly →
Full Text
66,257 characters extracted from source content.
Expand or collapse full text
RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance Xianbao Hou 1,2⋆ , Yonghao He 2⋆,† , Zeyd Boukhers 3 , John See 4 , Hu Su 5 , Wei Sui 2 , and Cong Yang 1 1 School of Future Science and Engineering, Soochow University, Suzhou, China 2 D-Robotics, Beijing, China 3 Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany 4 School of Mathematical and Computing Sciences, Heriot-Watt University Malaysia, Putrajaya, Malaysia 5 Institute of Automation, Chinese Academy of Sciences, Beijing, China xbhou2024@stu.suda.edu.cn, wei.sui@d-robotics.c, cong.yang@suda.edu.cn C-Diff MIGC FICGen a) original b) + RSGen (ours) Fig. 1: Visualization of controllability between existing L2I methods (MIGC [38], C- Diff [35], FICGen [24]) and the same methods equipped with RSGen. While the original methods (a) struggle to adhere to the specified bounding boxes, integrating our module (b) significantly enhances instance alignment and control precision. ⋆ Equal contribution. † Project leader. Corresponding authors. arXiv:2603.15484v2 [cs.CV] 17 Mar 2026 2X. Hou et al. Abstract. Diffusion models have significantly mitigated the impact of annotated data scarcity in remote sensing (RS). Although recent ap- proaches have successfully harnessed these models to enable diverse and controllable Layout-to-Image (L2I) synthesis, they still suffer from lim- ited fine-grained control and fail to strictly adhere to bounding box con- straints. To address these limitations, we propose RSGen, a plug-and- play framework that leverages diverse edge guidance to enhance layout- driven RS image generation. Specifically, RSGen employs a progressive enhancement strategy: 1) it first enriches the diversity of edge maps com- posited from retrieved training instances via Image-to-Image generation; and 2) subsequently utilizes these diverse edge maps as conditioning for existing L2I models to enforce pixel-level control within bounding boxes, ensuring the generated instances strictly adhere to the layout. Extensive experiments across three baseline models demonstrate that RSGen signif- icantly boosts the capabilities of existing L2I models. For instance, with C-Diff on the DOTA dataset for oriented object detection, we achieve remarkable gains of +9.8/+12.0 in YOLOScore (mAP 50 /mAP 50−95 ) and +1.6 in mAP on the downstream detection task. Our code will be pub- licly available: https://github.com/D-Robotics-AI-Lab/RSGen Keywords: Layout-to-Image· Remote Sensing· Object Detection 1 Introduction High-quality remote sensing (RS) data is crucial for improving downstream de- tection tasks, going beyond specific algorithmic enhancements [1]. As a result, many studies have investigated the potential of augmenting training sets with additional generated data [22, 23, 30, 35]. Traditional methods mainly use text prompts to control the semantics of the generated images [11,16]. However, a sig- nificant limitation of these approaches is that the generated data often requires manual annotation or additional processing to be effectively used. To mitigate this problem, some studies have focused on enhancing controllability by intro- ducing dense guidance [22,31]. Unfortunately, these strategies often compromise the diversity of the generated instances. Recently, layout-to-image (L2I) generation [25,37,38] has been introduced to achieve a balance between diversity and controllability by using specified object bounding boxes as spatial conditions. While these methods allow for control at the box level and enhance overall diversity, the precision of this control is still limited. As a result, there is often a misalignment between the generated in- stances and their corresponding bounding boxes Fig. 1 (a). Consequently, even though the generated images may look visually realistic, the corresponding an- notations tend to be inaccurate, which reduces their effectiveness for training purposes. In addition to issues with spatial misalignment, current L2I methods do not fully utilize the intrinsic information present in the training samples. They mainly depend on bounding boxes and class labels, neglecting the finer edge details that can contribute to better results. RSGen3 To address these issues, we introduce RSGen. RSGen consists of two modules: Edge2Edge and L2I FGControl (Frequency-Gated Control with Spatially Gated Injection). The rationale behind RSGen is to incorporate edge maps as auxil- iary conditions alongside box-level guidance to achieve pixel-level precision. This ensures that the generated instances align strictly with the specified bounding boxes and improves the quality and reliability of the resulting annotations, as demonstrated in Fig. 1 (b). Specifically, Edge2Edge retrieves Holistically-Nested Edge Detection (HED) [28] edge maps conditioned on the input boxes and per- forms Image-to-Image (I2I) generation with an SDXL [17] model fine-tuned via Low-Rank Adaptation (LoRA) [7], where diversity is promoted by varying seeds and text prompts for downstream data augmentation [3, 6]. Furthermore, the L2I FGControl module separates structure from semantics. FGControl extracts high-frequency structural residual features by filtering out low-frequency com- ponents, and then injects these residuals only within the bounding boxes via a spatially gated mechanism to keep the guidance spatially confined. Experiments conducted on three baselines (MIGC [38], C-Diff [35], FIC- Gen [24]) show consistent improvements in YOLOScore and mean Average Pre- cision (mAP). For example, with C-Diff on the DIOR-RSVG [33] dataset, we observe enhancements of +3.3/+6.5 in YOLOScore (mAP 50 /mAP 50−95 ), along with a +0.3 increase in mAP. The improvements are even more significant on the DOTA-v1.0 [26] dataset, which focuses on oriented object detection, producing notable increases of +9.8/+12.0 in YOLOScore and +1.6 in mAP. In summary, our key contributions are as follows: – We propose RSGen, a plug-and-play framework that enhances L2I models with fine-grained edge guidance. By providing pixel-level control, RSGen fully harnesses the potential of L2I generation. – To tackle the challenge of maintaining generation diversity while utilizing strong pixel-level control, we introduce Edge2Edge. This approach generates diverse edge priors at the source using varied prompts and seeds. Following this, FGControl incorporates these edges as guidance, effectively separating structure from semantics. This allows for precise control through spatial and frequency gating mechanisms. – Extensive experiments demonstrate the effectiveness and generalizability of RSGen. It significantly improves spatial alignment precision and enhances downstream detection performance for both horizontal bounding boxes (HBB) and oriented bounding boxes (OBB). 2 Related Work This section reviews the literature that forms the basis of our method, focusing on two areas: L2I generation and generative data augmentation. 2.1 Layout-to-Image Generation L2I generation aims to create images based on input bounding boxes and cat- egory labels. Early training-free approaches [2, 9, 27] intervene in the attention 4X. Hou et al. mechanisms of diffusion models [17,20] to restrict generation to specific regions, but they suffer from limited controllability. To address these issues, subsequent training-based methods explicitly integrate layout guidance into the model ar- chitecture. For example, GLIGEN [14] introduces gated attention to combine spatial information with visual features, achieving strong controllability. In re- mote sensing, AeroGen [23] advances this approach by enabling the generation of oriented bounding boxes, while C-Diff [35] enhances coherence between fore- ground instances and backgrounds. OF-Diff [30] utilizes prior masks for improved control, but acquiring these masks can be challenging compared to edge maps, especially for amorphous categories like golf courses. Furthermore, relying solely on simple spatial transformations can limit diversity. RSGen addresses these limitations in two ways. First, it employs edge guid- ance that is robust to boundary ambiguity, effectively managing amorphous cat- egories. Second, it utilizes a diffusion model to enhance the diversity of edge priors. By leveraging these diverse edges, our method guides the generation pro- cess to achieve both high diversity and precise structural control. 2.2 Generative Data Augmentation Generative data augmentation has become a popular strategy for addressing data scarcity during model training. By leveraging the generative capabilities of diffusion models, this approach generates novel samples to enrich existing datasets, and it is widely adopted across tasks such as classification [8], object detection [23, 30], and segmentation [36]. Some methods [3, 6] employ multi- stage, training-free pipelines, which efficiently avoid the need for model training. However, these methods incur high computational costs due to complex post- processing requirements. In contrast, L2I methods [35] fine-tune diffusion models to generate instances directly within bounding boxes, which reduces complex- ity compared to multi-stage processing. This approach facilitates a streamlined augmentation pipeline where generated samples are directly merged into the training dataset. Occasionally, a filtering step is incorporated to ensure quality. Similarly, our RSGen directly augments the training set, but keeps the base L2I model frozen and only trains the lightweight FGControl module and LoRA [7], resulting in minimal resource consumption. 3 Method In this section, we start by reviewing the fundamentals of Latent Diffusion Mod- els (LDMs) [20]. Next, we provide a detailed overview of the RSGen framework ( Fig. 2), which consists of two key components: the Edge2Edge module, de- signed for generating diverse edge maps, and the L2I FGControl module, which incorporates edge guidance to ensure accurate layout alignment. Together, these components address the challenges of limited diversity and spatial misalignment in remote sensing image generation. RSGen5 LoRA Training Dataset Reference Edge Generated Edge a) Edge2Edge Input Scale-Balanced Region Attention “This is an aerial image of...” Global Prompt Layout && Class CLS CLS CLS CLS Instance1 Instance2 Instance3 Instance4 Phrase Tokens Attention map Region Loss Latent ... guidance Input Layout SDXL U-Net Setps 1 ... U-Net Setps N CLS CLIP b) L2I FGControl ... ... ... L2I Model FGControl High-Pass Filter Spatially Gated && Control Residual Trainable Frozen U-net Feature Input Generated Edge Fig. 2: Overview of RSGen, which consists of the Edge2Edge module (a) and the L2I FGControl module (b), where “CLS” denotes the class label. The Edge2Edge module enhances the diversity of retrieved edge maps through an I2I process. Subsequently, these diverse edges and layout inputs guide the L2I FGControl module, which interacts with the base L2I model to achieve precise pixel-level control. Our framework signifi- cantly increases structural diversity while ensuring fine-grained spatial alignment. 3.1 Preliminary Denoising Diffusion Probabilistic Models [5] synthesize images by iteratively re- moving Gaussian noise directly in pixel space. In contrast, LDMs [20] signifi- cantly accelerate this process within a compressed latent space constructed by 6X. Hou et al. Fig. 3: Visualization of diverse edge maps generated by the Edge2Edge module. Our method employs distinct random seeds to generate varied structural details within the specified bounding boxes, significantly enhancing the diversity of the structural priors. a Variational Autoencoder (VAE) [12]. Specifically, an input image x is first en- coded into a latent representation z 0 = E(x) by a pre-trained VAE encoder E. Gaussian noise is then injected into z 0 to yield a noisy state z t = α t z 0 +σ t ε, with ε∼N(0,I), where α t and σ t are coefficients determined by the noise schedule. A denoising U-Net [21] ε θ is optimized to estimate the added noise ε conditioned on the timestep t (t ∈ 1,...,T) and auxiliary context c (e.g., layouts and text). The LDM loss, denoted as L LDM , is defined to minimize the Mean Squared Error (MSE) between the predicted noise and the actual noise, which can be expressed as follows: L LDM = E z 0 ,ε∼N(0,I),t,c ∥ε− ε θ (z t ,t,c)∥ 2 2 . (1) 3.2 Edge2Edge As highlighted in Fig. 2 (a), we construct a database of reference edges using HED [28] and retrieve the optimal candidates based on the class and aspect ratio of each input bounding box. The retrieved edges are then assembled into a composite edge map, which serves as the input for a fine-tuned SDXL [17] model within an I2I process. To ensure that the generated edge map aligns with the specified bounding boxes, we incorporate a novel Scale-Balanced Region Atten- tion mechanism. Additionally, by utilizing prompts derived from input instance classes and various random seeds, this module generates a diverse edge map, as illustrated in Fig. 3. LoRA Fine-tuning and Scale-Balanced Region Attention. The base SDXL model is fine-tuned via LoRA [7] to align with the visual domain of HED edges. By utilizing composite maps assembled from retrieved references as input, the model parameters are optimized to shift the distribution toward the target HED style. This parameter-efficient strategy ensures that the gener- ated edges maintain high fidelity to the desired structural patterns without the computational cost of full-parameter training. RSGen7 ControlNet ControlNet-XS FGControl Fig. 4: Comparison of ControlNet [34], ControlNet-XS [32], and our FGControl. Stan- dard global control methods suffer from feature entanglement, causing background chaos. Conversely, FGControl strictly confines high-frequency structural guidance within the layout bounding boxes, achieving fine-grained local control without inter- fering with the global semantic synthesis. However, although the fine-tuned SDXL model captures the visual charac- teristics of HED edges, standard I2I generation still lacks explicit spatial con- straints. This limitation leads to two critical issues: 1) Semantic Misalignment, where a single global prompt fails to restrict semantic attributes to their cor- responding bounding boxes; and 2) Boundary Overflow, where generated struc- tural edges drift beyond the limits of the input layout boxes. Such unconstrained generation produces noisy priors that significantly degrade the performance of the subsequent L2I FGControl module. Inspired by training-free layout control methods [2, 27], we propose Scale- Balanced Region Attention that intervenes in the U-Net [21] denoising process to enforce area-aware constraints, effectively mitigating the optimization bias toward larger bounding boxes. (i) Latent Guidance via Region Loss. In the early timesteps of the denois- ing process, we perform iterative latent updates to align the semantic concepts with their target spatial regions. To achieve this, we construct a comprehensive text prompt by concatenating the class names of all layout instances into the format: “hed edge map. [class 1], [class 2]...”. Subsequently, using the CLIP [18] text encoder, we tokenize the prompt and identify the specific token indices K i corresponding to the class name of each bounding box b i . During the U-Net forward pass, we extract both cross-attention and self- attention maps from designated low-resolution layers where layout features are concentrated. For cross-attention, let K i denote the set of token indices cor- responding to the class name of instance i. We obtain the aggregated spatial attention map A i ∈ R H×W (H,W are latent dimensions) by summing over all tokens k ∈ K i and averaging across all attention heads. For self-attention, A i is directly derived from the spatial region interactions without text tokens. 8X. Hou et al. For regional alignment, let M fg i ∈0, 1 H×W be the binary mask of bound- ing box b i , and M bg ∈0, 1 H×W be the global background mask. We compute the mean foreground activation (A fg i ) and background leakage (A bg i ): A fg i = P x,y A (x,y) i ⊙M fg,(x,y) i P x,y M fg,(x,y) i + ε , A bg i = P x,y A (x,y) i ⊙M bg,(x,y) P x,y M bg,(x,y) + ε .(2) where ⊙ denotes element-wise multiplication, (x,y) represents the spatial pixel coordinates, and ε = 10 −6 is a small constant. The region loss L reg is an Intersection over Union (IoU)-inspired objective to maximize foreground response and penalize background leakage. Computed per step and averaged across targeted layers, it is defined as: L reg = N X i=1 1− A fg i A fg i + N · A bg i ! 2 . (3) where N is the total number of bounding boxes. Crucially, scaling background leakage by N counters the area imbalance between instances and the global background, preventing it from dominating optimization. With this loss, we compute its gradient with respect to the input noisy latent z t and perform a gradient descent update: z t ← z t −λ∇ z t L reg , where λ is a step size dynamically decreasing from 8 to 2. Crucially, this updated latent dynami- cally shapes the subsequent attention maps. Since the spatial Query features (Q) in the cross-attention layers are directly projected from the latent z t , shifting z t along the negative gradient geometrically translates the high-activation visual features. When the modified z t is fed back into the U-Net, the resulting attention maps naturally concentrate the semantic response within the bounding boxes. (i) Region-Masked Attention. While the early timesteps establish the global layout via latent updates, the later steps require strict spatial confinement. To prevent boundary overflow, the attention computation is modified by introducing a masking matrixM to both cross-attention and self-attention layers. Based on the input layout,M ij is set to 0 if the query-key pair (i,j) aligns with the same instance, and −∞ otherwise. The attention score is then reformulated: Attention(Q,K,V ) = Softmax QK T √ d +M V.(4) By adding this large negative mask before the Softmax operation, information flow between unrelated regions is physically blocked. The design intentionally forces the model to focus on adhering to the strict edge constraints within the specified bounding boxes. To translate the diverse edge priors into precise pixel-level control, we intro- duce the L2I FGControl module ( Fig. 2 (b)). While retaining the lightweight RSGen9 Table 1: Quantitative comparison of baseline models with and without our proposed RSGen on DIOR-RSVG and DOTA datasets. Models equipped with RSGen achieve substantial improvements in layout consistency (YOLOScore) alongside highly com- petitive generation fidelity (FID). Method DIOR-RSVG (HBB)DOTA (OBB) FID↓ YOLOScore↑ FID↓ YOLOScore↑ mAP 50 mAP 50−95 mAP 50 mAP 50−95 MIGC [38]79.3263.238.466.8053.030.0 + Ours85.0468.745.868.0755.137.7 C-Diff [35] 66.7566.841.247.7257.734.0 + Ours68.1270.147.746.86 67.546.0 FICGen [24] 74.2664.939.748.4674.049.3 + Ours73.26 65.742.248.5675.253.2 efficiency of ControlNet-XS [32], FGControl fundamentally addresses the issues of standard global injections, which often cause background chaos and over- whelming structural control in L2I tasks, as shown in Fig. 4. To explicitly decou- ple structure from semantics, FGControl incorporates the generated edge maps as auxiliary conditions and utilizes a high-pass filter to extract high-frequency structural residuals. Through a spatially gated mechanism, these residuals are explicitly injected into the base model. This synergistic interaction ensures that the pixel-level structural guidance is strictly confined within the layout bounding boxes without affecting the background. 3.3 L2I FGControl (i) Frequency Gated Structure Decoupling. The fundamental goal of FGControl is to provide precise geometric edge guidance without interfering with the semantic synthesis of the base model. Since structural edges natu- rally correspond to high-frequency features, we propose a frequency-aware de- coupling strategy to explicitly extract sharp edge features. Given a control resid- ual h res from the control branch, we define the intermediate residual map as ∆h = ZeroConv(h res ). We then apply a Fast Fourier Transform (F) and a high- pass filter H to extract high-frequency structural edges: ∆h high =F −1 (F(∆h)⊙H).(5) Specifically, H zeroes out a central H d × W d low-frequency band (d = 16). We then apply soft thresholding (τ = 0.05) to eliminate negligible noise, yielding the purified structural residual ∆h str : ∆h str = sgn(∆h high )⊙ max(|∆h high |− τ, 0).(6) 10X. Hou et al. By completely discarding the low-frequency features, FGControl strictly dictates structural layouts while the base model governs the semantic content. (i) Spatially Gated Injection. Beyond frequency decoupling, strict spatial confinement is imperative to prevent structural artifacts from leaking into the background. For each instance, we dynamically resize its bounding box mask M fg to match the spatial resolution of the current U-Net [21] layer. A spatially gated mechanism is then formulated by explicitly multiplying the purified high- frequency residuals with this mask. The final injected residual is formulated as: ∆h final = ∆h str ⊙M fg .(7) Consequently, the base L2I model receives these precise structural constraints exclusively within the layout regions, achieving fine-grained pixel-level alignment while leaving the background generation completely unaffected. 4 Experiments In this section, we conduct comprehensive experiments to assess the effectiveness and generalization of RSGen. We incorporate our framework into various L2I baselines and evaluate its performance across a range of remote sensing datasets, focusing on both the quality of generation and its application in downstream object detection tasks. Additional implementation details, efficiency analysis, supplementary experiments, and limitations are provided in the Appendix. 4.1 Experimental Settings Datasets. Following C-Diff [35], we evaluate our proposed RSGen on two widely used remote sensing datasets: DIOR-RSVG [33] and DOTA-v1.0 [26]. Constructed based on the large-scale DIOR [13] dataset, DIOR-RSVG provides HBB annotations exclusively, serving as our primary benchmark for horizontal object generation. In contrast, DOTA-v1.0 is a challenging dataset comprising 15 categories, featuring dense scenes and small objects. We crop the images from DOTA to 512× 512 following C-Diff. Furthermore, we utilize this dataset to conduct a comprehensive dual verification for both HBB and OBB detection. Benchmarks. We evaluate our method across three L2I generation models orig- inally designed for distinct domains: MIGC [38] for natural images, C-Diff for remote sensing, and FICGen [24] for degraded scenes. Implementation. We train C-Diff and FICGen following their official settings, while MIGC is fine-tuned using its official weights under C-Diff’s settings, except for a reduced 50 training epochs. For FGControl, we freeze the base L2I models. Images are resized to 512× 512, and the module is optimized for 75,000 steps with a batch size of 8 and a fixed learning rate of 8e-5. Metrics. To comprehensively evaluate the performance of RSGen, we assess the generated images from three distinct perspectives: (1) Fidelity. We utilize RSGen11 Table 2: Comparison of downstream object detection performance. By mixing syn- thetic and real data at a 1:1 ratio, models trained with data generated by RSGen achieve overall accuracy improvements across both HBB and OBB settings. Method DIOR-RSVG (HBB)DOTA (HBB)DOTA (OBB) mAP mAP 50 mAP 75 mAP mAP 50 mAP 75 mAP 50 mAP 75 MIGC [38]54.3 78.959.838.3 64.539.253.55 19.68 + Ours55.0 78.7 61.138.9 64.5 40.553.19 21.92 C-Diff [35] 54.7 78.460.137.4 63.238.654.80 20.11 + Ours55.0 78.8 60.739.0 64.2 40.454.92 21.65 FICGen [24] 54.5 78.760.739.2 64.840.855.81 22.66 + Ours54.6 78.8 61.139.5 64.8 41.455.93 23.03 Table 3: Comparison of downstream object detection performance under different real- to-synthetic data ratios. All experiments are conducted on the DOTA dataset utilizing the C-Diff [35] baseline. MethodRatio (Real:Syn) DOTA (HBB)DOTA (OBB) mAP mAP 50 mAP 75 mAP 50 mAP 75 C-Diff [35] 1:3 39.065.239.855.7221.11 + Ours39.064.240.056.07 24.36 C-Diff [35] 1:2 38.865.439.754.7521.44 + Ours39.364.441.255.98 22.34 C-Diff [35] 1:1 37.463.238.654.8020.11 + Ours39.064.240.454.92 21.65 C-Diff [35] 0:1 16.230.914.722.079.49 + Ours21.938.621.333.24 12.16 the Fréchet Inception Distance (FID) [4] to assess the perceptual quality of the generated images. FID measures the distributional distance between generated and real features, providing a robust evaluation of contextual coherence. (2) Layout Consistency. The YOLOScore [15] is employed to quantify the align- ment between generated instances and spatial layouts. Specifically, we fine-tune a YOLOv8 detector [10] for HBB and a YOLOv8-OBB detector for OBB. The precision on the generated images directly reflects the control capability. (3) Trainability. To evaluate the effect of RSGen for data augmentation, we mix synthetic images with real training data to train downstream detectors. We em- ploy Faster R-CNN [19] for HBB tasks (reporting mAP, mAP 50 , mAP 75 .) and Oriented Faster R-CNN [29] for OBB tasks (reporting mAP 50 , mAP 75 ). 4.2 Main Results Observation 1: RSGen significantly enhances layout consistency with only marginal fluctuations in generation fidelity. As shown in Tab. 1, we 12X. Hou et al. evaluate the effectiveness by integrating RSGen into three baselines (MIGC, C- Diff, and FICGen). While the enforcement of strict structural edge constraints leads to slight increases in FID in certain cases (e.g., MIGC on DIOR-RSVG), the overall quality remains highly competitive, and in some instances (e.g., C- Diff on DOTA), the FID even improves. Specifically, whether evaluated under the HBB setting on DIOR-RSVG or the OBB setting on DOTA, the models equipped with our module achieve substantial improvements in YOLOScore compared to their original counterparts. Notably, the improvements in the comprehensive mAP 50−95 metric are even more pronounced than those in mAP 50 . For instance, integrating RSGen into C-Diff yields remarkable mAP 50−95 gains of +6.5 on DIOR-RSVG and +12.0 on DOTA. Such substantial increases strongly indicate that our method significantly enhances the fine-grained control precision within the bounding boxes. This consistent performance boost across distinct baselines confirms that our module effectively trades negligible variances in global image distribution for precise, pixel-level layout alignment. Observation 2: RSGen significantly boosts the accuracy of downstream object detection tasks. Following the experimental settings of C-Diff, we mix the synthetic images generated by our method with the real training data at a 1:1 ratio. As shown in Tab. 2, integrating RSGen into the base models leads to broad increases in the standard mAP under the HBB setting on both the DIOR-RSVG and DOTA datasets. Notably, when comparing different IoU thresholds, the improvements in mAP 75 are consistently more pronounced than those in mAP 50 . For example, adding RSGen to C-Diff on the DOTA dataset yields a substantial +1.8 gain in mAP 75 , compared to a +1.0 gain in mAP 50 . To provide a deeper comparison, we further evaluate the models under the more challenging OBB setting on DOTA. In this stringent scenario, the gains in mAP 50 are relatively marginal; for instance, C-Diff equipped with our module only shows a slight +0.12 increase over the baseline. However, the performance boost primarily manifests in the stricter mAP 75 metric, where it achieves a notable +1.54 improvement. These outsized gains at higher IoU thresholds across both HBB and OBB tasks conclusively demonstrate that our method provides substantially stronger and more precise pixel-level control capabilities. Observation 3: RSGen consistently improves downstream detection performance across various synthetic data ratios. To further validate the robustness and scalability of our method, we conduct extended experiments using the C-Diff baseline on the DOTA dataset. Specifically, we train the detectors under both HBB and OBB settings using different synthetic-to-real data mix- ing ratios, including 1:3, 1:2, and purely synthetic data. As reported in Tab. 3, the evaluation across different data mixing proportions reveals several insightful trends. First, under the HBB setting, simply increasing the proportion of syn- thetic data from 1:2 to 1:3 does not yield continuous performance gains. However, our method at a minimal 1:1 ratio (mAP 39.0, mAP 75 40.4) matches or exceeds the performance of the baseline trained with a heavily augmented 1:3 ratio (mAP 39.0, mAP 75 39.8). This comparison highlights the exceptional data efficiency RSGen13 w/o Edge2Edge w Edge2Edge Fig. 5: Qualitative comparison of generated instances with and without the Edge2Edge module. Incorporating the Edge2Edge module introduces rich structural variations, sig- nificantly enhancing both the structural and overall diversity of the generated instances within the specified bounding boxes. of our method, demonstrating that high-quality, strictly aligned instances can saturate the HBB detection performance with much less data volume. Conversely, under the OBB setting, increasing the synthetic data ratio yields sustained improvements. Compared to the baseline, RSGen boosts mAP 50 (e.g., +1.23 at 1:2 ratio), while the gains are even more pronounced in the stricter mAP 75 , peaking at 24.36 (+3.25) under the 1:3 ratio. This shows that for com- plex oriented object detection tasks, our fine-grained control ensures strict spa- tial alignment, providing highly accurate annotations for detector training. In the extreme purely synthetic setting, incorporating our module leads to sub- stantial performance gains over the baseline. Specifically, the detector achieves massive increases of +5.7 in mAP and +6.6 in mAP 75 for HBB, alongside an im- pressive +11.17 surge in mAP 50 and +2.67 in mAP 75 for OBB. Ultimately, these results convincingly confirm that RSGen generates highly accurate and struc- turally reliable instances, maximizing both training efficiency and fine-grained spatial alignment for downstream applications. 4.3 Ablation Study Effect of the Edge2Edge Module. Edge2Edge enriches structural diversity, as qualitatively shown in Fig. 5. As reported in Tab. 4, it significantly improves the YOLOScore, particularly in the comprehensive mAP 50−95 metric (+0.8). 14X. Hou et al. Table 4: Ablation of the Edge2Edge module on DIOR-RSVG. While maintaining nearly identical generation fidelity (FID), the module significantly boosts layout con- sistency (YOLOScore), particularly in the stricter IoU threshold. MethodFID↓ YOLOScore↑ mAP 50 mAP 50−95 w/o Edge2Edge68.4069.546.9 w/ Edge2Edge68.1270.147.7 Table 5: Cross-dataset validation. Models are trained on DIOR-RSVG and evaluated on DOTA. The Edge2Edge module improves AP across most categories, demonstrating its strong generalization capability, which is crucial for practical applications. Methodvehicle ship basketballcourt groundtrackfield harbor tenniscourt airplane w/o Edge2Edge 6.8 8.314.913.69.642.68.2 w/ Edge2Edge 7.1 8.315.513.510.2 44.68.4 Crucially, the FID remains nearly identical (68.12 and 68.40), demonstrating that diverse structural priors effectively enhance instance-layout alignment and realism without compromising generation fidelity of the base model. To verify that the structural variations introduce meaningful diversity rather than noise, we conduct a cross-dataset generalization test by training on DIOR- RSVG and evaluating on DOTA. As shown in Tab. 5, under this challenging setting, the model equipped with Edge2Edge consistently outperforms the base- line across most categories. These results confirm that the structural variations introduced by Edge2Edge enhance cross-dataset generalization, highlighting its robustness across domains and its importance for practical applications. Effect of the FGControl Module. To validate the design of the FGControl module, we progressively ablate its key components. As shown in Tab. 6, our baseline model is ControlNet-XS [32], a typical global control mechanism. How- ever, this global injection forces structural conditions into background regions, causing severe background confusion and degraded image quality. By introducing the Spatially Gated mechanism, we confine the residual fea- tures within the target bounding boxes, which effectively eliminates background noise and improves generation fidelity. Finally, the high-pass filter decouples structural guidance from semantic features. Despite a slight FID increase due to intensified structural focus, the complete FGControl module achieves the high- est layout consistency (mAP 50−95 46.9), demonstrating its distinct advantage in enabling precise, fine-grained control while preserving background coherence. 5 Conclusion In this paper, we introduced RSGen, a novel plug-and-play framework designed to resolve the misalignment between generated instances and given bounding RSGen15 Table 6: Ablation of the FGControl module on the DIOR-RSVG dataset. The spa- tially gated mechanism resolves background confusion, while the high-pass filter further decouples features to achieve the highest layout consistency. MethodFID↓ YOLOScore↑ mAP 50 mAP 50−95 Baseline (Global Control)112.4362.145.5 + Spatially Gated64.3868.745.7 + High-pass Filter (Full)68.4069.546.9 boxes in L2I generation. Within this framework, the Edge2Edge module enriches the structural diversity of the instances and significantly boosts the generaliza- tion capability of the model. Building upon this, the L2I FGControl module leverages these diverse edge priors to achieve precise, pixel-level layout control. 16X. Hou et al. References 1. Chen, Z., Chen, K., Lin, W., See, J., Yu, H., Ke, Y., Yang, C.: Piou loss: To- wards accurate oriented object detection in complex environments. In: European conference on computer vision. p. 195–211. Springer (2020) 2. Dahary, O., Patashnik, O., Aberman, K., Cohen-Or, D.: Be yourself: Bounded attention for multi-subject text-to-image generation. In: European Conference on Computer Vision. p. 432–448. Springer (2024) 3. Fan, C., Zhu, M., Chen, H., Liu, Y., Wu, W., Zhang, H., Shen, C.: Divergen: Improving instance segmentation by learning wider data distribution with more diverse generative data. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. p. 3986–3995 (2024) 4. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017) 5. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851 (2020) 6. Hou, X., He, Y., Boukhers, Z., See, J., Su, H., Sui, W., Yang, C.: Instada: Augmenting instance segmentation data with dual-agent system. arXiv preprint arXiv:2509.02973 (2025) 7. Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022), https://openreview.net/forum?id= nZeVKeeFYf9 8. Islam, K., Zaheer, M.Z., Mahmood, A., Nandakumar, K.: Diffusemix: Label- preserving data augmentation with diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 27621– 27630 (2024) 9. Jia, C., Luo, M., Dang, Z., Dai, G., Chang, X., Wang, M., Wang, J.: Ssmg: Spatial- semantic map guided diffusion model for free-form layout-to-image generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, p. 2480– 2488 (2024) 10. Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics yolo (2023), https://github.com/ ultralytics/ultralytics 11. Khanna, S., Liu, P., Zhou, L., Meng, C., Rombach, R., Burke, M., Lobell, D.B., Ermon, S.: Diffusionsat: A generative foundation model for satellite imagery. In: The Twelfth International Conference on Learning Representations (2024), https: //openreview.net/forum?id=I5webNFDgQ 12. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013) 13. Li, K., Wan, G., Cheng, G., Meng, L., Han, J.: Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS journal of photogrammetry and remote sensing 159, 296–307 (2020) 14. Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., Lee, Y.J.: Gligen: Open-set grounded text-to-image generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 22511–22521 (2023) 15. Li, Z., Wu, J., Koh, I., Tang, Y., Sun, L.: Image synthesis from layout with locality- aware mask adaption. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 13819–13828 (2021) RSGen17 16. Liu, C., Chen, K., Zhao, R., Zou, Z., Shi, Z.: Text2earth: Unlocking text-driven re- mote sensing image generation with a global-scale dataset and a foundation model. IEEE Geoscience and Remote Sensing Magazine (2025) 17. Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: SDXL: Improving latent diffusion models for high-resolution im- age synthesis. In: The Twelfth International Conference on Learning Representa- tions (2024), https://openreview.net/forum?id=di52zR8xgf 18. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. p. 8748–8763. PmLR (2021) 19. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object de- tection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence 39(6), 1137–1149 (2016) 20. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 10684–10695 (2022) 21. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. p. 234–241. Springer (2015) 22. Tang, D., Cao, X., Hou, X., Jiang, Z., Liu, J., Meng, D.: Crs-diff: Controllable remote sensing image generation with diffusion model. IEEE Transactions on Geo- science and Remote Sensing (2024) 23. Tang, D., Cao, X., Wu, X., Li, J., Yao, J., Bai, X., Jiang, D., Li, Y., Meng, D.: Aerogen: Enhancing remote sensing object detection with diffusion-driven data generation. In: Proceedings of the Computer Vision and Pattern Recognition Con- ference. p. 3614–3624 (2025) 24. Wang, W., Zhao, Y., Ma, M., Liu, M., Jiang, Z., Chen, Y., Li, J.: Ficgen: Frequency- inspired contextual disentanglement for layout-driven degraded image generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 19097–19107 (2025) 25. Wang, X., Darrell, T., Rambhatla, S.S., Girdhar, R., Misra, I.: Instancediffusion: Instance-level control for image generation. In: Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition. p. 6232–6242 (2024) 26. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: A large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 3974–3983 (2018) 27. Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., Shou, M.Z.: Boxdiff: Text- to-image synthesis with training-free box-constrained diffusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 7452–7461 (2023) 28. Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision. p. 1395–1403 (2015) 29. Xie, X., Cheng, G., Wang, J., Yao, X., Han, J.: Oriented r-cnn for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 3520–3529 (2021) 30. Ye, Z., Ma, S., Yang, J., Yang, X., Gong, Z., Yang, X., Wang, H.: Object fidelity diffusion for remote sensing image generation. In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum? id=ngfIm9aPsH 18X. Hou et al. 31. Yuan, Z., Hao, C., Zhou, R., Chen, J., Yu, M., Zhang, W., Wang, H., Sun, X.: Efficient and controllable remote sensing fake sample generation based on diffusion model. IEEE Transactions on Geoscience and Remote Sensing 61, 1–12 (2023) 32. Zavadski, D., Feiden, J.F., Rother, C.: Controlnet-xs: Rethinking the control of text-to-image diffusion models as feedback-control systems. In: European Confer- ence on Computer Vision. p. 343–362. Springer (2024) 33. Zhan, Y., Xiong, Z., Yuan, Y.: Rsvg: Exploring data and models for visual ground- ing on remote sensing data. IEEE Transactions on Geoscience and Remote Sensing 61, 1–13 (2023). https://doi.org/10.1109/TGRS.2023.3250471 34. Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 3836–3847 (2023) 35. Zhang, M., Liu, Y., Liu, Y., Zhao, Y., Ye, Q.: Cc-diff: enhancing contextual coher- ence in remote sensing image synthesis. arXiv preprint arXiv:2412.08464 (2024) 36. Zhao, H., Sheng, D., Bao, J., Chen, D., Chen, D., Wen, F., Yuan, L., Liu, C., Zhou, W., Chu, Q., et al.: X-paste: Revisiting scalable copy-paste for instance seg- mentation using clip and stablediffusion. In: International Conference on Machine Learning. p. 42098–42109. PMLR (2023) 37. Zheng, G., Zhou, X., Li, X., Qi, Z., Shan, Y., Li, X.: Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 22490–22499 (2023) 38. Zhou, D., Li, Y., Ma, F., Zhang, X., Yang, Y.: Migc: Multi-instance generation controller for text-to-image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 6818–6828 (2024) RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance Supplementary Material 1 Implementation Details The section provides additional implementation details. We first describe the ba- sic experimental configuration. Then we present the training settings for LoRA [7] and the YOLOv8 [8] model. After that, we introduce the evaluation for the Fréchet Inception Distance (FID) [6] and the cross-validation details used in our experiments. Finally, we provide the detailed training configurations for MMDe- tection [1] and MMRotate [28]. 1.1 Experimental Configuration We implement the training for the baseline models (C-Diff [26], MIGC [27], and FICGen [19]) as well as our FGControl module using PyTorch 2.1.0 and CUDA 12.1.0 on 8 NVIDIA H20 GPUs. The training follows the official settings of the respective methods. To ensure a fair comparison, we train FICGen for 100 epochs in our main experiments. We provide the evaluation of FICGen under its official 300 epochs training setting in the Additional Experimental Results sec- tion. Furthermore, we evaluate all metrics, including FID and YOLOScore [10], using 8 NVIDIA L20 GPUs with PyTorch 2.5.1 and CUDA 12.4 1.2 LoRA Training Details We fine-tune the SDXL [13] base model using the standard Low-Rank Adapta- tion (LoRA). The training dataset consists of the composited edge maps detailed in the main text. We freeze the text encoders and exclusively train the U-Net parameters [16]. We set the LoRA rank (dimension) to 64 and the network alpha to 8. The input training images are processed at a resolution of 1024× 1024. For the optimization, we utilize the AdamW [12] optimizer with a learning rate of 0.0004, managed by a cosine learning rate scheduler with a warmup phase. The training is conducted for a total of 7,500 steps with a batch size of 4 and 4 gradient accumulation steps. Furthermore, we employ FP16 mixed precision to accelerate the process and apply Min-SNR [4] weighting with a gamma value of 5 to improve convergence stability. 2Supplementary Material 1.3 YOLOScore Training Details We train distinct YOLO detectors tailored to the bounding box formats of each dataset. For the DIOR-RSVG [24] dataset, which relies on horizontal bound- ing boxes (HBB), we employ the YOLOv8 Nano (yolov8n). Conversely, for the DOTA-v1.0 [20] and the HRSC2016 [11] datasets (detailed in the Additional Experimental Results section), which require oriented bounding boxes (OBB), we utilize the YOLOv8 Medium OBB (yolov8m-obb). Across all datasets, models are trained on their respective train splits for 50 epochs using a batch size of 16 and an input image resolution of 512× 512. To evaluate layout consistency, images generated from specific spatial layouts are assessed against the corresponding ground-truth annotations. Because test set annotations are unavailable for DIOR-RSVG and DOTA-v1.0, we utilize their val sets for both generation and evaluation. For HRSC2016, we use the test set for both generation and evaluation. During these validation phases, we adjust the parameters to align with the settings of C-Diff: the DIOR-RSVG evaluation image size is scaled to 800× 800, while DOTA-v1.0 and HRSC2016 are maintained at 512× 512, all with a batch size of 16. 1.4 FID Computation The FID is computed using a pre-trained Inception-v3 network [17]. The real images from the DIOR-RSVG dataset are resized to 800× 800, while images from both the DOTA-v1.0 and HRSC2016 datasets are resized to 512× 512. 1.5 Cross-Validation To evaluate generalization, models trained on DIOR-RSVG are tested on DOTA- v1.0. We align the differing category sets by mapping shared DOTA categories to their DIOR-RSVG equivalents (e.g., merging "small-vehicle" and "large-vehicle" into "vehicle"). Annotations for unshared categories are filtered out to ensure a standardized and fair evaluation. 1.6 MMDetection and MMRotate Training Details To evaluate downstream object detection performance, all models are imple- mented using mmcv-full 1.7.2, MMDetection 2.28.2, and MMRotate 0.3.4. For HBB tasks on the DIOR-RSVG and DOTA datasets, we utilize Faster R-CNN [14], resizing input images to 800× 800 and applying standard data augmentations such as random horizontal flipping with a 0.5 probability. For OBB detection, ar- chitectures and angle representations are tailored to the specific datasets. Specif- ically, for DOTA-v1.0, we train a Rotated Faster R-CNN [21] with a ResNet-50 backbone [5] using the le90 angle definition and a 1024× 1024 image resolution. Conversely, for the HRSC2016 dataset, we employ S2ANet [3] with a ResNet-50 backbone, utilizing the le135 angle definition and an 800× 800 resolution. Both OBB configurations incorporate comprehensive spatial augmentations, including random horizontal, vertical, and diagonal flipping. Abbreviated paper title3 Table 1: Comparison of training times. Both FGControl and LoRA fine-tuning require significantly less time than the base L2I model. ComponentTime Base Model14h 43m FGControl (Ours) 8h 31m LoRA (Ours)4h 40m Table 2: Inference efficiency on the HRSC dataset. While the Edge2Edge module incurs computational costs, adding FGControl intro- duces negligible overhead. Model Config. VRAM (GB) Latency (s) Base Model10.189.58 + FGControl10.1910.16 Edge2Edge46.0541.04 2 Efficiency Analysis In this section, we provide a detailed efficiency analysis of the RSGen frame- work. First, we report the training time required for the FGControl module and fine-tuning the base model via LoRA. Subsequently, we evaluate the inference efficiency in terms of latency and peak memory utilization upon integrating our proposed modules. 2.1 Training Time To demonstrate the training efficiency of the proposed RSGen framework, we compare the training overhead of our modules against the base layout-to-image (L2I) model. As detailed in Tab. 1, training the base model requires approx- imately 14 hours and 43 minutes. In contrast, optimizing the lightweight FG- Control module takes only 8 hours and 31 minutes. Furthermore, fine-tuning the SDXL model via LoRA for the Edge2Edge module is highly efficient, requiring approximately 4 hours. These results indicate that our plug-and-play frame- work significantly enhances fine-grained layout control without introducing a prohibitive training burden. 2.2 Inference Efficiency As shown in Tab. 2, evaluations on the HRSC dataset reveal that integrating the lightweight FGControl module adds virtually no burden: VRAM consump- tion increases by a mere 0.01 GB, and latency rises by only 0.58 seconds per image. Furthermore, while the Edge2Edge module (operating at 50 steps with a 0.6 denoising strength) requires 46.05 GB of VRAM and approximately 41 sec- onds per image, it serves strictly as a one-time offline data augmentation step, introducing zero overhead during downstream inference. 3 Additional Experimental Results This section provides further experimental evidence to validate the proposed RSGen framework, including its generalization on the HRSC2016 dataset, effec- tiveness on performance-saturated models, additional ablation studies on core modules, and justification for the selection of SDXL. 4Supplementary Material Table 3: Performance comparison on the HRSC2016 dataset. Equipped with RSGen, the base model achieves superior layout consistency and downstream detection perfor- mance in complex backgrounds. MethodFID↓ YOLOScore (OBB)↑ mAP↑ mAP 50 mAP 50−95 Ori (Real Data Only)---37.71 FICGen101.2789.857.136.55 FICGen + Ours99.8590.861.238.39 Table 4: Evaluation on a performance-saturated model. Even when the base FICGen model is trained for 300 epochs to reach its performance bottleneck, integrating RSGen provides further improvements in layout consistency. MethodFID↓ YOLOScore (OBB)↑ mAP 50 mAP 50−95 FICGen41.2377.252.8 FICGen + Ours41.4177.254.8 3.1 Results on the HRSC2016 Dataset To evaluate the generalization capability of the proposed RSGen framework, we conduct experiments on the HRSC2016 dataset. The dataset focuses on ship detection in satellite images characterized by highly complex backgrounds. Using FICGen as the base L2I model, we assess FID, YOLOScore, and mAP. As shown in Tab. 3, integrating RSGen simultaneously improves FID (from 101.27 to 99.85) and YOLOScore, specifically achieving a +4.1 gain in mAP 50−95 . Crucially, evaluating the downstream detection performance reveals that simply augmenting the training set with FICGen degrades performance to 36.55 mAP, compared to 37.71 mAP when using solely real data. In contrast, incorporating RSGen significantly elevates generated data quality, reversing this degradation and boosting the overall mAP to 38.39. This confirms that our fine-grained control preserves structural integrity, enabling substantial gains for detectors even under severe background interference. 3.2 Improving Upon Performance-Saturated Model To investigate the effectiveness of our framework when applied to a model with strong inherent control capabilities, we evaluate RSGen on a baseline that has reached performance saturation. Specifically, FICGen is utilized as the base L2I model and trained on the DOTA dataset. The baseline is trained for 300 epochs to ensure its performance reaches saturation. FID and YOLOScore are then as- sessed under the OBB setting. As shown in Tab. 4, while the FID experiences a negligible increase (from 41.23 to 41.41), integrating RSGen yields a substantial Abbreviated paper title5 Table 5: Ablation study on different control modules. Compared to ControlNet and ControlNet-XS, our FGControl achieves superior layout consistency and fidelity. MethodFID↓ YOLOScore↑ mAP 50 mAP 50−95 ControlNet128.0450.537.5 ControlNet-XS112.4362.145.5 FGControl (Ours)68.4069.546.9 Table 6: Ablation on Scale Balanced Region Attention. Compared to standard SDXL I2I generation and Be Yourself, our method achieves the best layout consistency and generation fidelity. MethodFID↓ YOLOScore↑ mAP 50 mAP 50−95 SDXL I2I69.1867.344.7 Be Yourself68.4469.346.9 Ours68.1270.147.7 +2.0 gain in the mAP 50−95 metric, alongside a stable mAP 50 . The improvement demonstrates that our fine-grained edge guidance effectively elevates the gener- ation precision and overall performance ceiling, even for fully trained models. 3.3 Additional Ablation Studies Ablation on Control Modules. To evaluate the proposed FGControl, we compare it against the established global control mechanisms ControlNet [25] and ControlNet-XS [23]. As detailed in Tab. 5, FGControl outperforms these global approaches across all metrics. Specifically, our module achieves a highly competitive FID of 68.40 and a YOLOScore (mAP 50−95 ) of 46.9. Our approach yields a substantial improvement in generation fidelity alongside consistent gains in spatial alignment accuracy. This confirms that confining high-frequency struc- tural guidance within the layout bounding boxes achieves fine-grained local con- trol without interfering with the global semantic generation. Ablation on Scale-Balanced Region Attention. We evaluate the proposed Scale-Balanced Region Attention within the Edge2Edge module to assess its role in generating structural priors. The generation process utilizes 50 inference steps with a denoising strength of 0.6. To ensure a fair comparison, we adopt the hyperparameters established by Be Yourself [2]. Specifically, we utilize a dynamic step size that decays from 8 to 2, and we scale the maximum guidance iterations per step to 9 (proportionally adjusted from the original 15 to account for our 0.6 denoising strength). 6Supplementary Material SD1.5 SDXL GT Fig. 1: Qualitative comparison of standard I2I generation (without LoRA) between SD 1.5 and SDXL. SDXL successfully preserves the structural integrity while introducing meaningful diversity. As detailed in Tab. 6, standard SDXL I2I generation yields a low layout con- sistency (mAP 50−95 of 44.7) due to a lack of explicit spatial constraints, leading to semantic misalignment and boundary overflow. While Be Yourself introduces spatial guidance and improves the mAP 50−95 to 46.9, it suffers from an optimiza- tion bias that favors larger bounding boxes. Our area constraints mitigate this bias during latent updates, boosting mAP 50−95 to 47.7 while securing the best FID (68.12). These results demonstrate that our method ensures more balanced and precise spatial control across objects of varying scales, which provides the downstream FGControl module with accurate structural priors. 3.4 SDXL Selection To justify the foundational model choice for the Edge2Edge module, we quali- tatively compare the generation capabilities of SD 1.5 [15] and SDXL. For a fair evaluation, we conduct standard I2I generation using both models. As illustrated in Fig. 1, SD 1.5 struggles to preserve the structural integrity of the original instances. It frequently fails to maintain the basic shape of the air- plane and introduces noticeable noise. In contrast, SDXL successfully preserves the accurate structural contours of the airplane while concurrently introducing meaningful structural diversity. The qualitative comparison demonstrates that SDXL possesses a vastly superior inherent understanding of complex structures, making it the most suitable and robust foundation for generating high-quality edge priors in our framework. 4 Limitations and Long-tail Cases Despite its robust performance, our method has certain limitations. As an auxil- iary plug-and-play module, RSGen maximizes the layout consistency of existing L2I models but cannot fundamentally alter the inherent generative limits of the base model. Fig. 2 illustrates specific instances across different base models. Abbreviated paper title7 a) b) a) b) MIGC FICGen Fig. 2: Qualitative comparison of standard I2I generation (without LoRA) between SD 1.5 and SDXL. SDXL successfully preserves the structural integrity while introducing meaningful diversity. Table 7: Evaluation of RSGen on the GLIGEN baseline. The modified U-Net architec- ture of GLIGEN interferes with our feature injection mechanism, leading to suboptimal performance. MethodFID↓ YOLOScore↑ mAP 50 mAP 50−95 GLIGEN77.0878.650.8 GLIGEN + Ours81.2449.832.2 The MIGC results (Left) highlight the limitations of the base model in main- taining context coherence. Specifically, the generated harbor in image (a) fails to blend semantically with the surrounding environment. Adding RSGen (image b) successfully constrains the harbor within the box but fails to resolve it. The FICGen results (Right) demonstrate poor performance when handling extremely small objects (image a). The combination of RSGen in image (b) somewhat im- proves object generation but cannot completely resolve minute visual details. Furthermore, as detailed in Tab. 7, we observe suboptimal performance when integrating RSGen with GLIGEN [9]. This is primarily attributed to the modified U-Net architecture of GLIGEN, which inserts an additional Gated Self- Attention layer between the standard attention modules, thereby interfering with our feature injection mechanism. Finally, while RSGen is designed as a plug- and-play module within the diffusers library, it cannot be directly integrated into specialized remote sensing models, such as AeroGen [18] and OF-Diff [22]. 5 Qualitative Analysis To further demonstrate the effectiveness of our proposed framework, we provide qualitative visualizations from two perspectives: the quality of the generated images and the resulting improvements in downstream detection tasks. First, we showcase the high-quality synthetic data generated by RSGen. As illustrated in Fig. 3, we present a three-column visualization consisting of the in- 8Supplementary Material put spatial layouts, the diverse edge maps generated by our Edge2Edge module, and the final images generated by the L2I model. These results demonstrate that our framework produces highly realistic remote sensing data that strictly adheres to the given spatial constraints with pixel-level precision, successfully capturing both global layouts and fine-grained structural details. Notably ( Fig. 3, fifth row), the decoupling mechanism in FGControl naturally reduces reliance on guidance when edge maps are blurred. This prevents instance generation failure and ensures robust spatial alignment despite low-quality edges. Second, we present visual comparisons of detection performance to highlight the practical benefits of our framework, with FICGen serving as the base model. As shown in Fig. 4, we perform HBB detection on the DIOR-RSVG dataset and OBB detection on the DOTA dataset. The visualization confirms that augment- ing the training set with our generated data significantly enhances bounding box precision and overall recognition capability. Abbreviated paper title9 Layout Edge Ours Fig. 3: Visualization of the RSGen image generation process. Images generated by RSGen demonstrate the ability of the framework to maintain strict spatial alignment and high visual fidelity. 10Supplementary Material FICGen FICGen ＋RSGen ＋RSGen Fig. 4: Visual comparison of detection results on DIOR-RSVG (HBB) and DOTA (OBB) datasets using FICGen. Models trained with our augmented data exhibit higher localization precision and improved recognition of objects. Abbreviated paper title11 References 1. Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C.C., Lin, D.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019) 2. Dahary, O., Patashnik, O., Aberman, K., Cohen-Or, D.: Be yourself: Bounded attention for multi-subject text-to-image generation. In: European Conference on Computer Vision. p. 432–448. Springer (2024) 3. Han, J., Ding, J., Li, J., Xia, G.S.: Align deep features for oriented object detection. IEEE transactions on geoscience and remote sensing 60, 1–11 (2021) 4. Hang, T., Gu, S., Li, C., Bao, J., Chen, D., Hu, H., Geng, X., Guo, B.: Efficient diffusion training via min-snr weighting strategy. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 7441–7451 (October 2023) 5. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 770–778 (2016) 6. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017) 7. Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022), https://openreview.net/forum?id= nZeVKeeFYf9 8. Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics yolo (2023), https://github.com/ ultralytics/ultralytics 9. Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., Lee, Y.J.: Gligen: Open-set grounded text-to-image generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 22511–22521 (2023) 10. Li, Z., Wu, J., Koh, I., Tang, Y., Sun, L.: Image synthesis from layout with locality- aware mask adaption. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 13819–13828 (2021) 11. Liu, Z., Wang, H., Weng, L., Yang, Y.: Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds. IEEE geoscience and remote sensing letters 13(8), 1074–1078 (2016) 12. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017) 13. Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: SDXL: Improving latent diffusion models for high-resolution im- age synthesis. In: The Twelfth International Conference on Learning Representa- tions (2024), https://openreview.net/forum?id=di52zR8xgf 14. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object de- tection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence 39(6), 1137–1149 (2016) 15. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 10684–10695 (2022) 12Supplementary Material 16. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. p. 234–241. Springer (2015) 17. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the incep- tion architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 2818–2826 (2016) 18. Tang, D., Cao, X., Wu, X., Li, J., Yao, J., Bai, X., Jiang, D., Li, Y., Meng, D.: Aerogen: Enhancing remote sensing object detection with diffusion-driven data generation. In: Proceedings of the Computer Vision and Pattern Recognition Con- ference. p. 3614–3624 (2025) 19. Wang, W., Zhao, Y., Ma, M., Liu, M., Jiang, Z., Chen, Y., Li, J.: Ficgen: Frequency- inspired contextual disentanglement for layout-driven degraded image generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 19097–19107 (2025) 20. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: A large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 3974–3983 (2018) 21. Xie, X., Cheng, G., Wang, J., Yao, X., Han, J.: Oriented r-cnn for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 3520–3529 (2021) 22. Ye, Z., Ma, S., Yang, J., Yang, X., Gong, Z., Yang, X., Wang, H.: Object fidelity diffusion for remote sensing image generation. In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum? id=ngfIm9aPsH 23. Zavadski, D., Feiden, J.F., Rother, C.: Controlnet-xs: Rethinking the control of text-to-image diffusion models as feedback-control systems. In: European Confer- ence on Computer Vision. p. 343–362. Springer (2024) 24. Zhan, Y., Xiong, Z., Yuan, Y.: Rsvg: Exploring data and models for visual ground- ing on remote sensing data. IEEE Transactions on Geoscience and Remote Sensing 61, 1–13 (2023). https://doi.org/10.1109/TGRS.2023.3250471 25. Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 3836–3847 (2023) 26. Zhang, M., Liu, Y., Liu, Y., Zhao, Y., Ye, Q.: Cc-diff: enhancing contextual coher- ence in remote sensing image synthesis. arXiv preprint arXiv:2412.08464 (2024) 27. Zhou, D., Li, Y., Ma, F., Zhang, X., Yang, Y.: Migc: Multi-instance generation controller for text-to-image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 6818–6828 (2024) 28. Zhou, Y., Yang, X., Zhang, G., Wang, J., Liu, Y., Hou, L., Jiang, X., Liu, X., Yan, J., Lyu, C., Zhang, W., Chen, K.: Mmrotate: A rotated object detection bench- mark using pytorch. In: Proceedings of the 30th ACM International Conference on Multimedia (2022)