Paper deep dive
Generative Phomosaic with Structure-Aligned and Personalized Diffusion
Jaeyoung Chung, Hyunjin Son, Kyoung Mu Lee
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 4/10/2026, 3:46:32 AM
Summary
The paper introduces a generative framework for creating photomosaics that overcomes the limitations of traditional retrieval-based methods. By utilizing a diffusion-based approach with integral-noise subsampling, adaptive instance normalization for color consistency, and low-frequency structural guidance, the model synthesizes structurally aligned and semantically expressive tiles. It further supports personalized generation via few-shot LoRA fine-tuning, enabling user-specific or domain-adaptive photomosaic creation.
Entities (5)
Relation Signals (3)
Generative Photomosaic Framework → utilizes → Diffusion Models
confidence 98% · Our generative photomosaic framework synthesizes tile images using diffusion-based generation
Generative Photomosaic Framework → integrates → LoRA
confidence 95% · For personalized photomosaic generation, we tuned a separate LoRA module for each concept
Integral-Noise Subsampling → improves → Structural Consistency
confidence 92% · initializing tiles with this coherent multi-scale noise leads to significantly more stable denoising dynamics.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present the first generative approach to photomosaic creation. Traditional photomosaic methods rely on a large number of tile images and color-based matching, which limits both diversity and structural consistency. Our generative photomosaic framework synthesizes tile images using diffusion-based generation conditioned on reference images. A low-frequency conditioned diffusion mechanism aligns global structure while preserving prompt-driven details. This generative formulation enables photomosaic composition that is both semantically expressive and structurally coherent, effectively overcoming the fundamental limitations of matching-based approaches. By leveraging few-shot personalized diffusion, our model is able to produce user-specific or stylistically consistent tiles without requiring an extensive collection of images.
Tags
Links
- Source: https://arxiv.org/abs/2604.06989v1
- Canonical: https://arxiv.org/abs/2604.06989v1
Trouble viewing inline? Open PDF directly →
Full Text
46,624 characters extracted from source content.
Expand or collapse full text
Generative Phomosaic with Structure-Aligned and Personalized Diffusion Jaeyoung Chung ⋆1 , Hyunjin Son ⋆1 , and Kyoung Mu Lee 1 Seoul National University, Seoul, South Korea robot0321, hyunjin.son, kyoungmu@snu.ac.kr Fig. 1: Generative Photomosaic. We redefine photomosaic creation as a generative process. Each tile is synthesized by a diffusion model that maintains global structural alignment and reflects the local visual characteristics of the reference image. Abstract. We present the first generative approach to photomosaic cre- ation. Traditional photomosaic methods rely on a large number of tile images and color-based matching, which limits both diversity and struc- tural consistency. Our generative photomosaic framework synthesizes tile images using diffusion-based generation conditioned on reference images. A low-frequency–conditioned diffusion mechanism aligns global structure while preserving prompt-driven details. This generative formulation en- ables photomosaic composition that is both semantically expressive and structurally coherent, effectively overcoming the fundamental limitations of matching-based approaches. By leveraging few-shot personalized diffu- sion, our model is able to produce user-specific or stylistically consistent tiles without requiring an extensive collection of images. Project page. Keywords: Photomosaic· Generative model· Image Diffusion 1 Introduction A photomosaic is a form of image composition in which a large reference image, also called the main image, is reconstructed from a collection of small mosaic ⋆ indicates equal contribution. arXiv:2604.06989v1 [cs.CV] 8 Apr 2026 2J. Chung et al. images, referred to as tile images. Each tile contributes local visual details, while the overall arrangement collectively forms a coherent global structure. Owing to this dual-layered nature—global semantics emerging from local image com- position—photomosaic have found applications across a wide range of domains, including industrial design, digital art, education, and advertising [4, 31]. De- pending on the semantic relationship between the reference image and its con- stituent tiles, photomosaic can convey diverse meanings and aesthetic impres- sions. For example, a mosaic may be created by assembling personal photographs to depict a record of past travels, by composing concert images of an artist to form the artist’s portrait, or by arranging departmental photos of a company to construct its corporate logo. In artistic contexts, photomosaic is also used to produce paradoxical or ironic effects by intentionally contrasting the global image with the local tile content. This interplay between global coherence and local diversity makes photomosaic an expressive and versatile medium for both artistic storytelling and visual communication. Photomosaic aims to reconstruct the low-frequency structure of a reference image while allowing visual diversity in high-frequency details. Although pho- tomosaic has a long history as a creative image composition technique [1], con- ventional approaches suffer from several intrinsic limitations: the difficulty of finding tiles that are structurally aligned with reference blocks from a finite image pool. To mitigate this, prior work adopts finer block partitioning with color adjustment. While finer block partitioning improves low-frequency fidelity, it inherently fixes the perceptual scale of the mosaic, constraining the viewing distance at which individual tiles become discernible. This limits the ability to control how tile-level details emerge across different viewing scales. Moreover, as finer partitions require a larger number of tiles, the limited size of the image pool often leads to repetitive use of identical tiles. Consequently, these limita- tions motivate a shift beyond retrieval-based photomosaic methods toward more flexible and expressive alternatives. To overcome these limitations, we propose a generative photomosaic frame- work that leverages the expressive power of modern diffusion-based image gen- erators. Instead of depending on large external tile datasets, our method syn- thesizes diverse tile images conditioned on the reference image itself, ensuring structural alignment at the global level while maintaining stylistic flexibility at the local level. Specifically, we employ a frequency-aware generation strategy that aligns low-frequency components of the generated tiles with the reference structure, while allowing high-frequency variations to be guided by text con- ditions. This enables both coherent reconstruction and semantically meaningful diversity within the mosaic composition. Furthermore, to support personalized or domain-specific applications, we integrate few-shot image-conditioned diffusion models that can adapt to a small set of user-provided images. This personaliza- tion enables the photomosaic to embed personal memories, stylistic preferences, or brand-specific visual elements, thereby enriching its expressive and emotional depth. By unifying structural consistency, generative diversity, and user control- Generative Phomosaic with Structure-Aligned and Personalized Diffusion3 lability, our framework reinterprets photomosaic creation as a generative process rather than a combinatorial one. In summary, our main contributions are as follows: – We introduce the first generative approach for photomosaic that synthesizes structurally aligned tile images without relying on large-scale external tile collections. – We design a structural-guided diffusion control mechanism that harmonizes low-frequency structural alignment with high-frequency detail generation, achieving both global coherence and local diversity. – We employ a few-shot personalized diffusion model to incorporate user- specific or domain-adaptive imagery, enhancing semantic richness and in- dividuality. 2 Related work Photomosaic. A photomosaic represents a large image reconstructed from a collection of small, uniformly sized photographic tiles. The technique produces a coherent global perception when viewed from a distance, while each tile shows its own detail up close, making it popular in advertising, design, and digital art. The concept of photomosaics traces back to the 1990s, popularized by Robert Silvers’ work at MIT [30], which first formalized the idea of reconstructing a large image using numerous smaller photographic tiles. Early photomosaics were primarily artistic or commercial in nature, emphasizing aesthetic composition rather than algorithmic optimization. At that time, photomosaic was costly and time-consuming due to the large amount of computation required. Subsequent research transformed the process into a computational task involving color his- togram matching [4], tone mapping, and block-to-tile correspondence search. During the 2000s and 2010s, numerous heuristic and evolutionary algorithms were proposed to improve matching efficiency and reduce tile repetition [8,19]. These methods, however, fundamentally depended on the availability and diver- sity of external tile databases. Recent extensions explored semantic matching and hybrid approaches integrating clustering or deep features, yet all remained con- strained by the need to retrieve rather than synthesize tile imagery. In contrast, our work represents the first generative formulation of photomosaics, enabling tile synthesis through diffusion models and frequency-aware conditioning. Fur- thermore, to prevent redundant tile reuse that degrades the visual quality, we leverage personalized diffusion models to generate diverse, semantically aligned tile images that correspond to the content of the reference image. 2D diffusion models for Image Generation. Recent advances in generative modeling have fundamentally changed the landscape of image synthesis and com- position. Diffusion-based models such as Stable Diffusion [25], Imagen [29], and SDXL [23] demonstrate remarkable controllability over semantic and structural attributes, enabling precise manipulation of spatial layouts, textures, and visual styles. Several works explore structure-aware generation, where global layout 4J. Chung et al. or low-frequency components guide local detail synthesis [9,37]. These techniques highlight the potential of frequency-space control and cross-scale consistency for achieving spatial coherence. Building on these advances, our work introduces the first generative photo- mosaic framework, integrating personalized diffusion with frequency-wise con- ditioning to synthesize tile images that are semantically expressive, structurally aligned, and unconstrained by pre-existing datasets. Personalized image generation. Personalized image generation in diffusion models focuses on adapting text-to-image (T2I) systems to represent user-specific concepts from a few reference images while maintaining identity consistency un- der novel prompts. Early methods such as Textual Inversion (TI) [5] learned new text embeddings that encodes the subject, but were limited in identity fidelity. DreamBooth [27] fine-tuned the diffusion model on subject-specific images, im- proving identity preservation but tended to overfit to reference examples. Custom Diffusion [15] updated only cross-attention layers, enabling scalable and efficient personalization. Recent methods have leveraged Low-Rank Adaptation (LoRA) [10], a low- rank matrix decomposition technique that enables parameter-efficient fine-tuning. DB-LoRA [28] integrates LoRA into DreamBooth, achieving comparable fidelity to full weight fine-tuning in single-concept customization while being lightweight. Mix-of-Show [7] introduces ED-LoRA for single-concept tuning, which enhances expressiveness of the embedding through layer-wise embedding and multi-world representation. It further combines multiple LoRA modules via gradient fusion and regional sampling to enable coherent multi-concept composition. Following recent works [3,12,14,22,32] focused on achieving more flexible concept compo- sition and mitigating interference among multiple concepts. In this work, we generate personalized tile images using Mix-of-Show ap- proach, as it provides stable and high fidelity in single-conept customization. 3 Method We aim to construct a photomosaic image I mosaic that represents a given ref- erence image I ref through a set of tile images I tile k . The reference image I ref is partitioned into non-overlapping patches P ref k , and our goal is to preserve the color statistics and coarse structures of these patches while injecting target- prompt–aligned textures or fine details into each local tile. We propose several techniques that guide the generation of tile imagesI tile k such that both the global reference patches P ref k and the target text prompt are simultaneously satisfied. Section 3.1 presents a noise initialization strategy designed to improve consistency across tiles. Section 3.2 introduces latent distri- bution matching that aligns color distributions. Section 3.3 describes a gradient- descent–based latent adjustment that enforces coarse structural guidance during generation. Generative Phomosaic with Structure-Aligned and Personalized Diffusion5 ⋯ Reference Image (768 x 768) 푖-th Tile Image (768 x 768) 퐱 푡 ⋯ ⋯ 퐱 푇 퐱 0 DiT ⋯ ∫−푛표푖푠푒 (Sec 3.1) 96 x 96 DiT 휇 푘 tile ←휇 푘 ref 휎 푘 tile ←휎 푘 ref Color Dist. Align (Sec 3.2) Low-freq. Guide (Sec 3.3) 휇 푘 ref ,휎 푘 ref 2 − Predict 퐱 0 Backward “A cat with flower background” “A cat” Fig. 2: Method Overview. We generate photomosaic images using a diffusion-based framework. During the noise initialization stage, each partial reference block (e.g. 96×96) is expanded to proper resolution (e.g.768×768) using an integral-noise sub- sampling. At every denoising step, we align the color distribution of the evolving tile image with its corresponding reference block and apply a low-frequency guidance term to preserve coarse structural layout. As a result, the model produces tile images that both follow the structure of the reference image and comply with the target prompt. 3.1 Integral-Noise Subsampling for Initialization To obtain stable and spatially consistent tile-level latents, we generate fine-scale Gaussian noise conditioned on the global block rather than relying on indepen- dently sampled random initialization. Random latents injected per tile often introduce incoherent high-frequency patterns and step-to-step instability during denoising. Instead, we construct a coherent multi-scale noise field by adapting the integral-noise subsampling [2], which interprets each pixel as the integral of an underlying continuous white Gaussian field. Given a global latent B k ∈R h×w and a scale factor s that maps each global block to s 2 fine pixels (e.g., 96→ 768, so s = 8), the conditional distribution of the fine-scale latent over a block is W | X ∼ X N u + 1 √ N Z−⟨Z⟩u ,(1) where X denotes a coarse block-level noise variable, and W is the corresponding fine-scale noise conditioned to preserve the block integral. Here Z ∼ N (0,I) is Gaussian distribution, N = s 2 ,u is an all-ones vector of length N. This decomposition ensures that the fine-scale latent exactly preserves the coarse integral while the zero-sum residual recovers the correct conditional covariance of a Gaussian field. As a result, all tiles originate from a single coherent noise source, maintaining structural consistency across regions and avoiding texture discontinuities that commonly arise with random per-tile initialization. Diffusion models, however, assume a unit-variance per-pixel Gaussian prior, whereas the conditional form in Eq. 1 yields marginal variance (N−1)/N < 1. 6J. Chung et al. To align the subsampled latent with the diffusion prior, we apply a deterministic variance normalization in the zero-sum subspace: f W = X N u + 1 q 1− 1 N W − X N u .(2) where f W is the variance-normalized noise that matches the unit-variance diffu- sion prior while maintaining block-level coherence, following the presentations in [2]. This transformation preserves the coarse integral exactly while restoring unit variance at every fine-scale pixel, yielding a diffusion-ready coherent noise prior that is statistically compatible with the pretrained model. In practice, initializing tiles with this coherent multi-scale noise leads to sig- nificantly more stable denoising dynamics. During early timesteps, where the model is highly sensitive to high-frequency perturbations, the integral-noise ini- tialization reduces variance spikes, prevents boundary artifacts between tiles, and produces globally consistent structure compared to independent random noise sampling. 3.2 Color Distribution Alignment through Adaptive Instance Normalization To ensure coherent color appearance across tiles, we adopt a lightweight distri- bution matching strategy inspired by adaptive instance normalization (AdaIN) used in style transfer. While each tile in our framework is generated indepen- dently, direct composition without color normalization often yields noticeable tone inconsistencies and undesired color seams across tile boundaries. For each tile I tile k and the corresponding reference block B k from the global guidance image, we compute their first-order and second-order statistics: μ t k ,σ t k = Mean(I tile k ), Std(I tile k ),(3) μ r k ,σ r k = Mean(B k ), Std(B k ).(4) We then adjust each tile to match the global color distribution: ˆ I tile k = μ r k + σ r k ⊙ I tile k − μ t k σ t k .(5) This simple normalization aligns the per-tile color statistics with the global target, effectively harmonizing color tones across the entire mosaic. Despite its simplicity, we find that this distribution matching significantly reduces color drifting artifacts and improves global visual consistency. 3.3 Low-Frequency Structural Guidance with Gradient Descent To ensure global structural consistency across tiles while still allowing local ap- pearance diversity, we incorporate a tile-wise low-frequency guidance step into Generative Phomosaic with Structure-Aligned and Personalized Diffusion7 the denoising process. At each diffusion timestep t, given a local latent z t k for tile k, we first predict the corresponding clean latent ˆx 0,i (t) using the DDIM x 0 prediction formulation: ˆx 0,k (t) = α t z t k − σ t ε θ (z t k ,t,c),(6) where α t = √ ̄α t and σ t = √ 1− ̄α t follow the DDPM schedule, and ε θ denotes the noise prediction network conditioned on c. We decode ˆx 0,k (t) using the VAE decoder and resize the resulting RGB patch to the tile resolution. Letting G σ denote a fixed low-pass operator (e.g., Gaussian blur), we enforce consistency between the low-frequency components of the generated tile and those of the global guiding tile ̃y i : ℓ k (t) = G σ I tile k (t) −G σ (B k (t)) 2 2 . (7) We then directly update the latent z t i via gradient descent on ℓ i (t): z t k ← z t k − w∇ z t k ℓ k (t), w ← γw,(8) where w is a per-tile update magnitude and γ < 1 (e.g., γ = 0.95) applies exponential decay over diffusion steps. Importantly, unlike SyncDiffusion, which aligns perceptual similarity via LPIPS at the image level, our objective focuses solely on low-frequency structural alignment. This encourages global coherence and spatial layout consistency while leaving high-frequency details free to adapt to local content and style variations. 4 Experiments 4.1 Comparison Methods We compare our generative photomosaic framework with both traditional and diffusion-based baselines. As classical non-generative methods, we adopt feature matching and tone adjustment (Match & Tone) [16]. This approach replaces each image block with the most similar tile in terms of L2 distance within a given image pool, following the conventional photomosaic paradigm. The tone mapping method modifies the tone of a reference or arbitrary image to match each target block, enabling photomosaic construction even with a single source image. In contrast, histogram matching aligns the pixel intensity distribution of a reference image to each block, achieving more accurate tone transfer but still lacking spatial structure preservation. As we introduce the first generative approach to photomosaic creation, we explored and adapted a range of existing ideas and techniques that could be re- purposed for tile generation within the photomosaic pipeline, including Color ControlNet, AdaIN, NoiseBlend, and StreamDiffusion I2I. Color ControlNet (Stable Diffusion v1.4 [26] with T2Iadapter [21]) uses both a low-resolution reference and a text prompt as conditions, aiming to maintain color coher- ence while adhering to textual semantics. AdaIN (Adaptive Instance Normal- ization) [11] re-normalizes the mean and variance of denoised features at each 8J. Chung et al. (a) Reference(f) StreamDiffI2I(g) Ours(b) Match&Tone(c) Color ControlNet(d) AdaIN(d) NoiseBlend # 64 Local Tiles # 1024 Local Tiles # 4 Local Tiles Fig. 3: Qualitative Results across Different Mosaic Levels. Table 2: User Study Ours Win Rate (%) Global Local Match&Tone98.773.3 Color ControlNet 95.77.3 AdaIN96.712.7 NoiseBlend15.092.3 StreamDiffI2I23.088.7 (a) SSIM per Resolution(b) IQA Score vs 1-LPIPS Fig. 4: Comparison of Quantitative results. step to match those of the corresponding global block, facilitating color tone con- sistency across patches. NoiseBlend [17] blends denoised results from the global and local branches at every diffusion step in fixed ratios, seeking to preserve global structure while injecting local texture variations. Finally, StreamDiffu- sion Image-to-Image (I2I) [13] refines each low-resolution tile via the image-to- image mode of StreamDiffusion under text guidance, providing an alternative diffusion-based generation path focused on local refinement. 4.2 Experimental Setup Implementation Details. We build our generative photomosaic framework on Stable Diffusion 2.1, generating all images at 768× 768 resolution. We use 50 inference steps with guidance scale of 7.5. For photomosaic construction, we divide the image into 2 L × 2 L tiles, where L is the mosaic level (e.g., L = 3 yields an 8× 8 mosaic). w is fixed to 5000 for quantitative evaluation. All experiments are conducted on a single NVIDIA A6000 GPU, and generating one photomosaic image takes about 15 minutes. For personalized photomosaic generation, we tuned a separate LoRA module for each concept, injecting it into the linear layers of all attention modules in both Unet and text encoder, using a rank of r = 4, following the configuration used in Mix-of-Show [7]. Generative Phomosaic with Structure-Aligned and Personalized Diffusion9 Global Prompt: “A scenic view of the Eiffel Tower” / Local Prompt: “A young woman” Global Prompt: “A close-up photo of a dog in the forest” / Local Prompt: “A dog” Global Prompt: “A coffee shop interior with wooden furniture” / Local Prompt: “A mug cup in the room” Global Prompt: “A close-up portrait of Iron Man upper body and helmet, dramatic background with sparks and smoke” / Local Prompt: “Spider-Man” (a) Reference(f) StreamDiffI2I(g) Ours(b) Match&Tone(c) Color ControlNet(d) AdaIN(d) NoiseBlend Fig. 5: Qualitative Results of Generative Photomosaic. 10J. Chung et al. Table 1: Quantitative results Ref. Image (Global)Tile Image (Local) PSNR ↑ SSIM ↑ LPIPS ↓ HPSv2 ↑ IR ↑ BLIP ↑ CLIP ↑ IQA ↑ HPSv2 ↑ IR ↑ Match & Tone [16]16.719 0.366 0.206 0.129 -1.7830.625 0.612 0.596 0.174 -1.145 Color ControlNet [21]10.145 0.080 0.435 0.123 -2.2720.639 0.613 0.811 0.266 0.472 AdaIN [11]12.983 0.204 0.400 0.130 -2.2790.577 0.614 0.834 0.244 -0.106 NoiseBlend [17]11.590 0.205 0.479 0.117 -2.2750.239 0.597 0.385 0.195 -1.851 StreamDiffI2I [13]15.212 0.414 0.272 0.127 -1.3770.166 0.602 0.259 0.173 -1.073 Ours17.598 0.551 0.121 0.132 -2.2250.569 0.619 0.812 0.239 -0.317 Metric. We evaluate photomosaic quality using metrics that capture both global structural fidelity and local tile realism. PSNR and SSIM [34] measure low and mid frequency reconstruction quality, while LPIPS [38] evaluates percep- tual structural similarity. These metrics reflect how well the global structure of the reference image is preserved, where the reference images generated by Sta- ble Diffusion are used as ground truth. Also, BLIP [20] and CLIP [24] scores evaluate semantic alignment of the local tiles with their corresponding prompts, and CLIP-IQA [33] measures perceptual tile quality, for which we report the score of the "quality" attribute. We additionally evaluated metrics designed to align with human preference, thereby better capturing perceptual quality, such as Human Preference Score v2 (HPSv2) [35] and Image Reward (IR) [36]. Evaluation Details. We utilize 12 global and local prompt sets and evaluate all methods with 15 fixed random seeds for reproducibility, ensuring compara- ble quantitative metrics across methods. The evaluation prompts to generate photomosaic images are presented in supplementary material. For quantitative evaluation, we measure metrics for global structural fidelity across multiple res- olutions (32, 64, 128, 256). PSNR, SSIM, and LPIPS are reported at resolution 64 for the HPSv2 and IR metrics are reported at resolution 128. 4.3 Evaluation Figure 3 compares baseline methods derived from prior work across different mosaic levels. Table 1 and Figure 4 reports the quantitative comparisons. Our method maintains strong performance across both global and local metrics, with- out being biased toward either aspect. Figure 5 presents generative mosaics cre- ated with various prompts and evaluates their performance against existing base- lines. Our method preserves the global structure more faithfully while leveraging the reference block layout and generating tiles that follow the local prompts. Traditional methods such as feature matching, tone adjustment, and his- togram matching fail to maintain structural consistency between global and local regions. Because these approaches depend solely on local tone or color similarity, they require either a large number of tiles or very small patch sizes to preserve the recognizable global layout. As a result, both image collection and generation incur high computational and data costs. Generative Phomosaic with Structure-Aligned and Personalized Diffusion11 Table 3: Ablation Results. Ref. Image (Global)Tile Image (Local) PSNR ↑ SSIM ↑ LPIPS ↓ BLIP ↑ CLIP ↑ IQA ↑ Ours17.598 0.551 0.1210.569 0.619 0.812 w/o sec 3.116.224 0.510 0.1860.562 0.611 0.806 w/o sec 3.2 16.431 0.495 0.1610.566 0.612 0.564 w/o sec 3.313.142 0.314 0.3980.567 0.614 0.810 (a) Ours(b) w/o R -noise Init(c) w/o Color Align(d) w/o Gradient Guide Fig. 6: Visualization of Ablation Results. The diffusion-based methods demonstrate distinct trade-offs between struc- tural coherence, texture realism, and prompt alignment. Color ControlNet, al- though conditioned on a low-resolution reference, fails to adequately reflect its structural guidance. Regardless of the guidance strength, the model tends to prioritize text alignment, resulting in over-saturated or misaligned compositions. AdaIN successfully transfers the overall color tone of each global block but lacks geometric precision, making it more suitable for fine-grained mosaics rather than globally structured ones. NoiseBlend effectively preserves the structural layout of the global image while injecting detailed local textures. However, blending de- noised predictions at each step introduces noisy artifacts, which appear to stem from conflicting denoising directions guided by the global and local prompts. StreamDiffusion (I2I) maintains low-frequency consistency and generates coher- ent local textures but fails to fully reflect the intended prompt semantics, yielding visually plausible yet semantically under-constrained results. Human Preference Study. We conducted an A/B preference test with 100 participants, comparing our method against each baseline. For each of the five baselines, three comparison examples were included, resulting in a total of 15 questions. The Table 2 reports the percentage of responses in which our method was preferred over the baseline. Human A/B test results reveal a clear global–local trade-off in baselines. Baselines that achieve high global fidelity exhibit limited local tile quality, while locally emphasized methods degrade global semantic structure. Our method maintains a balanced performance across both aspects. 4.4 Ablation Study Component Ablation. We conducted an ablation study to evaluate the contri- bution of each proposed component, as illustrated in Figure 6. Figure 6b shows 12J. Chung et al. Table 4: Guidance Ablation Results. Ref. Image (Global)Tile Image (Local) PSNR ↑ SSIM ↑ LPIPS ↓ BLIP ↑ CLIP ↑ IQA ↑ Ours17.598 0.551 0.1210.569 0.619 0.812 LPIPS-guided 12.242 0.216 0.4560.373 0.603 0.656 YCbCr-guided14.107 0.361 0.3370.548 0.611 0.801 (a) Reference(b) Ours(c) LPIPS-guided(d) YCbCr-guided Fig. 7: Visualization of Guidance Ablation Results. the result obtained by replacing our integral noise subsampling with random initialization. The random initialization disrupts the latent distribution, leading to poor preservation of the structural layout of the reference image. Figure 6c presents the result without the proposed Color Alignment module. In this case, most color information vanishes and the output appears nearly monochromatic. This observation suggests that our gradient guidance not only constrains spa- tial structures but also implicitly enforces low-frequency consistency in the color space. In Figure 6d, the reference structure is almost entirely lost. Analysis on the Guidance Strategy. Our approach leverages tile-wise low- frequency guidance during denoising process to maintain global structural con- sistency across tiles. For comparison, we conducted additional experiments us- ing alternative guidance strategies, as shown in Figure 7 and Table 4. First, our MSE-based low-frequency structural guidance in is inspired by SyncDiffu- sion [18], which instead employs an LPIPS loss to enforce perceptual consistency. In practice, however, we observe that LPIPS tends to introduce undesirable arti- facts rather than encouraging adherence to low-frequency structure, whereas our MSE formulation provides a more stable and structurally faithful guidance sig- nal. Second, we further experimented with applying the structural guidance only on the luminance. Specifically, we converted the RGB outputs into the YCbCr color space and imposed the MSE loss exclusively on the luminance channel Y : Y = 0.299R + 0.587G + 0.114B,(9) L Y =∥Y tile k − Y ref.block k ∥ 2 2 . (10) The motivation was that enforcing the loss on luminance would emphasize low-frequency structural alignment while leaving chrominance relatively uncon- strained. However, this strategy did not yield noticeable improvement in color Generative Phomosaic with Structure-Aligned and Personalized Diffusion13 (b) Generated Photomosaic(a) Generation Process of Local Tile and Reference Local Tile Ref. Block Local Tile Ref. Block Fig. 8: Generation process of local tile and its corresponding reference block. (a) w = 0(b) w = 500(c) w = 2000(d) w = 5000(e) w = 10000(f) w = 20000 Fig. 9: Ablation Results on the Guidance Weight Magnitude. preservation; the resulting images still exhibited significant desaturation and chroma collapse. Denoising Process Visualization. Figure 8 illustrates the evolution of the tile image and its corresponding reference block throughout the diffusion pro- cess. As denoising progresses, the tile image is guided to gradually acquire the structural characteristics of the reference block, enabling the effective synthesis of photomosaic. Analysis on the Guidance Weight Magnitude. To analyze the effect of the low-frequency structural guidance, we conducted an ablation experiment by varying the guidance weight w used in Eq. (8), as shown in Figure 9. As w con- trols the magnitude of the gradient-based adjustment applied to each tile latent, it determines the strength of structural alignment enforced during denoising. When w is small, the guidance imposes only a mild constraint, preserving fine- grained visual details driven by the text prompt in local tiles. Larger values of w strengthen the low-frequency alignment, guiding each tile toward the structure of the reference. This improves the clarity of the global structure but diminishes local semantic detail, often producing overly smoothed textures within the tiles. These observations verify that w governs the trade-off between global structural preservation and local detail fidelity. 4.5 Application: Personal Photomosaic Generation We present personalized photomosaic results in Figure 10. To construct user- specific tile appearances, we collected 15 reference images per sample from the 14J. Chung et al. (a) Personalization Example 1: "Heungmin" (b) Personalization Example 2: "Harry" and "Hermione" (c) Personalization Example 3: "Nezuko" and "Tanjiro" Fig. 10: Application: Personalized Generative Photomosaic. internet and performed LoRA fine-tuning using the Mix-of-Show [7]. Each ED- LoRA weight required approximately twenty minutes of training. During gener- ation, the personalized LoRA weights are incorporated into the diffusion model so that each tile image reflects the target concept while still preserving the global structure of the reference image. As a result, the generated photomosaic simul- taneously maintains global visual fidelity to the target image and local semantic consistency with the personalized concept. In addition, the reference image it- self can also be generated using the same personalized concept, enabling a fully personalized photomosaic where both the global structure and local tiles share the same identity or theme. These results demonstrate that our framework can flexibly generate photomosaics that align with user-preferred visual targets, il- lustrating the adaptability of our approach to personalized content creation. 5 Conclusion We presented the first generative framework for photomosaic creation, replac- ing traditional matching-based pipelines with diffusion-based tile synthesis that preserves global structure and supports prompt-driven detail. Through low- frequency–conditioned guidance and color distribution matching, our method produces coherent, diverse, and user-adaptive photomosaics without requiring large tile collections. The results demonstrate the effectiveness of generative modeling for photomosaic composition and open new directions for controllable image assembly, including personalized photomosaic generation as a practical downstream application. References 1. Battiato, S., Di Blasi, G., Farinella, G.M., Gallo, G., et al.: A survey of digital mo- saic techniques. In: Eurographics Italian Chapter Conference. p. 129–135 (2006) Generative Phomosaic with Structure-Aligned and Personalized Diffusion15 2. Chang, P., Tang, J., Gross, M., Azevedo, V.C.: How i warped your noise: a temporally-correlated noise prior for diffusion models. In: The Twelfth Interna- tional Conference on Learning Representations (2024), https://openreview.net/ forum?id=pzElnMrgSD 3. Chen, S., Pan, Z., Cai, J., Phung, D.: Para: Personalizing text-to-image diffusion via parameter rank reduction. arXiv preprint arXiv:2406.05641 (2024) 4. Finkelstein, A., Range, M.: Image mosaics. In: International conference on raster imaging and digital typography. p. 11–22. Springer (1998) 5. Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., Cohen-Or, D.: An image is worth one word: Personalizing text-to-image gener- ation using textual inversion. arXiv preprint arXiv:2208.01618 (2022) 6. Geng, D., Park, I., Owens, A.: Factorized diffusion: Perceptual illusions by noise de- composition. In: European Conference on Computer Vision. p. 366–384. Springer (2024) 7. Gu, Y., Wang, X., Wu, J.Z., Shi, Y., Chen, Y., Fan, Z., Xiao, W., Zhao, R., Chang, S., Wu, W., et al.: Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Systems 36, 15890–15902 (2023) 8. He, Y., Zhou, J., Yuen, S.Y.: Composing photomosaic images using clustering based evolutionary programming. Multimedia Tools and Applications 78(18), 25919– 25936 (2019) 9. Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626 (2022) 10. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022) 11. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE international conference on computer vision (2017) 12. Jiang, J., Zhang, Y., Feng, K., Wu, X., Li, W., Pei, R., Li, F., Zuo, W.: Mcˆ 2: Multi-concept guidance for customized multi-concept generation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 2802–2812 (2025) 13. Kodaira, A., Xu, C., Hazama, T., Yoshimoto, T., Ohno, K., Mitsuhori, S., Sug- ano, S., Cho, H., Liu, Z., Tomizuka, M., et al.: Streamdiffusion: A pipeline-level solution for real-time interactive generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 12371–12380 (2025) 14. Kong, Z., Zhang, Y., Yang, T., Wang, T., Zhang, K., Wu, B., Chen, G., Liu, W., Luo, W.: Omg: Occlusion-friendly personalized multi-concept generation in diffu- sion models. In: European Conference on Computer Vision. p. 253–270. Springer (2024) 15. Kumari, N., Zhang, B., Zhang, R., Shechtman, E., Zhu, J.Y.: Multi-concept cus- tomization of text-to-image diffusion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 1931–1941 (2023) 16. Lee, H.Y.: Generation of photo-mosaic images through block matching and color adjustment. International Journal of Computer and Information Engineering 8(3), 457–460 (2014) 17. Lee, J., Kang, M., Han, B.: Diffusion-based image-to-image translation by noise correction via prompt interpolation. In: European Conference on Computer Vision. p. 289–304. Springer (2024) 16J. Chung et al. 18. Lee, Y., Kim, K., Kim, H., Sung, M.: Syncdiffusion: Coherent montage via syn- chronized joint diffusions. Advances in Neural Information Processing Systems 36, 50648–50660 (2023) 19. Li, C.L., Su, Y., Wang, R.Z.: Generating photomosaics with qr code capability. Mathematics 8(9), 1613 (2020) 20. Li, J., Li, D., Xiong, C., Hoi, S.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: International confer- ence on machine learning. p. 12888–12900. PMLR (2022) 21. Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., Shan, Y.: T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In: Proceedings of the AAAI conference on artificial intelligence (2024) 22. Po, R., Yang, G., Aberman, K., Wetzstein, G.: Orthogonal adaptation for modular customization of diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 7964–7973 (2024) 23. Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023) 24. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. p. 8748–8763. PmLR (2021) 25. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 10684–10695 (2022) 26. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022) 27. Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. p. 22500–22510 (2023) 28. Ryu, S.: Low-rank adaptation for fast text-to-image diffusion fine-tuning. Low-rank adaptation for fast text-to-image diffusion fine-tuning 3 (2023) 29. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text- to-image diffusion models with deep language understanding. Advances in neural information processing systems 35, 36479–36494 (2022) 30. Silvers, R.: Photomosaics: putting pictures in their place. Ph.D. thesis, Mas- sachusetts Institute of Technology (1996) 31. Silvers, R.: Photomosaics. Henry Holt and Co., Inc. (1997) 32. Simsar, E., Hofmann, T., Tombari, F., Yanardag, P.: Loraclr: Contrastive adapta- tion for customization of diffusion models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 13189–13198 (2025) 33. Wang, J., Chan, K.C., Loy, C.C.: Exploring clip for assessing the look and feel of images. In: Proceedings of the AAAI conference on artificial intelligence. vol. 37, p. 2555–2563 (2023) 34. WangZhou, B., Sheikh, H.R., et al.: Image qualityassessment: From errorvisibili- tytostructural similarity. IEEE Transon ImageProcessing 13(4), 600 (2004) 35. Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., Li, H.: Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 (2023) Generative Phomosaic with Structure-Aligned and Personalized Diffusion17 36. Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Imagere- ward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems 36, 15903–15935 (2023) 37. Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 3836–3847 (2023) 38. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 586–595 (2018) 18J. Chung et al. A Implementation Details A.1 Evaluation Details We evaluate all methods using 12 global and local prompt sets and 15 fixed ran- dom seeds to ensure reproducibility and fair comparison of quantitative metrics. The prompts used for photomosaic generation are shown in Figure 11. “Global Prompt” A cat with flower background A close-up photo of a dog in the forest A close-up photo of a teddy bear sitting on a bed, full body A car parked in front of a mountain A colorful bird sitting on a branch, surrounded by detailed leaves and trees A castle on top of a hill surrounded by clouds, dramatic lighting An anime girl with pink hair smiling under cherry blossoms, soft lighting A peaceful mountain landscape at sunrise, clear atmosphere A close-up of colorful coral reef underwater with fish swimming around A rustic coffee shop interior with wooden furniture A futuristic city at night with flying cars and neon lights A delicious hamburger with melting cheese and sesame bun “Local Prompt” A cat A dog A teddy bear A car A bird A chair An anime girl face A flower A fish A coffee cup A robot A hamburger Fig. 11: Global and Local Prompt Sets for Quantitative Evaluation. B Additional Experiments B.1 Comparison to Factorized Diffusion We conduct additional experiment by applying a recent technique, factorized diffusion (FD) [6] to generate photomosaic image. Factorized diffusion decom- poses an image into multiple components and performs diffusion sampling by conditioning each component on different prompts. This baseline relies on pixel diffusion, whereas our method and existing baselines perform diffusion in the latent space, we additionally report its results in Table 5 and Figure 12. We implement this baseline with its default settings within photomosaic framework. Factorized diffusion achieves strong global structural fidelity, but exhibits weak local tile semantic alignment. In particular, many tiles fail to generate the in- tended concepts, and the resulting tile quality is often degraded. Table 5: Comparison to Factorized Diffusion. Ref. Image (Global)Tile Image (Local) PSNR ↑ SSIM ↑ LPIPS ↓ HPSv2 ↑ IR ↑ BLIP ↑ CLIP ↑ IQA ↑ HPSv2 ↑ IR ↑ FD27.007 0.937 0.010 0.134 -2.1740.444 0.602 0.383 0.174 -1.574 Ours 17.598 0.551 0.121 0.132 -2.2250.569 0.619 0.812 0.239 -0.317 Generative Phomosaic with Structure-Aligned and Personalized Diffusion19 Fig. 12: Qualitative Result of Factorized Diffusion. Table 6: Low-Frequency Alignment Evaluation. Level-1 Level-2 Level-3 Level-4 Match & Tone [15]0.026 0.016 0.012 0.008 Color ControlNet [20]0.107 0.088 0.072 0.054 AdaIN [10]0.044 0.025 0.018 0.012 NoiseBlend [16]0.067 0.014 0.008 0.005 StreamDiffI2I [12]0.025 0.016 0.013 0.011 Ours0.025 0.009 0.005 0.003 B.2 Low-Frequency Alignment Analysis We evaluate global structural fidelity using a multi-scale Gaussian pyramid and measure the coarse-scale discrepancy between the reference and generated mo- saic images. We construct a pyramid by recursively applying Gaussian smooth- ing and 2× downsampling, producing progressively coarser representations that preserve only large-scale shape and tonal structure. We denote these pyramid levels as Level-1 through Level-4, each representing a lower-frequency band and capturing increasingly broader structural components of the image. Since high-frequency details are removed at coarser levels, this measure isolates global structural alignment. For each pyramid level k, we quantify the structural dif- ference between the reference and the mosaic image using mean squared error: E k =∥G k (x)− G k (ˆx)∥ 2 2 , (11) where x and ˆx denote the reference and mosaic images, and G k (·) denotes the k- th Gaussian-pyramid level. The metric E k reflects the discrepancy in large-scale structure and tone after high-frequency components have been removed. Lower values indicate better global structural consistency. As shown in Table 6, our method consistently achieves the lowest error, demonstrating superior alignment at coarse spatial scales. B.3 Computation Cost and Runtime We report the time required to construct a photomosaic composed of 64 tiles in Table 7, measured on an NVIDIA A6000 GPU; Model loading/image pool 20J. Chung et al. generation time are excluded. We employ Stable Diffusion 2.1 in our implemen- tation. However, we expect that incorporating fast generation strategies based on few-step diffusion models, such as SDXL-Turbo and LCM, could enable more efficient photomosaic generation in future work. Table 7: Runtime Comparison. M&T ControlNet AdaIN NoiseBlend StreamDiffI2I FDOurs runtime(s)1.81271891873.94361088 C Limitations and Future Work While this study primarily addresses reference image-conditioned tile image gen- eration, an interesting extension would be to incorporate a small set of real tile images as structural conditions for specific regions of the reference image. Such integration would allow the generated photomosaic to embed real imagery, en- hancing both its semantic coherence and artistic meaning. Exploring this setup could further enable fine-grained regional control, opening opportunities for more intentional photomosaic design. Additionally, video mosaics are often constructed using matching and tone- mapping approaches. In such cases, aligning motion becomes difficult, making the problem considerably more challenging. If a generative model could be used to create a video mosaic that accounts for both coarse structural consistency and optical flow, this could also be an interesting direction to explore. Societal Impact Our generative photomosaic framework may raise several so- cietal concerns. First, malicious users could embed misleading or manipulative content within photomosaic structures, exploiting their ability to hide detailed local imagery inside a global scene. Second, using personal or copyrighted im- ages as tiles poses risks of privacy violation and unauthorized identity or content misuse.