Paper deep dive

Dynamic Training-Free Fusion of Subject and Style LoRAs

Qinglong Cao, Yuntian Chen, Chao Ma, Xiaokang Yang

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 50

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/21/2026, 3:22:52 AM

Summary

The paper proposes a dynamic, training-free framework for fusing Subject and Style Low-Rank Adaptation (LoRA) models in diffusion-based image generation. Unlike static weight-heuristic methods, this approach dynamically selects LoRA weights during the forward pass by computing KL divergence between base and fine-tuned features to maximize feature perturbation. Additionally, it refines the generation trajectory during the reverse denoising stage using gradient-based corrections derived from objective metrics like CLIP and DINO scores, ensuring coherent subject-style synthesis without retraining.

Entities (10)

LoRA · technique · 99%Dynamic Training-Free Fusion Framework · method · 98%CLIP · modelmetric · 95%DINO · modelmetric · 95%KL Divergence · metric · 92%FLUX · base-model · 90%Stable Diffusion XL · base-model · 90%B-LoRA · baseline-method · 88%K-LoRA · baseline-method · 88%ZipLoRA · baseline-method · 88%

Relation Signals (8)

Dynamic Training-Free Fusion Framework → uses → KL Divergence

confidence 95% · we dynamically compute the KL divergence between the base model's original features and those produced by subject and style LoRAs

Dynamic Training-Free Fusion Framework → uses → CLIP

confidence 95% · dynamically applying gradient-based corrections derived from objective metrics such as CLIP and DINO scores

Dynamic Training-Free Fusion Framework → uses → DINO

confidence 95% · dynamically applying gradient-based corrections derived from objective metrics such as CLIP and DINO scores

Dynamic Training-Free Fusion Framework → appliedto → FLUX

confidence 90% · applied to both the Stable Diffusion XL v1.0 base model and the FLUX model

Dynamic Training-Free Fusion Framework → appliedto → Stable Diffusion XL

confidence 90% · applied to both the Stable Diffusion XL v1.0 base model

Dynamic Training-Free Fusion Framework → outperforms → B-LoRA

confidence 90% · our approach consistently outperforms state-of-the-art LoRA fusion methods

Dynamic Training-Free Fusion Framework → outperforms → ZipLoRA

confidence 90% · our approach consistently outperforms state-of-the-art LoRA fusion methods

Dynamic Training-Free Fusion Framework → →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Recent studies have explored the combination of multiple LoRAs to simultaneously generate user-specified subjects and styles. However, most existing approaches fuse LoRA weights using static statistical heuristics that deviate from LoRA's original purpose of learning adaptive feature adjustments and ignore the randomness of sampled inputs. To address this, we propose a dynamic training-free fusion framework that operates throughout the generation process. During the forward pass, at each LoRA-applied layer, we dynamically compute the KL divergence between the base model's original features and those produced by subject and style LoRAs, respectively, and adaptively select the most appropriate weights for fusion. In the reverse denoising stage, we further refine the generation trajectory by dynamically applying gradient-based corrections derived from objective metrics such as CLIP and DINO scores, providing continuous semantic and stylistic guidance. By integrating these two complementary mechanisms-feature-level selection and metric-guided latent adjustment-across the entire diffusion timeline, our method dynamically achieves coherent subject-style synthesis without any retraining. Extensive experiments across diverse subject-style combinations demonstrate that our approach consistently outperforms state-of-the-art LoRA fusion methods both qualitatively and quantitatively.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

48,691 characters extracted from source content.

Expand or collapse full text

Dynamic Training-Free Fusion of Subject and Style LoRAs Qinglong Cao1,2 Yuntian Chen2 Chao Ma1 Xiaokang Yang1 1Shanghai Jiao Tong University 2Eastern Institute of Technology, Ningbo *Corresponding author Corresponding author. Abstract Recent studies have explored the combination of multiple LoRAs to simultaneously generate user-specified subjects and styles. However, most existing approaches fuse LoRA weights using static statistical heuristics that deviate from LoRA’s original purpose of learning adaptive feature adjustments and ignore the randomness of sampled inputs. To address this, we propose a dynamic training-free fusion framework that operates throughout the generation process. During the forward pass, at each LoRA-applied layer, we dynamically compute the KL divergence between the base model’s original features and those produced by subject and style LoRAs, respectively, and adaptively select the most appropriate weights for fusion. In the reverse denoising stage, we further refine the generation trajectory by dynamically applying gradient-based corrections derived from objective metrics such as CLIP and DINO scores, providing continuous semantic and stylistic guidance. By integrating these two complementary mechanisms—feature-level selection and metric-guided latent adjustment—across the entire diffusion timeline, our method dynamically achieves coherent subject-style synthesis without any retraining. Extensive experiments across diverse subject–style combinations demonstrate that our approach consistently outperforms state-of-the-art LoRA fusion methods both qualitatively and quantitatively. 1 Introduction Figure 1: (a) Existing methods directly rely on properties of LoRA weights to achieve fusion. (b) Our method integrates feature-level selection in the forward pass and latent-level refinement in the reverse process to enable dynamic training-free LoRAs fusion. Diffusion models have demonstrated remarkable performance across a wide range of generative tasks [7, 20, 14, 40, 45, 26, 25]. Among these, personalized image generation [30, 34, 41] has garnered increasing attention, as it requires the model to synthesize high-quality images that reflect user-specified content or style. Here, content refers to the semantic structure and subject identity, while style captures visual properties such as color, texture, and patterns. Although substantial progress has been made in generating images conditioned on either content or style alone, producing images that faithfully integrate both a specific subject and a specific style remains a challenging and unsolved problem. Recently, Low-Rank Adaptation (LoRA) [18] has emerged as a popular and versatile technique for parameter-efficient fine-tuning, making it particularly appealing for personalized generation tasks. Leveraging the modular nature of LoRA, recent studies have explored the fusion of independently fine-tuned LoRAs to jointly generate specific subjects in specific styles. For example, ZipLoRA [32] proposes leveraging coefficient vectors to merge content and style LoRAs in each LoRA-applied layer. Differently, B-LoRA [11] investigates the impact of diverse LoRA layers and finds that modifying two distinct LoRA layers can effectively control the content and style of generated images. Furthermore, focusing on the intrinsic characteristics of LoRA weights, K-LoRA [28] selects LoRAs in each layer by comparing the Top-K elements of the weights. While these methods have demonstrated promising performance in LoRA fusion, as shown in Figure 1 (a), their core strategies remain grounded in statistical properties of LoRA weights, which diverge from the original intent of LoRA—learning additional features to adapt to diverse functions. This divergence suggests that the fine-tuned features themselves, rather than the LoRA weights alone, are the true key to effective fusion. More importantly, previous static fusion methods ignore the randomness of sampled latent inputs during generation, limiting their adaptability and leading to suboptimal performance. Inspired by these, as shown in Figure 1 (b), we propose a dynamic training-free fusion framework that operates throughout the diffusion process. In contrast to prior work that treats LoRA fusion as a static weight blend, our approach reinterprets fusion as a dynamic representation-aware decision process aligned with the generative dynamics of diffusion models. In particular, the previous method [28] argues that the absolute values of LoRA weights indicate their importance in the diffusion process. By contrast, we propose that the feature changes induced by LoRAs serve as a more direct and key indicator of their impact. In each LoRA-applied layer during the forward pass, we compute the fine-tuned features from both the style LoRAs and the content LoRAs, respectively. To better quantify the extent of feature change, we leverage the Kullback-Leibler (KL) divergence between the fine-tuned features and the original features, determining which LoRA is more suitable for each layer based on the magnitude of distributional change. In this way, though the sampled inputs vary, the most representative content and style features are adaptively retained in each LoRA-applied layer. Crucially, this layer-wise selection constitutes only the first half of our dynamic fusion strategy. To ensure global coherence and high-fidelity synthesis, we further introduce a metric-driven refinement mechanism that dynamically operates throughout the reverse denoising process. Specifically, we first generate two reference images using the subject and style LoRAs independently—serving as semantic and stylistic anchors. During denoising, at each timestep, we evaluate the intermediate prediction against these references using objective metrics such as CLIP [29] and DINO [5] scores, which quantify content fidelity and style consistency, respectively. These objective metrics are then used to compute a composite guidance signal, whose gradient is applied to guide the latent trajectory toward regions that better align with the desired subject–style composition. This training-free, step-wise correction enables continuous refinement without any additional supervision. By integrating feature-level selection in the forward pass and latent-level refinement in the reverse process, our method achieves effective integration of subject and style LoRAs, enabling high-quality image generation that preserves both subject fidelity and stylistic accuracy in a training-free manner. The main contributions of this work are summarized as follows: • We introduce a dynamic training-free LoRA fusion framework that shifts the paradigm from static weight-level heuristics to input-adaptive, representation-aware decisions throughout the generation process. • A KL divergence-based strategy adaptively selects the most informative LoRA at each layer based on feature perturbation magnitude. Complementarily, objective metric scores provide metric-guided refinement during denoising, enhancing both semantic and stylistic fidelity. • Our method is fully training-free, plug-and-play, and demonstrates superior performance across diverse subject–style combinations on multiple benchmarks, without requiring retraining or additional supervision. 2 Related work Diffusion Models for Custom Generation. With the rapid development of diffusion models [15, 4, 16, 25, 17, 42, 9], many researchers have introduced diverse approaches to fine-tuning large-scale diffusion models for custom generation, which aims to produce images of user-specified subjects or styles based on language descriptions. For instance, Textual Inversion [12] focuses on optimizing a single word embedding to capture unique and varied concepts. DreamBooth [30] designs text prompts containing a unique identifier to more effectively generate images with the desired subjects. CustomDiffusion [23] fine-tunes the cross-attention layers within the diffusion model to learn multiple concepts simultaneously. Additionally, some methods [2, 33, 38, 37] achieve custom generation without additional training, yet these typically target specific single tasks. Recently, parameter-efficient fine-tuning techniques such as LoRA [18] and StyleDrop [34] have gained popularity due to their ability to fine-tune models with low-rank adaptations, making them attractive for custom generation. LoRAs combination for image generation. Since the rise in popularity of LoRA applications [44, 46, 47], many studies on LoRA combinations have been proposed. Some methods [8, 13, 19, 39] focus on fusing multiple object LoRAs, enabling diffusion models to generate various new concepts and replace these objects through masking strategies. Meanwhile, several advanced methods address content-style LoRA fusion. For instance, Mixture-of-Subspaces [36] designs learnable mixer weights to fuse various LoRAs; ZipLoRA [32] leverages merge vectors across varying layers to linearly combine subject and style LoRAs; B-LoRA [11] investigates the impact of different LoRA layers and finds modifying two distinct layers can effectively control the content and style of generated images; and K-LoRA [28] selects the appropriate LoRAs in each layer by comparing top-K elements of different LoRA weights. Although these methods have shown promising performance, they directly rely on the properties of LoRA weights. The original intent of LoRA is to learn additional features to adapt to diverse tasks. Previous methods ignore the input randomness. Thus, dynamic fusion methods based on fine-tuned features could be more effective. To this end, we introduce a dynamic training-free LoRA fusion paradigm with input-adaptive, representation-aware decisions. 3 Preliminaries Diffusion Models. Diffusion models [31, 22, 6, 1] have demonstrated impressive performance across various generative tasks. They mainly consist of a forward noise addition process and a reverse denoising process. During the forward process, the original image is progressively transformed into Gaussian noise through incremental noise addition. In the reverse process, conditioned on paired text prompts, the diffusion network, typically a U-Net, gradually denoises the noisy input step-by-step, starting from randomly sampled pure noise. At inference time, the trained diffusion network achieves text-to-image generation based on the given textual input. LoRA. Low-Rank Adaptation (LoRA [18]) is a lightweight fine-tuning technique originally developed for large language and diffusion models. Rather than updating the full parameter matrix W0W_0, LoRA exploits the observation that the update ΔW∈ℝm×n W ^m× n often lies in a low-dimensional subspace. Concretely, one factorizes ΔW=BA W=BA with B∈ℝm×rB ^m× r and A∈ℝr×nA ^r× n for r≪min⁡(m,n)r (m,n), and only A,BA,B are learned while W0W_0 remains fixed. The tuned model thus uses weights W0+BAW_0+BA. In our setting, let D be a diffusion model with base weights W0W_0. To capture a new concept, we train a LoRA pair ΔWx W_x so that the adapted model is Dx=W0+ΔWxD_x=W_0+ W_x. Dynamic Representation Guidance. The primary goal of LoRA is to learn fine-grained feature adjustments that guide a diffusion model toward specific behaviors. To leverage this capability, we first generate the modified features produced by both the content and style LoRAs. While prior work [28] uses the static magnitudes of LoRA weight updates as a proxy for their influence, we argue that the actual dynamic change in feature distributions provides a more direct and interpretable measure of impact since the randomly sampled inputs. We therefore quantify this change via Kullback–Leibler (KL) divergence between the original and LoRA-modified features, and use it to select the most informative branch for fusion at each layer. In addition, objective metrics such as CLIP and DINO scores offer an effective way to assess the quality of LoRA fusion. Higher scores indicate better alignment with the intended semantics or style, and thus can serve as composite guidance signal to guide the diffusion network. The guidance score R based on the CLIP metric is computed as: R(x^0)=1−SCLIP(xref,x^0),R( x_0)=1-S_CLIP(x_ref, x_0), (1) where SCLIPS_CLIP computes the CLIP similarity score, xrefx_ref is the reference image, and x^0 x_0 is the predicted original image at step t. The guiding scores act as residuals that can be treated as virtually observed values [21] with R^=0 R=0 and virtual likelihood: p(R^=0∣xt)=(0∣R(x0),σr2I),p( R=0 x_t)=N(0 R(x_0), _r^2I), (2) where xtx_t is the intermediate result at timestep t, and σr _r is a predefined constant controlling the enforcement strength of the virtual observation. Although residuals are minimized during the generation process, they are not guaranteed to reach zero. To incorporate this guidance into the diffusion process, we apply Bayesian rule: p(xt∣R^=0)=p(xt)p(R^=0∣xt)p(R^=0).p(x_t R=0)= p(x_t)\,p( R=0 x_t)p( R=0). (3) Taking the gradient of the log-likelihood with respect to xtx_t, we obtain: ∇xtlog⁡p(xt∣R^=0)=∇xtlog⁡p(xt)+∇xtlog⁡p(R^=0∣xt), split _x_t p(x_t R=0)=& _x_t p(x_t)\\ &+ _x_t p( R=0 x_t), split (4) where the first term is the standard score function predicted by the diffusion model [35]. For the second term, substituting Eq. 2 yields: ∇xtlog⁡p(R^=0∣xt) _x_t p( R=0 x_t) =−1σr2∇xt∥R(x0)∥22 =- 1 _r^2 _x_t R(x_0) _2^2 ≈−1σr2∇xt∥R(x^0)∥22, ≈- 1 _r^2 _x_t R( x_0) _2^2, (5) where we approximate x0x_0 with x^0 x_0. In practice, because guidance scores range between [0,1][0,1], we simplify as: ∇xtlog⁡p(R^=0∣xt)∝−∇xtR(x^0). _x_t p( R=0 x_t) - _x_tR( x_0). (6) Thus, to implement the guidance, the denoising step is modified as: xt−1=xt−1ori−m∇xtR(x^0),x_t-1=x_t-1^ori-m _x_tR( x_0), (7) where xt−1orix_t-1^ori is the original output of the t-step denoising process, x^0 x_0 is the predicted original image at step t, and m is a predefined scaling factor. Figure 2: Overview of our method. By performing dynamic feature selection based on representation perturbation and applying metric-guided refinement throughout the denoising process, our framework enables training-free fusion of subject and style LoRAs. 4 Method As illustrated in Figure 2, our framework guides a base diffusion model D to generate a specified subject in a specified style by dynamically fusing content and style LoRAs without any additional training. Let the base model D consist of pre-trained weights W0iW_0^i at layer i. Applying the LoRA LxL_x with weight updates ΔWxi\ W_x^i\ yields the adapted model: DLx=D⊙Lx=W0+ΔWx.D_L_x=D L_x=W_0+ W_x. (8) In the experimental setting, we are given the content LoRA weights LcΔWciL_c\ W_c^i\, the style LoRA weights LsΔWsiL_s\ W_s^i\, and the base model D. Rather than merging LoRA weights heuristically, we perform dynamic, representation-aware fusion throughout the diffusion process: at each LoRA-applied layer, we select the branch that induces more significant feature perturbation; during denoising, we refine the latent trajectory using objective metric scores. This two-stage, dynamic training-free strategy enables coherent integration of subject and style in a single forward pass. 4.1 Feature-Level Selection During Forward Pass Prior work has used the absolute values of LoRA weight updates as a proxy for their importance in diffusion models [28]. However, the core function of LoRA is to induce feature-level adjustments rather than merely altering weight magnitudes, and the static methods ignore the randomness of sampled inputs. Motivated by this principle, we explicitly examine how content and style LoRA weight updates, ΔWci W_c^i and ΔWsi W_s^i, affect the base network’s i-th layer. Specifically, we apply these updates to the base weights W0iW_0^i and compute the corresponding fine-tuned feature maps: F^ci+1 F_c^\,i+1 =(W0i+ΔWci)Fi, = (W_0^i+ W_c^i )\,F_i, (9) F^si+1 F_s^\,i+1 =(W0i+ΔWsi)Fi, = (W_0^i+ W_s^i )\,F_i, (10) where FiF_i represents the original features at layer i. To quantify the impact of these modifications, we compute the KL divergence between each fine-tuned feature distribution and the original feature distribution Fi+1F_i+1: dci d_c^i =KL(F^ci+1∥Fi+1), =KL ( F_c^\,i+1\,\|\,F_i+1 ), (11) dsi d_s^i =KL(F^si+1∥Fi+1). =KL ( F_s^\,i+1\,\|\,F_i+1 ). (12) We then compare dcid_c^i and dsid_s^i to determine which adjustment induces a more significant feature change: Fi+1=F^ci+1,if dci≥dsi,F^si+1,otherwise.F^\,i+1\;=\; cases F_c^\,i+1,&if d_c^i≥ d_s^i,\\ F_s^\,i+1,&otherwise. cases (13) By performing this selection at each layer, we retain the most impactful content or style information, enabling a dynamic, training-free fusion of subject and style LoRAs. In contrast to weight-based fusion strategies, which are static and input-agnostic, our feature-based approach dynamically adapts to the input: as the prompt changes, so do the feature distributions and thus the fusion decisions. This input-conditional mechanism allows our method to flexibly handle generation tasks with diverse and evolving requirements. 4.2 Latent-Level Refinement In Denoising Stage As discussed in the preliminaries, objective metrics such as CLIP and DINO scores effectively assess the quality of LoRA fusion, where higher scores indicate better alignment with the desired content or style. To obtain the guidance scores, we first leverage the content LoRAs ΔWc W_c and style LoRAs ΔWs W_s with content descriptions lcl_c and style descriptions lsl_s to generate the new reference content image IcrefI_c^ref and reference style image IsrefI_s^ref: Icref=(W0+ΔWc)[lc]I_c^ref=(W_0+ W_c)[l_c] (14) Isref=(W0+ΔWs)[ls]I_s^ref=(W_0+ W_s)[l_s] (15) With the original output xt−1orix_t-1^ori at the t-th denoising step and the predicted original image x^0 x_0 at step t, we compute the CLIP and DINO scores [32] to evaluate the prediction performance. The scores are obtained by extracting embeddings from the generated and reference images and computing cosine similarity: SCLIPcontent=Simcos(ECLIP(Icref),ECLIP(x^0))S_CLIP^content=Sim_cos (E_CLIP(I_c^ref),E_CLIP( x_0) ) (16) SCLIPstyle=Simcos(ECLIP(Isref),ECLIP(x^0))S_CLIP^style=Sim_cos (E_CLIP(I_s^ref),E_CLIP( x_0) ) (17) SDINOstyle=Simcos(EDINO(Isref),EDINO(x^0))S_DINO^style=Sim_cos (E_DINO(I_s^ref),E_DINO( x_0) ) (18) where ECLIPE_CLIP and EDINOE_DINO denote the image encoders of CLIP [29] and DINO [5], respectively. To compute the final guidance score, we evenly weight the three metrics: R(x^0)=1−SCLIPcontent+SCLIPstyle+SDINOstyle3R( x_0)=1- S_CLIP^content+S_CLIP^style+S_DINO^style3 (19) With guidance scores and empirically setting scaling factor m=10m=10, we guide the diffusion step as: xt−1=xt−1ori−m∇xtR(x^0)x_t-1=x_t-1^ori-m _x_tR( x_0) (20) This training-free refinement is dynamically applied at every denoising step, providing continuous, objective feedback that steers the generation toward both subject fidelity and stylistic accuracy—without any retraining or additional supervision. In summary, our method achieves dynamic training-free subject–style LoRA fusion by operating throughout the diffusion process: it performs dynamic feature selection in the forward pass based on KL divergence to retain the most informative representation at each layer, and applies metric-guided latent refinement during denoising using objective metrics feedback to ensure global semantic and stylistic coherence. This dual-stage, input-adaptive strategy requires no retraining, yet enables high-fidelity, plug-and-play composition of independently trained LoRAs. Method Style Sim ↑ CLIP Score ↑ DINO Score ↑ Direct 48.9% 66.6% 43.0% B-LoRA [11] 58.0% 63.8% 30.6% ZipLoRA [32] 60.4% 64.4% 35.7% K-LoRA [28] 58.7% 69.4% 46.9% Ours 63.0% 78.5% 43.3% Table 1: Comparison of alignment results. Direct denotes direct arithmetic merging. 5 Experiments Method User Preference GPT-4o Feedback Qwen2.5-VL Feedback ZipLoRA [32] 13.80% 20.13% 3.40% B-LoRA [11] 21.89% 11.67% 9.11% K-LoRA [28] 11.11% 12.56% 21.82% Ours 53.20% 55.64% 65.67% Table 2: The Performance Comparison of user study results, GPT-4o and Qwen2.5-VL feedback. Figure 3: Qualitative comparisons. We present images generated by our method and the compared advanced generation methods. Through nput-adaptive, representation-aware decisions throughout the generation process, our method effectively enables training-free fusion of subject and style LoRAs. We evaluate the proposed LoRAs fusion approach under the experimental setup established by previous methods including K-LoRA, ZipLoRA, B-LoRA. Specifically, our method is applied to both the Stable Diffusion XL v1.0 base model and the FLUX model. Datasets. For training the local LoRAs, we follow the convention of previous works [28, 32]. To train content LoRAs, we select diverse image sets from the DreamBooth dataset [30], where each instance is represented by 4–5 images. For style LoRAs, we adopt the dataset introduced by the StyleDrop authors [34], which includes a wide variety of stylistic exemplars spanning classical art to modern creative styles. Each style LoRA is trained using a single reference image. Implementation Details.To obtain local LoRAs for experiments, we adopt the K-LoRA [28] strategy to fine-tune the SDXL v1.0 base model using a low-rank adaptation with rank set to 64. The LoRA weights—both style and content—are optimized using the Adam optimizer over 1000 steps with a batch size of 1 and a learning rate of 5e-5. For the FLUX model, we utilize publicly available, well-trained community LoRA weights obtained from HuggingFace. The corresponding experimental results on FLUX are provided in the supplementary material. 5.1 Results 5.1.1 Quantitative Comparisons To objectively evaluate the performance of our training-free generation method, we adopt commonly used metrics from prior works, including Style Similarity, CLIP Score, and DINO Score, to assess the quality of the generated images. Following previous methods, we randomly selected 30 unique content–style pairs, each of which consists of 10 images to perform quantitative comparisons. Specifically, CLIP [29] is employed to evaluate both the style alignment (Style Sim) and content preservation (CLIP Score), while DINO [43] is used to measure content consistency via DINO Score. Table 1 presents a detailed comparison between our method and existing state-of-the-art methods. Our method achieves the best performance in both Style Sim (63.0%) and CLIP Score (78.5%). Notably, it provides a substantial improvement of 9.1% in CLIP Score compared to the strongest baseline. Although our method does not achieve the top performance in DINO Score, it still ranks second, demonstrating a strong overall balance between style and content fidelity. These results validate the effectiveness of our dynamic fusion method. As a supplementary verification, we report further evaluations with extra metrics in the supplmentary material, which also demonstrate the effectiveness of our proposed method. Figure 4: Robustness Analysis. We present images generated by our method and K-LoRA with random seeds to analyze the robustness. 5.1.2 User Study and MLLM-based Evaluations To further assess the perceptual quality of generated images beyond conventional metrics, we conduct a comprehensive user study and multimodal large language model (MLLM)-based evaluations. As shown in Table 2, we collect human preferences and automatic feedback from two strong MLLMs, GPT-4o [27] and Qwen2.5-VL [3]. In the user study, participants were asked to choose their preferred images from outputs of four competing methods. Our proposed method is overwhelmingly favored, receiving 53.20% of total votes, outperforming all baselines. Similarly, in LLM-based evaluations, our proposed method is consistently ranked highest, achieving 55.64% preference by GPT-4o and an even more substantial 65.67% by Qwen2.5-VL. These results not only confirm the good quality of generated images in terms of human preference but also highlight our method’s effectiveness in producing stylistically and semantically coherent outputs that align well with multi-modal models. More setup details are provided in the supplementary material. Method Style Sim ↑ CLIP Score ↑ DINO Score ↑ Baseline 60.3% 75.6% 40.1% Only with FLS 59.5% 75.9% 43.7% Only with LLR 66.1% 78.9% 40.7% Ours(FLS+LLR) 64.0% 79.1% 43.4% Table 3: The ablation study of different components, including Feature-Level Selection (FLS) and Latent-Level Refinement (LLR). 5.1.3 Qualitative comparisons To visually assess the performance of different LoRA fusion methods, we present qualitative comparisons in Figure 3. Overall, our training-free LoRA fusion method demonstrates superior visual quality, effectively preserving both content and style information. In contrast, most existing methods tend to retain content reasonably well but struggle to capture the target style faithfully. For example, in the second row and third column, B-LoRA correctly identifies the ‘dog’ content but incorrectly applies a pink color inconsistent with the reference style. A similar issue is observed with K-LoRA in the fourth row and third column. Moreover, in the second-to-last row, K-LoRA produces a ‘cat’ whose head and body exhibit inconsistent styles, indicating a failure in achieving global style coherence. In the eighth row, K-LoRA also fails to preserve the oil painting style entirely. Interestingly, Zip-LoRA, despite being the only method with learnable parameters, performs relatively worse. It often fails to capture the desired style (e.g., the ‘dog’ in the fourth column) or generates semantically inaccurate content. These qualitative results further support the effectiveness of our dynamic fusion strategy throughout the generation process, which consistently delivers visually coherent outputs without the need for additional training. Divergence Style Sim ↑ CLIP Score ↑ DINO Score ↑ KL 59.5% 75.9% 43.7% JS 59.3% 75.3% 43.1% Cosine Similarity 58.9% 75.4% 43.3% Dot Product 58.4% 74.9% 43.8% Table 4: The ablation study of selection criteria in feature-level selection, including Kullback-Leibler (KL), Jensen-Shannon (JS) divergence, cosine similarity, and dot product. Scaling Factor m Style Sim ↑ CLIP Score ↑ DINO Score ↑ 1 62.1% 72.8% 32.4% 5 64.8% 74.1% 32.3% 10 64.0% 79.1% 43.4% 20 66.8% 76.8% 35.1% Table 5: The ablation study of scaling factor m for Latent-Level Refinement. Figure 5: LoRA Selection During the Generation Process. The vertical axis represents 50 diffusion steps, while the horizontal axis denotes varying LoRA layers. Dark green indicates the selection of subject LoRA, and light blue indicates the selection of style LoRA. 5.2 Ablation Studies To thoroughly analyze the contributions of each component in our dynamic training-free fusion framework, we conduct extensive ablation studies across three aspects: (1) the effect of feature-level selection and latent-level refinement, (2) the choice of feature divergence metric for selection, and (3) the impact of the guidance scaling factor. For each setting, we randomly sample 25 subject–style combinations to ensure robustness. Effect of Feature-Level Selection and Latent-Level Refinement. Table 3 presents the performance of different configurations. Using only feature-level selection (based on KL divergence) improves the DINO Score by 3.6% over the baseline, indicating better content consistency. In contrast, latent-level refinement alone significantly boosts Style Similarity (66.1%) and CLIP Score (78.9%), reflecting stronger semantic and stylistic alignment. When both components are combined, our method achieves the best overall performance—particularly in CLIP Score (79.1%) and DINO Score (43.4%)—demonstrating their complementary roles in local feature adaptation and global generation guidance. Choice of Selection Criteria. We evaluate alternative criteria for feature-level selection, including Kullback–Leibler (KL) divergence, Jensen–Shannon (JS) divergence, cosine similarity, and dot product (Table 4). Both KL and JS divergence yield superior performance compared to similarity-based metrics. KL divergence slightly outperforms JS in DINO Score (43.7% vs. 43.1%), likely because it is more sensitive to asymmetric distribution shifts—making it better suited for detecting meaningful feature perturbations in content representation. Effect of Scaling Factor. As shown in Table 5, we also explore the sensitivity of the scaling factor that balances the influence of latent-level refinement. Setting this factor too low (e.g., 1) leads to suboptimal CLIP and DINO Scores, while extremely high scaling (e.g., 20) causes instability in DINO Score. A moderate value of 10 yields the best trade-off, with a strong boost in both CLIP Score (79.1%) and DINO Score (43.4%), highlighting the importance of proper guidance calibration. Robustness Analysis. To further evaluate the robustness of our proposed framework, we visualize generated results under different random seeds and compare them with K-LoRA in Figure 4. While K-LoRA exhibits significant variability in scene layouts and fails to consistently preserve the target style, our method maintains both the semantic content and stylistic attributes across different sampling conditions. This stability under stochastic perturbations highlights the effectiveness of input-adaptive, representation-aware modulation in enforcing coherent and reliable image generation. Visualization of Dynamic Feature Selection. Figure 5 illustrates the dynamic feature-level selection mechanism employed by our method. Unlike K-LoRA, which relies solely on static weight elements for selection, our method performs input-conditional selection, dynamically choosing the more relevant LoRA weights based on input-dependent features. This adaptive strategy enables better alignment with the input semantics, thereby facilitating more effective and coherent style-content fusion. The superior performance observed in previous quantitative and qualitative comparisons further validates the advantages of this training-free, input-adaptive selection scheme. 6 Conclusion In this paper, we propose a dynamic training-free method for fusing subject and style LoRAs through put-adaptive, representation-aware decisions across the generation process. Specifically, during the forward pass, rather than relying on static weight-level heuristics, our approach adaptively selects LoRA branches at each layer based on the magnitude of feature perturbation—quantified via KL divergence. Moreover, in the denoising stage, objective metrics such as the CLIP and DINO scores are leveraged to dynamically refine the denoising trajectory using gradient-based feedback. This dual-stage strategy enables coherent integration of subject identity and artistic style without any retraining or additional supervision. Extensive experiments across multiple benchmarks demonstrate that our method consistently outperforms existing LoRA fusion approaches in both qualitative realism and quantitative metrics. References [1] T. Amit, T. Shaharbany, E. Nachmani, and L. Wolf (2021) Segdiff: image segmentation with diffusion probabilistic models. arXiv:2112.00390. Cited by: §3. [2] O. Avrahami, K. Aberman, O. Fried, D. Cohen-Or, and D. Lischinski (2023) Break-a-scene: extracting multiple concepts from a single image. In SIGGRAPH Asia 2023 Conference Papers, p. 1–12. Cited by: §2. [3] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin (2025) Qwen2.5-vl technical report. arXiv:2502.13923. Cited by: §5.1.2. [4] Q. Cao, D. Wang, X. Li, Y. Chen, C. Ma, and X. Yang (2024) Teaching video diffusion model with latent physical phenomenon knowledge. arXiv:2411.11343. Cited by: §2. [5] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin (2021) Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 9650–9660. Cited by: §1, §4.2. [6] S. Chen, P. Sun, Y. Song, and P. Luo (2023) Diffusiondet: diffusion model for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: §3. [7] S. Chen, M. Xu, J. Ren, Y. Cong, S. He, Y. Xie, A. Sinha, P. Luo, T. Xiang, and J. Perez-Rua (2024) Gentron: diffusion transformers for image and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6441–6451. Cited by: §1. [8] J. Dong, W. Liang, H. Li, D. Zhang, M. Cao, H. Ding, S. H. Khan, and F. Shahbaz Khan (2024) How to continually adapt text-to-image diffusion models for flexible customization?. Advances in Neural Information Processing Systems 37, p. 130057–130083. Cited by: §2. [9] Y. Dong, Z. Liu, H. Sun, J. Yang, W. Hu, Y. Rao, and Z. Liu (2025) Insight-v: exploring long-chain visual reasoning with multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 9062–9072. Cited by: §2. [10] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2021) An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations, Cited by: §1. [11] Y. Frenkel, Y. Vinker, A. Shamir, and D. Cohen-Or (2024) Implicit style-content separation using b-lora. In European Conference on Computer Vision, p. 181–198. Cited by: Table 1, §1, §2, §3, Table 1, Table 2. [12] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or (2022) An image is worth one word: personalizing text-to-image generation using textual inversion. arXiv:2208.01618. Cited by: §2. [13] Y. Gu, X. Wang, J. Z. Wu, Y. Shi, Y. Chen, Z. Fan, W. Xiao, R. Zhao, S. Chang, W. Wu, et al. (2023) Mix-of-show: decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Systems 36, p. 15890–15902. Cited by: §2. [14] A. Gupta, L. Yu, K. Sohn, X. Gu, M. Hahn, F. Li, I. Essa, L. Jiang, and J. Lezama (2024) Photorealistic video generation with diffusion models. In European Conference on Computer Vision, p. 393–411. Cited by: §1. [15] H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang (2024) Cameractrl: enabling camera control for text-to-video generation. arXiv:2404.02101. Cited by: §2. [16] H. He, C. Yang, S. Lin, Y. Xu, M. Wei, L. Gui, Q. Zhao, G. Wetzstein, L. Jiang, and H. Li (2025) CameraCtrl i: dynamic scene exploration via camera-controlled video diffusion models. arXiv:2503.10592. Cited by: §2. [17] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet (2022) Video diffusion models. Advances in Neural Information Processing Systems 35, p. 8633–8646. Cited by: §2. [18] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. (2022) Lora: low-rank adaptation of large language models.. International Conference on Learning Representations 1 (2), p. 3. Cited by: §1, §2, §3. [19] J. Jiang, Y. Zhang, K. Feng, X. Wu, W. Li, R. Pei, F. Li, and W. Zuo (2024) MC2: multi-concept guidance for customized multi-concept generation. arXiv:2404.05268. Cited by: §2. [20] Z. Jiang, C. Mao, Y. Pan, Z. Han, and J. Zhang (2024) Scedit: efficient and controllable image diffusion generation via skip connection editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8995–9004. Cited by: §1. [21] S. Kaltenbach and P. Koutsourelakis (2020) Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems. Journal of Computational Physics 419, p. 109673. Cited by: §3. [22] A. Kazerouni, E. K. Aghdam, M. Heidari, R. Azad, M. Fayyaz, I. Hacihaliloglu, and D. Merhof (2022) Diffusion models for medical image analysis: a comprehensive survey. arXiv:2211.07804. Cited by: §3. [23] N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J. Zhu (2023) Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1931–1941. Cited by: §2. [24] J. Li, D. Li, S. Savarese, and S. Hoi (2023) Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, p. 19730–19742. Cited by: §1. [25] S. Lu, Y. Li, Y. Xia, Y. Hu, S. Zhao, Y. Ma, Z. Wei, Y. Li, L. Duan, J. Zhao, et al. (2025) Ovis2. 5 technical report. arXiv:2508.11737. Cited by: §1, §2. [26] C. Ma, Y. Yang, C. Ju, F. Zhang, J. Liu, Y. Wang, Y. Zhang, and Y. Wang (2023) Diffusionseg: adapting diffusion towards unsupervised object discovery. arXiv:2303.09813. Cited by: §1. [27] OpenAI and Microsoft (2024-05) Introducing gpt-4o: openai’s new flagship multimodal model. Note: Azure Blog External Links: Link Cited by: §5.1.2. [28] Z. Ouyang, Z. Li, and Q. Hou (2025) K-lora: unlocking training-free fusion of any subject and style loras. arXiv:2502.18461. Cited by: Table 1, §1, §1, §2, §3, §3, §4.1, Table 1, Table 2, §5, §5. [29] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, p. 8748–8763. Cited by: §1, §4.2, §5.1.1. [30] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman (2023) Dreambooth: fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22500–22510. Cited by: §1, §2, §5. [31] C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi (2022) Palette: image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, Cited by: §3. [32] V. Shah, N. Ruiz, F. Cole, E. Lu, S. Lazebnik, Y. Li, and V. Jampani (2024) Ziplora: any subject in any style by effectively merging loras. In European Conference on Computer Vision, p. 422–438. Cited by: Table 1, §1, §2, §3, §4.2, Table 1, Table 2, §5. [33] J. Shi, W. Xiong, Z. Lin, and H. J. Jung (2024) Instantbooth: personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 8543–8552. Cited by: §2. [34] K. Sohn, L. Jiang, J. Barber, K. Lee, N. Ruiz, D. Krishnan, H. Chang, Y. Li, I. Essa, M. Rubinstein, et al. (2023) Styledrop: text-to-image synthesis of any style. Advances in Neural Information Processing Systems 36, p. 66860–66889. Cited by: §1, §2, §5. [35] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2020) Score-based generative modeling through stochastic differential equations. arXiv:2011.13456. Cited by: §3. [36] T. Wu, J. Wang, Z. Zhao, and N. Wong (2024) Mixture-of-subspaces in low-rank adaptation. arXiv:2406.11909. Cited by: §2. [37] G. Xiao, T. Yin, W. T. Freeman, F. Durand, and S. Han (2024) Fastcomposer: tuning-free multi-subject image generation with localized attention. International Journal of Computer Vision, p. 1–20. Cited by: §2. [38] S. Xie, Z. Zhang, Z. Lin, T. Hinz, and K. Zhang (2023) Smartbrush: text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22428–22437. Cited by: §2. [39] P. Xing, H. Wang, Y. Sun, Q. Wang, X. Bai, H. Ai, R. Huang, and Z. Li (2024) Csgo: content-style composition in text-to-image generation. arXiv:2408.16766. Cited by: §2. [40] Z. Xing, Q. Feng, H. Chen, Q. Dai, H. Hu, H. Xu, Z. Wu, and Y. Jiang (2024) A survey on video diffusion models. ACM Computing Surveys 57 (2), p. 1–42. Cited by: §1. [41] Y. Xu, W. Wang, Y. Zhang, B. Tang, P. Yan, F. Feng, and X. He (2025) Personalized image generation with large multimodal models. In Proceedings of the ACM on Web Conference 2025, p. 264–274. Cited by: §1. [42] F. Zhang, T. Zhou, B. Li, H. He, C. Ma, T. Zhang, J. Yao, Y. Zhang, and Y. Wang (2023) Uncovering prototypical knowledge for weakly open-vocabulary semantic segmentation. Advances in Neural Information Processing Systems 36, p. 73652–73665. Cited by: §2. [43] H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H. Shum (2022) Dino: detr with improved denoising anchor boxes for end-to-end object detection. arXiv:2203.03605. Cited by: §5.1.1. [44] L. Zhang, L. Zhang, S. Shi, X. Chu, and B. Li (2023) Lora-fa: memory-efficient low-rank adaptation for large language models fine-tuning. arXiv:2308.03303. Cited by: §2. [45] G. Zheng, X. Zhou, X. Li, Z. Qi, Y. Shan, and X. Li (2023) Layoutdiffusion: controllable diffusion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22490–22499. Cited by: §1. [46] H. Zhou, X. Lu, W. Xu, C. Zhu, T. Zhao, and M. Yang (2024) Lora-drop: efficient lora parameter pruning based on output evaluation. arXiv:2402.07721. Cited by: §2. [47] B. Zi, X. Qi, L. Wang, J. Wang, K. Wong, and L. Zhang (2023) Delta-lora: fine-tuning high-rank parameters with the delta of low-rank matrices. arXiv:2309.02411. Cited by: §2. Supplementary Material Method ViT Content ↑ ViT Style ↑ BLIP-2 Content ↑ BLIP-2 Style ↑ ZipLoRA [32] 23.2% 18.0% 48.2% 46.0% B-LoRA [11] 29.6% 19.9% 42.0% 54.3% K-LoRA [28] 32.8% 20.7% 49.4% 51.1% Ours 33.7% 21.8% 49.5% 51.4% Table 1: Comparison of alignment results with additional evaluation metrics. Figure 1: Additional results generated using FLUX. Each image corresponds to the object label indicated above and the style reference on the left. The results demonstrate the effects of applying different LoRA modules through our proposed method. Figure 2: Additional results generated using FLUX. Each image corresponds to the object label indicated above and the style reference on the left. The results demonstrate the effects of applying different LoRA modules through our proposed method. 1 Evaluation with Additional Metrics In addition to conventional measures such as Style Similarity, CLIP Score, and DINO Score, we further adopt ViT-based [10] and BLIP-2-based [24] content and style scores to provide a more comprehensive evaluation of our proposed method. As shown in Table 1, our method achieves state-of-the-art results in three out of four metrics. A closer look reveals that different baselines exhibit complementary strengths—ZipLoRA favors BLIP-2 content alignment, while B-LoRA excels in BLIP-2 style consistency. K-LoRA maintains relatively balanced performance across metrics. In contrast, our proposed method not only improves both ViT-based and BLIP-2-based scores, but also demonstrates a better balance between content fidelity and style preservation. We attribute this advantage to the proposed soft, inference-time metric guidance, which adaptively calibrates the generation process to maintain semantic and stylistic coherence without overfitting to a single representation space. These results suggest that our dynamic fusion framework is more robust across heterogeneous evaluation perspectives, highlighting its potential for broader generalization to multimodal generation scenarios. 2 Additional Experimental Results Based on Flux As discussed in the main experiments section, to more comprehensively illustrate the superior performance and generalization capability of our proposed method built upon the FLUX framework, we further conduct extensive qualitative evaluations using publicly available, well-trained LoRA (Low-Rank Adaptation) weights shared by the community on HuggingFace. Specifically, we selected a diverse set of LoRA weights corresponding to various object categories and artistic styles to systematically evaluate our model’s ability to integrate and synthesize complex cross-domain representations. The resulting fused images, presented in Figure 1 and Figure 2, showcase a wide range of combinations where object semantics and stylistic attributes are jointly encoded and rendered through our method. Our approach incorporates these LoRA weights by disentangling and recombining object- and style-specific latent representations in a manner that performs feature-level selection based on representation perturbation and applying metric-guided refinement during denoising. This enables the model to align and synthesize visual content in a controlled yet flexible fashion. The generated samples exhibit not only strong fidelity to the semantic structure of the target object but also high consistency with the desired style, demonstrating the model’s ability to preserve critical attributes from both input domains. Furthermore, the seamless integration of appearance and content substantiates the robustness of our framework in handling varied and unseen combinations, emphasizing its potential applicability in real-world generation tasks that demand stylistic generalization and compositional creativity. Overall, these visual results provide compelling evidence of the effectiveness of our method in producing coherent, high-quality outputs across a broad spectrum of challenging scenarios. 3 Setup Details for User Study and MLLM Evaluations Following the evaluation protocol of K-LoRA [28], we conducted a user study where participants were presented with a reference subject image, a reference style image, and two anonymized outputs—one generated by our method and the other by a randomly selected baseline (ZipLoRA [32], B-LoRA [11], or K-LoRA [28]). To mitigate presentation bias, the order of the two outputs was randomized across trials. Participants were asked: “Which image better reflects the given artistic style while preserving the subject identity?” We collected a total of 1,290 responses from 43 participants, with each participant evaluating a unique set of 30 trials. Beyond human evaluation, we further adopt GPT-4o and Qwen2.5-VL as multimodal large language model (MLLM) judges to assess perceptual alignment. For each trial, the prompt included the content image, the style reference, and four anonymized outputs from ZipLoRA, B-LoRA, K-LoRA, and our proposed method(randomized order). The LLMs were instructed to select the image that best balances style fidelity with subject preservation. To ensure robustness, we randomly sampled 100 subject–style pairs, and each pair was evaluated in three independent runs. The final score was determined via majority voting across runs, following standard practice in recent MLLM-based evaluation studies. Importantly, while the user study captures subjective human preference, the MLLM-based evaluation provides scalable and reproducible judgments, making the two evaluations complementary and mutually reinforcing.

Instant research discovery