Paper deep dive
Beyond Learning: A Training-Free Alternative to Model Adaptation
Namkyung Yoon, Kyeonghyun Yoo, Wooyong Jung, Sanghong Kim, Hwangnam Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/21/2026, 2:05:18 AM
Summary
This paper introduces 'model transplantation,' a training-free technique for improving language model performance by transplanting internal modules (layers) from a source model to a target model. The method identifies structurally compatible modules based on activation discrepancies and replaces them directly, achieving significant performance gains without fine-tuning or additional training.
Entities (9)
Relation Signals (10)
Model Transplantation β appliedto β Phi-3
confidence 95% Β· First, we conduct transplantation experiments within the Phi family using the open-source PHI-3-MINI-4K-COMMAND model.
Model Transplantation β appliedto β Phi 3.5
confidence 95% Β· Using the proposed pre-transplant diagnostic procedure, we identify PHI-3.5-MINI-COMMAND as a structurally compatible model for transplantation.
Model Transplantation β appliedto β Gemma-2-2B
confidence 95% Β· Specifically, we perform transplantation between GEMMA-2-2B and its instruction-tuned counterpart GEMMA-2-2B-IT.
Model Transplantation β appliedto β Gemma-2-2B-it
confidence 95% Β· Specifically, we perform transplantation between GEMMA-2-2B and its instruction-tuned counterpart GEMMA-2-2B-IT.
Model Transplantation β evaluatedon β MATH-500
confidence 95% Β· All experiments are conducted on the HuggingFaceH4/MATH-500 benchmark
Gemma-2-2B β transplantedwith β Gemma-2-2B-it
confidence 95% Β· Specifically, we perform transplantation between GEMMA-2-2B and its instruction-tuned counterpart GEMMA-2-2B-IT.
Phi-3 β transplantedwith β Phi 3.5
confidence 95% Β· Table 2 summarizes the best transplantation settings between Phi-3 and Phi-3.5 across decoding lengths
Model Transplantation β uses β Layer-wise Activation Evaluation (LAE)
confidence 95% Β· To quantitatively identify transplant candidates, we compare the activations of the source and target models using a layer-wise activation evaluation (LAE)
Model Transplantation β measuredby β Recovery
confidence 90% Β· Recovery: We introduce a recovery metric that quantifies target-to-source gap closure after transplantation
Model Transplantation β measuredby β Transplantation Improvement Ratio (TIR)
confidence 90% Β· Transplantation Improvement Ratio (TIR): A target-referenced metric newly introduced in this paper to quantify the normalized improvement ratio
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Despite the continuous research and evolution of language models, they sometimes underperform previous versions. Existing approaches to overcome these challenges are resource-intensive, highlighting the need for alternatives that enable immediate action. We assume that each language model has a local module inside that is suitable for a specific function. First, this work identifies a set of modules showing consistent and local activation changes under an inference workload through activation-based analysis. Subsequently, we transplant an internal module that is properly activated for a specific task into the target model, leading to immediate and measurable functional changes without additional training or fine-tuning. To experimentally demonstrate the effectiveness of the transplant technique, we quantify the relationship between transplant strength and performance improvement under different conditions for two language models. In the cross-generation setting, we find that transplanting activation-selected modules can substantially improve the underperforming model, reaching up to twice the target baseline and achieving gap-based recovery above 100%. Moreover, in transplant experiments between a base model and its instruction-tuned counterpart, transplantation improves the underperforming model toward the stronger baseline, yielding up to about 2.33 times the target baseline with gap-based recovery reaching up to 100% in the best case. These results show that meaningful capacity transfer can be realized through the implantation of highly localized modules implied by language models. Overall, this work provides empirical evidence for task-localized modularity in language models and presents a new research area: model transplantation.
Tags
Links
- Source: https://arxiv.org/abs/2602.16189v1
- Canonical: https://arxiv.org/abs/2602.16189v1
Trouble viewing inline? Open PDF directly β
Full Text
35,417 characters extracted from source content.
Expand or collapse full text
Graphical Abstract Beyond Learning: A Training-Free Alternative to Model Adaptation Namkyung Yoon, Kyeonghyun Yoo, Wooyong Jung, Sanghong Kim, Hwangnam Kim arXiv:2602.16189v1 [cs.CL] 18 Feb 2026 Highlights Beyond Learning: A Training-Free Alternative to Model Adaptation Namkyung Yoon, Kyeonghyun Yoo, Wooyong Jung, Sanghong Kim, Hwangnam Kim β’ We propose a transplant technique for improving language models without training. β’ We propose a compatibility diagnostic for model transplantation. β’ We validate the transplant technique across different conditions and models. β’ We analyze optimal transplant conditions with a new metric. Beyond Learning: A Training-Free Alternative to Model Adaptation Namkyung Yoon a , Kyeonghyun Yoo a , Wooyong Jung a , Sanghong Kim a and Hwangnam Kim a,β,1 a School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea A R T I C L E I N F O Keywords: Language models Model transplant Artificial Intelligence A B S T R A C T Despite the continuous research and evolution of language models, they sometimes underperform pre- vious versions. Existing approaches to overcome these challenges are resource-intensive, highlighting the need for alternatives that enable immediate action. We assume that each language model has a local module inside that is suitable for a specific function. First, this work identifies a set of modules showing consistent and local activation changes under an inference workload through activation-based analysis. Subsequently, we transplant an internal module that is properly activated for a specific task into the target model, leading to immediate and measurable functional changes without additional training or fine-tuning. To experimentally demonstrate the effectiveness of the transplant technique, we quantify the relationship between transplant strength and performance improvement under different conditions for two language models. In the cross-generation setting, we find that transplanting activation-selected modules can substantially improve the underperforming model, reaching up to twice the target baseline and achieving gap-based recovery above 100%. Moreover, in transplant experiments between a base model and its instruction-tuned counterpart, transplantation improves the underperforming model toward the stronger baseline, yielding up to about 2.33 times the target baseline with gap-based recovery reaching up to 100% in the best case. These results show that meaningful capacity transfer can be realized through the implantation of highly localized modules implied by language models. Overall, this work provides empirical evidence for task-localized modularity in language models and presents a new research area: model transplantation. 1. Introduction Recent advances in artificial intelligence and compu- tational infrastructure have led to the emergence of lan- guage models with strong reasoning ability and domain- specific expertise Kumar (2024). Despite these advances, such capabilities remain largely confined within the fixed parameterization of each individual model, limiting the di- rect reuse or transfer of learned behaviors across models Hadi et al. (2023); Ling et al. (2023). As a result, adapt- ing a model to a new domain typically requires additional training procedures, such as fine-tuning, parameter-efficient adaptation, model merging, or knowledge distillation, all of which modify parameters through optimization or statistical combination Han et al. (2024); Wang et al. (2025); Fang et al. (2025); Yang et al. (2024); Liu et al. (2023). While these approaches have proven effective, they do not provide a mechanism for directly transferring internal functional components between models. In contrast to bio- logical systems, where functional recovery or augmentation can be achieved through localized transplantation Murray et al. (2002), existing model adaptation techniques pro- vide limited interpretability regarding where and how new features are integrated Sun et al. (2024). This gap raises fundamental questions about whether pre-trained language models contain internal components that can be functionally communicated in a local and direct manner. β Corresponding author nkyoon93@korea.ac.kr (N. Yoon); seven1705@korea.ac.kr (K. Yoo); jy17347@korea.ac.kr (W. Jung); sanghongkim@korea.ac.kr (S. Kim); hnkim@korea.ac.kr (H. Kim) ORCID(s): 1 N. Yoon and H. Kim contributed equally to this work. In this work, we aim to address this gap by investigating the internal behavior of language models during inference, using activation-based analysis of their constituent compo- nents. We hypothesize that a small subset of modules within the language model exhibits localized activation changes that are specific to particular domain tasks. Inspired by these observations, we test whether func- tional behavior can be reproduced without training by se- lectively replacing a limited number of layers identified via activation-based analysis. We refer to this training-free adaptation as model transplantation, in which a small subset of internal components from a source model is directly trans- planted into a structurally compatible target model, where corresponding modules match in role and dimensionality. As a result, the transplanted model recovers task-relevant performance while retaining most of the existing functional- ity of the target model. This result suggests that, when main- taining architectural compatibility, the relevant modules for specific domain tasks can be recombined within a language model to improve performance. In addition, transplantation offers a potential mechanism for rapid rollback when post- training updates inadvertently degrade task-specific behav- ior. By restoring a small set of functionally relevant modules from a reference model, the degraded performance can be recovered without additional optimization. Collectively, these observations provide evidence of module-level portability across models. They also suggest task-dependent activation structures, analogous to functional specialization in biological systems Ellis and Aizenberg (2022). Namkyung et al.: Preprint submitted to ElsevierPage 1 of 7 Pattern Recognition Letters Figure 1: Illustration of module-level transplantability in language models. The rest of this paper is organized as follows. Section 2 reviews existing model adaptation techniques, such as fine- tuning. Section 3 describes the proposed transplant tech- nique between language models. Section 4 presents the re- sults of the experiments and evaluations. Section 5 discusses the conclusions. 2. Preliminary 2.1. Optimization-Based Model Adaptation Existing approaches for adapting or combining language models largely rely on optimization-based mechanisms. Parameter-efficient fine-tuning methods update a small sub- set of parameters, such as low-rank adapters or selected layers Wang et al. (2025); Hu et al. (2022), through gradient- based optimization using task-specific data. Model merg- ing techniques combine multiple trained models through parameter averaging or geometric alignment, producing a compromise model that blends behaviors across sources Yang et al. (2024). Knowledge distillation transfers knowl- edge by training a student model to approximate the outputs or representations of a teacher model, resulting in newly learned parameters Fang et al. (2025). Despite their differences, these approaches share a com- mon characteristic: model adaptation is achieved through additional training-based optimization. As a result, they do not directly reuse internal computations via weight-level module reuse, nor do they provide a mechanism to isolate and transfer specific layers responsible for task-specific func- tionality. In contrast, the approach studied in this work performs transplantation without optimization or retraining. By se- lectively replacing structurally compatible layers identified through consistent activation discrepancies, the proposed method enables immediate functional transfer through direct parameter reuse rather than learned approximation. 2.2. Motivation Language models are continuously studied and updated across generations to improve overall performance and gen- eralization. However, in practice, such updates can intro- duce performance regressions, whereby the new model per- forms worse than the previous version on tasks that were previously handled reliably. This can lead to critical out- comes, as task-specific reliability may degrade in real-world deployments despite apparent improvements in aggregate benchmark metrics. Resolving these performance regres- sions through traditional adaptation techniques usually re- quires additional training, task-specific data, or architec- tural modifications. These requirements impose significant latency and overhead, making them unsuitable for rapid recovery or real-time intervention. From a system perspective, an ideal solution should be able to selectively restore lost features while preserving most of the updated model behavior without retraining or global parameter modification. This requirement implicitly assumes that task-related computations are not uniformly distributed across the model, but are concentrated in a subset of inner layers that are disproportionately engaged depend- ing on the task. Based on these assumptions, it becomes pos- sible to identify and selectively replace only the components that contribute to the degraded behavior. Building on this motivation, this work investigates trans- plantation as a method for selective functional recovery and structured model editing in evolving language model systems. Namkyung et al.: Preprint submitted to ElsevierPage 2 of 7 Pattern Recognition Letters 3. Model Transplant The proposed model transplantation is a mechanism that changes operational behavior by replacing layers based on local activation scores as shown in Fig. 1 3.1. Pre-Transplant Diagnosis Our transplant procedure begins with identifying struc- turally compatible modules across models. In this paper, ξΉ ν and ξΉ ν are represented as the source and target models for transplantation. For a model ξΉ, we define the set of candidate modules as follows: ξΈ(ξΉ) = ν β£ν is a leaf n.Linear module in ξΉ , (1) where non-transferable components such as embedding lay- ers, normalization layers, dropout layers, and output heads are excluded. Each moduleν β ξΈ(ξΉ) is represented by its normalized name ν ν and weight matrixν ν β β ν out Γν in . A module pair (ν ν ,ν ν ) withν ν β ξΈ(ξΉ ν ) andν ν β ξΈ(ξΉ ν ) is considered compatible if ν ν ν = ν ν ν and shape(ν ν ν ) = shape(ν ν ν ). (2) The set of transplantable modules is thus defined as follows: ξ― = (ν ν ,ν ν ) β£ ν ν ν = ν ν ν , shape(ν ν ν ) = shape(ν ν ν ), (3) with transplantation implemented by direct weight copy as follows: ν ν ν βν ν ν , β(ν ν ,ν ν ) β ξΏ,(4) where ξΏ is a selected subset of compatible module pairs and is defined in the next section based on activation discrepan- cies. When present, the corresponding bias terms are copied in the same manner. All subsequent transplantation experiments operate only on module pairs drawn from ξ―, and we always select ξΏ β ξ―. No fine-tuning, architectural modification, or auxiliary adap- tation is applied after transplantation, ensuring that observed performance changes reflect functional transfer rather than structural effects. In practice, we extract only standalone linear modules, including attention projections and feed-forward layers, and normalize their namespaces to ensure one-to-one alignment between models. 3.2. Selection Criterion for Transplanting To quantitatively identify transplant candidates, we com- pare the activations of the source and target models using a layer-wise activation evaluation (LAE) based on forward- only inference. The LAE records activation levels across both models during autoregressive inference on input se- quences to address specific domains. This aims to distin- guish the local activation differences between the target model to be transplanted and the source model to which the transplant module is donated. For linear modules ν ν , such as attention projections or feedforward layers, the activation response at decoding step ν‘ is defined as follows: ν ν ν (ν‘) = 1 ν ββ ν ν (ν‘)β 1 ,(5) where β ν ν (ν‘) β β ν represents the post-linear hidden state of the last generated token at step ν‘. The activation mismatch between the source and target models is measured with LAE score, which is defined as: ν (ν ν ) = νΌ ν₯, ν‘β€ν [ |ν β ν ν (ν‘) β ν ν ν (ν‘)| ] ,(6) where ν β ν ν (ν‘) denotes the activation of the source model and ν ν ν (ν‘) denotes the activation of the target model under the same input ν₯. This metric provides a stable and interpretable measure of functional divergence by using last-token activations and averaging across decoding steps, capturing persistent task- induced differences while avoiding transient fluctuations. The layers are ranked according to ν (ν ν ), and the top νΎ layers with the greatest discrepancy are selected as can- didates for transplantation. The transplant is then applied only to this subset and modified locally while retaining most original structures, changing only some of the parameters of the target model. 3.3. Model Transplantation Procedure This section describes the model transplantation proce- dure, including the diagnosis of architectural compatibility between the source and target models and the criteria for selecting modules for transplantation. Model transplantation replaces the parameters of a target model with the corresponding parameters from a source model, restricted to structurally aligned linear modules with identical shapes. In this paper, we define ν as the target model parameters, ν β as the source model parameters, and ξΏ as a selected set of transplantable module pairs. We then select a subset ξΏ β ξ― based on LAE scores, and apply transplantation only to pairs in ξΏ. The transplanted model ν β² is constructed as follows: ν β² ν ν = ν β ν ν , (ν ν ,ν ν ) β ξΏ, ν ν ν , otherwise. (7) After this procedure, no additional optimization or fine- tuning is applied to the transplanted model, ensuring that any performance change arises solely from the transplanted modules. We ensure that for each selected module, the data type is maintained by transplanting weights and biases as follows: ν ν ν βν ν ν ,ν ν ν βν ν ν .(8) As a result, the transplanted model retains the existing architecture while inheriting the functionality of the source model. Namkyung et al.: Preprint submitted to ElsevierPage 3 of 7 Pattern Recognition Letters Table 1 Comparison of key architectural and training characteristics across Phi and Gemma model variants. ComponentPhi-3-mini-4kPhi-3.5-miniGemma-2-2BGemma-2-2B-IT Context Length4,096 tokens128K tokens8K tokens8K tokens Vocabulary Size32,064 tokens32,064 tokens256K tokens256K tokens ArchitectureDecoder-onlyDecoder-onlyDecoder-onlyDecoder-only Training Corpus Curated Phi-3 dataset Expanded Phi-3.5 data Gemma pretraining corpus Gemma pretraining corpus + instruction tuning 4. Experiment In this section, we present an evaluation designed to assess whether the proposed model transplant technique can effectively induce task-specific competency transfer in language models. 4.1. Implementation and Dataset Details We evaluate our method on a mathematical inference task that is known to induce heterogeneous and depth- dependent activation patterns in large language models Feng et al. (2023). First, we conduct transplantation experiments within the Phi family using the open-source PHI-3-MINI- 4K-COMMAND model. Using the proposed pre-transplant diagnostic procedure, we identify PHI-3.5-MINI-COMMAND as a structurally compatible model for transplantation. Phi-3.5 is an in-generation update that largely preserves the architecture of Phi-3 while extending context length and improving instruction following through additional training Abdin et al. (2024). In addition, we include a Gemma-model setting to iso- late the effect of instruction tuning Team et al. (2025). Specifically, we perform transplantation between GEMMA- 2-2B and its instruction-tuned counterpart GEMMA-2-2B- IT. This setting explicitly evaluates how specialization in- duced by instruction tuning affects module transferability. The architectural and training characteristics of Phi-3, Phi-3.5, and Gemma models are summarized in Table 1. We design a targeted diagnostic workload based on func- tional specialization to analyze transferability. Using the pre- transplant diagnostic technique described in Section 3.1, we find that all model pairs used in our experiments achieve compatibility over the candidate linear modules. All experiments are conducted on the HuggingFaceH4/ MATH-500 benchmark, which consists of 500 problems span- ning seven mathematical subjects HuggingFaceH4 (2024): Algebra, Intermediate Algebra, Prealgebra, Precalculus, Geometry, Number Theory, and Counting & Probability. On these datasets, we extract 50 questions by sampling and use them to evaluate transplantation between models. At this point, we use the same prompt that requires a single boxed final answer from all models for the reliability of the evaluation: You are a careful math problem solver. Solve step by step and give ONLY the final answer wrapped in .... Notably, on the selected mathematical reasoning bench- marks, PHI-3-MINI-4K-INSTRUCT sometimes exhibits stronger task performance than the newer PHI-3.5-MINI-INSTRUCT, despite PHI-3.5 being an updated variant. This performance reversal provides a controlled setting in which the newer model is not necessarily superior in task accuracy, allowing us to test whether model transplantation can recover task- relevant behavior by transferring only localized components rather than relying on global fine-tuning or full-model replacement. For the Gemma setting, we evaluate transplantation be- tween the base model GEMMA-2-2B and its instruction- tuned counterpart GEMMA-2-2B-IT. Because their relative performance can vary across decoding lengths and eval- uation conditions, we first evaluate both models for each experimental configuration and then perform transplantation from the better-performing model to the weaker model. We evaluate transplantation performance by varying the decod- ing length using max_new_tokens set to 32, 64, 128, 256, and 512, together with the number of transplanted layers K set to 8, 16, 32, 64, and 128. Unless otherwise stated, we fix SEED to 42, disable sampling, and set top_p to 1.0. Accuracy is defined as the fraction of correct predictions over the evaluation set. Given the model output Μν¦ ν and the ground-truth answer ν ν for the ν-th problem, accuracy is computed as Acc = 1 ν ν β ν=1 ν [ norm ( boxed(Μν¦ ν ) ) = norm(ν ν ) ] , where ν denotes the number of evaluated problems. We do not assume that more recent or fine-tuned models necessarily achieve superior performance. For each experi- mental configuration, models are first evaluated, and layer transplantation is performed from the empirically better- performing model to the weaker model. 4.2. Results and Analysis In this section, we examine whether the proposed trans- plant technique can transfer task-relevant behavior between language models. Furthermore, we investigate transplan- tation performance under different experimental settings, providing insights into how task-specific functionality is localized within model components. First, we investigate how transplantation performance varies with respect to the number of selected layers νΎ. For each decoding length, modules are transferred from the better-performing model to the weaker one, isolating capability transfer through structural intervention alone. To clarify the notation in the experimental analysis, each term is defined as follows: Namkyung et al.: Preprint submitted to ElsevierPage 4 of 7 Pattern Recognition Letters Table 2 Best-performing transplantation settings between Phi-3 and Phi-3.5 across decoding token lengths. Model PairTokens Dir. Acc target (%) Acc source (%) Acc after (%) Best νΎ TIR Recovery (%) Intervention Type Phi-3 (A)β Phi-3.5 (B) 32BβA6.08.012.0162.00300.0FFN + Attention 64AβB16.020.020.0161.25100.0FFN + Attention 128 AβB16.030.026.0641.62571.4FFN + Attention 256 AβB34.034.036.081.059βFFN + Attention 512 BβA36.040.040.081.111100.0FFN + Attention Table 3 Intervention composition as a function of transplantation scale νΎ for Phi-3β Phi-3.5. νΎ FFN count Attn count Attn ratio 8710.125 161240.250 3214180.563 6425390.609 12864640.500 Figure 2: The TIR and Recovery of Phi-3β Phi-3.5 as a function of the number of transplanted modules K for different decoding lengths with max_new_tokens. β’ Tokens: The maximum decoding length used during evaluation, which determines the generation budget. β’ Dir: the transplantation direction from the source to the target. β’ Best νΎ: The number of transplanted modules that maximizes Acc for the given decoding length. β’ Transplantation Improvement Ratio (TIR): A target- referenced metric newly introduced in this paper to quantify the normalized improvement ratio of a trans- planted model over the target baseline, defined as TIR = Acc after Acc Target .(9) Positive TIR indicates a relative improvement over the target model. β’ Recovery: We introduce a recovery metric that quanti- fies target-to-source gap closure after transplantation, measured as the fraction of the performance gap be- tween the target and source baselines that is closed by the transplanted model. Recovery = 100 Γ Acc after β Acc Target Acc Source β Acc Target . (10) A recovery value of 100 indicates that the transplanted model reaches the source baseline, and a value above 100 indicates that it surpasses the source baseline. β’ Intervention Type: Indicates which module is config- ured when performing the transplant operation. 4.2.1. Results between Model Generations Table 2 summarizes the best transplantation settings between Phi-3 and Phi-3.5 across decoding lengths, re- porting post-transplant accuracy and the proposed metric, TIR. Across token budgets, activation-selected transplanta- tion yields measurable gains over the target baseline, while the optimal direction and transplantation scale depend on the decoding horizon. Under short decoding, the best configuration favors transfer from the stronger model to the weaker model. At 32 tokens, the best setting with νΎ=16 improves accuracy from 6 to 12, corresponding to TIR = 2.0 which means Acc after is two times higher than Acc Target . At 64 and 128 tokens, the best settings occur at moderate transplant scales and remain beneficial, with TIR peaking at 1.25 and 1.625, respectively. Recovery provides a complementary view based on the gap closure between the target and source baselines. At 64 and 512 tokens, Recovery = 100.0, showing complete recovery to the source level. At 32 tokens, the target-to-source gap is more than fully recovered, with Recovery = 300.0, indicating that the transplanted model surpasses the source baseline. This over-recovery suggests that transplantation is not a mere functional copy of the source behavior, but a compositional recombination in which transplanted modules interact with the remaining target circuitry to yield a synergistic improvement beyond either baseline. At 128 tokens, Recovery = 71.4, indicating partial recovery of the target-to-source gap despite a clear TIR gain. At 256 tokens, Recovery is not defined because the target and source accuracies are identical, making the denominator zero. The intervention composition provides insight into the under- lying mechanism. As shown in Table 3, small νΎ sets are Feedforward Network (FFN)-dominant, whereas attention modules become increasingly prevalent with larger values of νΎ grows. This suggests that improvements at small scales are Namkyung et al.: Preprint submitted to ElsevierPage 5 of 7 Pattern Recognition Letters Table 4 Best-performing transplantation settings between Gemma-2-2B and Gemma-2-2B-IT across decoding token lengths. Model PairTokens Dir. Acc B (%) Acc A (%) Acc after (%) Best νΎ TIR Recovery (%) Intervention Type Gemma-2-2B-IT (A)β Gemma-2-2B (B) 32BβA10.06.010.01281.667100.0FFN + Attention 64AβB8.08.08.0321.000βFFN + Attention 128AβB6.022.010.0641.66725.0FFN + Attention 256AβB6.036.012.0642.00020.0FFN + Attention 512AβB6.030.014.0642.33333.3FFN + Attention Table 5 Intervention composition as a function of transplantation scale νΎ for Gemma-2-2B-ITβ Gemma-2-2B. νΎ FFN count Attn count Attn ratio 8081.000 162140.875 322300.938 647570.891 12839890.695 Figure 3: The TIR and Recovery of Gemma-2-2B-ITβ Gemma-2-2B as a function of the number of transplanted modules νΎ for different decoding lengths with max_new_tokens. primarily driven by localized feed-forward transformations, while larger transplant scales increasingly engage attention pathways that mediate token-to-token interactions. Increasing νΎ does not yield monotonic gains at longer decoding lengths. Fig. 2 shows that TIR often peaks at intermediate νΎ and can diminish at larger transplant scales. This non-monotonic pattern indicates that beneficial transfer is selective in both scale and layer type, and that broader replacement does not guarantee additional gains, even under high structural compatibility. 4.2.2. Results between Base and Tuned Models Table 4 and Table 5 summarize transplantation between Gemma-2-2B model and Gemma-2-2B-IT model. Because the two models share the same backbone architecture, this setting isolates the effect of post-training on transferability under high structural compatibility. Table 4 reports the best configuration at each decoding token length. At 64 tokens, the two models exhibit the same baseline accuracy, and the best post-transplant accuracy remains unchanged, yielding TIR = 1.0. This indicates a saturation regime with little headroom for relative improve- ment under the given decoding constraint. For moderate and long decoding token budgets, the optimal direction is pre- dominantly ITβBASE, reflecting that the instruction-tuned model is substantially stronger. The transplant composition further indicates that transfer in this setting involves broad coordination across attention pathways. As shown in Table 5, the optimal sets remain attention-dominant across νΎ, with high attention ratios even at large intervention scales. At 512 tokens, the best configuration is GEMMA-2-2B- ITβGEMMA-2-2B with νΎ=64, as shown in Fig. 3. Under the proposed TIR metric, this corresponds to TIR = 2.333, indicating a large relative gain over the target baseline at long decoding lengths. Recovery provides a complementary gap- based view between the target and source baselines. At 32 tokens, Recovery = 100.0, showing complete recovery to the source level. At 128, 256, and 512 tokens, Recovery re- mains partial at 25.0, 20.0, and 33.3, respectively, indicating that transplantation improves the target baseline while not fully closing the target-to-source gap under longer decoding horizons. At 64 tokens, Recovery is not defined because the target and source accuracies are identical, making the denominator zero. Fig. 3 shows that increasing νΎ does not yield monotonic gains and that TIR peaks at intermediate νΎ before diminishing at larger transplant scales for longer decoding lengths. This is consistent with instruction tuning, which can induce broad changes in generation behavior and token-to-token interaction patterns beyond task-specific computation Team et al. (2024). The component differences in transplantation across νΎ reported in Table 3 and Table 5 arise from how activation mismatches distribute across modules for each model pair. Phi-3 and Phi-3.5 form a cross-generation setting with dif- ferences in architecture and training, whereas Gemma-2- 2B and Gemma-2-2B-IT primarily differ in the behavioral policies induced by instruction tuning. Accordingly, in the Phi setting, performance-related activation differences are concentrated in localized representation transforms, and the largest activation mismatches at small νΎ tend to appear in FFN projections. As a result, activation-based selection pref- erentially yields FFN-dominant intervention sets for small νΎ, while attention modules are gradually included as νΎ increases. In the Gemma base versus instruction-tuned setting, gen- eration policies such as output format, response length, ter- mination propensity, and conversational alignment differ and are more directly expressed through attention pathways that Namkyung et al.: Preprint submitted to ElsevierPage 6 of 7 Pattern Recognition Letters mediate token-to-token interactions. Consequently, activation- based selection consistently yields attention-dominant inter- vention sets across νΎ. 4.3. Discussion Our experiments show that transplantation induces a localized functional shift: the transplanted modules move the target model responses toward those of the source model, while the remaining untransplanted modules largely pre- serve their original characteristics. This supports a modular view in which task-relevant computation is concentrated in a sparse subset of internal modules rather than being uniformly distributed across the network. An important implication is that effective transfer is selective not only in scale νΎ, but also in model type and decoding token length. In the Phi setting, the largest gains over the target baseline are achieved at small to moderate transplant scales, and the optimal transplant composition shifts as the decoding horizon increases. In the Gemma base versus instruction-tuned setting, transplantation consistently improves the target model at long decoding lengths, while gains can saturate when the two models have similar base- lines under a constrained token budget. Overall, these results indicate that transfer is most ef- fective when we transplant modules whose activations best match the target task under the given decoding length. These findings motivate future work on identifying functionally equivalent substructures beyond strictly aligned architec- tures. 5. Conclusion In this work, we provided experimental evidence of portability between language models for performance im- provement. We demonstrated immediate functional changes by directly transplanting a small subset of internal modules. Using the TIR metric, we show that transplantation based on activation-based analysis can partially recover domain- specific features without retraining or parameter optimiza- tion. Complementing TIR, the Recovery metric quantifies how much of the target-to-source performance gap is closed after transplantation, enabling a normalized assessment of transfer effectiveness across model pairs and decoding hori- zons. Our results indicate that transferable computation is lo- calized within specific layers, revealing an implicit modular structure inside pretrained language models. These findings offer insight into how internal representations are organized and suggest that localized parameter replacement may serve as a viable mechanism for model editing. We plan to extend transplantation to heterogeneous model pairs with architectural mismatches by developing compatibility mappings between non-identical modules. Finally, we will broaden the evaluation beyond mathematical reasoning to diverse domains and tasks to better characterize when and how localized module replacement yields reliable capability transfer. Acknowledgements This work was supported by the Korea Institute of En- ergy Technology Evaluation and Planning(KETEP) and the Ministry of Climate, Energy & Environment(MCEE) of the Republic of Korea (RS-2022-KP002860), and also sup- ported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Cen- ter) support program (IITP-2025-RS-2021-I211835) super- vised by the IITP (Institute of Information & Communica- tions Technology Planning & Evaluation). References Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R.J., Javaheripi, M., Kauffmann, P., et al., 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905 . Ellis, D.G., Aizenberg, M.R., 2022. Structural brain imaging predicts individual-level task activation maps using deep learning. Frontiers in Neuroimaging 1, 834883. Fang, L., Yu, X., Cai, J., Chen, Y., Wu, S., Liu, Z., Yang, Z., Lu, H., Gong, X., Liu, Y., et al., 2025. Knowledge distillation and dataset distillation of large language models: Emerging trends, challenges, and future directions. arXiv preprint arXiv:2504.14772 . Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., Wang, L., 2023. Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems 36, 70757β70798. Hadi, M.U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M.B., Akhtar, N., Wu, J., Mirjalili, S., et al., 2023. Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Authorea preprints 1, 1β26. Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q., 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 . Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al., 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 3. HuggingFaceH4, 2024. Math-500 dataset card. https://huggingface.co/ datasets/HuggingFaceH4/MATH-500. Accessed: 2026-01. Kumar, P., 2024. Large language models (llms): survey, technical frame- works, and future challenges. Artificial Intelligence Review 57, 260. Ling, C., Zhao, X., Lu, J., Deng, C., Zheng, C., Wang, J., Chowdhury, T., Li, Y., Cui, H., Zhang, X., et al., 2023. Domain specialization as the key to make large language models disruptive: A comprehensive survey. ACM Computing Surveys . Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G., 2023. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys 55, 1β35. Murray, M., Kim, D., Liu, Y., Tobias, C., Tessler, A., Fischer, I., 2002. Transplantation of genetically modified cells contributes to repair and recovery from spinal injury. Brain research reviews 40, 292β300. Sun, H., Zhuang, Y., Wei, W., Zhang, C., Dai, B., 2024. Bbox-adapter: Lightweight adapting for black-box large language models. arXiv preprint arXiv:2402.08219 . Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., RamΓ©, A., RiviΓ¨re, M., et al., 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786 . Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., RiviΓ¨re, M., Kale, M.S., Love, J., et al., 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 . Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., Yang, F., 2025. Parameter-efficient fine-tuning in large language models: a survey of methodologies. Artificial Intelligence Review 58, 227. Yang, E., Shen, L., Guo, G., Wang, X., Cao, X., Zhang, J., Tao, D., 2024. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666 . Namkyung et al.: Preprint submitted to ElsevierPage 7 of 7