Paper deep dive

Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters

Junyi Zou

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 35

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/26/2026, 1:34:10 AM

Summary

The paper investigates the reliability of nominal labels (e.g., 'instruction-tuned') for LoRA adapters by evaluating them across tasks. It identifies a 'capability drift' phenomenon where nominal training objectives fail to predict realized cross-task performance, particularly regarding strict, automatically verifiable instruction following (IFEval). The study provides empirical evidence across multiple seeds, base models, and LoRA settings, demonstrating that instruction-tuned adapters can significantly improve off-target numeric reasoning benchmarks while failing to improve verifiable instruction-following compliance.

Entities (5)

IFEval · benchmark · 100%LoRA · methodology · 100%Capability Drift · phenomenon · 95%Qwen3-8b · model · 95%Numeric Reasoning Benchmark · benchmark · 90%

Relation Signals (3)

Instruction-tuned adapter → failstoimprove → IFEval

confidence 95% · while not improving verifiable instruction following on IFEval

Instruction-tuned adapter → improves → Numeric Reasoning Benchmark

confidence 95% · an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance

LoRA adapter → exhibits → Capability Drift

confidence 90% · We refer to this nominal–realized mismatch pattern as capability drift as a descriptive label.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Adapters are often selected and deployed based on nominal labels (e.g., instruction-tuned), which implicitly suggest what capability improves after adaptation. We test whether nominal training objectives reliably align with realized cross-task capability gains by evaluating the same LoRA adapter across tasks. Our strongest evidence is tied to strict, automatically verifiable instruction following as measured by IFEval: across multiple seeds, base models, and LoRA settings, nominal labels recurrently but not universally fail to predict improvements on this verifiable target, with clear configuration sensitivity including a near-zero or negative case. As an illustrative strongest-case example in a controlled instruction-versus-numeric setting, an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance from 0.133 to 0.632 while not improving verifiable instruction following on IFEval (ILA: 0.313 to 0.271; PLA: 0.250 to 0.143; values rounded to three decimals). We refer to this nominal-versus-realized mismatch pattern as capability drift as a descriptive label. The mismatch is visible in the raw cross-task performance matrix; we use a drift score only as a compact summary in the same units as the underlying metrics, not as a new formal metric contribution. Evidence from broader instruction-following benchmarks is benchmark-dependent and mixed, reflecting heterogeneity in how instruction following is operationalized; we therefore do not treat cross-benchmark agreement as a premise. Overall, the practical takeaway is to perform routine cross-task evaluation before deployment and to avoid treating nominal labels as reliable capability proxies.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

34,832 characters extracted from source content.

Expand or collapse full text

Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters Junyi Zou Zjydiary Group zoujunyi@zjydiary.cn Abstract Adapters are often selected and deployed based on nominal labels (e.g., instruction-tuned), which implicitly suggest what capability improves after adaptation. We test whether nominal training objectives reliably align with realized cross-task capability gains by evaluating the same LoRA adapter across tasks. Our strongest evidence is tied to strict, automatically verifiable instruction following as measured by IFEval: across multiple seeds, base models, and LoRA settings, nominal labels recurrently—but not universally—fail to predict improvements on this verifiable target, with clear configuration sensitivity including a near-zero or negative case. As an illustrative strongest-case example in a controlled instruction-versus-numeric setting, an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance (0.133→ 0.632) while not improving verifiable instruction following on IFEval (ILA: 0.313→ 0.271; PLA: 0.250→ 0.143; values rounded to three decimals). We refer to this nominal–realized mismatch pattern as capability drift as a descriptive label. The mismatch is visible in the raw cross-task performance matrix; we use a drift score only as a compact summary in the same units as the underlying metrics, not as a new formal metric contribution. Evidence from broader instruction-following benchmarks is benchmark-dependent and mixed, reflecting heterogeneity in how instruction following is operationalized; we therefore do not treat cross-benchmark agreement as a premise. Overall, the practical takeaway is to perform routine cross-task evaluation before deployment and to avoid treating nominal labels as reliable capability proxies. 1 Introduction LoRA adapters (Hu et al., 2021) are often trained with a nominal task label—instruction-tuned, numeric-reasoning-tuned, etc.—and then treated as if that label reliably describes what improves after deployment. In this paper, our strongest evidence is explicitly bounded to strict, automatically verifiable instruction following (IFEval) rather than to all notions of instruction following. We ask a deployment-facing question: do nominal training objectives reliably predict realized cross-task capability gains, especially on verifiable compliance that can be audited automatically? We frame the work as an empirical cross-task diagnosis: evaluate the same adapter across tasks and compare its nominal-objective gain to its realized gains elsewhere. Across multiple seeds, base models, and LoRA settings in our runs, we find the nominal–realized mismatch recurrent but not universal and clearly configuration-sensitive, including a near-zero or slightly negative case. Table˜1 provides an illustrative strongest-case example: an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance while not improving strict verifiable instruction following on IFEval (Zhou et al., 2023). The primary empirical support, however, comes from the robustness pattern across seeds/models/settings summarized in Table˜2 and Appendix Table˜6. Instruction tuning is widely used to improve controllability (Zhang et al., 2023), and instruction following can sometimes be elicited via prompting even without instruction tuning (Brown et al., 2020). But improved helpfulness or instruction-following behavior in broad settings does not imply improved verifiable compliance under strict constraints (Zhou et al., 2023; Zhang et al., 2023). We therefore treat benchmark heterogeneity as central rather than incidental: different benchmarks operationalize “instruction following” differently, so agreement across them should not be assumed a priori. We focus on empirical cross-task diagnosis rather than adapter composition, mechanistic attribution, or metric innovation. We use capability drift only as a descriptive label for the observed nominal–realized mismatch pattern. The practical motivation is that nominal labels are used in model selection and deployment; cross-task evaluation provides a low-cost check for unintended or off-target shifts. Contributions. We make three restrained contributions aligned with our experimental evidence: 1. We formulate a deployment-relevant cross-task diagnostic framing: test whether an adapter’s nominal label/training objective aligns with realized cross-task capability gains under evaluation. 2. We provide robustness evidence across seeds, base models, and LoRA settings, showing the mismatch recurrent but configuration-sensitive in magnitude, including a near-zero or slightly negative case. 3. We give a controlled counterexample: an instruction-tuned adapter can substantially improve off-target NM-based numeric benchmark performance while not improving strict, automatically verifiable instruction following on IFEval. We additionally report supplementary probing and breakdown analyses, but we treat them as descriptive/exploratory rather than as mechanistic evidence. 2 Related Work Adapters, composition, and merge side effects. Parameter-efficient adaptation via inserted modules (adapters) (Hu et al., 2021; Houlsby et al., 2019) supports modular reuse and composition. Prior work explores sharing and composing task-specific components (Pfeiffer et al., 2020a, b), including dynamic composition of LoRA modules (Huang et al., 2023), as well as weight-space mixing and merging of fine-tuned models (Wortsman et al., 2022; Ilharco et al., 2022; Yadav et al., 2023). These lines emphasize interoperability and sometimes analyze when combinations succeed or fail; in contrast, our setting does not involve merging multiple trained adapters or composing multiple tasks at inference time, and we instead diagnose the mismatch between a single adapter’s nominal objective and its realized cross-task capability gains. Negative transfer and cross-task trade-offs. Multi-task and transfer learning can exhibit cooperation as well as competition, where optimizing for one objective harms performance on another (Caruana, 1997; Ruder, 2017; Standley et al., 2019). Our empirical phenomenon is related but more specific: we diagnose a divergence between the nominal training objective (e.g., instruction tuning) and the most improved measured capability under cross-task evaluation, even without explicit multi-task training. We use capability drift as shorthand for this mismatch; the contribution is the cross-task diagnostic framing and controlled robustness evidence rather than introducing a new formal object. Instruction tuning and evaluation mismatch. Instruction tuning is broad and evaluated with heterogeneous benchmarks and protocols (Ouyang et al., 2022; Zhang et al., 2023; Wang et al., 2023). Strict, automatically verifiable instruction-following benchmarks such as IFEval (Zhou et al., 2023) capture a narrower notion of compliance than preference-based or open-ended evaluations (Zheng et al., 2023). Recent work further highlights that verifiable instruction-following performance can be benchmark-dependent and may not generalize to unseen constraint families (Pyatkin et al., 2025). Our main quantitative evidence is therefore anchored on verifiable instruction following, and we treat mixed evidence across additional benchmarks as contextual rather than as generalizing to all notions of instruction following. Overlap, interference, and localized versus distributed explanations. Comparing learned changes across fine-tuning runs is often framed in terms of representation similarity and shared directions (Kornblith et al., 2019). Related to this, weight-space “task vectors” (Ilharco et al., 2022) provide a lens for describing learned updates and their interactions. More broadly, interpretability work on superposition and polysemantic features suggests why changes may be distributed rather than localized to a single unit or module (Elhage et al., 2022). Our probing follows this spirit but remains deliberately conservative: we test whether a simple localized account emerges, and interpret negative results as evidence against a single-module explanation rather than as a definitive mechanistic conclusion. 3 Setup 3.1 Base model and LoRA adapters We consider a fixed base model and two primary LoRA adapters trained with different nominal objectives: reason (numeric-reasoning-tuned) and instr (instruction-tuned). We also train/evaluate a domain adapter as a secondary setting, but it is not a core pillar of the main claim and is deferred to the appendix. We do not retrain large models; all analyses reuse existing training and evaluation artifacts. 3.2 Tasks and metrics We evaluate three task families: (i) numeric reasoning benchmark, (i) verifiable instruction following, and (i) domain QA (secondary). Numeric reasoning benchmark. We report numeric match (NM), which measures whether the final numeric answer matches the reference. Throughout the paper, mentions of “reasoning” refer to this benchmark’s NM-based operationalization, rather than a claim about broad reasoning ability. Verifiable instruction following (IFEval). IFEval (Zhou et al., 2023) evaluates strict, automatically verifiable instruction constraints. We report two verifiable metrics: instruction-level accuracy (ILA) and prompt-level accuracy (PLA). ILA captures per-instruction constraint satisfaction (aggregated over instructions), while PLA captures whether the full prompt satisfies all constraints. 3.3 Drift score Let a denote an adapter and let b denote a destination task/metric. Let ℳt(⋅)M_t(·) be the scalar metric value for task t. For instruction tuning, we use IFEval prompt-level accuracy (PLA) as the target metric in drift score because it directly reflects strict, end-to-end verifiable compliance; we still report ILA throughout. We define: TargetGain(a) (a) =ℳt(a)(a)−ℳt(a)(base), =M_t(a)(a)-M_t(a)(base), (1) OffTargetGain(a,b) (a,b) =ℳb(a)−ℳb(base), =M_b(a)-M_b(base), (2) DriftScore(a→b) (a→ b) =OffTargetGain(a,b)−TargetGain(a), =OffTargetGain(a,b)-TargetGain(a), (3) where t(a)t(a) is the nominal target task of adapter a. Intuitively, a large positive DriftScore(a→b)DriftScore(a→ b) indicates that off-target gains exceed target-task gains (or that target-task gains are absent while off-target gains are large). Metric choice rationale. We use a strict, automatically verifiable target metric for instruction tuning (IFEval PLA) because our central question is whether nominal instruction tuning translates into verifiable compliance rather than broader helpfulness; PLA is an end-to-end criterion that is sensitive to any violated constraint, while ILA provides a complementary per-constraint view. For the numeric reasoning benchmark, we use numeric match (NM) because it is simple, task-aligned, and directly comparable across model variants in our setup. We define drift score as a difference of gains to keep the diagnostic interpretable in the same units as the underlying metrics and to avoid introducing additional normalization choices that would require extra assumptions. We emphasize that drift score is a diagnostic summary of cross-task mismatch in this study, not a universal or uniquely correct capability metric, and the central mismatch is already visible in the raw cross-task performance matrix. 4 Cross-task mismatch under cross-task evaluation 4.1 Reading the cross-task table Our primary empirical support is the robustness pattern: across multiple seeds, base models, and LoRA settings, nominal labels can diverge from realized cross-task capability gains in a recurrent but configuration-sensitive way (Table˜2 and Appendix Table˜6). Table˜1 is an illustrative strongest-case example that makes the nominal–realized mismatch immediately visible in raw cross-task metrics. We refer to this mismatch as capability drift as a descriptive label. Table 1: Illustrative strongest-case cross-task evaluation (main setting). We report NM-based numeric benchmark performance (NM) and IFEval strict, automatically verifiable instruction following (ILA/PLA), with values rounded to three decimals. This example provides a concrete view of the mismatch in raw cross-task metrics; robustness results are summarized in Table˜2 and Appendix Table˜6. Model Nominal task Numeric NM IFEval ILA IFEval PLA Realized change Base – 0.133 0.313 0.250 Reference point. Reason adapter Numeric 0.309 0.271 0.179 Numeric gain aligned with target. Instruction adapter Instruction 0.632 0.271 0.143 Numeric gain, no IFEval gain. 4.2 Illustrative strongest-case example We emphasize that Table˜1 is illustrative; the claim is established by robustness results in Section˜5. The illustrative mismatch is explicit in Table˜1: the instruction-tuned adapter improves off-target NM-based numeric benchmark performance substantially while failing to improve strict, automatically verifiable instruction following on IFEval. NM is a benchmark-specific operationalization of numeric performance in our setup (numeric match), and we do not interpret NM gains as a claim about broad reasoning shifts. Concretely, NM rises from 0.133 (base) to 0.632 (instruction adapter), yet IFEval decreases (ILA: 0.313→ 0.271; PLA: 0.250→ 0.143; values rounded to three decimals). By contrast, the numeric-reasoning-tuned adapter achieves a smaller NM gain (0.309) and does not exhibit a comparably large off-target improvement. Alternative operationalizations (e.g., EM) may change the magnitude of effects, but the nominal–realized mismatch is already visible in the raw cross-task matrix. 4.3 Quantifying drift with drift score Figure˜1 summarizes drift scores computed from the same evaluation metrics. Instruction→ reasoning is the most pronounced case: because the instruction adapter’s target-task metric (IFEval PLA) does not improve while its off-target NM gain on the numeric reasoning benchmark is large, the drift score is high and positive. The drift score is a compact summary; the central empirical point does not depend on introducing a new metric beyond the underlying cross-task matrix. Figure 1: Drift score summary. Heatmap of DriftScore(a→b)DriftScore(a→ b) computed from cross-task evaluation metrics. Compact overview of score magnitudes from the same evaluations; see §5 for context. 5 Robustness Across Seeds, Models, and Benchmarks Nominal labels can diverge from realized cross-task gains across seeds, base models, and LoRA settings, in a recurrent but configuration-sensitive way that includes a near-zero or slightly negative case. Throughout this section, drift score uses the numeric reasoning benchmark NM as the off-target metric and IFEval PLA as the verifiable target metric (equation˜3). Table˜2 summarizes the most directly comparable robustness slices. Figure˜2 visualizes this primary claim directly using the same robustness evaluations summarized in Table˜2 and Appendix Table˜6. −0.2-0.2−0.18-0.18−0.16-0.16−0.14-0.14−0.12-0.12−0.1-0.1−8⋅10−2-8· 10^-2−6⋅10−2-6· 10^-2−4⋅10−2-4· 10^-2−2⋅10−2-2· 10^-202⋅10−22· 10^-200.10.10.20.20.30.30.40.40.50.50.60.6TargetGain on IFEval PLA (instr−-base)OffTargetGain on numeric NM (instr−-base)Seed sweepModel/setting sweepBenchmark-suite slice Figure 2: Robustness quadrant plot (primary evidence). Per-run target gain on IFEval PLA (instr−-base) versus off-target gain on numeric NM (instr−-base) for the robustness evaluations underlying Table˜2 and Appendix Table˜6. Most points lie above y=xy=x (positive drift), with occasional near-zero or negative cases. Multi-seed robustness. On Qwen3-8B, the drift score remains positive across five seeds under two LoRA ranks, with non-trivial but bounded variance (e.g., mean 0.511± 0.178 for r=16r=16). This supports that the main mismatch is not a single-seed artifact in this setting. Multi-model and multi-setting robustness. Across multiple base models under a shared setting (r=16r=16 with attention+MLP modules), drift scores remain positive with small across-seed variance for models where multiple seeds are available. Across additional LoRA settings, most observed drift scores remain positive, but we also identify configuration sensitivity: one model/setting yields a slightly negative drift score. Overall, drift magnitude depends on the base model and adapter setting, including a near-zero or slightly negative case. Additional verifiable instruction-following benchmarks. We also run a benchmark suite including FollowBench and IFBench for the same base/adapters. The results are benchmark-dependent: IFBench and IFEval both emphasize stricter, automatically verifiable compliance and are qualitatively consistent here (no improvement for the instruction adapter), while FollowBench operationalizes a different notion and improves. We treat this heterogeneity as an insight rather than as a contradiction: cross-benchmark agreement is not a premise because “instruction following” is not a single operationalization. Accordingly, we report the multi-benchmark evidence as contextual and supplementary (Table˜5) and keep the strongest claim explicitly tied to strict, automatically verifiable instruction following on IFEval. Table 2: Robustness summary of drift score (instr→ reasoning). Drift score remains positive across five seeds (Qwen3-8B) and across multiple base models and LoRA settings, with one model/setting showing near-zero or slightly negative drift (configuration sensitivity). Slice Model LoRA setting n Drift score (↑ ) Seeds qwen3_8b r16_attnmlp_do005_lrmain 5 0.511± 0.178 Seeds qwen3_8b r8_attnmlp_do005_lrmain 5 0.495± 0.150 Models qwen3_14b r16_attnmlp_do005_lrmain 2 0.667± 0.014 Models qwen3_8b r16_attnmlp_do005_lrmain 2 0.597± 0.010 Models llama31_8b_instruct r16_attnmlp_do005_lrmain 2 0.394± 0.016 Models qwen25_7b_instruct r16_attnonly_do0_lrsmall 1 -0.040 6 Selective Shifts in Verifiable Instruction Following The aggregate IFEval metrics (ILA/PLA) hide which verifiable constraints change. We therefore inspect IFEval category- and type-level breakdowns for the instruction adapter relative to base (figure˜3). These breakdowns are descriptive and qualitative: low-support categories or types should be interpreted cautiously, and per-type differences are not intended as high-confidence estimates. Two representative degradations illustrate the pattern. At the category level, language constraints drop sharply (rate 0.5→ 0.0; Δ=−0.5 =-0.5), and detectable_format also decreases (Δ=−0.2 =-0.2). At the type level, keywords:existence shows a large negative shift (Δ=−1.0 =-1.0). At the same time, some types improve (e.g., keywords:letter_frequency and punctuation:no_comma in the top positive shifts). Overall, the aggregate non-improvement on IFEval is not uniformly distributed across verifiable instruction types; this breakdown is descriptive and does not by itself identify why the mismatch occurs. (a) Category-level Δ (instr−-base). (b) Type-level top positive/negative shifts. Figure 3: Selective shifts in verifiable instruction following (IFEval). Category-level and type-level shifts for the instruction adapter relative to base on IFEval. This breakdown is qualitative; low-support types should be interpreted cautiously. 7 Probing Possible Localized Explanations This section is supplementary and exploratory. We probe whether a simple localized account emerges (e.g., a small set of modules or a dominant direction), but we do not treat this as mechanistic or causal evidence. Summary. We observe modest geometric overlap between instruction- and numeric-reasoning-tuned LoRA updates in a small number of upper-layer modules, but the overlap is not uniformly high across modules. Takeaway. Localized functional tests do not provide strong causal evidence linking any particular module/direction to the drift behavior. Overall, these probes do not support a simple localized explanation in this setup; they should be read as exploratory diagnostics, with additional details in Appendix Figures˜4 and 5. 8 Discussion and Limitations Nominal labels can diverge from realized gains. In our evaluated settings, an adapter’s nominal label or nominal training objective does not reliably predict realized cross-task capability gains under deployment-relevant evaluation. Cross-task evaluation can reveal nominal–realized mismatches that are invisible when only the nominal target task is reported. Instruction tuning vs verifiable instruction following. Instruction tuning should not be conflated with improved strict, automatically verifiable instruction following. In our illustrative strongest-case example, off-target NM-based numeric benchmark performance improves substantially while IFEval does not. IFEval targets strict, automatically verifiable compliance and is narrower than broader notions of helpful or conversational instruction following; benchmark-dependent operationalizations therefore matter when interpreting whether a nominal–realized mismatch is present. Plausible hypotheses (speculative). One possibility is that instruction tuning shifts response priors (e.g., completion style or answer-format regularity) in ways that help a benchmark-specific numeric task while leaving strict constraint satisfaction unchanged. We do not adjudicate such hypotheses here, and we avoid interpreting the results as evidence about broad reasoning shifts. Limitations. First (empirical scope), our findings are empirical and specific to the models, tasks, and adapters evaluated here. Second (claim scope), our strongest claim is tied to strict, automatically verifiable instruction following (IFEval) rather than to broader notions of instruction following. Third (benchmark heterogeneity), broader instruction-following benchmarks operationalize different targets and yield benchmark-dependent evidence in our suite, so cross-benchmark agreement should not be assumed. Fourth (probing inconclusive), our exploratory probing remains non-causal and inconclusive: it does not support a simple localized explanation, but it does not establish a mechanism. Fifth (metric choice), we operationalize benchmark-specific numeric performance via NM and summarize mismatch with drift score; alternative operationalizations (e.g., EM or different instruction-following aggregations) can change magnitudes, as shown in Appendix Tables˜3 and 4. We also observe configuration sensitivity, including a near-zero or slightly negative case, which should be interpreted as evidence that the pattern is recurrent but not universal. 9 Conclusion We present an empirical cross-task diagnosis for LoRA adapters. Across multiple seeds, base models, and LoRA settings, we observe a recurrent but not universal nominal–realized mismatch, with the strongest evidence tied to strict, automatically verifiable instruction following (IFEval). An illustrative case shows large off-target NM-based numeric gains without IFEval improvement. The practical takeaway is to perform routine cross-task evaluation before deployment and avoid treating nominal labels as reliable capability proxies. References T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020) Language models are few-shot learners. External Links: 2005.14165, Link Cited by: §1. R. Caruana (1997) Multitask learning. Machine Learning 28 (1), p. 41–75. External Links: Link Cited by: §2. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah (2022) Toy models of superposition. External Links: 2209.10652, Link Cited by: §2. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly (2019) Parameter-efficient transfer learning for NLP. External Links: 1902.00751, Link Cited by: §2. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021) LoRA: low-rank adaptation of large language models. External Links: 2106.09685, Link Cited by: §1, §2. C. Huang, Q. Liu, B. Y. Lin, T. Pang, C. Du, and M. Lin (2023) LoraHub: efficient cross-task generalization via dynamic lora composition. External Links: 2307.13269, Link Cited by: §2. G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi (2022) Editing models with task arithmetic. External Links: 2212.04089, Link Cited by: §2, §2. S. Kornblith, M. Norouzi, H. Lee, and G. E. Hinton (2019) Similarity of neural network representations revisited. External Links: 1905.00414, Link Cited by: §2. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe (2022) Training language models to follow instructions with human feedback. External Links: 2203.02155, Link Cited by: §2. J. Pfeiffer, A. Kamath, A. Rücklé, K. Cho, and I. Gurevych (2020a) AdapterFusion: non-destructive task composition for transfer learning. External Links: 2005.00247, Link Cited by: §2. J. Pfeiffer, A. Rücklé, C. Poth, A. Kamath, I. Vulić, S. Ruder, K. Cho, and I. Gurevych (2020b) AdapterHub: a framework for adapting transformers. External Links: 2007.07779, Link Cited by: §2. V. Pyatkin, S. Malik, V. Graf, H. Ivison, S. Huang, P. Dasigi, N. Lambert, and H. Hajishirzi (2025) Generalizing verifiable instruction following. External Links: 2507.02833, Link Cited by: §2. S. Ruder (2017) An overview of multi-task learning in deep neural networks. External Links: 1706.05098, Link Cited by: §2. T. Standley, A. R. Zamir, D. Chen, L. Guibas, J. Malik, and S. Savarese (2019) Which tasks should be learned together in multi-task learning?. External Links: 1905.07553, Link Cited by: §2. Y. Wang, H. Ivison, P. Dasigi, J. Hessel, T. Khot, K. R. Chandu, D. Wadden, K. MacMillan, N. A. Smith, I. Beltagy, and H. Hajishirzi (2023) How far can camels go? exploring the state of instruction tuning on open resources. External Links: 2306.04751, Link Cited by: §2. M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, and L. Schmidt (2022) Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. External Links: 2203.05482, Link Cited by: §2. P. Yadav, D. Tam, L. Choshen, C. Raffel, and M. Bansal (2023) TIES-merging: resolving interference when merging models. External Links: 2306.01708, Link Cited by: §2. S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, and G. Wang (2023) Instruction tuning for large language models: a survey. External Links: 2308.10792, Link Cited by: §1, §2. L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging LLM-as-a-judge with MT-bench and chatbot arena. External Links: 2306.05685, Link Cited by: §2. J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou (2023) Instruction-following evaluation for large language models. External Links: 2311.07911, Link Cited by: §1, §1, §2, §3.2. Appendix A Secondary results and additional figures A.1 Domain adapter as a secondary setting We evaluate a domain-tuned adapter as an additional setting, but do not use it as core evidence for the main claim. Table˜3 provides an extended cross-task view including the domain adapter and additional metrics. Table 3: Extended cross-task evaluation including a domain adapter (secondary). This table is provided for completeness and is not central to the paper’s primary robustness claim, which is established by Table˜2 and Appendix Table˜6; Table˜1 is illustrative. All values are reported as fractions in [0,1][0,1]; numeric benchmark performance is reported using both exact match (EM) and numeric match (NM). Numeric EM Numeric NM IFEval ILA IFEval PLA Domain Acc Domain MC base 0.000 0.133 0.312 0.250 0.000 0.000 reason 0.042 0.306 0.271 0.179 0.000 0.000 instr 0.000 0.640 0.271 0.143 0.000 0.000 domain 0.000 0.281 0.345 0.148 0.000 0.000 A.2 Additional benchmarks: FollowBench and IFBench To test whether the instruction adapter improves instruction-following beyond IFEval, we report two additional benchmarks from a benchmark suite: FollowBench (rule-based score summary) and IFBench (strict prompt-level accuracy). Across three base models under the same setting, IFBench aligns more closely with IFEval because both emphasize stricter, automatically verifiable compliance, and we observe no improvement for the instruction adapter on these strict criteria. By contrast, FollowBench operationalizes a different notion and improves. We report these results as contextual and supplementary: benchmark-dependent operationalizations matter, so cross-benchmark agreement is not assumed and does not expand the paper’s strongest claim beyond IFEval. A.3 Metric sensitivity: target metric PLA vs ILA Our main diagnostic uses IFEval PLA as the target metric for instruction tuning because it is an end-to-end strict criterion, and we report ILA throughout as a complementary view. To illustrate sensitivity to this choice without introducing new experiments, we recompute the instruction→ reasoning drift score in the main setting using ILA in place of PLA (all values derived from Table˜1). The mismatch direction is unchanged: off-target NM gains on the numeric reasoning benchmark remain large while verifiable instruction-following target gains remain absent, and the drift score remains strongly positive; only the magnitude varies with the target aggregation. Table 4: Target-metric sensitivity check (PLA vs ILA). Instruction→ reasoning drift score recomputed from Table˜1 using IFEval PLA or ILA as the instruction target metric. Target metric OffTargetGain (NM) TargetGain (IFEval) DriftScore (instr→ reasoning) PLA 0.499 −0.107-0.107 0.606 ILA 0.499 −0.042-0.042 0.541 Table 5: Supplementary multi-benchmark instruction following. FollowBench rule-based mean and IFBench strict PLA for base vs instruction adapter. Values are computed from benchmark-suite outputs. FollowBench, IFBench, and IFEval differ in scale and emphasis, so absolute values should not be compared across benchmarks; in particular, FollowBench rule mean should be interpreted according to its scoring definition. Model IFBench PLA (strict) base IFBench PLA (strict) instr FollowBench rule mean base FollowBench rule mean instr Qwen3-8B 0.122 0.112 0.000 4.000 Qwen3-14B 0.143 0.116 0.000 2.000 Llama-3.1-8B-Instruct 0.139 0.119 4.000 4.500 A.4 Probing figures These figures are placed in the appendix to keep the main paper focused on the empirical diagnosis and robustness results. They provide supporting context for Section˜7: how to read them is as diagnostics for whether a simple localized account might emerge. Consistent with the main takeaway, the patterns suggest at most modest geometric overlap in a few upper-layer modules, while the functional interventions do not yield strong localized causal evidence; we therefore avoid interpreting these plots as identifying a single-module mechanism for drift. Figure 4: Upper-layer geometry diagnostics. Top modules by geometric similarity between instruction- and numeric-reasoning-tuned LoRA updates. This figure provides preliminary evidence but does not establish a localized causal mechanism. Figure 5: Functional probing summary with null or small effects. Largest observed effect sizes from localized functional probing. We do not find strong localized causal evidence that explains the mismatch pattern in this setup. A.5 Robustness tables This subsection provides the full robustness numbers referenced in Section˜5, including per-seed and per-setting slices. They are intended as supporting material for transparency and reproducibility, not as additional main claims beyond the summaries already reported in the main text. Table 6: Robustness: drift score summary (instr→ reasoning). Drift score mean and standard deviation across seeds (when n>1n>1); values are rounded to three decimals. Stage Model Setting n Drift score (mean ± std) benchmark_suite llama31_8b_instruct r16_attnmlp_do005_lrmain 1 0.4850.485 benchmark_suite qwen3_14b r16_attnmlp_do005_lrmain 1 0.6740.674 benchmark_suite qwen3_8b r16_attnmlp_do005_lrmain 1 0.6060.606 model_sweep deepseek_r1_distill_qwen_7b r16_attnonly_do0_lrsmall 1 0.2840.284 model_sweep deepseek_r1_distill_qwen_7b r32_attnmlp_do005_lrmain 1 0.3500.350 model_sweep llama31_8b_instruct r16_attnmlp_do005_lrmain 2 0.394±0.0160.394± 0.016 model_sweep llama31_8b_instruct r16_attnonly_do0_lrsmall 1 0.4260.426 model_sweep llama31_8b_instruct r32_attnmlp_do005_lrmain 1 0.3360.336 model_sweep llama31_8b_instruct r8_attnonly_do0_lrmain 2 0.451±0.0360.451± 0.036 model_sweep qwen25_7b_instruct r16_attnonly_do0_lrsmall 1 −0.040-0.040 model_sweep qwen25_7b_instruct r32_attnmlp_do005_lrmain 1 0.4710.471 model_sweep qwen3_14b r16_attnmlp_do005_lrmain 2 0.667±0.0140.667± 0.014 model_sweep qwen3_14b r16_attnonly_do0_lrsmall 1 0.6170.617 model_sweep qwen3_14b r32_attnmlp_do005_lrmain 1 0.6720.672 model_sweep qwen3_14b r8_attnonly_do0_lrmain 2 0.611±0.0060.611± 0.006 model_sweep qwen3_8b r16_attnmlp_do005_lrmain 2 0.597±0.0100.597± 0.010 model_sweep qwen3_8b r16_attnonly_do0_lrsmall 1 0.4810.481 model_sweep qwen3_8b r32_attnmlp_do005_lrmain 1 0.6260.626 model_sweep qwen3_8b r8_attnonly_do0_lrmain 2 0.495±0.0060.495± 0.006 seed_sweep qwen3_8b r16_attnmlp_do005_lrmain 5 0.511±0.1780.511± 0.178 seed_sweep qwen3_8b r8_attnmlp_do005_lrmain 5 0.495±0.1500.495± 0.150 Benchmark-suite run status (bookkeeping). All benchmark-suite evaluations referenced in the paper completed successfully for the included model/seed/setting combinations; we omit the full per-run status table here because it is bookkeeping rather than a scientific result.

Instant research discovery