Paper deep dive
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
Atul Anand, Sourav Chattaraj
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/5/2026, 3:39:38 AM
Summary
This paper investigates 'Instruction Stacking Collapse,' where Large Language Models (LLMs) fail to follow multiple simultaneous instructions. The authors introduce a benchmark stacking 24 verifier-checked instructions and evaluate three production-tier models (Claude Sonnet 4.6, GPT-5-mini, Gemini 2.5 Flash). Results show non-linear degradation in instruction-following rates as constraints accumulate, driven by pairwise conflicts (e.g., JSON output conflicting with markdown formatting). A training-free 'instruction compiler' that rewrites prompts to resolve conflicts and structure instructions recovers performance significantly for weaker models (up to +11 points) but has negligible or negative effects on stronger models, suggesting the benefit is capability-dependent.
Entities (9)
Relation Signals (7)
Instruction Stacking Collapse → affects → Instruction-Following
confidence 95% · Instruction-following degrades non-linearly: the follow rate falls from ~96% to as low as 20%
Instruction Stacking Collapse → causedby → Pairwise Conflict
confidence 93% · driven by a structured and reproducible set of pairwise conflicts.
Instruction Compiler → benefits → GPT-5-mini
confidence 92% · It recovers up to +11 points of follow rate for weaker models
JSON → conflictswith → Markdown Instructions
confidence 91% · valid JSON cannot also carry markdown headers
Instruction Compiler → mitigates → Instruction Stacking Collapse
confidence 90% · It recovers up to +11 points of follow rate for weaker models
Instruction Compiler → haslittleeffecton → Claude Sonnet 4.6
confidence 88% · leaving stronger models, which already internalise the same structure, essentially unchanged.
IFEval → comparedto → Instruction Stacking Collapse
confidence 85% · Existing benchmarks do not chart this. IFEval... measures one or two instructions per prompt
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Production prompts rarely carry a single instruction. One system message may require valid JSON, a word limit, three citations, and a fixed tone at the same time. We study how instruction-following degrades as such constraints accumulate. We introduce a benchmark that stacks 24 verifier-checked instructions, one to twenty at a time, and evaluate three production-tier LLMs (Claude Sonnet 4.6, GPT-5-mini, Gemini 2.5 Flash). Instruction-following degrades non-linearly: the follow rate falls from ~96% to as low as 20%, driven by a structured and reproducible set of pairwise conflicts. A single "output JSON" constraint, for example, is jointly unsatisfiable with nine others. We then evaluate a training-free remedy: an instruction compiler that rewrites the stacked prompt in a single LLM call and is reused across queries. Its benefit is capability-graded. It recovers up to +11 points of follow rate for weaker models, which are also the models most often deployed at scale, while leaving stronger models, which already internalise the same structure, essentially unchanged. Cluster-robust tests, same-baseline controls, and a within-family scaling ladder attribute the gain to the rewrite itself rather than to additional tokens, reordering, or measurement headroom. We release the benchmark, verifiers, and cached runs for full reproduction.
Tags
Links
- Source: https://arxiv.org/abs/2608.02639v1
- Canonical: https://arxiv.org/abs/2608.02639v1
Trouble viewing inline? Open PDF directly →
Full Text
39,186 characters extracted from source content.
Expand or collapse full text
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation Atul Anand Sourav Chattaraj Abstract Production prompts rarely carry a single instruc- tion. One system message may require valid JSON, a word limit, three citations, and a fixed tone at the same time. We study how instruction- following degrades as such constraints accumu- late.We introduce a benchmark that stacks 24 verifier-checked instructions, one to twenty at a time, and evaluate three production-tier LLMs (Claude Sonnet 4.6, GPT-5-mini, Gem- ini 2.5 Flash). Instruction-following degrades non- linearly: the follow rate falls from∼96%to as low as20%, driven by a structured and reproducible set of pairwise conflicts. A single “output JSON” constraint, for example, is jointly unsatisfiable with nine others. We then evaluate a training-free remedy: an instruction compiler that rewrites the stacked prompt in a single LLM call and is reused across queries. Its benefit is capability-graded. It recovers up to+11points of follow rate for weaker models, which are also the models most often deployed at scale, while leaving stronger models, which already internalise the same struc- ture, essentially unchanged. Cluster-robust tests, same-baseline controls, and a within-family scal- ing ladder attribute the gain to the rewrite itself rather than to additional tokens, reordering, or measurement headroom. We release the bench- mark, verifiers, and cached runs for full reproduc- tion. 1. Introduction Large language models reliably follow a single instruction, but production prompts rarely contain only one. A deployed system prompt may at once require a specific output format, a length limit, citations, and a fixed tone. When many such constraints are combined, models satisfy fewer of them, and the violations are silent: no error is raised when an instruc- tion is dropped. It is widely reported that compliance falls as the number of constraints grows, yet there is no systematic account of how quickly it falls, which instructions interfere with one another, or whether the loss can be recovered at Table 1. One response, eight stacked instructions, four silent fail- ures. Satisfying F1 (JSON) made four markdown-/case-dependent instructions impossible, a cascade from a single structural choice (GPT-5-mini, advice task). Instruction (8 stacked)VerdictWhy C1 reference a year✓ passmatched “2021” F1 valid JSON✓ passparses L1≤50 words✓ pass29 words X1 use “therefore”✓ pass ×1 F4 “Summary:” line✗ failJSON has no bare trailing line L4 title≤10 words✗ failfirst line is the 29-word JSON R2 ##Assumptions✗ failno markdown header inside JSON X4 all-lowercase✗ failJSON keys force an uppercase char inference time. Table 1 makes the failure concrete. Given eight stacked instructions, GPT-5-mini obeyed “output JSON”, and that one choice silently broke four others. No error was raised; four constraints simply went unmet. Section 4.2 shows this is the rule, not the exception. Existing benchmarks do not chart this. IFEval (Zhou et al., 2023) measures one or two instructions per prompt; Fol- lowBench (Jiang et al., 2024) stacks constraints but only within a single category. Both answer “can the model obey instructionX?” Neither answers “what happens when a dozen instructions from different categories must all hold at once?” That is the question deployment actually poses. This paper maps the collapse and tests a fix. We build a benchmark of 24 atomic instructions across six categories (format, length, lexical, structural, reasoning, content), each paired with a released verifier, and stack them uniformly at random into prompts of size 1–20. We evaluate three production-tier models, one per major provider. We then ask three questions and, because we pre-registered our pre- dictions, report plainly where the data confirmed them and where it did not. Collapse (RQ1). Follow rate decays smoothly but non- linearly with stack size, from∼0.96at one instruction to 0.60(Sonnet),0.43(Gemini), and0.20(GPT-5-mini) at twenty. A sigmoid (flat near baseline, then a bend) fits two of three models, and a bootstrap over the model-selection itself picks it in 93–100% of refits, so the shape is not an 1 arXiv:2608.02639v1 [cs.SE] 31 Jul 2026 Instruction Stacking Collapse Stacked prompt 1–20 instructions JSON·≤50w·5 bul- lets·##Assumptions· LLM Collapse (Section 4) follows 20–60% at 20 instructions Instruction compiler one LLM call: cluster·merge·precedence LLM Capability-graded recovery (Section 5) weak+11p·mid+3p strong−1p Figure 1. Overview. Stacking many instructions into one prompt collapses instruction-following (top). A training-free instruction compiler rewrites the stack in one LLM call; its benefit is capability-graded: large for weak models, neutral-to-negative for strong ones (bottom). artifact of fitting four parameters to seven points. Conflict (RQ2). The cause is interference between instruc- tions. Testing all 231 pairs and separating the 15 that are log- ically impossible (valid JSON cannot also carry markdown headers) from genuine behavioral failures, we find∼12% of the satisfiable pairs (95% CI[4.6, 11.6]) still fail far more than their individual rates predict. The structure is inter- pretable and reproducible: “output JSON” is the dominant conflict hub, “cite named sources” the dominant synergy, and the whole interaction landscape correlates across both tasks and all three models (ρ = +0.23 to +0.28). Mitigation (RQ3). We then ask whether the loss can be recovered without training. We evaluate an instruction com- piler: a single LLM call that rewrites the raw stack by clustering instructions by category, merging redundant ones, and inserting a precedence note where two rules conflict. The rewrite is computed once per instruction set and reused across queries. Its effect is capability-graded. It recovers +11.0p of follow rate for the weakest target (GPT-5-mini) and+3.3p for the middle one (Gemini), while leaving the strongest target (Sonnet) essentially unchanged (−1.2p, not statistically robust). The recovery is thus concentrated on the smaller models that are most often deployed at scale, where the need is greatest. The effect is smaller than the d≥ 0.5we pre-registered, but its source is well identified: three controls attribute it to the semantic rewrite rather than to additional tokens, reordering, or the compiler grading its own output, and a within-family scaling ladder reproduces the ordering (Spearman−0.85between raw strength and recovery). Interpretation. Two readings are consistent with this pat- tern. The practical reading: because the rewrite is computed once and reused, it is an inexpensive intervention precisely for the smaller, faster models that serve most production traf- fic, and is unnecessary for top-tier models. The mechanistic reading: a strong model reconstructs the stack’s structure on its own while reading the raw prompt, whereas a weaker model does not, so making that structure explicit helps the weaker model and is redundant for the stronger one. We do not overstate the magnitude: even in the best case the weak- est model still leaves about two-thirds of its instructions unsatisfied at the largest stack size. Contributions. 1.A stacked-instruction benchmark (24 verifier-backed instructions, six categories, random stacks of size 1–20) and an audited verifier suite (Section 3). 2.A characterization of collapse across three models and three task types, with bootstrap-stable functional forms and the finding that collapse is universal but per-model ordering is task-dependent (Sections 4 and 4.3). 3. A conflict topology that separates designed-in impos- sibilities from behavioral conflicts and is shared across tasks and models (Section 4.2). 4. A training-free instruction compiler and a controlled evaluation of it: a capability-graded recovery that is largest for weaker models, with the active mechanism isolated by ablation and a within-family ladder that re- moves the cross-lab confound (Section 5). To our knowl- edge this is the first analysis of how the value of a prompt rewrite depends on target capability. All code, cached responses, verifiers, and analysis scripts are released. 2. Related Work Instruction-following benchmarks. IFEval (Zhou et al., 2023) pioneered regex-verifiable instructions but places only one or two per prompt, so it cannot see degrada- tion with count. FollowBench (Jiang et al., 2024) stacks constraints within a single category; ours is the random cross-category analog, exposing the cross-category interac- tions (Section 4.2) a within-category design cannot. RE- CAST (Guo et al., 2025) and MulDimIF (Ye et al., 2026) push multi-constraint training data; we target evaluation and mitigation (Table 2; full version in Section B). 2 Instruction Stacking Collapse Table 2. Benchmark comparison (full version in Section B). PropertyIFEvalFollowB.RECASTMulDimOurs Instr./prompt1–21 cat30+multi1–20 Cross-categorynonopart.part.yes Pairwise conflictnonodatadataFDR Imposs.-vs-behav.nonononoyes Train-free mitig.nonononoyes Prompt rewriting and inference-time mitigation. Sev- eral methods improve constraint-following by editing the prompt at inference time. PRewrite (Kong et al., 2024) trains a prompt rewriter with reinforcement learning; De- CRIM (Ferraz et al., 2024) decomposes an instruction into atomic constraints and runs an iterative critique-and-refine loop; RECAST (Guo et al., 2025) and MulDimIF (Ye et al., 2026) instead curate multi-constraint training data. All of these require either model training or several model calls per query. Our instruction compiler differs in being a sin- gle, zero-shot, training-free rewrite that is computed once per instruction set and reused across all queries, adding no per-query cost. More importantly, prior work treats prompt rewriting as a uniform improvement; we are not aware of any study that characterises how its value varies with the capability of the target model. That dependence, rather than the rewrite itself, is the methodological contribution of Section 5. Capability-dependent prompt engineering. Our central claim, that one compilation step helps the weak model far more than the strong one, is the mirror image of Hakim (2026), where brevity constraints help the strong model by curbing over-elaboration. Both say a prompt-level fix has model-dependent value. This is not inverse scaling on the task (McKenzie et al., 2023), since the task is fixed, but inverse scaling on the value of a prompt-engineering move. Qi et al. (2026) note a related paradox where models fail by adding a self-evident constraint already met implicitly. 3. The Stacking-Collapse Benchmark Instructions. 24 atomic instructions, four per category (Table 3). Each has one canonical phrasing shown verbatim and a verifier mapping a response to pass/fail. Two (analogy, counterargument) need an LLM judge and are excluded, leaving 22 deterministic instructions as the active pool; the full list is in Section A. Verifiers. Each verifier is strict on intent, lenient on cos- metic markup (case-insensitive headers; any list marker counts; an optional code-fence around JSON is stripped). These policies came from a pilot audit that flipped 8.0% of verdicts. A human audit (∼500labels over 19 verifiers) finds 18/19 agree with humans at 100%, one (C4) at 80%, and caught and fixed a real bug in L4 that was re-scored Table 3. The six categories (representative instructions; full set in Section A). CategoryExamples Format (F1–4)valid JSON; code-fence;≥3 bullets; “Summary:” line Length (L1–4) ≤50 words;≥150 words; exactly 5 bullets; title≤10 Lexical (X1–4)use “therefore”; avoid “however”; 3 citations; lowercase Structural (S1–4)begin “Answer:”; ##Pros/##Cons; 2nd-person; no “?” Reasoning (R1–4)##Reasoning; ##Assumptions; Confidence:0.X; ##Alt. Content (C1–4)a year; analogy ∗ ; counterarg ∗ ;≥2 sources ∗ require an LLM judge; excluded from the active pool. 0.02.55.07.510.012.515.017.520.0 Number of Stacked Instructions 0.0 0.2 0.4 0.6 0.8 1.0 Instruction-Following Rate Instruction-Following Degrades with Stack Size Claude Sonnet 4.6 GPT-5-mini Gemini 2.5 Flash Figure 2. Instruction-following rate vs. stack size (advice). Each point is a mean over 30 random stacks×10 items (∼300trials). Follow rate falls from∼0.96to0.20–0.60; GPT-5-mini collapses sharpest. from cache with no new calls. Three verifiers (R1, R3, S2) are spot-checked only; a full 100-label-per-instruction vali- dation is owed (Section 6). Details and the agreement table are in Section C. Grid. Stacks are sampled uniformly at random (not screened for feasibility, since real prompts are not), at sizes 1, 3, 5, 8, 12, 16, 20, 30 stacks/size×10 items. Three models: Claude Sonnet 4.6, GPT-5-mini, Gemini 2.5 Flash, temperature 0. The primary task is open-ended advice; GSM8K math and HumanEval code are general- ization checks (Section 4.3). All responses are cached by (model, system, user) hash for exact reproducibility. 4. Characterizing the Collapse 4.1. Degradation is real and non-linear Follow rate falls from∼0.96at one instruction to0.604 (Sonnet),0.433(Gemini), and0.201(GPT-5-mini) at twenty (Table 4, Figure 2). Cluster-bootstrap CIs (resampling re- sponses) are mutually non-overlapping at stack≥ 16, so the ordering is reliable. The shape is non-linear: an AIC comparison of linear / exponential / sigmoid selects sig- moid for Sonnet and GPT-5-mini, and a bootstrap over the model selection itself confirms it is stable (sigmoid wins 93% of refits for Sonnet, 100% for GPT-5-mini). Gemini 3 Instruction Stacking Collapse Table 4. Mean follow rate, cluster-bootstrap 95% CI (advice). Modelstack=1stack=5stack=16stack=20 Sonnet 4.6.964 [.94,.99].827 [.81,.85].621 [.60,.64].604 [.59,.62] GPT-5-mini.964 [.94,.99].845 [.82,.87].259 [.24,.28].200 [.19,.21] Gemini 2.5 F.959 [.93,.99].823 [.80,.84].535 [.52,.56].433 [.41,.45] F1F2F3F4 L1L2L3L4 X1X2X3X4 S1S2S3S4 R1R2R3R4 C1C4 F1 F2 F3 F4 L1 L2 L3 L4 X1 X2 X3 X4 S1 S2 S3 S4 R1 R2 R3 R4 C1 C4 F F L L X X S S R R C C Pairwise Interaction (observed P(A)·P(B)) 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 interaction Blue = conflict (joint follow-rate lower than independent), red = synergy. Dots: BH-significant at FDR 0.05. Figure 3. Pairwise interaction for the 231 pairs (Sonnet), ordered by category. Blue=conflict (joint rate below independent base- line); red=synergy; dots are BH-significant at FDR 0.05. F1 (JSON) is the dominant conflict row; C4 (named sources) anchors the synergies. is non-sigmoidal: its decline begins immediately and is best described as linear-to-exponential (sigmoid in< 2%of refits). Categories degrade unevenly: lexical survives best, format and length collapse hardest (Section D). We operationalize target strength as a model’s raw stacked- following rate (Sonnet>Gemini>GPT-5-mini), the axis Section 5 grades the compiler against. 4.2. The failure is structured: a conflict topology Recall Table 1: one structural commitment, emitting JSON, silently broke four other instructions in a single response. That cascade is not an accident of one prompt; it is the aggregate pattern, which we now quantify across all pairs. Testing all 231 pairs (interaction= observed(A∧ B)− P(A)P(B) , bootstrap CI, BH-corrected at FDR 0.05; Fig- ure 3), we separate two things a raw count conflates: • 15 logical impossibilities (e.g. JSON×any markdown instruction;≤50 vs.≥150 words) sit at the−1.0floor by construction: the compositional ceiling, reported not hidden. •Behavioral conflicts: of the 216 satisfiable pairs,∼12% (95% CI[4.6, 11.6]) fail significantly more than their parts predict, that is, pairs the model could satisfy but does not (e.g. “exactly 5 bullets”דAssumptions sec- tion”, joint rate 0.00). The structure is interpretable: F1 (JSON) is the dominant conflict hub, C4 (named sources) the dominant synergy (a required section gives citations a home). Crucially it is reproducible: the interaction landscape correlates across all three models (Spearmanρ = +0.27, +0.28, +0.23; Sec- tion E) and across tasks (advice↔mathρ = +0.44). The exact conflict count is sensitive to the significance rule; the topology (rank agreement) is not, so we lead with the latter. 4.3. What generalizes across tasks (and what does not) Re-running the curve on GSM8K math and HumanEval code (Section F): collapse is universal: every model loses heavily by stack=20 on all three tasks. But two things are task-dependent. The curve shape shifts from sigmoidal (prose) to linear (code). And the per-model ordering is not task-invariant: GPT-5-mini, the worst follower on advice/- math (0.20), nearly ties Sonnet on code (0.44) and overtakes Gemini. “ModelXis the weakest follower” is therefore false in general; weakness is joint in in model and task. 5. Instruction Compilation: Method and Results 5.1. The compiler One LLM call rewrites the raw stack via three transforms: (1) cluster + merge by category; (2) conflict resolution, in- serting a precedence note where two rules conflict; (3) struc- tural reformat, producing a numbered checklist, reasoning first (nearest “thinking”), format last (nearest token pro- duction). It is training-free (any chat API), amortized (compiled once, reused across queries), and falsifiable (pre- registered paired d≥ 0.5). The full prompt: System: You are an expert prompt engineer. Restructure instructions for maximum compliance. User: You will receive a list of instructions that must ALL be followed. Rewrite them as a single optimized instruction set that maximizes the chance all constraints are satisfied simultaneously. Apply these rules: 1. Group by type: REASONING -> CONTENT -> STRUCTURAL -> LEXICAL -> LENGTH -> FORMAT 2. Within each group, merge instructions that overlap into one clearer statement. 3. If two instructions conflict, add: "When X conflicts with Y, prioritize X." 4 Instruction Stacking Collapse 8121620 Stack size (instructions) 5 0 5 10 15 Recovery: compiled raw (p) Recovery by target: the cross-model targets gain, the same-model target loses Claude Sonnet 4.6 (same-model) GPT-5-mini (cross-model) Gemini 2.5 Flash (cross-model) Pre-specified threshold (+10p) Figure 4. Recovery (compiled−raw) by stack size per target; dashed line is the pre-registered+10p threshold. GPT-5-mini clears it from stack=12; Sonnet goes negative. Table 5. Compiler recovery, pooled stacks 8–20. Cluster-bootstrap resamples whole stacks; contrast with the naive t-test p. TargetMean (p) dNaive pCluster CI / p Sonnet (same)−1.2 −0.06 0.03[−4.7, +2.2] / 0.49 Gemini (cross)+3.3+0.20 <10 −11 [+1.2, +5.5] / 0.001 GPT-5-mini (cross)+11.0+0.43 <10 −46 [+7.9, +14.5] / <0.001 4. Output as a numbered checklist with category headers. 5. End with: "Before finalizing: verify EVERY numbered constraint is satisfied." Do NOT add or remove constraints. Only restructure and clarify. Raw instructions: instructions In all experiments the compiler is Claude Sonnet 4.6. We test four conditions at stacks8, 12, 16, 20:raw, compiled,expanded_raw(token-padded control), andreorder_only(deterministic category-reorder, no LLM). Targets span same-model (Sonnet→Sonnet) and cross-model (Sonnet→GPT-5-mini, Sonnet→Gemini). 5.2. Recovery is capability-graded The compiler produces a recovery that grows as the target model weakens (Table 5). We report it with cluster-robust inference (resampling whole stacks, since the items un- der one compiled prompt are not independent and a naive n=1,200t-test overstates significance). The recovery is smaller than thed ≥ 0.5effect we pre-registered (Sec- tion 5.5), but it is statistically significant for the two weaker targets and is concentrated where the need is greatest. The two cross-model gains are robust; the strong-model “harm” is not: null when pooled and only marginal at the hardest stack size (clusterp = 0.037at stack=20, stack- level sign test 14/20,p = 0.11). So the accurate statement is: compilation is neutral for the strongest target and increasingly helpful as the target weakens. The ordering tracks target strength across providers. Table 6. Stack=20 follow rate by condition (advice). ConditionSonnetGeminiGPT-5-mini raw0.6270.4530.204 compiled0.5600.5030.322 expanded_raw0.4860.4820.212 reorder_only0.5010.4630.214 0.20.30.40.50.60.7 Raw stacked-following rate at stack=20 (capability proxy ) 7.5 5.0 2.5 0.0 2.5 5.0 7.5 10.0 12.5 Compiler recovery at stack=20 (compiled raw, p) pre-specified +10p Haiku 4.5 Sonnet 4.6 Opus 4.6 5-nano 5-mini GPT-5.5 Flash-Lite Flash Pro Within-family scaling ladder: recovery shrinks as the target gets stronger Anthropic (Claude) OpenAI (GPT-5) Google (Gemini) Figure 5. Recovery at stack=20 vs. raw rate, one line per family. Filled markers are the production-tier anchors; open markers the added rungs. Recovery falls as the target strengthens; a floor effect appears at the very bottom. 5.3. The mechanism is the LLM rewrite Four alternatives, all ruled out (Table 6): • Not extra tokens / not reordering.expanded_raw andreorder_onlybarely move the weak targets. Because they share the same baseline as raw, “com- piled beats them” cannot be a headroom artifact, yet compiled exceeds both for every model (GPT-5-mini +10.0/+12.1p; Sonnet+6.2/+4.1p; allp < 10 −24 ). This is the clustering-robust not-headroom argument. •Not leakage. If the compiler merely fed Sonnet its own reasoning, the same-model case would gain most; instead it gains least, while two different models gain more. •Not a ceiling artifact. The within-family ladder (Sec- tion 5.4) shows the ordering where only capability varies. 5.4. Within-family ladder Holding the provider fixed and varying only size (Anthropic Haiku→Sonnet→Opus; OpenAI nano→mini→5.5; Google Flash-Lite→Flash→Pro), recovery at stack=20 falls as the target strengthens (Figure 5, Table 7). Across the nine targets, Spearman(raw rate, recovery)= −0.85(p = 0.004 ); the top rung of every family gains the least (Opus −2.0, GPT-5.5+4.0, Gemini-Pro−1.0p), and recovery peaks for weak-but-not-floored targets, with a floor effect at 5 Instruction Stacking Collapse Table 7. Ladder recovery (p) at stack=20, compiled by Sonnet 4.6. Providersmallmid (anchor)large AnthropicHaiku +9.0Sonnet−6.7Opus−2.0 OpenAInano +6.0mini +11.85.5 +4.0 GoogleFlash-Lite +7.0Flash +5.0Pro−1.0 the bottom (GPT-5-nano+6.0 <GPT-5-mini+11.8). This is single-seed, stack=20 only, and pooling families re-mixes the cross-lab confound, so we read it as corroboration, not proof. 5.5. Pre-registration scorecard We fixed three predictions before running the experiments, and report the outcome of each. (1) Decay is sigmoidal: ✓for two of three models; Gemini is non-sigmoidal. (2) At least 15% of pairs conflict:✗on the satisfiable set (∼12%; 17.7% if the designed-in impossibilities are counted). (3) The compiler recovers≥50% of the loss with d ≥ 0.5:✗for every model (bestd = +0.43). The con- tributions we claim are the predictions the data confirmed, together with the capability-graded recovery that the third prediction under-anticipated. 6. Discussion Why does the benefit depend on capability? Two read- ings are consistent with the data, and they are not mutually exclusive. The practical one is about cost: the rewrite is computed once and reused, so it is an inexpensive interven- tion for the small, fast models that serve most production traffic, and it is simply not needed for top-tier models. The mechanistic one is about what the rewrite supplies: a strong model already recovers the stack’s structure as it reads the raw prompt, grouping the format rules and settling prece- dence on its own, whereas a weaker model does not. Writ- ing that structure out therefore helps the weaker model and adds little for the stronger one. The conflict topology of Section 4.2 also explains why the recovery is bounded: a precedence note cannot satisfy the 15 logically impossible pairs, and only a verifier-guided compiler that dropped one side of each would close that part of the gap. Limitations. (i) Construct: verifiers capture the letter, not the intent, of an instruction; “target strength” is a within- benchmark proxy, making the strength–recovery link partly circular (broken by the cross-model and same-baseline- control evidence). (i) Validation: the verifier audit is abbre- viated (∼500labels, 3 verifiers spot-checked only); absolute levels carry small unaudited bias, but comparative claims cancel it. (i) Baseline: the compiler is only compared to machine-degraded versions of a naive dump, and we do not compare against a competently hand-written prompt, and a single model (Sonnet) does all compiling. (iv) Scope: com- piler recovery is shown on advice (and code as a robustness check); the ladder is single-seed at stack=20. None of the headline claims extend beyond the tested grid. Future work. The compiler studied here is deliberately min- imal: it groups and merges constraints, but it never removes one. Because 15 of the conflicting pairs are logically un- satisfiable (Section 4.2), no prompt-only rewrite can satisfy both sides, which sets a ceiling on the achievable recovery. The natural next step is a verifier-guided compiler that de- tects such pairs, drops the lower-priority instruction, and reports the omission to the caller, so that the residual conflict is made explicit rather than silently violated. Two further directions would sharpen the present results: a comparison against a competently hand-written prompt, rather than only the machine-degraded baselines used here, and a multi-seed run of the within-family ladder across all stack sizes. 7. Conclusion Stacking many instructions into one prompt reliably de- grades instruction-following. The decline is non-linear, and it runs through an interpretable, reproducible set of pairwise conflicts rather than uniform forgetting. A single training- free rewrite of the prompt offers a targeted remedy: it returns a meaningful share of the lost follow rate to weaker mod- els (+11p for the weakest), while strong models, which already impose this structure themselves, are left essentially unchanged. The gain comes from the rewrite rather than from extra tokens, reordering, or measurement headroom, and it holds down three within-family scaling ladders. Be- cause the rewrite is paid for once and reused at no per-query cost, it is most worth applying precisely where instruction- following is weakest. Impact Statement This work studies the reliability of instruction-following in deployed language models and a cheap mitigation for it. The likely effect is positive: better adherence to safety-, format-, and policy-constraints on the small models that serve most production traffic. Two cautions apply. First, our benchmark measures compliance, not the correctness or safety of content, so higher follow rates must not be read as higher trustworthiness. Second, the capability-graded finding could be misread as “frontier models need no prompt hygiene”; our data show only that one specific restructuring helps them little, not that they are immune to instruction collapse. No human subjects or private data are involved, and all artifacts are released to support scrutiny. 6 Instruction Stacking Collapse Software and Data All code, the 24-instruction benchmark, deterministic veri- fiers, cached model responses, per-row result parquets, and the analysis scripts (including the cluster-robust re-analysis of Section 5) are released to enable exact reproduction with- out new API calls. References Ferraz, T. P., Mehta, K., Lin, Y.-H., Chang, H.-S., Oraby, S., Liu, S., Subramanian, V., Chung, T., Bansal, M., and Peng, N. Llm self-correction with DeCRIM: De- compose, critique, and refine for enhanced following of instructions with multiple constraints. arXiv preprint arXiv:2410.06458, 2024.URLhttps://arxiv. org/abs/2410.06458. Guo, Z., Liu, W., Xie, M., Xu, J., Huang, Z., Tian, M., Xu, J., Shen, Y., Qian, Q., Wu, M., Wang, X., Lv, C., Wang, H.-D., Yao, H., Zheng, X., and Huang, X. RECAST: Expanding the boundaries of LLMs’ com- plex instruction following with multi-constraint data. arXiv preprint arXiv:2505.19030, 2025. URLhttps: //arxiv.org/abs/2505.19030. Hakim, M. D. A.Brevity constraints reverse perfor- mance hierarchies in language models. arXiv preprint arXiv:2604.00025, 2026.URLhttps://arxiv. org/abs/2604.00025. Jiang, Y., Wang, Y., Zeng, X., Zhong, W., Li, L., Mi, F., Shang, L., Jiang, X., Liu, Q., and Wang, W. Follow- Bench: A multi-level fine-grained constraints following benchmark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (ACL 2024), 2024. URLhttps: //aclanthology.org/2024.acl-long.257/. Kong, W., Hombaiah, S., Zhang, M., Mei, Q., and Ben- dersky, M. PRewrite: Prompt rewriting with reinforce- ment learning.In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis- tics (ACL 2024, Short Papers), 2024. URLhttps: //aclanthology.org/2024.acl-short.54/. McKenzie, I. R., Lyzhov, A., Pieler, M., Parrish, A., Mueller, A., Prabhu, A., McLean, E., Kirtland, A., Ross, A., Liu, A., Gritsevskiy, A., Wurgaft, D., Kauffman, D., Recchia, G., Liu, J., Cavanagh, J., Weiss, M., Huang, S., The Floating Droid, Tseng, T., Korbak, T., Shen, X., Zhang, Y., Zhou, Z., Kim, N., Bowman, S. R., and Perez, E. Inverse scaling: When bigger isn’t better. Transactions on Machine Learning Research (TMLR), October 2023. URL https://arxiv.org/abs/2306.09479. Qi, Y., Peng, H., Shi, X., Xin, A., Wang, X., Xu, B., Hou, L., and Li, J. On the paradoxical interference between instruction-following and task solving. arXiv preprint arXiv:2601.22047, 2026.URLhttps:// arxiv.org/abs/2601.22047. Ye, J., Huang, C., Chen, Z., Fu, W., Yang, C., Yang, L., Wu, Y., Wang, P., Zhou, M., Yang, X., Gui, T., Zhang, Q., Shi, Z., Fan, J., and Huang, X. MulDimIF: A multi-dimensional constraint framework for evaluat- ing and improving instruction following in large lan- guage models.In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin- guistics (ACL 2026), 2026. URLhttps://arxiv. org/abs/2505.07591 . Forthcoming. Preprint at arXiv:2505.07591. Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., and Hou, L. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023.URLhttps://arxiv. org/abs/2311.07911. 7 Instruction Stacking Collapse A. Full Instruction Set The 24 atomic instructions; the 22 deterministic ones form the active pool. C2 and C3 (starred) require an LLM judge and are excluded from all reported experiments. • Format. F1 valid JSON (json.loads, optional code-fence stripped); F2 wrap whole response in a markdown code fence; F3≥ 3 markdown bullets; F4 trailing “Summary: . . . ” line on its own. • Length. L1≤ 50 words; L2≥ 150 words; L3 exactly 5 bullets; L4 title (first line)≤ 10 words. • Lexical. X1 use “therefore”≥ 1; X2 avoid “however” entirely; X3 exactly three numbered citations[1] [2] [3]; X4 all lowercase. • Structural. S1 begin with “Answer:”; S2 include “## Pros” and “## Cons”; S3 second-person≥ 3; S4 no question marks. •Reasoning. R1 “## Reasoning” header before the answer; R2 “## Assumptions” (≥ 2items); R3 “Confidence: 0.X” line; R4 “## Alternatives” (≥ 2 items). • Content. C1 reference a year 1900–2025; C2 ∗ exactly one analogy/metaphor; C3 ∗ a counterargument; C4≥ 2named sources. B. Extended Related-Work Comparison Table 8. Full benchmark comparison. PropertyIFEvalFollowBenchRECASTMulDimIFOurs Instructions/prompt1–2incr., 1 cat30+multi1–20, 6 cat Cross-category stackingnonopartialpartialyes (uniform) Deterministic verifiersyespartialpartialpartial22/24 Pairwise conflict analysisnonodatadata231 pairs, FDR Impossible-vs-behavioralnonononoyes Primary purposeevalevaltraintraineval + mitig. Training-free mitigationnonononoyes (compiler) C. Verifier Validation Each verifier was audited against hand labels formed blind to its verdict. After an L4 bug fix (a leading code-fence line was wrongly counted as a short title; corrected and re-scored from cache), 18 of 19 audited verifiers match humans exactly; C4 is over-strict by three false negatives in fifteen. R1, R3, S2 were spot-checked only; a full 100-label-per-instruction audit is owed. Table 9. Verifier-vs-human agreement. Verifiern/sheetagreement F1–F4, L1–L4, X1–X4, S1, S3, S4, R2, R4, C1 (18)15–311.000 C4 (≥ 2 named sources)150.800 D. Category and Per-Instruction Degradation E. Conflict Topology: Details The 15 logical impossibilities. F1 (JSON)×F3,F4,L3,R1,R2,R3,R4, S1,S2; X4 (lowercase)×S1,F4,R3; F2 (code fence)× S1,F4; and L1× L2. 8 Instruction Stacking Collapse 2.55.07.510.012.515.017.520.0 Number of Stacked Instructions 0.0 0.2 0.4 0.6 0.8 1.0 Instruction-Following Rate Degradation by Instruction Category Content Format Length Lexical Reasoning Structural Figure 6. Mean follow rate by category, pooled across models. Lexical survives; format and length collapse hardest. Table 10. Ten strongest behavioral conflicts on Sonnet 4.6 (impossibilities excluded), sorted by observed− expected. pairP(A)P(B)obsexpinteraction F1–L41.001.000.001.00 −1.00 L3–R21.001.000.001.00 −1.00 F4–R31.000.950.000.95 −0.95 F1–X31.000.900.100.90 −0.80 L3–R41.000.750.000.75 −0.75 X3–C40.900.850.050.77 −0.71 F2–R41.000.750.050.75 −0.70 X3–R10.901.000.300.90 −0.60 L1–R11.001.000.401.00 −0.60 F3–X31.000.900.450.90 −0.45 F. Multi-Task Degradation G. Compiler: Conditions and Ablations H. Statistical Methods All inference is reproduced by scripts/extra_stats.py on the released parquets (numpy/scipy only). Cluster bootstrap. Because the 15 items under one stack share a single compiled prompt and instruction draw, items are not independent. We resample whole stacks with replacement (3,000 reps) and recompute the statistic, giving CIs and two-sided p-values that respect within-stack correlation. The naiven=1,200-paired-observation t-test overstates significance; under the cluster bootstrap the two cross-model recoveries survive (p≤ 0.001) while the same-model “harm” does not (p = 0.49 pooled; p = 0.037 at stack=20 only). Model-selection bootstrap. For each degradation curve we resample responses within each stack size, recompute the mean curve, refit linear/exponential/sigmoid, and record the AIC winner (400 reps). Sigmoid wins 93% (Sonnet) and 100% (GPT-5-mini) of refits; Gemini selects sigmoid in < 2% (linear 64%, exp 35%). Interaction inference. For each pair, interaction= observed(A∧ B)− P(A)P(B)withP(·)from singleton baselines; 9 Instruction Stacking Collapse 1358121620 Stack Size X2 L4 F2 R1 S2 X1 C1 X4 L1 R2 F3 F4 L3 S1 S4 X3 S3 C4 F1 R4 R3 L2 Instruction ID 1.001.001.000.980.980.991.00 1.001.000.940.910.830.610.48 1.001.000.930.810.610.390.22 1.001.000.750.790.690.550.38 1.001.000.680.720.610.400.35 1.001.000.950.960.860.760.62 1.000.990.950.950.820.730.59 1.000.950.890.800.810.760.71 1.000.930.570.290.180.250.32 1.000.900.840.780.560.400.34 1.000.860.900.910.630.450.37 1.000.810.860.470.270.230.13 1.000.790.680.510.210.080.02 1.000.780.740.410.140.100.02 1.001.001.001.000.991.00 0.970.850.810.670.650.450.30 0.930.991.000.940.860.690.55 0.870.800.750.760.700.470.39 0.800.540.630.380.340.180.11 0.730.810.970.890.740.440.34 0.670.710.700.760.660.530.37 0.630.880.770.770.700.630.53 Per-Instruction Follow Rate by Stack Size 0.0 0.2 0.4 0.6 0.8 1.0 Follow Rate Figure 7. Per-instruction follow rate by stack size. L3, S1, F4, F1 fall fastest; S4, X2, X4 stay high. significance by Benjamini–Hochberg at FDR 0.05. The feasible-only conflict rate is∼12%with a pair-resampling 95% CI of [4.6, 11.6]. Cross-model Spearman of all 231 interactions: +0.27, +0.28, +0.23 (all p < 10 −3 ). Ladder. Spearman between raw stacked-following rate and recovery across the nine ladder targets is−0.85(p = 0.004, n = 9, single seed). 10 Instruction Stacking Collapse Table 11. Significant pairwise interactions per model (advice, 231 pairs, FDR 0.05). GPT-5-mini has the fewest conflicts despite collapsing hardest—accumulation, not a denser conflict set, drives its aggregate collapse. ModelConflictsimpossib.behav.Synergies Sonnet 4.641152642 Gemini 2.5 Flash48143448 GPT-5-mini36132327 F1F2F3F4 L1L2L3L4 X1X2X3X4 S1S2S3S4 R1R2R3R4 C1C4 F1 F2 F3 F4 L1 L2 L3 L4 X1 X2 X3 X4 S1 S2 S3 S4 R1 R2 R3 R4 C1 C4 Sonnet 4.6 41 conflicts / 42 synergies F1F2F3F4 L1L2L3L4 X1X2X3X4 S1S2S3S4 R1R2R3R4 C1C4 F1 F2 F3 F4 L1 L2 L3 L4 X1 X2 X3 X4 S1 S2 S3 S4 R1 R2 R3 R4 C1 C4 Gemini 2.5 Flash 48 conflicts / 48 synergies F1F2F3F4 L1L2L3L4 X1X2X3X4 S1S2S3S4 R1R2R3R4 C1C4 F1 F2 F3 F4 L1 L2 L3 L4 X1 X2 X3 X4 S1 S2 S3 S4 R1 R2 R3 R4 C1 C4 GPT-5-mini 36 conflicts / 27 synergies 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 interaction Pairwise interaction (observed P(A)·P(B)) by target model advice task Blue = conflict, red = synergy. Dots: BH-significant at FDR 0.05. F1 (JSON) is the dominant conflict hub in every panel. Figure 8. Per-model conflict heatmaps. F1 is the dominant conflict hub in every panel; GPT-5-mini (right) is the sparsest despite collapsing hardest in aggregate. Table 12. Mean follow rate (95% CI), GSM8K math. Modelstack=1stack=5stack=16stack=20 Sonnet 4.60.9550.8530.6060.591 GPT-5-mini0.9360.8410.3230.206 Gemini 2.5 F0.9180.8140.5570.431 Table 13. Mean follow rate (95% CI), HumanEval code. Per-model ordering flips: GPT-5-mini ties Sonnet; Gemini is weakest. Modelstack=1stack=5stack=16stack=20 Sonnet 4.60.9640.8370.5770.447 GPT-5-mini0.9140.7930.5210.443 Gemini 2.5 F0.8410.7770.4870.344 11 Instruction Stacking Collapse 0.02.55.07.510.012.515.017.520.0 Number of stacked instructions 0.0 0.2 0.4 0.6 0.8 1.0 Instruction-following rate Advice task (open-ended) 0.02.55.07.510.012.515.017.520.0 Number of stacked instructions GSM8K math task 0.02.55.07.510.012.515.017.520.0 Number of stacked instructions HumanEval code task Claude Sonnet 4.6 GPT-5-mini Gemini 2.5 Flash Collapse reproduces on all three tasks, but the per-model ordering shifts on code Figure 9. Degradation on advice / math / code. The fact of collapse generalizes; the per-model ordering does not. 8121620 Stack size (instructions) 0.0 0.2 0.4 0.6 0.8 1.0 Instruction-following rate Claude Sonnet 4.6 (same-model) 8121620 Stack size (instructions) GPT-5-mini (cross-model) 8121620 Stack size (instructions) Gemini 2.5 Flash (cross-model) Raw stack Compiled (ours) Expanded raw Reorder only Follow rate by condition: compilation helps the cross-model target, not the same-model one Figure 10. Follow rate by condition×stack size per target.compiledseparates from controls on the cross-model targets; on Sonnet raw is best. Claude Sonnet 4.6 (same-model)GPT-5-mini (cross-model)Gemini 2.5 Flash (cross-model) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Instruction-following rate at stack=20 0.63 0.20 0.45 0.56 0.32 0.50 0.49 0.21 0.48 0.50 0.21 0.46 Ablation at stack=20: only LLM rewriting helps the weaker target; every intervention hurts the stronger one Raw stack Compiled (ours) Expanded raw Reorder only Figure 11. Stack=20 conditions per target. Only the LLM rewrite (compiled) separates fromraw; token-padding and reordering do not. 12