Paper deep dive

How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs

Ali Alfageeh, Rahul Gopinath, Amin Alipour

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 59

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Prompts are the primary mechanism for directing the behavior of large language models (LLMs). Yet the internal structure and causal hierarchy of prompts remain poorly understood: which parts are causally necessary and which are redundant is an open question. This opacity can have severe consequences. Subtle prompt variations can silently shift model outputs in critical software systems, and engineers lack techniques to reason about prompt reliability. We present \framework, a blackbox prompt-minimization framework that reduces few-shot prompts to their necessary minimal subset. We use a case study to apply \framework to a few-shot learning system and demonstrate the insights that this framework can provide. Our experiments show that few-shot exemplars can be reduced by a mean of 65.3\%~$\pm$~15.8\% in character count while fully preserving propositional output fidelity. The models preferentially retain logical identifiers and constraint declarations while discarding natural language prose and cross-prompt relational annotations. Our analysis also shows that some models are universal encoders, able to produce highly legible yet minimized prompts, while others are universal decoders, able to interpret minimized prompts from most other models. By identifying which components are indispensable, \framework provides a principled basis for prompt compression and structural analysis of few-shot exemplars.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

55,529 characters extracted from source content.

Expand or collapse full text

How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs Ali Alfageeh University of Houston Rahul Gopinath The University of Sydney Amin Alipour University of Houston Abstract—Prompts are the primary mechanism for directing the behavior of large language models (LLMs). Yet the inter- nal structure and causal hierarchy of prompts remain poorly understood: which parts are causally necessary and which are redundant is an open question. This opacity can have severe consequences. Subtle prompt variations can silently shift model outputs in critical software systems, and engineers lack tech- niques to reason about prompt reliability. We present D-FSM, a blackbox prompt-minimization frame- work that reduces few-shot prompts to their necessary minimal subset. We use a case study to apply D-FSM to a few-shot learning system and demonstrate the insights that this framework can provide. Our experiments show that few-shot exemplars can be reduced by a mean of 65.3%± 15.8% in character count while fully preserving propositional output fidelity. The models preferentially retain logical identifiers and constraint declarations while discarding natural language prose and cross- prompt relational annotations. Our analysis also shows that some models are universal encoders, able to produce highly legible yet minimized prompts, while others are universal decoders, able to interpret minimized prompts from most other models. By identifying which components are indispensable, D- FSM provides a principled basis for prompt compression and structural analysis of few-shot exemplars. Index Terms—Prompt minimization, few-shot learning sys- tems, Prompt compression, LLM I. INTRODUCTION The integration of large language models (LLMs) into production software systems has elevated the prompt to a crit- ical engineering artifact. Two paradigms dominate inference- time control: few-shot prompting [1], in which a model is contextualized using example inputs and outputs, and Chain- of-Thought (CoT) reasoning [2], in which intermediate rea- soning steps are explicitly demonstrated to elicit structured multi-step reasoning. Industry analyses estimate that by 2025, over 80% of enterprises will incorporate prompt engineering into their workflows [3], with few-shot strategies accounting for roughly 40% of deployed techniques and CoT variants comprising nearly a quarter of enterprise applications [4]. With over 67% of organizations deploying LLM-powered products in production [5], prompts have become the cornerstone of software engineering. Despite this scale, prompts are fragile. LLMs are sensitive to superficial perturbations such as the ordering of examples [6], [7], the phrasing of instructions, and even the presence of seemingly irrelevant formatting tokens [8], all of which can shift model outputs in ways invisible to developers. Pearce et al. [9] found that approximately 40% of programs generated by GitHub Copilot contained security vulnerabilities from the CWE Top 25, illustrating that prompt-driven code generation can silently produce exploitable output. Given the obscurity of the mapping from natural-language prompts to model behavior, such failures can emerge at deployment without warning. Current approaches to reliability assessment, including static aggregate benchmarks such as MMLU and HumanEval and qualitative human review, treat the prompt–model interaction as a monolithic event. As a result, they cannot answer a fundamental software engineering question: which specific component of a prompt caused a given output? Recent work on automatic prompt optimization, including DSPy [10] and Automatic Prompt Engineering (APE) [11], addresses prompt construction but not prompt explanation: these methods maxi- mize task performance without revealing why a prompt works or which of its components are causally necessary. Without constraint-level observability, engineering robust LLM appli- cations remains a precarious trial-and-error process. This paper proposes treating prompts as analyzable, for- mally verifiable software artifacts. We introduce D-FSM (Delta Debugging for Few-Shot Minimization), a black- box framework based on delta-debugging [12] that reduces prompts to their minimal necessary subset. We use a few-shot system as a case study to demonstrate the analyses that this framework can produce to better understand and evaluate few-shot systems. Input minimization requires a precise notion of behavioural preservation. We capture this using Prompt2Propositions (P2P) [13], which maps prompts to propositional constraints P ≡ (C 1 ∧·∧C k ). This allows direct equivalence checks between the baseline output and that of reduced prompts. Minimization retains a reduction only if equivalence holds; otherwise, it is reverted. An example prompt and its minimization are given in Figure 1. Experiments across six models and two temperature settings show that few-shot exemplars can be reduced by a mean of 65.3% ± 15.8% of characters while fully preserving propo- sitional output fidelity. The models preferentially retain the logical skeleton of exemplars: prompt and constraint identi- fiers, propositional declarations, and logical expressions, while discarding natural language prose and cross-prompt relational annotations (Table I). Regression analysis demonstrates that model scale is the dominant factor in achievable compression, with each additional 10B parameters corresponding to 4.3 arXiv:2609.36289v1 [cs.SE] 28 Sep 2026 Role Assignment YouareanassistantthatanalyzesPythonpromptsandgeneratespropositionallogicformalizationsandrelationships betweenprompts. Task Description Followthisexampleformatstrictly. Few-shot ExamplesPrompts: PromptID: P1 Text:WriteaPythonfunctionnamedmath(numbersinput)thattakes a list of numbers as input and returns a new list consisting of the addition of each consecutive numbers in the input. C1: Python function C2: named math(numbersinput) C3: takes a list of numbers as input C4: returns a new list C5: the addition C6: each consecutive numbers LogicalExpression: P 1→ (C1∧ C2∧ C3∧ C4∧ C5∧ C6) PromptID: P2 Text:WriteaPythonfunctionnamedmath(numbersinput)thattakesalistofnumbersasinputandreturnsanewarray containingof the product of each consecutive numbers in the input. C1: Python function C2: named math(numbersinput) C3: takes a list of numbers as input C7: returns a new array C8: theproduct C6:eachconsecutivenumbers LogicalExpression:P2(C1 C2 ∧C3∧ C7∧ C8∧ C6) LogicalRelationshipWithPreviousPrompt: PreviousPrompt: P1 SemanticRefinement: C4 evolves into C7 from list to array, and C5 evolves into C8 from addition to product. NewAddition:Onlyrefinementsareintroduced;noindependentnewconstraints. CoreContinuation:C1∧ C2∧ C3∧ C6remainunchanged. Constraints ENDOFEXAMPLE.Donotincludetheexampleinyourresponse.Do not generate any explanation more than the example structure. Only produce the formalization of the following two consecutive prompts. The logical expressions should be in C-form. Fig. 1: Anatomy of a minimized prompt.Green shading marks content retained after minimization;red strikethrough marks removed spans. The role assignment, task description, and post-example constraints are eliminated, while the few-shot exemplar retains its logical skeleton of identifiers, constraints, and expressions. percentage points of additional removal. Further, we find that some models serve as universal encoders, producing minimized prompts that retain their fidelity when consumed by other models, while others serve as universal decoders, able to reconstruct the intended output from minimized prompts produced by most other models, regardless of the degree of minimization. Contributions. 1) D-FSM, a blackbox delta-debugging pipeline for few- shot prompt minimization leveraging an oracle based on propositional constraints. 2) An empirical characterization of which prompt com- ponents survive minimization, attributing retained and removed content to specific nonterminals of the P2P grammar. 3) A regression analysis that separates the effects of model scale, sampling temperature, and architecture family on minimization, showing that family-level variation dominates raw parameter count. 4) A cross-model transfer study that identifies universal encoders (models whose minimized prompts travel well to other models) and universal decoders (models that reliably consume minimized prompts produced by other models). The rest of the paper is organized as follows. Section I provides the motivation and research questions. Section I discusses the design and implementation. Section IV presents the results, followed by Section V, which discusses the impli- cations. Section VI covers related work. Section VII discusses threats to validity, and Section VIII concludes. I. MOTIVATION AND RESEARCH QUESTIONS Existing work on few-shot prompt optimization and exam- ple compression often relies on heuristic pruning, embedding similarity, or end-to-end LLM evaluation [14], [15]. Although these approaches can shorten prompts while preserving task performance, they provide limited insight into what semantic content survives compression, why certain tokens remain indispensable, or how task intent persists even as symbolic structure collapses. This lack of interpretability limits criti- cal understanding of in-context learning and weakens repro- ducibility across model runs. Our Delta Debugging for Few-Shot Minimization (D- FSM) framework tackles this by applying delta debugging directly to prompts to find the smallest subset that preserves the original output. This approach reveals what essential information survives as formatting and peripheral text are removed. D-FSM gives a clear method for understanding how task intent survives during significant prompt reduction. A. Research Questions Prompt minimization is only useful if reduction is achiev- able without sacrificing output fidelity, and the reduction possible quantifies the redundancy present in typical prompts. Hence we ask: RQ1. To what extent can a prompt be minimized while fully preserving propositional output fidelity? For insight into in-context learning, we need to know what is being preserved. The types of content that survive minimiza- tion reveal which structural or semantic elements LLMs treat as indispensable, and which are genuinely redundant. Hence, we ask: RQ2. Which parts of prompts survive minimization? Preserving propositional output fidelity is a formal criterion, but practitioners care about downstream task accuracy. A minimized prompt that satisfies the oracle but degrades real- world performance would have limited practical value. Hence, our question is: RQ3. How does the task accuracy of the minimized prompt compare to that of the full prompt? A minimized prompt derived from one model may have exploited model-specific behaviors rather than identifying genuinely redundant content. Testing transfer across models determines whether the minimal core is universal or model- dependent. This is what we verify next. RQ4. Does the minimized prompt preserve correctness across large language models? I. DESIGN AND IMPLEMENTATION We first provide an overview of the algorithm, how we adapt it into the D-FSM pipeline, and the models and dataset used in the evaluation. Delta Debugging. Zeller and Hildebrandt [12] introduced the minimizing delta debugging algorithm ddmin (Algorithm 1). It takes a predicate and a satisfying starting input and iteratively simplifies the input until it reaches the minimal input that still satisfies the predicate. To identify which parts of the few-shot exemplar guide the model to reproduce the same propositional output as the manually authored baseline, we integrate ddmin into D-FSM for few-shot exemplar reduction, as described below. Workflow. Figure 3 shows a high-level overview of the D- FSM workflow. Framework. The framework targets the setting studied by Alfageeh et al. [13], who examined the interaction between students and LLMs on programming tasks using logical rep- resentations. The logical representations were generated by the LLM using few-shot examples. They found that 98% of Algorithm 1 Delta Debugging (ddmin) Require: C such that TEST(C) = FAIL Ensure: C ′ is 1-minimal failing subset n← 2▷ Initial granularity while |C|≥ 2 do Partition C into n segments ∆ 1 , ∆ 2 ,..., ∆ n somereduction← false for all ∆ i do if test(∆ i ) = FAIL then C ← ∆ i n← 2 somereduction← true break if not somereduction then for all ∆ i do if TEST(C\ ∆ i ) = FAIL then C ← C\ ∆ i n← max(n− 1, 2) somereduction← true break if not somereduction then if n <|C| then n← min(|C|, 2n) else break▷ Result is 1-minimal return C the sample correctly mapped the student’s prompts to the logical representations. We therefore seek to understand how the model generated these logical representations using the few-shot examples provided to the LLM. 1) Ground Truth. In this setup, we manually coded two inquiries, each inquiry having two prompts, into logical expressions. These logical expressions serve as an initial check for the model when it uses the full few-shot examples to generate the baselines for each inquiry. 2) Delta Debugging. In this step, ddmin begins minimizing the full few-shot example. The minimized few-shot example is sent to the model, followed by the two inquiries. The minimized few-shot is categorized into one of the outcome classes discussed in Section I-C. 3) Verification. Lastly, we verify the minimal example returned by ddmin for further analysis. A. Evaluated Models To broaden the generalizability of our findings, we selected six large language models spanning four model families and parameter counts from 8B to 72B. We evaluate each model at two temperature settings (T = 0.0 and T = 0.5). The models are listed in Table I. B. Analysis Instrumentation D-FSM records the result of each minimization run as a JSON parse tree in which every internal node is labeled with its P2P grammar nonterminal (see Figure 2) and every leaf ⟨PromptSeq⟩ → ⟨Text⟩ ‘Prompts:’ ⟨Prompt⟩* ‘END OF EXAMPLE’ ⟨Text⟩ ⟨Prompt⟩ → ‘PromptID:’ ⟨PID⟩ ‘Text:’ ⟨Text⟩ ⟨Constraint⟩* ‘LogicalExpression:’ ⟨LExp⟩ [⟨LRel⟩] ⟨LExp⟩ → ⟨PID⟩ ‘->’ ‘(’ ⟨Conjuncts⟩ ‘)’ ⟨LRel⟩ → ‘LogicalRelationshipWithPreviousPrompt:’ ‘PreviousPrompt:’ ⟨PID⟩ ‘Refinement:’ ⟨Text⟩ ‘Addition:’ ⟨Text⟩ ‘CoreContinuation:’ ⟨Conjuncts⟩ ⟨Conjuncts⟩ → ⟨CID⟩ ∧ ⟨Conjuncts⟩ | ⟨CID⟩ ⟨Constraint⟩ → ⟨CID⟩ ‘:’ ⟨Text⟩ ⟨PID⟩ → ‘P1’ | ‘P2’ | ‘P3’ | . . . ⟨CID⟩ → ‘C1’ | ‘C2’ | ‘C3’ | . . . Fig. 2: Grammar for prompt minimization formalization (P2P) Same Propositions? Reject Reduction (PASS) Accept Reduction (FAIL) Start: Full Prompt (C full ) D Module C ′ = C full − i LLM Oracle Evaluate Candidate C ′ Stop: Minimal Correct Prompt No Yes Fig. 3: D-FSM workflow for prompt minimization and evaluation. TABLE I: Large Language Models Used in Evaluation. Model NameFamilyParameters Llama-3.1-8BLlama8B gemma-2-9b-itGemma9B Mistral-Small-24BMistral24B Qwen-2.5-32bQwen32B Llama-3.3-70BLlama70B Qwen-2.5-72bQwen72B is either a retained terminal string or a deletion marker. After minimization is complete, a post-hoc analysis script traverses this tree recursively: for each nonterminal N it accumulates (i) the total character count of all terminals in the subtree rooted at N , and (i) the count of characters carried by deletion markers in that subtree. The ratio of deleted to total characters gives the removal rate for N . Aggregating these rates across all 12 model/temperature runs yields the mean and standard deviation reported in Table I. Because the P2P grammar is hierarchical, a character deleted inside a〈Constraint〉 subtree is also counted under every ancestor—〈ConstraintList〉,〈Prompt〉, and〈PromptSeq〉. The per-nonterminal rates therefore reflect the aggregate be- havior of all content governed by that symbol, not just its immediate children. C. Verdict Classification in Delta Debugging The framework classifies each minimization step into one of three verdicts. These verdicts reflect how the Delta De- bugging (D) algorithm interprets the behavior of a min- imized few-shot exemplar relative to predefined baselines. Note that in Zeller’s original formulation, the verdict names FAIL/PASS carry their conventional software-testing mean- ing (a FAIL means the bug is still triggered). To avoid confusion in the prompt-minimization context, we adopt the aliases PRESERVED, LOST, and UNRESOLVED throughout this paper. 1) PreservedState:TargetBehaviorRetained:A PRESERVED outcome (corresponding to FAIL in classical D) indicates that the reduced input still preserves the target behavior. 1) PRESERVED (doubly verified match): The reduced prompt successfully generates logical expressions that match the baselines for both Inquiry 1 and Inquiry 2. Additionally, the result is fully deterministic: when re- evaluated during Verification, the same matching outputs are reproduced. 2) DDInterpretation: This outcome is treated as PRESERVED, meaning the target property is retained. The algorithm continues minimizing the input. 2) Lost States: Behavior No Longer Preserved: A LOST outcome (corresponding to PASS in classical D) indicates that the minimized prompt no longer preserves the target behavior. These cases typically arise when the prompt becomes too degraded for reliable parsing. 1) LOST (<2 expressions Inq1): For the first run, the model fails to produce exactly two structured expres- sions. 2) LOST (<2 expressions Inq2): Inquiry 1 succeeds, but Inquiry 2 fails to produce two valid expressions. 3) D Interpretation: Both cases are treated as LOST, indicating the absence of the target property. The con- figuration is discarded. 3) Ambiguous States: Divergent or Non-Deterministic Be- havior: An UNRESOLVED outcome indicates that the model produces outputs, but they either deviate from the baseline or exhibit instability. Divergent Outputs. 1) LOST (diff Inq1): Inquiry 1 produces valid expressions that do not match the baseline. 2) LOST (diff Inq2): Inquiry 1 matches the baseline, but Inquiry 2 produces non-matching expressions. 3) D Interpretation: Although labeled as LOST in logs, these cases are internally treated as UNRESOLVED. Flaky Behavior If a configuration initially matches both baselines but fails to do so consistently during verification, it is classified as flaky: 1) UNRESOLVED – Flaky Inq1 (<2 expressions Inq1 Verif): Re-evaluation of Inquiry 1 fails to extract valid expressions. 2) UNRESOLVED – Flaky Inq1 (diff Inq1 Verif): Re- evaluation of Inquiry 1 produces different expressions. 3) UNRESOLVED – Flaky Inq2 (<2 expressions Inq2 Verif): Re-evaluation of Inquiry 2 fails to extract valid expressions. 4) UNRESOLVED – Flaky Inq2 (diff Inq2 Verif): Re- evaluation of Inquiry 2 produces different expressions. 5) D Interpretation: All such cases are treated as UNRESOLVED, indicating instability or lack of repro- ducibility. IV. RESULTS A. RQ1: Extent of Minimization Table I shows the proportion of characters removed by each model at each sampling temperature. Every model achieves substantial reduction while fully preserving propositional out- put fidelity, confirming that typical few-shot prompts carry significant redundancy. Across all 12 model/temperature com- binations the mean removal is 65.3%± 15.8%, with individual runs ranging from 38.4% (Mistral-Small-24B, T = 0.0) to 87.3% (Llama-3.3-70B, T = 0.0). Model parameter size seems to be the dominant fac- tor in how much can be removed. The two 70B-class models—Llama-3.3-70B and Qwen-2.5-72b—average 85.8% and 79.1% removal, respectively, whereas the 8B-class models achieve far less reduction: Llama-3.1-8B averages 51.7% and Mistral-Small-24B averages 44.1%. Qwen-2.5-32b and TABLE I: Removal statistics by model and sampling temper- ature. ModelTTotalRemovedRemoved (%) gemma-2-9b-it0.01749111063.5 gemma-2-9b-it0.51749127372.8 Llama-3.1-8B0.0174984448.3 Llama-3.1-8B0.5174996455.1 Mistral-Small-24B0.0174967238.4 Mistral-Small-24B0.5174986949.7 Qwen-2.5-32b0.01749123170.4 Qwen-2.5-32b0.5174997555.7 Llama-3.3-70B0.01749152687.3 Llama-3.3-70B0.51749147484.3 Qwen-2.5-72b0.01749139679.8 Qwen-2.5-72b0.51749137278.4 Average—1749114265.3 St. Dev.—26515.8 01020304050607080 30 40 50 60 70 80 90 Model size (billions of parameters) Characters removed (%) Fit (r=0.74, R 2 =0.54, p=0.006) gemma-2-9b-it Llama-3.1-8BMistral-Small-24B Qwen-2.5-32b Llama-3.3-70B Qwen-2.5-72b Fig. 4: Scatter plot of achievable compression (% characters removed) against model size (billions of parameters) for all 12 model/temperature runs. The line shows the ordinary least- squares fit (y = 0.43x + 50.0, r = 0.74, R 2 = 0.54, p = 0.006). Each model contributes two points (temperatures T = 0.0 and T = 0.5). gemma-2-9b-it fall in between at 63.1% and 68.2%. This gradient suggests that larger models maintain propositional fidelity even when the scaffolding around the core examples is stripped away, while smaller models rely more heavily on the examples and their surrounding structure. Sampling temperature has a comparatively modest and inconsistent effect. For Mistral-Small-24B, raising T from 0.0 to 0.5 increases removal by 11.3 percentage points, while for Qwen-2.5-32b the same increase in temperature reduces removal by 14.7 points. No systematic direction is apparent, implying that the achievable compression is governed primar- ily by model capacity rather than by stochasticity in decoding. TABLE I: Characters removed (%) per nonterminal symbol NonterminalMean (%)Std. Dev. 〈PromptSeq〉65.315.8 〈Prompt〉60.719.5 〈Constraint〉45.522.7 〈Conjuncts〉61.127.7 〈LExp〉50.622.9 〈LRel〉72.626.3 〈PID〉42.617.1 〈CID〉53.616.7 RQ1. D-FSM reduces prompts by an average of 65.3% of characters while fully preserving propositional output fidelity. Reduction scales with model size: 70B-class mod- els tolerate up to 87% removal, whereas 8B-class models plateau near 48–55%. Temperature has no consistent direc- tional effect on compression. B. RQ2: What Prompt Content Survives? Table I breaks the removal rate down by nonterminal symbol in the P2P grammar. Relational annotations are the most removable con- tent. Among the grammar symbols in Table I,〈LRel〉 (72.6% ± 26.3%) is the most heavily pruned.〈LRel〉 en- codes the logical relationship between consecutive prompts (“LogicalRelationshipWithPreviousPrompt”); its high removal rate indicates that models do not rely on explicit cross-prompt relationship annotations to reproduce correct propositional output. The logical skeleton is preserved. At the other end of the spectrum, prompt and constraint identifiers survive far more often.〈PID〉 (prompt identifiers such as P1, P2) is removed at only 42.6% ± 17.1%, and〈CID〉 (constraint iden- tifiers such as C1, C2) at 53.6% ± 16.7%.〈Constraint〉 text (45.5% ± 22.7%) and〈ConstraintList〉 (45.9% ± 24.7%) are also relatively preserved, as is the〈LExp〉 logical expression structure (50.6%± 22.9%). Together, these nonterminals form the logical skeleton of the few-shot exemplar. RQ2. Minimization preferentially eliminates natural lan- guage prose and cross-prompt relational annotations while retaining the logical skeleton. C. RQ3: Task Accuracy of the Minimized Prompt To answer RQ3, we generated 10 independent responses for each model using its minimized prompt and compared the resulting constraint sets against the baselines obtained from the full prompt. The diagonals in Figure 5 summarizes accuracy per model. Considering T=0.0, four models achieved 100% accuracy across all 10 runs, reproducing exactly the baseline constraints on both inquiries: gemma-2-9b-it, Qwen-2.5-32b, Llama-3.3- 70B, and Llama-3.1-8B. The remaining two models exhibited distinct failure modes. Considering T=0.5, only three models achieved 100% accuracy: gemma-2-9b-it, Mistral-Small-24B, and Qwen-2.5-72b. Qwen-2.5-72b. For Inquiry 1 the model produced the correct number of constraints, but the logical expression was rendered in free-form prose rather than the required →,∧ notation; we count such outputs as correct. For Inquiry 2, under the full prompt the model produced 5 constraints for Prompt 1 and 6 for Prompt 2, whereas under the minimized prompt it produced only 4 and 5, respectively. Mistral-Small-24B. The model had a single failure mode: for Inquiry 1 it generated a logical expression for Prompt 2 in only one of ten runs. All other runs matched the baseline. Overall, four of six models reproduce the full-prompt be- havior exactly after minimization, with output-format post- processing. RQ3. Four of six models reproduce full-prompt task accuracy exactly under their minimized prompts. The remaining two show specific, interpretable failure modes such as inconsis- tent output formatting, or omitted logical expressions rather than a general collapse in accuracy. D. RQ4: Cross-Model Transfer of the Minimized Prompt To answer RQ4, we cross-execute every minimized prompt on every evaluated model. For each ordered pair (M s ,M e )— source model M s that produced the minimized prompt, exe- cution model M e that consumes it—we ran 10 independent generations and measured constraint fidelity against the full- prompt baseline. Figure 5 reports the resulting 6×6 transfer matrices for T = 0.0 and T = 0.5; diagonal cells repeat the single-model self-accuracy from RQ3, while off-diagonal cells isolate the transfer effect. Transfer is the common case, not the exception. At T = 0.0, of the 30 off-diagonal pairs, 22 yield a non-empty output that parses into the expected propositional form, and the average accuracy of those 22 successful transfers is 78.1%. Eight of the 30 off-diagonal pairs meet 100% constraint fidelity. That is, they reproduce the full-prompt baseline as exactly as the source model does on itself, and thirteen reach at least 90%. The minimized core is therefore not a model-specific artifact. In the majority of cases it captures content that is sufficient for a different model to reconstruct the same propositional output. At T = 0.5 (Figure 5b), 28 of the 30 off-diagonal pairs yield non-empty output that parses into the expected propositional form, with a mean accuracy of 75.1%; seven of those 28 reach 100% fidelity. Source models differ sharply in how portable their prompts are. Reading Figure 5a by column isolates how well each source model’s minimized prompt travels. Two source models produce universally portable prompts at T = 0.0: every other execution model was able to use the minimized prompts from gemma-2-9b-it (mean 96.7% across five targets, all five non-empty) and from Mistral-Small-24B (mean 88.8%, all five non-empty). Turning to Figure 5b, at T = 0.5 the prompts from Llama-3.1-8B and Qwen-2.5-32b were success- fully consumed by all models, with mean accuracies of 95.3% 90 113 29 1536248 Remaining tokens (out of 266) 100.0 92.9 37.7100.00.046.4 83.7100.00.050.00.025.0 100.00.0100.0 93.8 0.00.0 100.0 92.947.9 77.50.084.3 100.0 42.939.3 100.0100.0 91.3 100.0100.0 90.8 100.00.0 90.8 gemma-2-9b-it Llama-3.1-8B Llama-3.3-70B Mistral-Small-24B Qwen-2.5-32bQwen-2.5-72b gemma-2-9b-it Llama-3.1-8B Llama-3.3-70B Mistral-Small-24B Qwen-2.5-32b Qwen-2.5-72b Execution Model (a) (T = 0.0) 68 109 38 9397 48 100.0 92.9 3.6 92.9 100.0 42.9 20.8 96.4 50.018.4 96.4 0.0 0.0100.0 95.0 80.0100.054.2 92.9 83.7 92.9 100.070.470.4 92.9 100.075.346.0 91.9 100.0 75.0100.0 91.7 58.3100.0100.0 gemma-2-9b-it Llama-3.1-8B Llama-3.3-70B Mistral-Small-24B Qwen-2.5-32bQwen-2.5-72b 0 20 40 60 80 100 (b) (T = 0.5) Source Model (Minimized Prompt Generator) Fig. 5: Cross-Model Transfer Accuracy (% Constraint Fidelity). Rows index the execution model; columns index the source model that produced the minimized prompt. Diagonal cells show each model’s 10-run self-accuracy under its own minimized prompt; off-diagonal cells show transfer accuracy when the minimized prompt from the column model is executed by the row model. Empty (zero) cells indicate that no valid output was produced. The red numbers above each column show the minimized prompt length (in tokens) produced by that source model. llama 3.3 70b qwen 72b qwen 32b gemma 2 9b it llama 3.1 8b mistral 24b 0 20 40 60 80 100 120 140 160 Source Model Example Length (Tokens) Information Density vs. Cross-Model Robustness T=0.0T=0.5 0 20 40 60 80 100 120 Universal Transfer Accuracy (%) Fig. 6: Information density versus cross-model robustness. For each source model (x-axis, ordered by example length), blue bars show the length of its minimized exemplar in tokens (left axis), and the red dashed line shows the mean transfer accuracy of that prompt when executed by the other five models (right axis). and 93.4% respectively. At the opposite extreme, the Qwen- 2.5-32b prompt at T = 0.0 fails to transfer to any other model (0/5 non-empty transfers). Llama-3.1-8B, Llama-3.3- 70B, and Qwen-2.5-72b sit in between, with four of five targets producing valid output at T = 0.0. At T = 0.5, Llama- 3.1-8B, Llama-3.3-70B, Mistral-Small-24B, and Qwen-2.5- 32b successfully transferred to all models. Source portability is therefore not monotone in model size: the largest source model in the study (Qwen-2.5-72b) is less portable than a 9B source (gemma-2-9b-it), and the single worst source (Qwen- 2.5-32b) sits in the middle of the size range. Execution models differ in what they can consume. Reading the matrix in Figure 5a by row isolates the execution side. gemma-2-9b-it, Mistral-Small-24B, and Qwen-2.5-72b each successfully consume prompts from four of the other five sources at T = 0.0; at T = 0.5 (Figure 5b) they consume prompts from all sources, as does Qwen-2.5-32b. The two Llama variants consume two and three of five sources respec- tively at T = 0.0, but reach four of five at T = 0.5. Qwen-2.5- 32b was able to consume all five in both temperature settings. The high execution-side scores of Qwen-2.5-72b (mean non- empty transfer of 95.4%) indicate that these models tolerate a wide variety of minimization styles, even when their own minimized prompt is less universally portable. Information density tracks transfer robustness. Figure 6 overlays, for each source model, the length in tokens of its minimized exemplar (left axis, blue and orange bars) against the mean transfer accuracy of that prompt when executed by the other five models (right axis, dashed lines). Very short minimization fails to carry enough signal for other models: Llama-3.3-70B, which produced the shortest minimized ex- emplar (≈ 29 tokens), reaches only about 49% at T = 0.0 and 62.7% at T = 0.5 mean transfer accuracy, and Qwen-2.5- 32b, whose minimization is similarly aggressive at T = 0.0, transfers to no other model at all. Qwen-2.5-32b, however, transfers to all models at 93.4% mean accuracy at T = 0.5, in which case its minimized prompt is considerably longer (≈ 97 tokens). In contrast, gemma-2-9b-it (≈ 90 tokens at T = 0.0) and Mistral-Small-24B (≈ 153 tokens at T = 0.0) achieve universal transfer at or above 88.0% mean accuracy. When temperature is raised to 0.5, these models compress their prompts further (to 68 and 93 tokens respectively), and their universal transferability decreases accordingly. Longer minimized exemplars transfer more reliably: gemma-2-9b-it (≈ 90 tokens) and Mistral-Small-24B (≈ 153 tokens) achieve universal transfer at or above 90% mean accuracy. In other words, a source model that compresses too aggressively over- fits to its own priors and strips content that other models still need, whereas a source that leaves more structural scaffolding in place produces a more model-agnostic minimal core. Transferred prompts can outperform self-accuracy. Eight off-diagonal cells reached 100%; notably, Mistral-Small-24B’s minimized prompt was successfully executed by Llama-3.3- 70B, Qwen-2.5-72b, and gemma-2-9b-it at 100.0%. Some models, conditioned on a different model’s minimization, re- produce the baseline constraints more reliably than when con- ditioned on its own minimization (e.g. Qwen-2.5-72b achieves 90.8% self-accuracy at T = 0.0 but executes gemma-2-9b- it, Llama-3.1-8B, and Mistral-Small-24B prompts at 100.0%). This phenomenon persists across sampling temperatures; at T = 0.5, Llama-3.1-8B achieves 96.4% self-accuracy under its own prompt, yet that same minimized prompt serves as a perfect instructional scaffold (100.0% execution accuracy) for Qwen-2.5-32b, Qwen-2.5-72b, and Llama-3.3-70B. This is consistent with the reading that minimization is not strictly a per-model operation: a prompt minimized by one model can serve as a better scaffold for a second model than the second model’s own minimization. RQ4. The minimized prompt transfers across models in (22 of 30 at T=0.0; 28 of 30 at T=0.5) off-diagonal pairs. Transfer is not determined by model size: aggressive minimization (fewer tokens) overfits the prompt to the source model’s priors, while prompts that retain more structural scaffolding (more tokens) serve as highly portable instructions that occasionally outperform a target model’s own self-accuracy. V. DISCUSSION A. Model Scale as a Proxy for Scaffold Independence We model achievable compression as a function of model size S (in billions of parameters) and sampling temperature T , using ordinary least squares across all 12 model/temperature runs (n = 12). The saturated model is μR| S,T = β 0 + β 1 S + β 2 T + β 3 (S× T ) and is reported in Table IV. TABLE IV: Saturated model: R∼ S +T +S×T . R 2 = 0.566, Adj. R 2 = 0.403. EstimateStd. ErrortPr(>|t|) (Intercept)46.418.475.48< 0.001 S0.510.192.670.029 T14.4023.950.600.564 S×T−0.320.54 −0.600.564 TABLE V: Reduced model: R ∼ S + T . R 2 = 0.546, Adj. R 2 = 0.446. EstimateStd. ErrortPr(>|t|) (Intercept)49.316.707.36< 0.001 S0.430.133.290.009 T2.7713.630.200.844 VIF values for S, T , and S × T are 2.00, 2.87, and 3.87, indicating no problematic multicollinearity. The interaction term S × T has the highest p-value (0.564) and is removed first, giving Table V. Temperature remains far from significance (p = 0.844). An ANOVA comparison confirms that adding T to a size- only model contributes no explanatory power (F = 0.04, p = 0.844). Removing T yields the final model in Table VI. The final equation is μR | S = 0.43· S + 50.0, shown as the fitted line in Figure 4. Each additional 10B parameters corresponds to 4.3 percentage points of additional achievable compression. There is no significant effect of sampling tem- perature on compression rate in any model formulation tested. However, the size-only model explains just over half the variance (R 2 = 0.544), and conflates parameter count with architecture, training data, and alignment strategy. As we show below, a categorical family variable captures substantially more variance (R 2 = 0.912), indicating that raw parameter count is a useful but crude first-order proxy for scaffold independence. Log-scale comparison. Because model parameter counts often span orders of magnitude, we also fit the log-linear family R∼ logS +T + logS×T and reduce it by the same stepwise procedure. Neither T (p = 0.867) nor logS× T (p = 0.565) are significant, yielding R ∼ logS as the log-scale final model. Table VII compares the two final single-predictor models directly via AIC. The linear model is preferred by ∆AIC = 3.83, confirming that the relationship between capacity and compression is better described as linear than as logarithmic across the 8B–72B range studied. Architecture family. The large residuals from the linear model—gemma-2-9b-it exceeds its size-based prediction by 14.3 points, Mistral-Small-24B falls short by 16.2—suggest that model architecture and training data contribute to scaffold independence independently of parameter count. To quantify this, we replace S with a categorical family variable F (six levels; Llama-3.1-8B as reference), resulting in the saturated family model R ∼ F + T + F × T . Following the same reduction procedure, F × T is not significant (F = 0.29, p = 0.883 for the joint interaction test), and T is again not significant after removing the interaction (p = 0.743). The TABLE VI: Final model: R ∼ S. R 2 = 0.544, Adj. R 2 = 0.499. EstimateStd. ErrortPr(>|t|) (Intercept)50.005.499.11< 0.001 S0.430.123.460.006 TABLE VII: AIC comparison of final single-predictor models (n = 12). Lower AIC indicates a better fit penalized for model complexity. ModelR 2 Adj. R 2 AIC∆AIC R∼ S0.5440.49993.900.00 R∼ log S0.3730.31097.73+3.83 reduced family model R∼ F is reported in Table VIII. Table IX summarizes all the models. The family model im- proves fit dramatically over the size model (∆AIC =−11.71, R 2 = 0.912 vs. 0.544), demonstrating that architecture and training choices explain a large share of the variance that the raw parameter count cannot capture. Within the family model, two contrasts are particularly informative. First, gemma-2-9b-it achieves 16.5 percentage points more compression than Llama-3.1-8B despite having essentially the same parameter count (9B vs. 8B; p = 0.042). This rules out size as the sole explanation and implicates architectural or training-data differences between the two fam- ilies. Second, Mistral-Small-24B performs 7.7 points below the reference 8B model (not significant, p = 0.275), despite being three times larger. The Mistral family therefore does not gain the scaffold independence that its size would predict under the linear model. The two large models—Llama-3.3-70B and Qwen-2.5-72b—both show strongly significant positive contrasts (p = 0.002 and p = 0.005), consistent with the size effect identified earlier, but their magnitude differs (34.1 vs. 27.4 points above reference), again pointing to family-level variation beyond size. B. Universal Encoders and Universal Decoders The cross-model transfer matrix (Figure 5) exposes an asymmetry that a single-model view of minimization hides: source portability and execution robustness are two distinct model properties, and a model can be strong on one axis and weak on the other. Universal encoders. Inspecting Figure 5 columnwise, two source models produce minimized prompts that every other model evaluated can successfully consume: gemma-2-9b-it (mean transfer accuracy 96.7% across five targets) and Mistral- Small-24B (88.8%, all five targets non-empty). Neither is the largest model in the study. Their minimized prompts are comparatively long (roughly 90 and 154 tokens respectively, see Figure 6), indicating that these models leave enough structural scaffolding in place to serve as a model-agnostic minimal core. We call such models universal encoders. Universal decoders. Inspecting Figure 5 rowwise, some models reliably consume minimized prompts produced by TABLE VIII: Family model: R ∼ F (Llama-3.1-8B as reference). R 2 = 0.912, Adj. R 2 = 0.838. EstimateStd. ErrortPr(>|t|) (I) [Llama-3.1-8B]51.704.5111.47< 0.001 gemma-2-9b-it+16.456.372.580.042 Mistral-Small-24B −7.656.37 −1.200.275 Qwen-2.5-32b+11.356.371.780.125 Llama-3.3-70B+34.106.375.350.002 Qwen-2.5-72b+27.406.374.300.005 TABLE IX: Summary of all regression models (n = 12). ModelR 2 Adj. R 2 AIC∆AIC R∼ S0.5440.49993.900.00 R∼ log S0.3730.31097.73+3.83 R∼ S + T0.5460.44695.85+1.95 R∼ log S + T0.3750.23699.69+5.79 R∼ F0.9120.83882.19 −11.71 R∼ F + T0.9140.81183.90 −10.00 other sources regardless of how aggressively those sources compress. Qwen-2.5-72b is the clearest example, reaching a mean non-empty transfer accuracy of 95.4% on prompts from four of the five other source models, notably including gemma- 2-9b-it, Llama-3.1-8B, and Mistral-Small-24B prompts, on which it scores 100%, despite achieving only 90.8% self- accuracy on its own minimized prompt. Qwen-2.5-32b is a second example, which shows a high ability to execute on all sources (74.7% mean transfer, five of five sources). We call such models universal decoders. Interaction with compression aggressiveness. The Qwen- 2.5-32b row is informative: its own minimized prompt is aggressive enough that no other model can consume it (0/5 non-empty transfers), yet Qwen-2.5-32b itself consumes other models’ minimizations on all five targets. Aggressive self- compression and poor portability therefore co-occur, consistent with the interpretation that over-compression encodes model- specific shortcuts. Figure 6 makes this quantitative: the two shortest minimized exemplars (Llama-3.3-70B at ≈29 tokens, Qwen-2.5-32b at ≈63 tokens) correspond to the two worst mean transfer accuracies (43% and 0%), while the two longest exemplars (gemma-2-9b-it and Mistral-Small-24B) correspond to the two universal encoders. This asymmetry has a practical consequence. If a minimized prompt must be deployed across heterogeneous models, a com- mon case in production pipelines with model-swap or fallback logic, the minimization should be produced on a universal- encoders model rather than on the aggressively compressing ones, even when the latter achieves a higher self-compression rate. C. Implications for Prompt Engineering Our findings have three practical implications for prompt engineering. 1) Cross-prompt annotations are redundant. The high removal rate of〈LRel〉 (72.6%) is perhaps the most actionable finding for prompt designers. The LogicalRelationshipWithPreviousPrompt block was included in the P2P exemplar to help the model understand how consecutive prompts relate to one another—refinements, additions, and shared conjuncts across examples. Its near-complete dispensability indi- cates that models process each exemplar largely inde- pendently rather than chaining them relationally. Prompt designers can therefore omit cross-prompt relationship annotations without loss of output fidelity, reducing prompt length and engineering overhead simultaneously. 2) Logical identifiers are the causal core. The sur- vival of〈PID〉,〈CID〉,〈Constraint〉, and〈LExp〉 sug- gests that models attend to the structural pattern—the typed slot names, the identifier tokens, and the logical connectives—rather than to the relational scaffolding that surrounds them. This aligns with prior work show- ing that labels in few-shot examples can be randomly permuted with only modest accuracy loss [8], and ex- tends that observation to a finer-grained, grammar-level analysis: it is the schema of the exemplar, not its content, that is causally necessary. 3) Minimize on a universal encoder, deploy on a univer- sal decoder. When a single prompt must serve multiple downstream models, the minimization source matters more than the minimization rate. A prompt minimized by gemma-2-9b-it or Mistral-Small-24B transfers uni- versally, whereas an equally or more compressed prompt from Qwen-2.5-32b fails to transfer to any other model. Pairing a universal-encoder source with a universal- decoder target (e.g. Mistral-Small-24B → Qwen-2.5- 72b) yields fidelity that can exceed the target’s self- accuracy. VI. RELATED WORK A. Delta Debugging and Prompt Minimization The core of our D-FSM pipeline is delta debugging, a technique for automatically isolating the minimal cause of a failure. Zeller and Hildebrandt [12] introduced the minimizing delta debugging algorithm (ddmin) to find the minimal cause of a bug in a failing program. This technique was further improved by Zeller [16] so that ddmin can isolate cause-effect chains in program execution. Furthermore, exploiting struc- tural grammar was an improvement introduced by Misherghi and Su [17] through Hierarchical Delta Debugging. Beyond Delta Debugging, several techniques were intro- duced for prompt minimization. Su et al. [18] introduced Selective Annotations, which select the most informative examples. This technique led to effectively minimizing the number of examples without losing performance. Additionally, RLPrompt is a reinforcement-learning technique introduced by Deng et al. [19] that identifies the most effective combinations of tokens to optimize and simplify prompts for a given task. Shi et al. [15] introduced Prompt Space, a mathematically grounded framework that selects few-shot exemplars. Prompt Space showed improved performance compared to heuristic prompting strategies such as zero-shot and few-shot. B. Chain-of-Thought and Few-shot Learning Brown et al. [1] established few-shot prompting as the standard approach to in-context learning. Few-shot prompting allows models to complete tasks by learning from a number of examples included in the prompt. Chain-of-Thought (CoT) prompting emerged as a major development in this field. Wei et al. [2] showed that when the model is asked to show its intermediate reasoning steps, it can achieve much higher performance on difficult math and logic problems. Because LLMs can be unpredictable (stochastic), Wang et al. [20] proposed Self-Consistency to handle such cases. Self- Consistency demonstrates that even if the reasoning process varies, the final logical result should stay consistent across different attempts. Although long CoT prompts are effective, recent studies have shown that models might not actually need the entire CoT text. Kojima et al. [21] showed that LLMs can act as Zero-Shot Reasoners when asked to use a simple phrase like “Let’s think step by step.” This suggests that a large portion of the CoT text in few-shot examples may be redundant or unnecessary. Additionally, Reynolds and McDonell [22] argued that few- shot examples serve as “locative” signals that guide the model to the correct task environment, rather than teaching it new information. These studies provide the theoretical foundation for our D-FSM framework. This suggests that within every complex CoT prompt there may be a “minimal causal core” that can be extracted without damaging the model’s logical performance. C. Prompt Sensitivity and Reliability The non-determinism of LLMs has been highlighted in re- cent research. This phenomenon creates challenges for down- stream tasks. Although deterministic-behavior settings such as setting temperature to zero exist, the model can still produce different outputs. Astekin et al. [23] showed that LLMs produced inconsistent log-parsing templates across multiple runs, which raises serious concerns about reproducibility in software engineering. Similarly, Atil et al. [24] conducted a systematic study to quantify output variability across different benchmarks. Regardless of the deterministic configurations, they found accuracy variation of up to 15%. Prompts are highly sensitive to small changes in their structure. Zhao et al. [7] showed that the order and formatting of examples can play a large role in accuracy. A model may therefore produce the right answer using one example but fail if one important word is removed. This instability can lead to security risks. Pearce et al. [9] found that a model can produce insecure code because of prompt variations. Moreover, Lu et al. [6] found that prompt structure often matters more than the semantic meaning of the text. These findings motivate the need for an approach that tells researchers which specific part of a prompt is important to maintain stable output. VII. THREATS TO VALIDITY A. External Validity Our experiments cover six models and two temperature settings. While this range spans 8B to 72B parameters and includes four distinct model families (Llama, Gemma, Qwen, and Mistral), the results may not generalize to models trained on different corpora, to proprietary frontier models, or to models using alternative decoding strategies (for example, nucleus or beam search). The evaluation dataset itself is also narrow: two inquiries of two prompts each, drawn from a programming-tasks domain. Findings about which structural layers are preserved may shift for prompts in natural-language, mathematical, or multimodal domains, and we mitigate this only partially by releasing the D-FSM pipeline so other researchers can run the same analysis on their own datasets and models. B. Internal Validity The P2P formalization layer introduces a potential source of error. If the formalization itself is imperfect, the comparison between baseline and minimized outputs will be unreliable. We validated P2P on a separate dataset [13] and found it to be highly accurate. A second internal threat is the non- determinism of the models themselves: a configuration that matches the baseline on one run may not match it on re- evaluation. To mitigate this, D-FSM classifies any such non-reproducible configuration as UNRESOLVED rather than PRESERVED (see Section I-C), so flaky runs do not cause over-reduction, although they may cause under-reduction. C. Construct Validity “Propositional output fidelity” is a proxy for “task cor- rectness.” We have shown that minimized prompts preserve propositional output, but it is possible that they fail to capture some subtle aspects of task intent that are not reflected in the constraint set. Our RQ3 accuracy experiments provide partial evidence that the proxy is reasonable—four of six models reproduce full-prompt behavior exactly—but the two inquiries we evaluate are too few to claim construct validity at scale. A further construct risk is that D-FSM measures characters removed rather than semantic information removed: two reductions that delete the same number of characters can differ substantially in their effect on model behavior. We partially mitigate this in RQ2 by attributing removals to grammar nonterminals, which at least groups characters by their semantic role in the prompt. VIII. CONCLUSION This paper presents D-FSM, a blackbox minimization framework for identifying the minimal causal core of prompts. We show that few-shot exemplars can be reduced by a mean of 65.3% ± 15.8% in character count while fully preserv- ing propositional output fidelity. Analysis of the surviving structure reveals that models preferentially retain the logical skeleton of exemplars while discarding natural language prose and cross-prompt relational annotations. Regression analysis demonstrates that model scale is the dominant factor in achievable compression, with each additional 10B parameters corresponding to 4.3 percentage points of additional removal. Temperature has no consistent directional effect on compres- sion rate. Furthermore, model architecture and training choices explain a large share of the variance that raw parameter count cannot capture. Our findings have three practical implications for prompt engineering: cross-prompt annotations are redundant, logical identifiers constitute the causal core of few-shot exemplars, and minimization should be produced on a universal encoder and deployed on a universal decoder. Future work will ex- plore whether these findings generalize to other domains and whether minimized prompts can be used for safety verification of LLM-based systems. REFERENCES [1] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, p. 1877–1901, 2020. [2] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, p. 24 824–24 837, 2022. [3] Gartner, Inc., “Predicts 2024: The impact of generative AI on the enterprise,” Gartner, Tech. Rep., 2024. [4] SQ Magazine, “The state of prompt engineering in enterprise AI: Usage and market share,” Software Quality Magazine, 2024. [5] Index.dev, “State of AI adoption in 2026,” Index.dev Industry Reports, 2026. [6] Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp, “Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity,” in Proc. Annu. Meet. Assoc. Comput. Linguist. (ACL), 2022, p. 8086–8098. [7] Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, “Calibrate before use: Improving few-shot performance of language models,” in Proc. Int. Conf. Mach. Learn. (ICML), 2021, p. 12 697–12 706. [8] S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer, “Rethinking the role of demonstrations: What makes in-context learning work?” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2022, p. 11 048–11 064. [9] H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, and R. Karri, “Asleep at the keyboard? Assessing the security of GitHub Copilot’s code contributions,” in Proc. IEEE Symp. Secur. Privacy (SP), 2022, p. 754– 768. [10] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Mober et al., “DSPy: Compiling declarative language model calls into self-improving pipelines,” arXiv preprint, vol. arXiv:2310.03714, 2023. [11] Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large language models are human-level prompt engineers,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023. [12] A. Zeller and R. Hildebrandt, “Simplifying and isolating failure-inducing input,” IEEE Trans. Softw. Eng., vol. 28, no. 2, p. 183–200, 2002. [13] A. Alfageeh, S. A. K. Zarkouei, D. Nam, D. Prol, M. Amoozadeh, S. Chattopadhyay, J. Prather, P. Denny, J. Leinonen, M. Hilton, S. S. Ragavan, and A. Alipour, “From prompts to propositions: A logic-based lens on student-LLM interactions,” in Proc. Koli Calling Int. Conf. Comput. Educ. Res., 2025. [14] D. Aftab and S. Davy, “Tailored-LLaMA: Optimizing few-shot learning in pruned LLaMA models with task-specific prompts,” arXiv preprint, vol. arXiv:2410.19185, 2024. [15] F. Shi, P. Qing, D. Yang, N. Wang, Y. Lei, H. Lu, X. Lin, and D. Li, “Prompt space optimizing few-shot reasoning success with large language models,” in Findings Assoc. Comput. Linguist.: NAACL, 2024, p. 1836–1862. [16] A. Zeller, “Isolating cause-effect chains from computer programs,” ACM SIGSOFT Softw. Eng. Notes, vol. 27, no. 6, p. 1–10, 2002. [17] G. Misherghi and Z. Su, “HDD: Hierarchical delta debugging,” in Proc. Int. Conf. Softw. Eng. (ICSE), 2006, p. 142–151. [18] H. Su, J. Kasai, C. H. Wu, W. Shi, T. Wang, J. Xin, R. Zhang, M. Ostendorf, L. Zettlemoyer, N. A. Smith et al., “Selective annotation makes language models better few-shot learners,” arXiv preprint, vol. arXiv:2209.01975, 2022. [19] M. Deng, J. Wang, C.-P. Hsieh, Y. Wang, H. Guo, T. Shu, M. Song, E. Xing, and Z. Hu, “RLPrompt: Optimizing discrete text prompts with reinforcement learning,” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2022, p. 3369–3391. [20] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint, vol. arXiv:2203.11171, 2022. [21] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, p. 22 199–22 213, 2022. [22] L. Reynolds and K. McDonell, “Prompt programming for large language models: Beyond the few-shot paradigm,” in Extend. Abstr. ACM CHI Conf. Hum. Factors Comput. Syst., 2021, p. 1–7. [23] M. Astekin, M. Hort, and L. Moonen, “An exploratory study on how non-determinism in large language models affects log parsing,” in Proc. ACM/IEEE Int. Workshop Interpret. Robust. Benchmark. Neural Softw. Eng. (InteNSE), 2024, p. 13–18. [24] B. Atil, S. Aykent, A. Chittams, L. Fu, R. J. Passonneau, E. Radcliffe, G. R. Rajagopal, A. Sloan, T. Tudrej, F. Ture et al., “Non-determinism of “deterministic” LLM settings,” arXiv preprint, vol. arXiv:2408.04667, 2024.

Instant research discovery