Paper deep dive
A Theory of LLM Information Susceptibility
Zhuo-Yang Song, Hua Xing Zhu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/26/2026, 1:48:49 AM
Summary
The paper proposes a theory of 'LLM information susceptibility' to understand the limits of LLM-mediated optimization in agentic systems. It hypothesizes that for a fixed LLM, the performance susceptibility (the rate of performance gain relative to computational budget) cannot exceed that of the base strategy in the large-budget limit. The authors validate this theory across diverse domains (Tetris, Knapsack, Ranking, AIME) and demonstrate that while fixed LLM layers are constrained by this susceptibility bound, nested, co-scaling architectures can overcome it, suggesting a structural requirement for open-ended agentic self-improvement.
Entities (6)
Relation Signals (3)
LLM Information Susceptibility Theory → predictslimitationsof → Fixed LLM Layers
confidence 95% · the susceptibility of the LLM-derived strategy does not exceed that of the base strategy
Nested Architectures → enablesimprovementbeyond → Susceptibility Bound
confidence 92% · nested, co-scaling architectures open response channels unavailable to fixed configurations.
Statistical Physics → providestoolsfor → AI System Design
confidence 90% · demonstrating that tools from statistical physics can provide predictive constraints for the design of AI systems.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are increasingly deployed as optimization modules in agentic systems, yet the fundamental limits of such LLM-mediated improvement remain poorly understood. Here we propose a theory of LLM information susceptibility, centred on the hypothesis that when computational resources are sufficiently large, the intervention of a fixed LLM does not increase the performance susceptibility of a strategy set with respect to budget. We develop a multi-variable utility-function framework that generalizes this hypothesis to architectures with multiple co-varying budget channels, and discuss the conditions under which co-scaling can exceed the susceptibility bound. We validate the theory empirically across structurally diverse domains and model scales spanning an order of magnitude, and show that nested, co-scaling architectures open response channels unavailable to fixed configurations. These results clarify when LLM intervention helps and when it does not, demonstrating that tools from statistical physics can provide predictive constraints for the design of AI systems. If the susceptibility hypothesis holds generally, the theory suggests that nested architectures may be a necessary structural condition for open-ended agentic self-improvement.
Tags
Links
- Source: https://arxiv.org/abs/2603.23626v1
- Canonical: https://arxiv.org/abs/2603.23626v1
Trouble viewing inline? Open PDF directly →
Full Text
55,547 characters extracted from source content.
Expand or collapse full text
Preprint A Theory of LLM Information Susceptibility Zhuo-Yang Song 1,B Hua Xing Zhu 1,2,B 1 School of Physics, Peking University, Beijing 100871, China 2 Center for High Energy Physics, Peking University, Beijing 100871, China B e-mail: zhuoyangsong@stu.pku.edu.cn; zhuhx@pku.edu.cn Large language models (LLMs) are increasingly deployed as optimization modules in agentic systems, yet the fundamental limits of such LLM-mediated improvement remain poorly understood. Here we propose a theory of LLM information susceptibility, centred on the hypothesis that when computational resources are sufficiently large, the intervention of a fixed LLM does not increase the performance susceptibility of a strategy set with respect to budget. We develop a multi-variable utility-function framework that generalizes this hypothesis to architectures with multiple co-varying budget channels, and discuss the conditions under which co-scaling can exceed the susceptibility bound. We validate the theory empirically across structurally diverse domains and model scales spanning an order of magnitude, and show that nested, co-scaling architectures open response channels unavailable to fixed configurations. These results clarify when LLM intervention helps and when it does not, demonstrating that tools from statistical physics can provide predictive constraints for the design of AI systems. If the susceptibility hypothesis holds generally, the theory suggests that nested architectures may be a necessary structural condition for open-ended agentic self-improvement. Introduction Large language models (LLMs) are rapidly becoming core components of agentic systems, especially when combined with search, planning, verification, memory and tool-use modules 1–8 . Such systems often outperform pure language modeling or traditional pipelines alone, motivating growing interest in agents that iteratively improve their own strategies, modules or internal organization 9–15 . Meanwhile, the empirical success of LLM-mediated optimization has outpaced our theoretical understanding of its limits. Existing work has focused primarily on specific prompting, training or inference schemes 16–25 , but a general theoretical framework for understanding the fundamental limits of LLM-mediated optimization remains absent. Here we propose a hypothesis about the limits of LLM- mediated optimization and, drawing on linear response the- ory 26,27 , develop a framework to understand its applicability across different agent architectures. We treat an agent as produc- ing a strategy set together with a utility function퐽defined over that set, and study how퐽changes with respect to computational variables that the architecture can control. This viewpoint is inherently broad: depending on the task and agent structure, 퐽may denote score, accuracy, ranking quality or another op- erational measure of performance, while the relevant budget variableBmay denote beam width, search depth, sample count, model size, verification effort or other architecture-dependent resources. Within this formulation, we hypothesize that a fixed LLM-derived mapping cannot increase the performance susceptibility of the strategy set with respect to budget. When there is only a single budget variable, this hypoth- esis can be equivalently expressed as a relative sensitivity훼 that has an upper bound of one in the large-budget regime. This hypothesis is significant because it separates two ques- tions often conflated in discussions of agentic improvement: whether LLMs help in finite-budget settings (empirically, often yes 9,10,17,19 ) and whether a fixed LLM layer can improve the asymptotic response of performance to additional computation. Our experiments address the latter question and give a negative answer in the fixed-architecture setting. This provides a more precise way to reason about the design and optimization of high-compute pipelines and self-evolving agents 4,28–31 . A self-evolving agent cannot simply be under- stood as a system that repeatedly applies the same optimization layer to its own outputs; rather, it must be a system whose performance-relevant components and computational channels change as complexity grows 7,10,12,13,15 . We argue that, if the susceptibility hypothesis holds generally, nested architectures may be a necessary structural condition for overcoming the sus- ceptibility bound imposed by a fixed LLM layer. More broadly, the framework developed here demonstrates that theoretical tools from statistical physics can provide a priori constraints and predictive structure in the design of complex agentic sys- tems 27–32 . Results A theory of LLM information susceptibility Consider a base strategy setP B generated under a compu- tational budgetBto maximize a utility function퐽(P B )(see Fig. 1, right). As the computational resources increase with- out bound,퐽(P B→∞ )approaches its optimal value퐽 ∞ . Now introduce a fixed LLM that reads the base strategy setP B and outputs a derived strategy setP ′ B . We hypothesize that, when computational resources are sufficiently large, the performance susceptibility ofP ′ B does not exceed that ofP B : lim B→∞ 휕퐽(P B ) 휕B ≥ lim B→∞ 휕퐽(P ′ B ) 휕B ,(1) where⟨·⟩denotes the average over different random seeds or experimental repetitions. This is the central hypothesis of 1 arXiv:2603.23626v1 [cs.LG] 24 Mar 2026 Preprint the theory: the susceptibility휕퐽/휕Bunder the LLM-derived strategy cannot exceed that under the base strategy in the large- budget limit. The use of partial derivatives is deliberate:퐽 may in general depend on multiple budget variables, and this formulation provides the basis for the multi-variable gener- alization developed below. As a hypothesis, equation (1) is empirically testable and carries concrete implications for agent design: it implies that fixed LLM layers cannot improve the asymptotic scaling trajectory of the base strategy. Importantly, the asymptotic regime sets in at practically relevant budget levels: as shown in Fig. 3, the relative sensitivity훼defined in equation (2) crosses below 1 at푘 ∼ 12independent samples, after which the susceptibility bound is already operative. This rapid onset means the bound is not merely a theoretical limit but a constraint that governs real-world agent performance. The intuition behind this claim rests on two arguments. First, asB →∞, the performance퐽(P B )converges toward the global optimum퐽 ∞ , so the residual improvable gap퐽 ∞ −퐽(P B ) shrinks. Any mapping applied toP B , including the LLM, can only redistribute probability mass among strategies already present in or reachable fromP B ; it cannot inject strategies that are not computable from the information contained inP B and the LLM’s fixed parameters. Second, a fixed LLM can be viewed as a deterministic (or fixed-distribution) channel with finite capacity 33 : it compresses the input strategy set through a fixed-dimensional representation, based on its context window and parameters, and outputs a derived set. When the base set already encodes near-optimal information at largeB, the channel cannot amplify the marginal information content of additional budget. Since the mutual information between the derived set and the optimal strategy cannot exceed that between the base set and the optimal strategy by data-processing- inequality reasoning 34 , the marginal return on budget cannot increase through the LLM intervention. This argument is not a formal proof, but it motivates why the bound훼 ≤ 1should hold generically rather than being an artefact of specific tasks. Figure 1 shows representative results for the Tetris domain (see Methods for full experimental details). The performance of the base strategy set (beam search with depth-first back- tracking, hereafter DFS) increases monotonically with beam width, while the LLM-derived strategy set exhibits a consis- tently lower susceptibility across all five Qwen-series models ranging from 7B to∼ 200B parameters. A linear fit yields an average slope of 1.4 for the base algorithm versus 0.5 for the LLM-derived strategies, indicating that the LLM transforms each unit increase in beam width into about one third the perfor- mance gain of the base algorithm. This pattern is remarkably consistent: all five models, despite their order-of-magnitude difference in parameter count, fall within the same narrow performance band at each beam width, suggesting that the susceptibility bound is not merely a consequence of insufficient model capacity but reflects a structural property of the fixed- LLM intervention. We define the normalized performance gap asΔ(B) = 퐽(P B )− 퐽(P ′ B ) / 퐽(P B ), where퐽(P B )is the mean of the base performance over budget levels. The per-model breakdown ofΔ(B)across all four domains is shown in Extended Data Fig. 1 and Extended Data Fig. 2, confirming that this pattern holds at the level of individual models. When the utility function퐽depends on a single budget variableB, the hypothesis can equivalently be expressed in terms of a relative sensitivity: 훼(B) = 푑퐽(P ′ B ) 푑퐽(P B ) = 휕퐽(P ′ B )/휕B ⟨ 휕퐽(P B )/휕B ⟩ ≤ 1 (B →∞). (2) Here훼(B)admits a natural interpretation: computational re- sources increase the mutual information between the strategy set and the optimum, while the fixed LLM channel cannot am- plify this information gain (by the data-processing inequality), so that훼 ≤ 1when computational resources are sufficiently large. Robustness of the susceptibility bound A natural concern is whether the observed susceptibility gap is an artefact of specific prompt engineering choices or reward function design. We tested both systematically in the Tetris domain (Fig. 2). Four prompt variants were evaluated: minimal (JSON-only output), standard (full analysis), chain-of-thought (5-step reasoning) and expert (domain-specific strategy). All variants exhibit the same qualitative behaviour: the susceptibil- ity of the LLM-derived strategy does not exceed that of the base strategy (Fig. 2a). The observation that the minimal prompt, which provides the least guidance to the LLM, nearly matches the DFS baseline implies that the gap arises from active repro- cessing rather than a passive information bottleneck. Three distinct reward functions likewise show qualitative invariance: in all cases the DFS baseline outperforms the LLM-derived strategy and the gap grows with budget (Fig. 2b), confirming that the susceptibility bound is a structural property of the fixed- LLM intervention, independent of prompt design or reward signal. Empirical characterization of the sufficiency condition The theory predicts that the susceptibility bound holds when computational resources are “sufficiently large”, but does not specify the threshold a priori. To characterize this transition empirically, we designed an experiment using 60 mathematics problems from AIME 2024 and 2025 35,36 . In this domain the performance depends on three variables:퐽 = 퐽(푘,B gen ,B sel ), where푘is the number of independent solution attempts,B gen is the generator model size andB sel is the selector model size. A generator LLM of sizeB gen produces푘independent solution attempts, and the base strategy applies majority vote 17 . A fixed selector LLM of sizeB sel then reads the candidate answers and selects one, forming the derived strategy setP ′ B . This generate-then-select architecture has been widely adopted in competitive programming 37 , mathematical reasoning 38,39 and scientific discovery 12,13 . To isolate the effect of the sample budget푘, we average over all five selector model sizesB sel and all five generator model sizesB gen . This yields an average sensitivity ̄훼(푘) = ⟨훼(B gen ,B sel ; 푘)⟩ B gen ,B sel that characterizes how the relative advantage of the LLM selector evolves as the base strategy aggregates more samples. Figure 3 shows ̄훼(푘)as a function of푘. At low푘(≤ 5), the selector LLM could outperform majority vote ( ̄훼 > 1), reflecting the regime in which the LLM’s world knowledge and reasoning provide a genuine advantage over a sparse vote distribution. As푘increases, ̄훼crosses below 1 and continues to 2 Preprint 12481632 Computational budget B (beam width) 0 2 4 6 8 10 12 Performance J (lines cleared) P B LLM P ′ B J(P ′ B ) Derived J(P B ) Base DFS fit LLM avg fit P B (DFS) P ′ B (7B) P ′ B (14B) P ′ B (32B) P ′ B (72B) P ′ B (Qwen3) Fig. 1|Framework and representative results. Performance퐽(lines cleared) versus computational budgetB(beam width) in the Tetris domain for the base algorithm (DFS, blue circles) and LLM-derived strategies (red markers; five Qwen models: 7B, 14B, 32B, 72B and Qwen3-Max). Dashed lines show linear fits for DFS and the LLM average. Error bars indicate the standard error of the mean across 40 random seeds. The schematic on the right illustrates the two evaluation paths: the base strategy setP B is evaluated directly by the utility function퐽(base path, blue), or first processed by a fixed LLM to produce a derived setP ′ B (derived path, red). 12481632 B (beam width) 2 4 6 8 10 12 J (lines cleared) a DFS (base) LLM (minimal) LLM (chain-of-thought) LLM (expert) LLM (standard) 12481632 B (beam width) 2 4 6 8 10 12 J (lines cleared) b DFS (aggressive) LLM (aggressive) DFS (conservative) LLM (conservative) DFS (default) LLM (default) Fig. 2|Robustness of the susceptibility bound. a, Four prompt variants compared against the DFS baseline in the Tetris domain (Qwen-32B). The minimal prompt nearly matches DFS at highB, while more elaborative prompts amplify the gap. b, Three reward functions overlaid for both DFS (grey shades) and LLM (red shades, Qwen-32B). The susceptibility bound holds across all prompt and reward configurations. decline, marking the onset of the large-budget regime in which majority vote becomes statistically robust and the fixed selector can no longer improve upon it. This crossover 40 provides an empirical operationalization of “sufficiently large”: where the base strategy’s aggregation of diverse samples begins to dominate the LLM’s judgement. Cross-domain validation To test the universality of the hypothesis, we conducted experiments across four task domains that differ substantially in their structure and the role of LLM knowledge: Tetris (combinatorial game-playing), 0/1 Knapsack 41 (combinatorial optimization), world-knowledge Ranking (factual recall under noise) and AIME mathematics (multi-step reasoning). Full experimental configurations are provided in Methods; results are shown in Fig. 4. Across all domains, the base strategy set’s performance increases monotonically with computational budget, while the LLM-derived strategy set’s susceptibility is generally not larger, validating equation (1). The Ranking domain is particularly instructive: at low budgets, the LLM significantly outperforms the noisy algorithmic baseline because it can draw on world 3 Preprint 35915171921 k (number of samples) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 α ( k ) α = 1 α(k) (linear fit) Fig. 3|Transition of the relative sensitivity훼. Average훼versus the number of samples푘. For each푘,훼is estimated by fitting 퐽 agent = 훼· 퐽 MV + 훽across five generator model sizes, where퐽 MV is the majority-vote accuracy,퐽 agent is the LLM-selector accuracy and훽 is the regression intercept (see Methods). As푘increases,훼decreases and falls below 1 around푘 ∼ 12, marking the onset of the large-budget regime where the susceptibility bound takes effect. knowledge (for example, identifying China as more populous than Japan regardless of the noisy score estimate). However, as the signal-to-noise ratio increases, the algorithmic ranking converges to the ground truth and the LLM advantage vanishes, consistent with the hypothesis’s prediction that the susceptibility advantage of the base strategy dominates at large budget. This pattern, greater LLM advantage at low budget and greater algorithmic advantage at high budget, is precisely the signature predicted by the theory and is observed across all four domains. These results underscore that the utility function퐽is flexible and task-dependent: it may represent game score, solution qual- ity, ranking performance or answer accuracy, depending on the domain. Likewise,Bshould be interpreted as the controllable budget associated with the underlying agent. The Knapsack do- main deserves particular comment: the performance gapΔ(B) is nearly zero across all budget levels and model sizes (Extended Data Fig. 1). This is consistent with the theory (훼≤ 1) but does not exhibit the dramatic separation seen in Tetris. The likely explanation is that the LLM acts approximately as an identity mapping in this domain: because the beam-search candidates are already sorted by value density and the packing structure is opaque to the LLM without explicit combinatorial reasoning, the model largely defers to the algorithmic ranking rather than reprocessing it. This “pass-through” regime is similar to the minimal prompt regime in Tetris and represents a qualitatively different manifestation of the susceptibility bound, one in which the LLM neither helps nor hurts, because it recognizes the limits of its own intervention. The empirical evidence therefore supports a general statement: the hypothesis applies whenever one can define a strategy set, a utility function over that set and a meaningful computational variable with respect to which susceptibility is measured. Generality: 퐽 as a multi-variable utility function The experiments also clarify the scope of the framework. The basic formulation in equation (1) concerns a fixed derivation mapping responsive to a single effective budget variable. More generally, the utility function퐽depends on all architectural budget variables:퐽 = 퐽(B 1 ,B 2 ,...,B 푛 ). By analogy with linear response theory 26 , the gradient∇ B 퐽is the susceptibility vector; each component휕퐽/휕B 푖 measures how efficiently one budget channel converts additional compute into performance. Equation (2) is the푛 = 1special case in which a single budget variable controls the entire system. When the architecture is extended so that additional com- putational variables become relevant, the utility function can be correspondingly generalized. If we write퐽for the seed- averaged derived-strategy performance and퐽 base for that of the base strategy, both as deterministic functions of the budget variables, the generalized total sensitivity follows by summing over all budget channels that co-vary with a reference budget B ref : 훼 total = 푛 ∑︁ 푖=1 휕퐽/휕B 푖 휕퐽 base /휕B ref · 푑B 푖 푑B ref .(3) As a concrete example, in the AIME domain, if the selector LLM is allowed to vary with the generator LLM, then the utility of the derived strategy set becomes퐽(P ′ B gen ,B sel ). Here the results are averaged over large values of푘(푘 ∈ 15, 17, 19, 21), which is therefore not treated as a co-varying budget channel. Setting푛 = 2,B 1 = B gen ,B 2 = B sel andB ref = B gen , the relative sensitivity reduces to 훼(B gen ,B sel ) = 휕퐽(P ′ B gen ,B sel )/휕B gen 휕퐽(P B gen )/휕B gen + 휕퐽(P ′ B gen ,B sel )/휕B sel 휕퐽(P B gen )/휕B gen · 푑B sel 푑B gen .(4) The first term on the right-hand side is the fixed-architecture contribution constrained by the hypothesis (훼≤ 1), while the second term appears only when the architecture itself is allowed to vary with budget. Here푑B sel /푑B gen is the rate at which the selector’s budget changes when the generator’s budget is increased: it equals zero in the fixed-selector configuration and one when generator and selector are co-scaled. Note that equation (3) has a covariant-contravariant structure: the susceptibility vector휕퐽/휕B 푖 characterizes the local geome- try of the performance landscape (partial derivatives hold other budget variables fixed), while the scaling protocol푑B 푖 /푑B ref is a design choice specifying how budget channels co-vary. Their contraction yields the scalar훼 total , which depends on both the landscape and the chosen scaling path. This viewpoint reveals three distinct coupling regimes (Fig. 5). (i) Decoupled (푑B sel /푑B ref = 0): each budget chan- nel operates independently, and the hypothesis훼≤ 1applies to each channel separately; this is the regime described by equation (2). (i) Negative coupling: In this regime, co-scaling the selector with the generator reduces the marginal return of additional budget, analogous to Le Chatelier’s principle 42 , so that the total slope훼 total < 훼 gen ≤ 1falls below that of the fixed-selector curve, where훼 gen denotes the first term on the right-hand side of equation (4), the contribution from the 4 Preprint 12481632 B (beam width) 2 4 6 8 10 J (a) Tetris (lines cleared) J(P B ) J(P ′ B ) 1248163264 B (beam width) 1550 1560 1570 1580 1590 (b) 0/1 Knapsack (total value) J(P B ) J(P ′ B ) 1248163264128 B (SNR) 0.5 0.6 0.7 0.8 0.9 1.0 J (c) Ranking (accuracy) J(P B ) J(P ′ B ) 7B14B32B72BQwen3 B (model size) 0.10 0.15 0.20 0.25 (d) AIME Math (accuracy) J(P B ) J(P ′ B ) Fig. 4|Cross-domain validation. Performance퐽versus computational budgetBfor the base strategy set (blue circles) and the LLM-derived strategy set (red squares) across four domains: Tetris, Knapsack, Ranking and AIME mathematics. In the AIME domain, the derived strategy set averages over all five selector models and over 푘 ∈ 15, 17, 19, 21. generator channel alone (Fig. 5b). This occurs when휕퐽/휕B sel and푑B sel /푑B ref have opposite signs, so that their product contributes a negative term to훼 total . (i) Positive coupling: co-scaling increases the marginal return, so that훼 total can ex- ceed 1 (Fig. 5c). This occurs when a stronger selector genuinely complements a stronger generator, as demonstrated empirically in the nested AIME configuration (equation (4) and Fig. 6). The sign of the inter-layer coupling can be estimated em- pirically from how the utility function changes with different budget combinations: positive coupling indicates that increas- ing the generator’s capability amplifies the marginal return of the selector, and vice versa. When the coupling is positive, co-scaling is beneficial and a nested architecture is preferred; when it is near zero or negative, independent scaling of in- dividual components may be more efficient. This provides a concrete, measurable design criterion: before committing to a nested agent architecture, evaluate훼 total from a small grid of budget combinations and check whether co-scaling improves the marginal return. Figure 6 illustrates this in the AIME domain: we compare a “nested” configuration, in which the generator and selector are the same model and thus co-scale, against “fixed” configura- tions, in which the selector is held constant while the generator varies. The nested curve intersects each fixed-selector curve at the model size of the respective fixed selector, since the two configurations coincide at that point. Crucially, the nested curve can exceed any individual fixed-selector curve in the large- generator regime, demonstrating that co-scaling architectural components opens a response channel that is not available to the fixed-layer configuration. The fixed-architecture hypothesis applies to each individual fixed-selector curve, but does not constrain the nested curve, which can exceed the envelope of the fixed-selector family and thereby explore a fundamentally different region of the architectural parameter space. Discussion A theory of LLM information susceptibility addresses a question that is increasingly pressing as LLM-based agents are deployed in high-compute settings: does inserting a fixed LLM layer into an optimization pipeline improve how efficiently additional computation is converted into performance? Our results give a negative answer for fixed architectures and a conditionally positive answer for nested, co-scaling ones. This finding has a natural interpretation in terms of the susceptibility framework. The utility function퐽is not defined independently of architecture: the structure of the agent deter- mines which budget variables are available, how they couple to one another and which response channels contribute to per- formance 26–30,43 . The generalized susceptibility (equation (3)) makes this dependence explicit: the contraction of the sus- ceptibility vector휕퐽/휕B 푖 with the scaling protocol푑B 푖 /푑B ref determines whether co-scaling helps or hurts (Fig. 5). If the agent structure is held fixed and only the budget along one response channel is increased, then LLM intervention can improve constants or finite-budget behaviour, but it does not increase the large-budget susceptibility. By contrast, nesting changes the relationship between퐽and its budget variables by allowing the capability of one component to scale with the complexity induced by another, a regime characterized by 5 Preprint B ref Gen Sel B gen B sel fixed J a 246810 B ref 0 10 20 30 40 50 J B ref Gen Sel B gen B sel − J b 246810 B ref 0 10 20 30 40 50 J B ref Gen Sel B gen B sel + J c 246810 B ref 0 10 20 30 40 50 J Fig. 5|Inter-layer coupling regimes. Each panel shows an architecture diagram (top) and illustration of퐽versusB ref (bottom). Faded blue lines represent three fixed-selector configurations; solid coloured lines show the nested (co-scaled) configuration. Dots mark intersection points where configurations coincide. a, Decoupled: only the generator scales withB ref ; the selector remains fixed. The nested line coincides with one of the fixed lines. b, Negative coupling: both components scale, but co-scaling reduces marginal return (훼 total < 1). The nested line falls below the fixed line. c, Positive coupling: co-scaling amplifies marginal return (훼 total can exceed 1). The nested line exceeds all fixed lines, opening a response channel unavailable to fixed architectures. 7B14B32B72BQwen3 Generator model size 0.075 0.100 0.125 0.150 0.175 0.200 0.225 0.250 〈 J 〉 k (accuracy) MV (each model) Agent (nested derived) Agent (sel=7B) Agent (sel=14B) Agent (sel=32B) Agent (sel=72B) Agent (sel=Qwen3) Fig. 6|Nested versus fixed architectures in the AIME domain. Accuracy퐽(averaged over푘 ∈ 15, 17, 19, 21) versus model size for the nested derived strategy set (generator and selector co-scaled) and fixed derived strategy sets (fixed selector, varying generator). The curves intersect at the model size of the respective fixed selector, showing that co-scaling architectural components can exceed the susceptibility bound. 6 Preprint positive inter-layer coupling. This perspective is consistent with the potential-landscape analysis of Song et al. 31 , which shows that within a fixed LLM-driven agent, optimization is constrained by an intrinsic landscape. Our results complement that picture at the system level: repeated optimization by a fixed layer is fundamentally limited both by internal model structure and by external response structure. These findings carry practical implications for agent design. First, when the target application operates in a large-budget regime, investing computation in the base strategy-generation process, like stronger search, better proposal generation or more reliable verification, may be more effective than relying on a fixed LLM wrapper to amplify gains 28–30,40,43 . Second, static LLM selection modules are most useful in low- or intermediate- budget regimes, where world knowledge and heuristic com- pression still provide noticeable improvements 9,10,17 . Third, if the goal is to build systems capable of open-ended im- provement, designers should allocate budget so that generator, selector, verifier, memory and tool-use components can co- scale 4,7,13,15,30,44,45 . More broadly, the susceptibility-based viewpoint developed here suggests a quantitative language for comparing agent architectures: rather than asking only whether an LLM helps, one can ask which architectural variables ap- pear in퐽, how those variables couple through the scaling protocol and which susceptibilities dominate in the regime of interest 30,32 . Beyond these design implications, the results bear directly on a fundamental question in AI: whether LLMs can achieve open-ended self-evolution 11,46,47 (see Extended Data Fig. 3 for a detailed phenomenological model). Consider a scenario in which an LLM attempts to improve its own strategies by using itself as the optimization layer. If the LLM mediation cannot increase asymptotic susceptibility (훼 ≤ 1), then self- guided improvement is expected to saturate once the model’s capability exceeds a threshold, because the fixed LLM layer cannot increase the rate at which performance responds to additional computation; the feedback loop of self-improvement is inherently bounded. A related limitation has been observed in unsupervised reinforcement learning, where initial training gains are followed by collapse once the self-generated reward signal diverges from the true objective at sufficient scale 48 . Conversely, if a nested architecture enables훼 total > 1, the LLM can alter its own strategy distribution in a way that increases marginal return: as the LLM’s capability grows, its ability to guide its own improvement strengthens in turn, potentially creating a positive feedback loop. Figure 6 provides empirical evidence for this logic: the nested configuration’s accuracy is approaching and poised to exceed the majority-vote baseline, indicating that the LLM’s ability to reshape its own distribution through nested co-scaling is nearing a critical crossover. In the current experiments, the nested curve for Qwen3-Max is close to but has not yet crossed this threshold. Contingent on the susceptibility hypothesis holding, this suggests that nested, co-scaling architectures are not merely sufficient for exceeding the susceptibility bound, but constitute a necessary structural condition for open-ended self-evolution: if fixed architectures cannot achieve훼 > 1, only architectures whose components co-scale can sustain unbounded improvement. Several directions for future work emerge naturally from this study. First, the theory is stated as an empirical hypothesis supported by experiments; developing a formal proof would place the bound on firmer theoretical ground. Second, the four domains tested, though structurally diverse, do not cover settings with very long horizons, multi-agent interaction or continuous action spaces, where the relationship between bud- get and performance may differ qualitatively; exploring these settings would clarify the boundary conditions of the frame- work. Third, the framework suggests a practical engineering methodology: by measuring the susceptibility of individual architectural layers and combining these measurements with the known compositional structure of the architecture, one could in principle, if the inter-layer coupling structure is known, reconstruct the full utility function퐽across the entire budget space; this would reduce system-level performance prediction from costly end-to-end evaluation to composable single-layer characterizations, offering more efficient guidance for engi- neering design. Finally, the nested-architecture experiments demonstrate that co-scaling can exceed the susceptibility bound, but do not yet characterize the rate at which it does so; deriving a quantitative scaling law for nested susceptibility is perhaps the most important open question, as it would provide concrete guidance for allocating compute across co-scaling components. Beyond these future directions, the framework offers a con- crete criterion for evaluating when LLM intervention is worth the cost: compute the sensitivity훼in the target budget regime. If훼 < 1, the LLM layer is consuming resources without pro- portionally improving the scaling trajectory, and the design should either move to a nested architecture or redirect com- putation to the base strategy. This criterion is measurable, domain-agnostic and complementary to standard metrics such as absolute accuracy or win rate that do not distinguish between constant offsets and scaling improvements. More generally, the susceptibility-based approach demonstrates that tools from statistical physics can provide a predictive framework for the study of AI systems, one that constrains design choices be- yond post-hoc rationalization of empirical results. Among its concrete, hypothesis-dependent predictions is that open-ended self-evolution may require nested co-scaling, a claim that is already approaching testability with current models. Methods Models and infrastructure All experiments use five Qwen-series models: Qwen-2.5-7B- Instruct (7B), Qwen-2.5-14B-Instruct (14B), Qwen-2.5-32B- Instruct (32B), Qwen-2.5-72B-Instruct (72B) and Qwen3-Max (∼200B) 49,50 . Decoding parameters are specified per domain below. All domains use the same models and API, ensuring that the observed effects are not artefacts of a particular model. Tetris Environment. A10× 20Tetris board with 6 pre-filled garbage lines. Pieces are drawn from 18 fixed orientations (I, O, T, S, Z, L, J variants); no rotation is performed during play. Each game lasts at most 50 steps. The utility function퐽is the number of lines cleared. Base strategyP B . Beam search 51 with depth-first back- tracking and a lookahead depth of 3. At each step, the algorithm expands all legal placements to depth 3, evaluates terminal states using a heuristic combining aggregate height, hole count, 7 Preprint bumpiness and lines cleared, and retains the top-Bcandidates (beam width). The top 3 placements are returned as candidates. Beam widths tested: B ∈ 1, 2, 4, 8, 16, 32. Derived strategyP ′ B . Each LLM receives the current board state (ASCII grid), the current piece and the top 3 DFS candidates with their heuristic scores. The LLM selects one placement. Decoding: temperature= 0.1, max tokens= 500, timeout = 15 s, max retries = 2. Prompt variants. Four prompt designs were tested: minimal (JSON-only output format), standard (full board analysis), chain-of-thought (explicit 5-step reasoning) and expert (domain- specific Tetris strategy). The main text reports results using the standard prompt with the aggressive reward function as the representative case showing the strongest susceptibility gap; robustness across all prompt and reward configurations is reported in Fig. 2. Reward functions. Three heuristic evaluation functions were tested: aggressive (prioritizing line clearing with weight 5.0), conservative (prioritizing hole avoidance with weight 3.0) and default (balanced weights). The qualitative pattern of the susceptibility bound is invariant across all three. Statistics. 40 independent random seeds per (model,B) pair. Error bars in Fig. 1 are standard errors of the mean over seeds. AIME mathematics Problem set. 60 problems from AIME 2024 (30 problems) and AIME 2025 (30 problems). Each answer is an integer in [0, 999]. Base strategyP B . For each problem, a generator LLM of sizeB gen produces푘independent solution attempts at tem- perature 0.7 (max tokens= 1,500). The base strategy applies majority vote 17,52 : answers are grouped by approximate equal- ity (|푎− 푏| < 0.5) and the most common group is selected, with random tie-breaking. Here푘serves as a control parameter that tunes the statistical power of the majority vote, while the generator model sizeB gen determines the quality of individual attempts. Values tested:푘 ∈ 1, 3, 5, 9, 15, 17, 19, 21; all 21 samples are generated once and subsampled for each 푘 . Derived strategyP ′ B . A selector LLM of sizeB sel reads the 푘candidate answers (deduplicated, without frequency counts) and selects one. The “fixed derived” configuration uses each of the five models as a fixed selector while varying the generator model; the reported ̄훼(푘)averages over all five generator sizes and all five selectors. Agent selection uses temperature= 0.1. The generation temperature of 0.7 ensures diversity across the 푘independent attempts, while the low selection temperature yields deterministic selector behaviour. The prompt does not strictly adhere to the official AIME format; this is intentional, to minimize wording differences between the majority-vote and LLM-selector conditions. Estimation of훼. For each푘and each fixed selector, five data points(퐽 (푖) MV , 퐽 (푖) agent ) are obtained, one per generator model sizeB gen . A linear model퐽 agent = 훼· 퐽 MV + 훽is fitted using ordinary least squares. The slope훼and its standard error are reported. The average ̄훼(푘)shown in Fig. 3 is obtained by first averaging the agent’s accuracy over all five selectors for each generator size, then fitting a single linear model across the five generator sizes. Statistics. The accuracy for each (model,푘) pair is the mean correctness over 60 problems. Error bars in Figs. 4 and 6 are binomial standard errors √︁ 푝(1− 푝)/푛, where푝is the observed accuracy and푛is the number of independent trials. For the majority-vote baseline,푛 = 60× |퐾|(60 problems times the number of푘values averaged over); for the LLM agent,푛 = 60× 5×|퐾|(additionally averaged over five selector configurations). The binomial standard error is used because each problem outcome is a Bernoulli trial (correct or incorrect), and the standard error quantifies the uncertainty due to finite sample size. 0/1 Knapsack 50 items with weights푤 푖 ∈ [1, 50]and values푣 푖 ∈ [1, 100], capacity= 0.3 Í 푤 푖 . The base strategy is beam search over the item-selection tree, with items sorted by value density푣 푖 /푤 푖 41 . The LLM receives the top 3 packings and selects one.퐽= total value,B= beam width∈ 1, 2, 4, 8, 16, 32, 64. Statistics: 50 problem instances; error bars are standard errors of the mean over instances. World-knowledge Ranking Four real-world ranking datasets (GDP of 15 countries, population of 15 countries, diameters of 8 planets, weights of 12 animals). For each item, a noisy score estimate is generated:ˆ푠 푖 = 푠 푖 +N(0,휎/ √ B), where휎is a dataset-specific baseline noise scale chosen so that the algorithmic success rate is approximately 50% atB = 1. The top 5 candidates by noisy score are presented to the LLM, which selects the item it believes ranks first using world knowledge.퐽= fraction correctly identifying the true rank-1 item,B = signal-to-noise ratio∈ 1, 2, 4, 8, 16, 32, 64, 128. Statistics: 100 noise seeds × 4 datasets× 8 SNR levels. Error bars are standard errors of the mean over noise seeds and datasets. Data availability Allexperimentaldatageneratedinthisstudy arepubliclyavailableonHuggingFaceathttps: //huggingface.co/datasets/Nondegeneracy/LLM- Susceptibility-theory under the C BY 4.0 license. Code availability The code used to run the experiments and produce all figures is available on GitHub athttps://github.com/ SonnyNondegeneracy/LLM-Susceptibility-theoryun- der the MIT license. Acknowledgements This work is supported by National Natural Science Founda- tion of China under contract No. 12425505. Competing interests The author declares no competing interests. References [1] Wang, L. et al. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 186345 (2024). URLhttps://link.springer.com/ article/10.1007/s11704-024-40231-1. 8 Preprint [2]Xi,Z. et al.The rise and potential of large language model based agents:A sur- vey.Science China Information Sciences 68, 121101 (2025). URLhttps://link.springer.com/ article/10.1007/s11432-024-4222-0. [3] Yao, S. et al. React: Synergizing reasoning and acting in language models. In Proceedings of the Eleventh International Conference on Learning Representations (2023). URLhttps://openreview.net/forum?id= WE_vluYUL-X. [4] Schick, T. et al. Toolformer: Language models can teach themselves to use tools. In Advances in Neural Informa- tion Processing Systems, vol. 36 (2023). URLhttps: //proceedings.neurips.c/paper_files/paper/ 2023/hash/d842425e4bf79ba039352da0f658a906- Abstract-Conference.html. [5]Park, J. S. et al. Generative agents: Interactive simu- lacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23 (Association for Comput- ing Machinery, New York, NY, USA, 2023). URL https://doi.org/10.1145/3586183.3606763. [6]Bran, A. M. et al. Chemcrow: Augmenting large-language models with chemistry tools (2023). URLhttps:// arxiv.org/abs/2304.05376. 2304.05376. [7] Wang, G. et al. Voyager: An open-ended embodied agent with large language models (2023). URLhttps: //arxiv.org/abs/2305.16291. 2305.16291. [8]Durante, Z. et al. Agent ai: Surveying the horizons of multimodal interaction (2024). URLhttps://arxiv. org/abs/2401.03568. 2401.03568. [9]Madaan, A. et al. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, vol. 36 (2023).URLhttps: //proceedings.neurips.c/paper_files/paper/ 2023/hash/91edff07232fb1b55a505a9e9f6c0f3- Abstract-Conference.html. [10] Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K. & Yao, S. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Informa- tion Processing Systems, vol. 36 (2023). URLhttps: //proceedings.neurips.c/paper_files/paper/ 2023/hash/1b44b878b782e6954cd888628510e90- Abstract-Conference.html. [11] Zelikman, E., Wu, Y., Mu, J. & Goodman, N. D. STaR: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems, vol. 35 (2022). URL https://arxiv.org/abs/2203.14465. [12] Romera-Paredes, B. et al. Mathematical discoveries from program search with large language models. Nature 625, 468–475 (2024). URLhttps://doi.org/10.1038/ s41586-023-06924-6. [13] Cui, C. et al. Alphaevolve: A learning framework to discover novel alphas in quantitative investment. In Pro- ceedings of the 2021 International Conference on Manage- ment of Data, SIGMOD ’21, 2208–2216 (Association for Computing Machinery, New York, NY, USA, 2021). URL https://doi.org/10.1145/3448016.3457324. [14]Liu, F. et al. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In Proceedings of the 41st International Conference on Machine Learning, vol. 235 of PMLR, 32201–32223 (2024).URLhttps://proceedings.mlr.press/ v235/liu24bs.html. [15]Song, Z.-Y. et al. Iterated agent for symbolic regression (2025). URLhttps://arxiv.org/abs/2510.08317. 2510.08317. [16]Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Infor- mation Processing Systems, vol. 35 (2022). URLhttps: //proceedings.neurips.c/paper_files/paper/ 2022/hash/9d5609613524ecf4f15af0f7b31abca4- Abstract-Conference.html. [17]Wang, X. et al. Self-consistency improves chain of thought reasoning in language models. In Proceedings of the Eleventh International Conference on Learning Representations (2023). URLhttps://openreview. net/forum?id=1PL1NIMMrw. [18] Ouyang,L. et al.Training language mod- elstofollowinstructionswithhumanfeed- back.In Advances in Neural Information Pro- cessing Systems, vol. 35 (2022).URLhttps: //proceedings.neurips.c/paper_files/paper/ 2022/hash/b1efde53be364a73914f58805a001731- Abstract-Conference.html. [19]Yao, S. et al. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Infor- mation Processing Systems, vol. 36 (2023). URLhttps: //proceedings.neurips.c/paper_files/paper/ 2023/hash/271db9922b8d1f4d7aaef84ed5ac703- Abstract-Conference.html. [20] Wang, L. et al. Plan-and-solve prompting: Improving zero- shot chain-of-thought reasoning by large language models (2023). URLhttps://arxiv.org/abs/2305.04091. 2305.04091. [21]Chen, W., Ma, X., Wang, X. & Cohen, W. W. Pro- gram of thoughts prompting: Disentangling compu- tation from reasoning for numerical reasoning tasks (2023). URLhttps://arxiv.org/abs/2211.12588. 2211.12588. [22]Gao, L. et al. PAL: Program-aided language mod- els.In Proceedings of the 40th International Con- ference on Machine Learning, vol. 202 of Proceed- ings of Machine Learning Research, 10764–10799 (PMLR, 2023).URLhttps://proceedings.mlr. press/v202/gao23f.html. 9 Preprint [23]Besta, M. et al. Graph of thoughts: Solving elaborate problems with large language models. Proceedings of the AAAI Conference on Artificial Intelligence 38, 17682– 17690 (2024). URLhttps://ojs.aaai.org/index. php/AAAI/article/view/29720. [24]OpenAI. OpenAI o1 system card (2024). URLhttps: //arxiv.org/abs/2412.16720. 2412.16720. [25]DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning (2025). URLhttps://arxiv.org/abs/2501.12948.2501. 12948. [26]Kubo, R. Statistical-mechanical theory of irreversible processes. I. General theory and simple applications to magnetic and conduction problems. Journal of the Physical Society of Japan 12, 570–586 (1957). URL https://doi.org/10.1143/JPSJ.12.570. [27]De Nittis, G. & Lein, M. Linear response theory: an analytic-algebraic approach (Springer, 2017). [28] Kaplan, J. et al. Scaling laws for neural language models (2020). URLhttps://arxiv.org/abs/2001.08361. 2001.08361. [29]Hoffmann, J. et al. Training compute-optimal large language models.In Advances in Neural Informa- tion Processing Systems, vol. 35 (2022). URLhttps: //arxiv.org/abs/2203.15556. [30] Kim, Y. et al. Towards a science of scaling agent systems (2025). URLhttps://arxiv.org/abs/2512.08296. 2512.08296. [31]Song, Z.-Y., Cao, Q.-H., xing Luo, M. & Zhu, H. X. Detailed balance in large language model-driven agents (2025). URLhttps://arxiv.org/abs/2512.10047. 2512.10047. [32]Wolpert, D. H. & Macready, W. G. No free lunch theorems for optimization. IEEE transactions on evolutionary com- putation 1, 67–82 (2002). URLhttps://ieeexplore. ieee.org/abstract/document/585893. [33] Shannon, C. E. A mathematical theory of communi- cation. The Bell System Technical Journal 27, 379– 423 (1948). URLhttps://ieeexplore.ieee.org/ document/6773024. [34] Cover, T. M. & Thomas, J. A. Elements of Informa- tion Theory (Wiley-Interscience, Hoboken, NJ, 2006), 2 edn. URLhttps://onlinelibrary.wiley.com/ doi/book/10.1002/047174882X. [35] Hendrycks, D. et al. Measuring mathematical problem solving with the math dataset (2021). URLhttps:// arxiv.org/abs/2103.03874. 2103.03874. [36]Hugging Face H4.Aime 2024 dataset.https: //huggingface.co/datasets/HuggingFaceH4/ aime_2024 (2024). Accessed: 2025-05-16. [37] Li, Y. et al. Competition-level code generation with AlphaCode. Science 378, 1092–1097 (2022). URL https://arxiv.org/abs/2203.07814. [38]Cobbe, K. et al. Training verifiers to solve math word prob- lems (2021). URLhttps://arxiv.org/abs/2110. 14168. 2110.14168. [39] Brown, B. et al.Large language monkeys: Scal- ing inference compute with repeated sampling (2024). URLhttps://arxiv.org/abs/2407.21787.2407. 21787. [40] Hogg, T., Huberman, B. A. & Williams, C. P. Phase transi- tions and the search problem. Artificial Intelligence 81, 1– 15 (1996). URLhttps://w.sciencedirect.com/ science/article/pii/0004370295000445 .Fron- tiers in Problem Solving: Phase Transitions and Com- plexity. [41]Kellerer, H., Pferschy, U. & Pisinger, D.Knap- sack Problems (Springer, Berlin, 2004).URL https://link.springer.com/book/10.1007/978- 3-540-24777-7. [42] Le Chatelier, H. L. Sur un ́ enonc ́ e g ́ en ́ eral des lois des ́ equilibres chimiques. Comptes rendus de l’Acad ́ emie des sciences 99, 786–789 (1884). URLhttps://gallica. bnf.fr/ark:/12148/bpt6k3055h/f786.item. [43]Snell, C., Lee, J., Xu, K. & Kumar, A. Scaling llm test-time compute optimally can be more effective than scaling model parameters (2024). URLhttps://arxiv. org/abs/2408.03314. 2408.03314. [44]Hu, S., Lu, C. & Clune, J. Automated design of agentic systems (2025). URLhttps://arxiv.org/abs/2408. 08435. 2408.08435. [45]Hosseini, A. et al. V-star: Training verifiers for self- taught reasoners (2024). URLhttps://arxiv.org/ abs/2402.06457. 2402.06457. [46]Good, I. J. Speculations concerning the first ultraintelli- gent machine. In Advances in Computers, vol. 6, 31–88 (Academic Press, 1966). URLhttps://doi.org/10. 1016/S0065-2458(08)60418-0. [47]Singh, A. et al. Beyond human data: Scaling self-training for problem-solving with language models. In Advances in Neural Information Processing Systems, vol. 37 (2024). URL https://arxiv.org/abs/2312.06585. [48]He, B. et al. How far can unsupervised rlvr scale llm train- ing? (2026). URLhttps://arxiv.org/abs/2603. 08660. 2603.08660. [49]Team, Q. Qwen2.5 technical report (2025). URLhttps: //arxiv.org/abs/2412.15115. 2412.15115. [50]Yang, A. et al.Qwen3 technical report (2025). URLhttps://arxiv.org/abs/2505.09388.2505. 09388. 10 Preprint [51]Russell,S.&Norvig,P.ArtificialIn- telligence:AModernApproach(Pearson, Hoboken,NJ, 2021),4 edn.URLhttps: //w.pearson.com/en-us/subject-catalog/ p/artificial-intelligence-a-modern- approach/P200000003500/9780137505135. [52] de Condorcet, M. J. A. N. d. C.Essai sur l’application de l’analyse ` a la probabilit ́ e des d ́ ecisions rendues ` a la pluralit ́ e des voix (Imprimerie Royale, Paris, 1785). URLhttps://gallica.bnf.fr/ark: /12148/bpt6k417181. Reprinted by Chelsea, New York, 1972. 11 Preprint Extended Data 12 Preprint 12481632 B (beam width) 0.0 0.2 0.4 0.6 0.8 ∆( B ) (a) Tetris 1248163264 B (beam width) 0.010 0.005 0.000 0.005 0.010 0.015 (b) 0/1 Knapsack 1248163264128 B (SNR) 0.3 0.2 0.1 0.0 ∆( B ) (c) Ranking 7B14B32B72BQwen3 B(Model size) 0.0 0.1 0.2 0.3 0.4 (d) AIME Math (fixed derive) Extended Data Fig. 1|Averaged performance gap across domains. The normalized performance gapΔ(B) = 퐽(P B )− 퐽(P ′ B ) /퐽(P B ), averaged over all five LLMs, as a function of computational budgetBfor four domains. Blue shading indicates the regime where the base algorithm outperforms the LLM-derived strategy (Δ > 0); red shading indicates the opposite. In Tetris,Δgrows monotonically. In Knapsack,Δis negligible. In Ranking,Δtransitions from negative (LLM advantage at low SNR) to near zero. In AIME,Δ(averaged over푘 ∈ 15, 17, 19, 21) remains positive across model sizes. 13 Preprint 12481632 B (beam width) 0.00 0.25 0.50 0.75 1.00 ∆( B ) (a) Tetris 7B 14B 32B 72B Qwen3 1248163264 B (beam width) 0.01 0.00 0.01 0.02 (b) 0/1 Knapsack 7B 14B 32B 72B Qwen3 1248163264128 B (SNR) 0.4 0.2 0.0 ∆( B ) (c) Ranking 7B 14B 32B 72B Qwen3 7B14B32B72BQwen3 B(Model size) 0.2 0.0 0.2 0.4 (d) AIME Math k = 3 k = 5 k = 9 k = 15 k = 21 Extended Data Fig. 2|Per-model performance gap across domains. The normalized performance gapΔ(B)broken down by individual model size (7B through Qwen3-Max) for each domain. In Tetris, 72B models show largest gaps. In Knapsack, all models produce negligible gaps. In Ranking, all models converge from negative to near-zeroΔas SNR increases. In AIME, the gap varies with both generator model size and number of samples 푘 , with larger 푘 showing an increasing tendency with model size. 14 Preprint 2 3 4 5 6 7 8 9 10 Data quality b ∗ a p′> l′: unstable fixed point p(b) (pipeline) l(b) (model) 1 2 3 4 5 6 7 8 9 Data quality b ∗ b p′< l′: stable fixed point p(b) (pipeline) l(b) (model) 0.51.01.52.02.53.03.54.04.55.0 Model capability b 2 1 0 1 2 db/dr ∝ p ( b ) − l ( b ) Collapse Self-evolution 0.51.01.52.02.53.03.54.04.55.0 Model capability b 2 1 0 1 2 db/dr ∝ p ( b ) − l ( b ) Bounded improvement Bounded degradation Extended Data Fig. 3|Illustration of the phenomenological theory of self-evolution dynamics. Top row: data quality functions푝(푏) (pipeline, blue) and푙(푏)(model output, red) versus model capability푏. Bottom row: phase portrait푑푏/푑푟 ∝ 푝(푏)− 푙(푏), with arrows indicating the flow direction. Open circle: unstable fixed point; filled circle: stable fixed point. a,푝 ′ (푏) > 푙 ′ (푏): the fixed point is a repeller, giving rise to a collapse phase (푏 < 푏 ∗ ) and a self-evolution phase (푏 > 푏 ∗ ). b, 푝 ′ (푏) < 푙 ′ (푏): the fixed point is an attractor; improvement and degradation are both bounded. See Supplementary Note 1 for the full derivation. 15 Preprint Supplementary Information Supplementary Note 1: Phenomenological theory of self- evolution The self-evolution argument in the main text can be for- malized with a minimal dynamical model. Let푏denote the capability of a model,푝(푏)the quality of training data pro- duced by a data-generation pipeline constructed using a model of capability푏, and푙(푏)the quality of output generated directly by a model of capability푏. When the model is trained on its own pipeline-generated data, the capability evolves according to 푑푏 푑푟 = 휂[푝(푏)− 푙(푏)],(S1) where푟is the cumulative training resource and휂 > 0is a learning-rate constant. The driving term푝(푏)−푙(푏)represents the gap between what the pipeline can produce and what the model currently outputs: when the pipeline generates higher- quality data than the model’s own output (푝 > 푙), training improves the model; when the pipeline produces lower-quality data (푝 < 푙), training degrades it. A fixed point푏 ∗ satisfies푝(푏 ∗ ) = 푙(푏 ∗ ): the pipeline output quality matches the model’s own output, so training produces no net change in capability. The stability of this fixed point is determined by the sign of 푝 ′ (푏 ∗ )− 푙 ′ (푏 ∗ ). Case 1:푝 ′ (푏) > 푙 ′ (푏)(repeller). Since푝−푙is an increasing function of푏, the fixed point푏 ∗ is unstable (Extended Data Fig. 3a). For푏 < 푏 ∗ ,푝(푏) < 푙(푏)and푑푏/푑푟 < 0: the pipeline produces data of lower quality than the model’s own output, so training degrades capability, which further widens the gap (collapse phase with positive feedback). For푏 > 푏 ∗ ,푝(푏) > 푙(푏)and푑푏/푑푟 > 0: the pipeline data quality exceeds the model’s output, so training continually improves the model and the improvement accelerates as the gap widens (self-evolution phase). The system thus exhibits a phase transition: whether the initial capability푏 0 lies above or below the critical point 푏 ∗ determines whether the model undergoes unbounded self- evolution or irreversible collapse. Case 2:푝 ′ (푏) < 푙 ′ (푏)(attractor). Since푝−푙is a decreasing function of푏, the fixed point푏 ∗ is stable (Extended Data Fig. 3b). For푏 < 푏 ∗ ,푝(푏) > 푙(푏)and the model improves, but the improvement decelerates as푏approaches푏 ∗ (bounded improvement). For푏 > 푏 ∗ ,푝(푏) < 푙(푏)and the model degrades, but the degradation likewise decelerates (bounded degradation). In both cases the system converges to푏 ∗ . There is no phase transition; training always produces a finite, bounded change in capability. Marginal case:푝 ′ (푏) = 푙 ′ (푏). When the two slopes are equal,푝(푏) − 푙(푏)is a constant independent of푏. If this constant is positive, the system is in a global self-evolution phase; if negative, it collapses globally. No fixed point exists and no phase transition occurs. In the linear model this case is degenerate, as it requires two parallel lines whose fate is determined entirely by the sign of the global offset. Connection to the susceptibility framework. In the frame- work developed in the main text, the pipeline quality푝(푏) corresponds to the effective performance of a nested architec- ture in which a model of capability푏serves as both generator and selector, while푙(푏)corresponds to the performance of the base strategy (e.g., majority vote). The condition푝 ′ (푏) > 푙 ′ (푏) is then equivalent to the nested total sensitivity훼 total > 1(posi- tive coupling regime, equation (4) in the main text), whereas 푝 ′ (푏) < 푙 ′ (푏)corresponds to훼 total < 1(negative coupling or decoupled regime). Within the hypothesis framework of the main text, the requirement of nested co-scaling to realize 훼 total > 1can thus be restated dynamically: self-evolution is possible only when the pipeline’s data quality responds to model capability faster than the model’s own output quality does. 16