Paper deep dive
Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs
Neil F. Johnson, Frank Yingjie Huo, Bella Xinrui Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Increasing the temperature of an ordinary many-state system increases access to a wider range of states and hence increases its entropy. We find the opposite in ChatGPT-like AIs, even though raising the decoder temperature likewise increases access to a wider range of states (next-token choices). Across 12,000 continuations from 11 AIs, autoregressive feedback drives the long-time output population through an entropy maximum and into population inversion. The transition features frozen states, cycles, intermittency and noise-induced ordering. We present evidence of a hidden coordinate that acts as the state variable of an effective nonlinear map. Its trajectory average strongly predicts output repetition in separate test trajectories. ChatGPT-like AIs therefore behave not as `stochastic parrots', but as a new class of controllable nonlinear physical systems whose internal dynamics can be measured and perturbed.
Tags
Links
- Source: https://arxiv.org/abs/2608.00939v1
- Canonical: https://arxiv.org/abs/2608.00939v1
Trouble viewing inline? Open PDF directly â
Full Text
39,137 characters extracted from source content.
Expand or collapse full text
Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs Neil F. Johnson*, Frank Yingjie Huo, Bella Li Department of Physics, The George Washington University Washington, DC 20052, USA *Correspondence: neiljohnson@gwu.edu Abstract Increasing the temperature of an ordinary many-state system increases access to a wider range of states and hence increases its entropy. We find the opposite in ChatGPT-like AIs, even though raising the decoder temperature TdT_d likewise increases access to a wider range of states (next-token choices). Across 12,000 continuations from 11 AIs, autoregressive feedback drives the long-time output population through an entropy maximum and into population inversion. The transition features frozen states, cycles, intermittency and noise-induced ordering. We present evidence of a hidden coordinate xnx_n that acts as the state variable of an effective nonlinear map. Its trajectory average strongly predicts output repetition in separate test trajectories. ChatGPT-like AIs therefore behave not as âstochastic parrotsâ, but as a new class of controllable nonlinear physical systems whose internal dynamics can be measured and perturbed. ChatGPT-like AIs are doing increasingly consequential work in medicine and health, finance, business, law, science and defence [42, 28, 10, 26, 47, 37]. Yet all can produce fluent but undesirable output that can negatively affect diagnoses, livelihoods, legal rights, scientific conclusions and security decisions. Safe use requires more than performance benchmarks: it demands a reproducible account of how these systems generate and change their output [8]. Existing studies chart output failures and architectural pathologies through benchmarks, theoretical analysis and model-specific diagnostics. These include text degeneration and repetition, hallucination, loss of fidelity to earlier input, representation drift, and rank or depth collapse [20, 14, 16, 11, 45, 13, 49, 5, 2]. Mechanistic-interpretability methods resolve individual internal features and circuits through controlled probes, sparse-autoencoder feature dictionaries, circuit tracing and attention-head visualization [34, 35, 36, 12, 3, 29]. The Jacobian Lens adds a layer-resolved readout, using averaged downstream Jacobians to expose interpretable hidden directions that can affect present and future output [17]. Yet exactly how these components combine to produce the observed output remains a mystery. In the absence of a system-level account, one influential metaphor treats ChatGPT-like AIs as âstochastic parrotsâ: systems that recombine statistical patterns in training text without grounded understanding [6]. At each next-token generation step n, a ChatGPT-like AI uses the prompt and all tokens generated before that step to assign a numerical score ziz_i, called a logit, to every possible next token i, and then selects one according to the resulting probabilities. The negative logit acts as an effective energy, Ei=âziE_i=-z_i, so the decoder temperature TdT_d enters exactly as temperature does in a Boltzmann factor: token i receives a weight proportional to expâĄ(âEi/Td) (-E_i/T_d), normalized over all tokens [24, 18, 22, 25]. For fixed logits, increasing TdT_d spreads probability across a wider range of possible next tokens, just as raising physical temperature spreads occupation across energy levels. With a large vocabulary providing many such effective levels, one would naturally expect increasing TdT_d to produce a monotonic increase in the entropy of the long-time output. As we show, this is not what happens. Across 11 AIs, we show that decoder temperature TdT_d acts as a controlled physical parameter that drives the long-time output population through an entropy maximum and into inversion, with the transition structured by frozen states, cycles, intermittency and noise-induced ordering. Focusing on the large open-source model Llama-3.1-70B, we identify a hidden coordinate xnx_n that predicts repetition and can be causally steered. Together, these results show that the popular âstochastic parrotâ metaphor is misleading: apparent randomness is instead organized by measurable transitions, memory and causal response. Our findings bring the massive societal challenge of understanding ChatGPT-like AIs into the established physics of controllable stochastic nonlinear systems whose internal dynamics can be measured and perturbed. Nonlinear dynamics explains how mechanistic update rules generate fixed points, cycles, intermittency, bifurcations and chaos [44]. It has expanded from low-dimensional maps to high-dimensional network control [31, 32] and brain dynamics [43]. ChatGPT-like AIs â autoregressive decoders that generate each next token from the prompt and all preceding tokens â constitute a new class of iterative nonlinear systems with recomputed interactions and perturbable internal states. Prior work reports nonlinear autoregressive formulations [27], temperature-driven transitions and criticality [4, 33, 30, 46, 41], attractor cycles [48], chaos diagnostics [23] and causal repetition units [19]. By contrast, our work explicitly unifies three previously separate levels: external control by the decoder temperature, collective output inversion and internal nonlinear dynamics along a measurable, perturbable hidden coordinate. Temperature drives population inversion Figure 1 illustrates how increasing decoder temperature TdT_d changes GPT-2 output for the prompt âAI is âŠâ. At low TdT_d, a trajectory remains trapped in one repetitive basin over a range of temperatures. Raising TdT_d opens other recurrent patterns and switching routes. At still higher TdT_d, recurrence and novelty coexist in prose with the balance of order and disorder characteristic of human language. This suggests a transfer of the output-text population between a recurrent sector and a complex/noisy sector, where âpopulationâ means the collection of complete texts generated over many runs. Figure 1: Illustrative GPT-2 output for the prompt âAI is âŠâ. As TdT_d rises, generation moves from a single repetitive attractor-like pattern through additional periodicities and switching, and eventually to regularity and novelty coexisting in human-like output. For convenience, we coarse-grain sentences to symbols. Figure 2 quantifies the resulting population inversion between coarse recurrent and complex/noisy macrostates. The key nonlinear ingredient is autoregressive feedback: each sampled token is appended to the input for the next step, giving the AI an evolving memory of its own output. If CnC_n is the prompt together with all tokens generated before step n, generation is the stochastic map CnâŠCn+1C_n C_n+1: sample the next token from softmaxâĄ[zâ(Cn)/Td]softmax[z(C_n)/T_d], where zâ(Cn)z(C_n) is the vector of logits, append it to CnC_n, and repeat. Changing TdT_d therefore changes both the current choice and the sequence of inputs subsequently explored, allowing the long-time output population to reorganize. Figure 2: Maximum entropy marks ChatGPT-like AIsâ output inversion. a, Normalized mixing entropy for the two output sectors. Its maximum is near equal occupation. The dashed orange curve is the occupancy-defined dimensionless parameter ÎČeffâÎ _eff , which changes sign at the mean crossing Tdâ=0.517T_d^*=0.517; the grey band spans the 11 individual-AI crossings. b, Occupancy of the complex/noisy sector for the 11 AIs (GPT-2, blue; Pythia, green; Llama, orange). Black is the unweighted mean across AIs. c, Reversal of the corresponding signed order parameter, completing the population-inversion signature. Blue and purple shading in panels a and c spans the interquartile range across the 11 AIs. To quantify this at the output level, we generated 12,000 continuations from four GPT-2 AIs, six Pythia AIs and Llama-3.1-70B using five fixed prompts [39, 7, 15]. Each continuation was classified by its sentence-level recurrence as frozen, sparse, periodic, intermittent, complex or noisy. Classifying completed sentences in this way averages over fast token variation to expose slower recurrent output structures. Supplementary Note 2 gives the full construction. AI-resolved crossings and family-level occupancy curves are reported in Supplementary Table S2 and Supplementary Figure S1. The complete 605-cell common-grid atlas is Supplementary Figure S2, while Supplementary Table S1 provides the complete evidence map. For the population analysis, frozen, sparse, periodic and intermittent output effectively form a recurrent sector, while complex and noisy output effectively form a complex/noisy sector. Let q be the fraction in the complex/noisy sector. The corresponding coarse two-sector variables are m=1â2âq,S2=âqâlnâĄqâ(1âq)âlnâĄ(1âq),ÎČeffâÎ=lnâĄ1âq.m=1-2q, S_2=-q q-(1-q) (1-q), _eff = \! 1-qq. (1) Here m is the signed order parameter and S2S_2 is the entropy of the coarse two-sector population split. Î>0 >0 is the effective level gap between the two sectors, with the complex/noisy sector higher, while ÎČeff _eff is the effective inverse temperature inferred from their relative occupancies. Only the dimensionless product ÎČeffâÎ _eff is determined by q. Thus ÎČeff _eff is an occupancy-defined effective quantity, not the inverse decoder temperature 1/Td1/T_d. Every AI crosses equal occupation. The individual crossings lie between Td=0.425T_d=0.425 and 0.562, the unweighted mean across AIs crosses at 0.517, and the mean complex/noisy occupation reaches 0.932 at Td=1T_d=1 (Fig. 2). The entropy therefore rises to its maximum and then falls as the complex/noisy sector becomes overpopulated. The robust transfer of the output population through equal occupation is the central empirical result; the entropy maximum and change of sign of ÎČeffâÎ _eff are its statistical-mechanical signatures. Beyond equal occupation, raising the positive decoder temperature reduces the coarse mixing entropy even though, at any specified next-token generation step n, it increases the entropy of the distribution for token n+1n+1 when CnC_n (the prompt and all tokens generated before step n) is held fixed. In this two-sector mapping, the dimensionless inverse-temperature parameter ÎČeffâÎ _eff changes sign and becomes negative. The coarse output population therefore displays a bounded-spectrum inversion [38, 40, 9, 1]: as TdT_d increases, it passes from a recurrent-dominated sector through maximum mixing and becomes concentrated in the complex/noisy sector. This collective reversal occurs despite the broadening of the token-n+1n+1 distribution at each individual next-token generation step n. Structured routes through the transition Figure 3: Different structured routes through the same population inversion. P1, P4 and P5 denote prompts 1, 4 and 5 in the fixed five-prompt set listed in Supplementary Note 2. a, Frozen output is the most common regime for Llama-3.1-70B at low and intermediate temperatures, before stronger noise destroys that ordering. At each TdT_d, the marker position and background colour identify the most common regime among ten runs; marker size gives the fraction of runs in that regime, with percentages shown when this fraction is below 65%. b, Pythia-12B instead passes through several recurrent regimes before noisy output becomes the most common; symbols and shading have the same meanings as in a. c, Representative trajectories show frozen, sparse, intermittent, complex and noisy dynamics. Each horizontal strip represents one run, and each coloured block represents one generated sentence. Repeated colours within a strip identify sentences belonging to the same recurrent family; pale grey marks unused positions after generation ended. This population inversion (Fig. 2a) is not a featureless crossover. The six output classes reveal frozen or sparse recurrence, short cycles, intermittent switching, complex recurrence and high-diversity noisy motion. Which regimes dominate depends on the AI and prompt, so the same inversion can be reached through different dynamical routes. We use two prompt-conditioned ensembles to illustrate these structured routes (Fig. 3). For Llama-3.1-70B prompt P4, frozen output is the most common regime from Td=0.1T_d=0.1 to 0.8 before noisy output becomes the most common at higher temperature. Extending this comparison to near-greedy decoding, independent word n-gram statistics establish that the frozen interval is a noise-induced ordering window [21], without sentence splitting or regime labels (Supplementary Fig. S3). Prompt-resolved occupancies show the heterogeneous routes hidden by averages over AIs (Supplementary Fig. S4). Pythia-12B prompt P1 instead passes through predominantly frozen, sparse and intermittent ensembles before complex and noisy output dominate. Decoder temperature TdT_d therefore reorganizes the recurrent structures sustained by feedback, rather than simply increasing disorder. Hidden coordinate predicts and steers output The output transition also has an internal dynamical coordinate xnx_n, measured immediately before token n+1n+1 is selected at next-token generation step n. For base Llama-3.1-70B under a fixed prompt, we generated trajectories of up to 300 tokens at 14 positive temperatures, using ten seeds to define the hidden direction and ten disjoint seeds to test the resulting coordinate; the full experimental configuration is listed in Supplementary Table S3. Repetition was measured directly as the fraction of repeated four-word sequences. From training data only, we formed the hidden direction by joining the average normalized hidden states of the upper and lower quartiles of trajectories ranked by repetition. We obtained xnx_n by centring each later normalized hidden state on the training mean and projecting it onto this direction. No test trajectory was used to define the direction, centre or thresholds. On 140 separate test trajectories, the value of xnx_n averaged over each trajectory predicts repetition with rank correlation 0.933 (95% interval, 0.914â0.948; Fig. 4); Supplementary Table S4 gives the complete temperature sweep for the test trajectories. The correlation remains 0.565 after subtracting the mean within each temperature, so the coordinate distinguishes trajectories generated at the same external temperature. The median of these trajectory-averaged values also changes sign across the sharp loss of repetition between Td=0.7T_d=0.7 and 0.8. Because the macroscopic population and hidden coordinate xnx_n are constructed independently, their aligned change from recurrent to non-recurrent behaviour links the population inversion to the AIâs internal dynamics. Figure 4: A hidden coordinate defined from training data predicts repetition in separate test trajectories. a, Repetition falls sharply between Td=0.7T_d=0.7 and 0.8 in separate training and test runs. Lines show medians across ten runs at each temperature; shading spans the interquartile range. b, Repetition in each of the 140 separate test trajectories versus the value of xnx_n averaged over that trajectory; colour denotes decoder temperature. c, The median trajectory-averaged value of xnx_n reverses across the same sharp repetition change; shading spans the interquartile range across the ten test runs at each temperature. A generic effective map follows by projecting the transformer operations onto the hidden direction (see Supplementary Note 6 for the details). During each next-token generation step n, the attention calculation supplies learned tokenâtoken interactions through the softmax operation, a normalized exponential competition; layer normalization and the multilayer perceptron (MLP) then rescale and reshape the internal state. Projected onto a direction separating two output sectors that compete near the transition, the attention update has an S-shaped response whose leading nonlinear saturation near balance is cubic. Appending the sampled token changes the attention weights at next-token generation step n+1n+1, converting the nonlinear calculation during step n into a map from xnx_n to xn+1x_n+1. Slower internal variables appear as memory and sampling as noise. This yields xn+1âa+bâxnâcâxn3+Îșâxnâ1+ηn,c>0.x_n+1 a+bx_n-cx_n^3+Îș x_n-1+ _n, c>0. (2) Here n indexes successive next-token generation steps and xnx_n is the token-level hidden coordinate, oriented so that positive xnx_n corresponds to greater repetition. The constant a biases the next step: a>0a>0 favours repetition, whereas a<0a<0 favours non-repetitive output. The coefficient b is the feedback gain for small displacements about the expansion point: b>0b>0 tends to carry a displacement forward, whereas b<0b<0 tends to reverse it and promote alternating motion. Its magnitude, together with Îș, controls whether a displacement decays or grows. The cubic term with c>0c>0 limits that growth, Îș represents short-term memory, and ηn _n contains unresolved fluctuations. The coefficients may change with decoder temperature. Equation (2) therefore captures the leading bias, gain, saturation, memory and noise of the reduced dynamics. We tested six candidate forms of this nonlinear map, fitting each to the training trajectories and testing them on entirely separate trajectories. Maps using only the current coordinate, whether linear, polynomial or spline, perform similarly, whereas including xnâ1x_n-1 improves prediction on the separate test trajectories at all 14 temperatures by 1.1â8.0% (median 4.6%) and reduces residual correlation. Full diagnostics are shown in Supplementary Figure S6. The hidden coordinate xnx_n therefore changes predictably and retains information from next-token generation step nâ1n-1: it is not merely a label assigned after the text has been generated. To verify that this hidden direction actually influences the output, we perturbed the AI along it at five steering strengths, α=â2,â1,0,1,2α=-2,-1,0,1,2, during text generation, repeating the experiment at three values of TdT_d. The hidden coordinate xnx_n is the scalar position along this direction at next-token generation step n. The dimensionless parameter α sets the perturbation: its sign determines whether the perturbation is positive or negative along the hidden direction, while |α||α| sets its strength. As a control, runs with α=0α=0 reproduced the corresponding unperturbed text sequences exactly. Repetition and the later value of xnx_n both increased monotonically on average: their mean slopes were 0.056 and 0.041 per unit α, respectively, with aggregate estimates reported in Supplementary Table S5. At Td=1T_d=1, where the output population is strongly inverted, the largest positive steering strength, α=2α=2, increased repetition by 0.255 (95% interval, 0.097â0.433), pushing trajectories back towards the recurrent sector (Fig. 5). Figure 5: Steering the AIâs hidden state changes repetition. a, Change in repetition as the hidden state is steered with different strengths along the hidden direction. b, Resulting change in the hidden coordinate xnx_n at later next-token generation steps, measured immediately before the perturbation is applied at each step. Points show averages across ten separate test runs; error bars show 95% bootstrap confidence intervals. Four matched orthogonal perturbations provide a broader demonstration that the different microscopic interventions are organized by the hidden coordinate xnx_n (Supplementary Table S6). Across 36 combinations of perturbation direction, steering strength and decoder temperature, behavioural change correlates 0.842 with the later change in xnx_n but only 0.199 with its immediate change (Fig. 6). Interventions that produce similar later changes in xnx_n produce similar behavioural effects, showing that the hidden coordinate organizes the evolving system. Figure 6: Behavioural effects align with the later change in the hidden coordinate xnx_n across perturbations. a, Behavioural change closely tracks the later change in xnx_n after autoregressive evolution. b, It is only weakly related to the immediate change in xnx_n. Each point represents one combination of perturbation direction, steering strength and decoder temperature, averaged over ten independent runs. Squares show perturbations along the hidden direction and circles show the four matched orthogonal perturbations; colour indicates decoder temperature. A laboratory for nonlinear physics We establish a unified physical picture of ChatGPT-like AIsâ output process. Increasing decoder temperature TdT_d broadens the distribution of possible next tokens at each next-token generation step n, but autoregressive feedback reorganizes the long-time dynamics. ChatGPT-like AIs are therefore unusual systems for physics: greater randomness in each next-token choice does not simply increase long-time disorder, but instead reorganizes the collective output. Beyond the societal need to understand their output, ChatGPT-like AIs provide a new controlled laboratory for nonlinear physics. Decoder temperature and initial conditions can be tuned, internal states can be observed, and perturbations can be applied as the output evolves. This creates a direct experimental route from collective output transitions to the hidden dynamics that generate them. Methods Cross-AI survey. The survey used GPT-2 (124M, 355M, 774M and 1.5B parameters), Pythia (160M, 410M, 1.0B, 1.4B, 2.8B and 12B) and Llama-3.1-70B, with five fixed prompts and up to 300 generated tokens. Conditions through Pythia-2.8B used 20 seeds and the two largest AIs used ten. Within each continuation, sentence similarity was calculated from term-frequencyâinverse-document-frequency (TFâIDF) vectors of character 3â5-grams and converted into a symbolic recurrence trajectory. A fixed rule set classified frozen, sparse, periodic, intermittent, complex and noisy regimes. The complex/noisy fraction was pooled across prompts within each AI and temperature; every AI received equal weight in cross-AI summaries. Hidden-state experiment. The exact base Llama-3.1-70B checkpoint and tokenizer versions are recorded in Supplementary Table S3. Generation used 4-bit weights, bfloat16 computation, full-vocabulary sampling and deterministic algorithms. Ten training seeds and ten disjoint test seeds were fixed before analysis. The first 16 next-token generation steps were excluded as an initial settling period; hidden-state measurements began at step 17. The hidden direction was the normalized difference between training-state averages from the upper and lower quartiles of trajectory repetition. The coordinate xnx_n was obtained by centring each later normalized hidden state on the training mean and projecting it onto this direction. Bootstrap refits confirmed that this direction was stable. Dynamics and interventions. We compared six candidate dynamical maps for predicting xn+1x_n+1 from the current hidden coordinate xnx_n. The candidates used linear, polynomial or spline dependence on xnx_n; extended maps also included xnâ1x_n-1 or additional directions extracted from the training hidden states. The hidden direction and four orthogonal control directions defined from training data were then applied after the initial settling period. Each perturbed run was compared with its exact zero-strength control at the same temperature and seed. Aggregate intervals resampled the ten test seeds. Complete classifier rules, dynamical-map diagnostics and intervention tests are given in the Supplementary Information. Statistics. Cross-AI summaries give each of the 11 AIs equal weight. Hidden-coordinate correlations use 140 separate test trajectoriesâten seeds at each of 14 positive temperaturesâwith 95% intervals obtained by bootstrap resampling entire seed clusters. Intervention effects are paired within the ten test seeds at each condition, and their 95% intervals are percentile bootstrap intervals obtained by resampling seeds. Wilcoxon diagnostics reported in the Supplementary Information are two-sided. Exact sample sizes and definitions of independent runs are given with each figure and in Supplementary Notes 2â9. Use of generative AI tools. OpenAI ChatGPT and Codex were used to assist with rephrasing the flow of the arguments, checking the consistency of the notation and proof-reading the manuscript and code. Data availability The manuscript package contains the archived hidden-state analysis, macroscopic survey and sensitivity results, full experimental configuration and integrity checks, and the numerical data used to produce the figures. For reproducibility, Supplementary Note 10 and the accompanying file list record the archiveâs complete contents and its SHA-256 digital checksum. Code availability The manuscript package is posted on Zenodo at 10.5281/zenodo.21752973. It includes the executed Colab notebook, the same notebook as a Python script, and a script that rebuilds the order-parameter, hidden-state, memory and intervention-response figures. Author contributions N.F.J. and F.Y.H. conceived and designed the study. N.F.J. and B.L. ran the experiments and generated the data. N.F.J., F.Y.H. and B.L. analysed the data, and N.F.J. prepared the figures. N.F.J. wrote the initial draft and supervised the project. Competing interests N.F.J. is a co-founder of d-AI-ta Consulting LLC, which provides advice on AI deployment. The company was not involved in this work, and its activities are unrelated to the research reported here. F.Y.H. and B.L. declare no competing interests. References [1] E. Abraham and O. Penrose (2017) Physics of negative absolute temperatures. Physical Review E 95 (1), p. 012125. External Links: Document Cited by: Temperature drives population inversion. [2] A. Alansari and H. Luqman (2025) Large language models hallucination: a comprehensive survey. arXiv preprint arXiv:2510.06265. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [3] E. Ameisen, J. Lindsey, A. Pearce, et al. (2025) Circuit tracing: revealing computational graphs in language models. Note: Accessed 28 March 2025 External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [4] J. Arnold, F. Holtorf, F. SchĂ€fer, and N. Lörch (2024) Phase transitions in the output distribution of large language models. arXiv preprint arXiv:2405.17088. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [5] Y. Bang, Z. Ji, A. Schelten, A. Hartshorn, T. Fowler, C. Zhang, N. Cancedda, and P. Fung (2025) HalluLens: LLM hallucination benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, p. 24128â24156. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [6] E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell (2021) On the dangers of stochastic parrots: can language models be too big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, p. 610â623. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [7] S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. OâBrien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, A. Skowron, L. Sutawika, and O. Van Der Wal (2023) Pythia: a suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 2397â2430. External Links: Link Cited by: Temperature drives population inversion. [8] R. Bommasani, D. A. Hudson, E. Adeli, et al. (2021) On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [9] S. Braun, J. P. Ronzheimer, M. Schreiber, S. S. Hodgman, T. Rom, I. Bloch, and U. Schneider (2013) Negative absolute temperature for motional degrees of freedom. Science 339 (6115), p. 52â55. External Links: Document Cited by: Temperature drives population inversion. [10] E. Brynjolfsson, D. Li, and L. R. Raymond (2025) Generative AI at work. The Quarterly Journal of Economics 140 (2), p. 889â942. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [11] Z. Chen and T. Luo (2025) From condensation to rank collapse: a two-stage analysis of transformer training dynamics. In Advances in Neural Information Processing Systems, Vol. 38. External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [12] H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey (2024) Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations, External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [13] Y. Dong, J. Cordonnier, and A. Loukas (2021) Attention is not all you need: pure attention loses rank doubly exponentially with depth. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 2793â2803. External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [14] Z. Fu, W. Lam, A. M. So, and B. Shi (2021) A theoretical analysis of the repetition problem in text generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, p. 12848â12856. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [15] A. Grattafiori et al. (2024) The llama 3 herd of models. arXiv preprint arXiv:2407.21783. External Links: Document Cited by: Temperature drives population inversion. [16] R. Guo, Y. Liu, G. Ma, Y. Wang, Y. Zhang, L. Xia, K. Chen, Z. Sun, and D. Shi (2026) When less is more: the LLM scaling paradox in context compression. arXiv preprint arXiv:2602.09789. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [17] W. Gurnee, N. Sofroniew, A. Pearce, M. Piotrowski, et al. (2026) Verbalizable representations form a global workspace in language models. arXiv preprint arXiv:2607.15495. External Links: Document, Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [18] G. E. Hinton, O. Vinyals, and J. Dean (2015) Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [19] T. Hiraoka and K. Inui (2025) Repetition neurons: how do language models produce repetitions?. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), p. 483â495. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [20] A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi (2020) The curious case of neural text degeneration. In International Conference on Learning Representations, External Links: 1904.09751, Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [21] W. Horsthemke and R. Lefever (1984) Noise-induced transitions: theory and applications in physics, chemistry, and biology. Springer Series in Synergetics, Vol. 15, Springer-Verlag, Berlin, Heidelberg. External Links: Document Cited by: Structured routes through the transition. [22] F. Y. Huo and N. F. Johnson (2026) Physics of generative aiâs atom: repetition, bias, and beyond. AIP Advances 16 (3), p. 035305. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [23] G. Jaca, K. Benedek, and J. Török (2026) Chaos in reason: how chain-of-thought LLMs can look for an answer. arXiv preprint arXiv:2607.27805. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [24] E. T. Jaynes (1957) Information theory and statistical mechanics. Physical Review 106 (4), p. 620â630. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [25] N. F. Johnson and F. Y. Huo (2026) Simple picture of how output from chatgpt-like ai shifts from good to bad. PNAS Nexus 5 (5), p. pgag148. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [26] D. M. Katz, M. J. Bommarito, S. Gao, and P. Arredondo (2024) GPT-4 passes the bar exam. Philosophical Transactions of the Royal Society A 382 (2270), p. 20230254. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [27] V. Krishnamurthy (2026) LLMs as high-dimensional nonlinear autoregressive models with attention: training, alignment and inference. arXiv preprint arXiv:2602.00426. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [28] A. Lopez-Lira and Y. Tang (2026) Can ChatGPT forecast stock price movements? return predictability and large language models. Journal of Financial Economics 184, p. 104335. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [29] R. Luger, H. Kamath, D. Finkbeiner, P. Goel, A. Jermyn, S. Zimmerman, J. Batson, and T. Conerly (2026) HeadVis: an interactive tool for investigating attention heads. Note: Transformer Circuits Thread; accessed 31 July 2026 External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [30] N. Mikhaylovskiy (2025) States of llm-generated texts and phase transitions between them. In MathAI 2025, External Links: 2503.06330, Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [31] A. E. Motter and R. Albert (2012) Networks in motion. Physics Today 65 (4), p. 43â48. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [32] A. E. Motter (2015) Networkcontrology. Chaos 25 (9), p. 097621. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [33] K. Nakaishi, Y. Nishikawa, and K. Hukushima (2024) Phase transition in large language models and the criticality of natural languages. arXiv preprint arXiv:2406.05335. Note: version 3, revised 2026 External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [34] N. Nanda, L. Chan, T. Lieberum, J. Smith, and J. Steinhardt (2023) Progress measures for grokking via mechanistic interpretability. In International Conference on Learning Representations, External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [35] N. Nanda and T. Lieberum (2022) A mechanistic interpretability analysis of grokking. Note: Accessed 7 May 2024 External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [36] N. Nanda (2023) Paper replication walkthrough: reverse-engineering modular addition. Note: Accessed 7 May 2024 External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [37] North Atlantic Treaty Organization (2024) Summary of NATOâs revised artificial intelligence strategy. Technical report NATO. Note: Accessed 31 July 2026 External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [38] E. M. Purcell and R. V. Pound (1951) A nuclear spin system at negative temperature. Physical Review 81 (2), p. 279â280. External Links: Document Cited by: Temperature drives population inversion. [39] A. Radford et al. (2019) Language models are unsupervised multitask learners. Technical report OpenAI. Cited by: Temperature drives population inversion. [40] N. F. Ramsey (1956) Thermodynamics and statistical mechanics at negative absolute temperatures. Physical Review 103 (1), p. 20â28. External Links: Document Cited by: Temperature drives population inversion. [41] H. Ruan, J. Li, X. Guo, and L. Wang (2026) Generative criticality in large language model temperature scaling. arXiv preprint arXiv:2606.06238. Note: Contributed to PAI 2026 External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [42] K. Singhal, S. Azizi, T. Tu, et al. (2023) Large language models encode clinical knowledge. Nature 620, p. 172â180. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [43] O. Sporns (2010) Networks of the brain. MIT Press, Cambridge, MA. Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [44] S. H. Strogatz (2024) Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. 3 edition, Chapman and Hall/CRC. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [45] W. Sun, X. Song, P. Li, L. Yin, Y. Zheng, and S. Liu (2025) The curse of depth in large language models. In Advances in Neural Information Processing Systems, Vol. 38. External Links: Link Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [46] Y. Sun and B. Haghighat (2025) Phase transitions in large language models and the Oâ(N)O(N) model. arXiv preprint arXiv:2501.16241. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [47] H. Wang, T. Fu, Y. Du, et al. (2023) Scientific discovery in the age of artificial intelligence. Nature 620, p. 47â60. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [48] Z. Wang, Y. Li, J. Yan, Y. Cheng, and Y. Zhang (2025) Unveiling attractor cycles in large language models: a dynamical systems view of successive paraphrasing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12740â12755. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs. [49] Z. Wei, S. Wang, X. Rong, X. Liu, and H. Li (2025) Shadows in the attention: contextual perturbation and representation drift in the dynamics of hallucination in LLMs. arXiv preprint arXiv:2505.16894. External Links: Document Cited by: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs.