Paper deep dive
Estimating Rare Events in Language Models with Proper Evaluation
Nikita Y. Parulekar, Anqi Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/22/2026, 2:37:22 AM
Summary
This paper addresses the challenge of estimating rare failure probabilities in language models, where standard sampling is infeasible. The authors propose Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS), a method that operates in the continuous activation space using a gradient-based MCMC kernel to avoid zero-estimate collapse. Additionally, they introduce the Shifted-Power Bregman (SPB) Loss, a proper scoring rule that handles zero estimates and allows for tunable asymmetry in penalty costs. Experiments show GA-AMLS outperforms baselines under symmetric evaluation, while methods with overestimation bias may be preferred for conservative safety audits.
Entities (6)
Relation Signals (6)
GA-AMLS → operatesin → activation space
confidence 95% · GA-AMLS uses a gradient-based MCMC kernel to navigate activation space, eliminating the zero-estimate collapse of input-space search
SPB Loss → handles → zero estimates
confidence 94% · SPB remains finite when an estimator returns exactly zero
GA-AMLS → useskernel → MALA
confidence 92% · GA-AMLS uses a gradient-based Metropolis-adjusted Langevin algorithm (MALA) kernel to guide the sampler within each level.
SPB Loss → generalizes → Itakura-Saito Loss
confidence 88% · The unshifted power family also recovers standard losses as special cases, including ... the Itakura–Saito loss used by Wu and Hilton (2025)
QLD → suffersfrom → overestimation bias
confidence 85% · empirically, the resulting distortion surfaces mainly as overestimation on rare events.
GA-AMLS → outperforms → QLD
confidence 80% · GA-AMLS achieves the lowest loss under symmetric evaluation, reducing average log-space squared error relative to the strongest baseline
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Quantifying the risk of rare failures in language models, such as those triggered by adversarial distribution shifts or very large-scale deployments, requires estimating probabilities far too small for random sampling. While recent work has formalized Low Probability Estimation, existing pipelines remain fragile in the rarest regimes: estimators can suffer zero-estimate collapse or systematic bias, and standard evaluation losses can become unstable or poorly matched to asymmetric safety costs. In this work, we introduce Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS), which adapts rare-event Monte Carlo methods to the continuous activation space of language models. Specifically, GA-AMLS uses a gradient-based MCMC kernel to navigate activation space, eliminating the zero-estimate collapse of input-space search and replacing the independence assumptions of prior activation-space estimators with conditional sampling under an explicit, heavier-tailed activation prior. We also propose the Shifted-Power Bregman (SPB) Loss, a proper scoring rule that remains finite for zero-estimates and offers tunable asymmetry between underestimation and overestimation penalties. Experiments on small transformer models reveal a bias-variance tradeoff: GA-AMLS achieves the lowest loss under symmetric evaluation, reducing average log-space squared error relative to the strongest baseline across model sizes, while methods with overestimation bias prevail under asymmetric penalties. Our findings highlight that estimator choice should be matched to deployment context. More broadly, our work establishes activation space as a tractable domain for rare-event estimation in language models, circumventing the brittleness of discrete input-space search.
Tags
Links
- Source: https://arxiv.org/abs/2607.18454v1
- Canonical: https://arxiv.org/abs/2607.18454v1
Trouble viewing inline? Open PDF directly →
Full Text
94,773 characters extracted from source content.
Expand or collapse full text
Estimating Rare Events in Language Models with Proper Evaluation Nikita Y. Parulekar nparule2@jhu.edu Johns Hopkins University Anqi Liu aliu.cs@jhu.edu Johns Hopkins University Abstract Quantifying the risk of rare failures in language models, such as those triggered by adversarial distribution shifts or very large-scale deployments, requires estimating probabilities far too small for random sampling. While recent work has formalized Low Probability Estimation, existing pipelines remain fragile in the rarest regimes: estimators can suffer zero-estimate collapse or systematic bias, and standard evaluation losses can become unstable or poorly matched to asymmetric safety costs. In this work, we introduce Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS), which adapts rare-event Monte Carlo methods to the continuous activation space of language models. Specifically, GA-AMLS uses a gradient-based MCMC kernel to navigate activation space, eliminating the zero-estimate collapse of input-space search and replacing the independence assumptions of prior activation-space estimators with conditional sampling under an explicit, heavier-tailed activation prior. We also propose the Shifted-Power Bregman (SPB) Loss, a proper scoring rule that remains finite for zero-estimates and offers tunable asymmetry between underestimation and overestimation penalties. Experiments on small transformer models reveal a bias-variance tradeoff: GA-AMLS achieves the lowest loss under symmetric evaluation, reducing average log-space squared error relative to the strongest baseline across model sizes, while methods with overestimation bias prevail under asymmetric penalties. Our findings highlight that estimator choice should be matched to deployment context. More broadly, our work establishes activation space as a tractable domain for rare-event estimation in language models, circumventing the brittleness of discrete input-space search. 1 Introduction Modern large language models (LLMs) are trained to generalize to new inputs by minimizing expected loss during training. However, this averaging objective leaves them susceptible to producing highly undesirable outputs on a small subset of rare inputs. While these events may appear statistically negligible during training, the rarity of a failure is often relative. In deployment, distribution shifts, whether manufactured through adversarial jailbreaks or incidental due to goal misgeneralization (Shah et al., 2022), can make these catastrophic failures significantly more common. Furthermore, in safety-critical applications such as autonomous vehicles or medical infrastructure, even a single failure can be unacceptable. Additionally, the large-scale use of language models can make low-probability events eventual certainties. Therefore, there is a growing need for accurate low probability estimation (LPE) methods: the ability to quantify the likelihood of a model producing a specific harmful output, even when that probability is extremely small, such as between 10−510^-5 and 10−910^-9 , ranges where naive sampling is computationally infeasible. Following the problem setting introduced by Wu and Hilton (2025), we study LPE in the context of argmax sampling, where our goal is to estimate the probability that a specific target token will have the largest output logit within a computationally constrained budget as shown in Figure 1(a). The status quo established by Wu and Hilton (2025) comprises two classes of methods. Input-space Importance Sampling searches the discrete token space; for the rarest targets it often fails to find any triggering input, collapsing estimates to zero, and it additionally requires expensive full backward passes and knowledge of exact input-sequence probabilities. The activation-space alternative, Quadratic Logit Decomposition (QLD), avoids zero estimates by recombining whitened activation components as if they were independent, producing n2n^2 synthetic candidates from n empirical activations. But whitening guarantees only uncorrelatedness, not independence, and language-model activations carry heavy-tailed, outlier-dominated structure (Sun et al., 2024; Dettmers et al., 2022) that can survive whitening; empirically, the resulting distortion surfaces mainly as overestimation on rare events. Finally, the metrics used to compare these methods, Itakura-Saito (IS) loss and squared error loss in log space, are undefined at zero estimates, which Wu and Hilton (2025) patch with an affine calibration that itself requires known ground-truth probabilities. (a) Low-probability estimation setup. (b) GA-AMLS and QLD estimates on the colon distribution and 1-layer model. Each point represents a different target token. Figure 1: Overview of the low-probability estimation setup and GA-AMLS estimates. In this paper, we address both the algorithmic and evaluative limitations of current LPE approaches through two coupled contributions: an activation-space estimation algorithm, and a numerically stable evaluation metric that makes comparing all estimators, including those whose estimates collapse to zero, well-posed. First, we propose Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS), which adapts the Adaptive Multi-Level Splitting (AMLS) algorithm (Guyader et al., 2011) to operate entirely within the continuous activation space of language models. While Webb et al. (2019) previously applied AMLS to computer vision models in the input space, ours is, to our knowledge, the first method to apply it in the activation space of language models. Figure 2 gives a schematic overview of GA-AMLS, while Figure 1(b) previews its comparison with the strongest activation-space baseline, QLD, on a representative distribution. By working in activation space, we can estimate tail probabilities without the expensive full backward passes required by Importance Sampling. We also avoid needing to compute the input sequence probability under the original distribution, which Importance Sampling methods require for importance weights. Instead, we decompose the search for activations that trigger the target token, into a sequence of intermediate levels, using a gradient-based Metropolis-adjusted Langevin algorithm (MALA) kernel (Roberts and Tweedie, 1996; Srinivasan et al., 2025) to guide the sampler within each level. The MALA kernel’s drift term allows the evolved activations to stay “close” to the typical activations of the input distribution, while the adaptive levels progressively steer towards the failure region. This replaces QLD’s independence-based recombination with conditional sampling under an explicit prior. Idealized AMLS is an unbiased estimator of the tail probability of its sampling distribution (Cérou et al., 2019); because our MCMC kernel targets a fitted activation prior rather than the true activation distribution, GA-AMLS inherits this guarantee only up to prior mismatch (Section 7). Second, we introduce Shifted Power Bregman (SPB) Loss, a family of proper scoring-rule divergences designed for evaluating rare-probability estimates. SPB remains finite when an estimator returns exactly zero, supports tunable asymmetry between underestimation and overestimation through an interpretable asymmetry parameter, and approximately preserves scale invariance across several orders of magnitude of the ground-truth probability. The unshifted power family also recovers standard losses as special cases, including squared error and the Itakura–Saito loss used by Wu and Hilton (2025) (Appendix B.8). Thus SPB Loss is a strict generalization of existing evaluation losses, with an explicit finiteness correction and a tunable asymmetry knob. A comparison with standard losses is presented in Table 1. Empirically, the estimator ranking depends on the evaluation cost model. Under approximately symmetric penalties, including log-space squared error and corresponding SPB settings, GA-AMLS has lower bias and outperforms the baselines. Under strongly asymmetric penalties that heavily punish underestimation, including Itakura–Saito loss and corresponding SPB settings, QLD can outperform GA-AMLS because its positive bias acts as a conservative hedge against false negatives. This rank inversion is visible under pre-existing metrics as well as under SPB; SPB makes the tradeoff explicit while avoiding the numerical instability caused by zero estimates. Consequently, the appropriate estimator depends on the deployment context: conservative catastrophic-risk audits may prefer overestimation, whereas applications sensitive to excessive false alarms may prefer the lower-bias GA-AMLS diagnostic. Figure 2: GA-AMLS in activation space. The rare-event probability is decomposed into a product of conditional probabilities over adaptive score levels. At each level, activations below the threshold are discarded and survivors are resampled. MALA rejuvenation uses the fitted activation-prior gradient to improve mixing within the constrained level. Property Squared Error IS Loss Log Squared Error SPB Loss Scale invariance ✗ ✓ ✓ ✓ ≈ exact Defined when estimate =0=0 ✓ ✗ ✗ ✓ Asymmetric: FN >> FP penalty ✗ ✓ ✗ ✓ tunable via α Proper scoring rule ✓ ✓ ✗ ✓ Table 1: SPB Loss is the only loss satisfying all four desiderata for LPE evaluation: scale invariance, finiteness at zero estimates, tunable asymmetry between underestimation and overestimation penalties via α, and strict propriety. 2 Background and Problem Setup 2.1 Problem Setup Low-probability estimation. We adopt the low-probability estimation (LPE) setting of Wu and Hilton (2025). Let M be a language model with vocabulary V, and let x∼x be an input sequence drawn from a specified input distribution. We consider deterministic argmax decoding: M(x)M(x) is the token with maximal output logit. For a target token t∈t , the rare-event probability of interest is qt=Prx∼[M(x)=t].q_t= _x [M(x)=t]. In our experiments, qtq_t lies in the extreme-tail regime 10−9≤qt≤10−510^-9≤ q_t≤ 10^-5, where naive Monte Carlo sampling is infeasible. Activation-space formulation. GA-AMLS operates on internal activations rather than discrete text. Let f:∗→ℝdf:V^* ^d map an input sequence to its final-layer pre-LayerNorm, pre-unembedding activation a=f(x)a=f(x). Given unembedding matrix WUW_U, the logits are z(a)=LayerNorm(a)WU.z(a)=LayerNorm(a)W_U. The event M(x)=tM(x)=t can therefore be written as an activation-space constraint: qt=Prx∼[zt(f(x))>maxi≠tzi(f(x))].q_t= _x [z_t(f(x))> _i≠ tz_i(f(x)) ]. Existing estimators. We compare against the methods introduced by Wu and Hilton (2025). Input-space importance sampling methods such as ITGIS and MHIS search over discrete token sequences and reweight samples to estimate qtq_t. These estimators can be unbiased in principle, but empirically in the extreme-tail regime they often fail to find triggering inputs, producing zero estimates. QLD avoids this failure mode by operating in activation space. It whitens pre-unembedding activations, decomposes each activation as u=a+bu=a+b into a target-direction component and an orthogonal residual, and estimates qtq_t by recombining components a(i)+b(j)a^(i)+b^(j). This converts n activations into n2n^2 candidate activations, but relies on treating the whitened components as independent. Since whitening guarantees uncorrelatedness rather than independence except under stronger assumptions such as joint Gaussianity, non-Gaussian activation structure can introduce bias. 2.2 Sampling Methods The Adaptive Multilevel Splitting (AMLS) Algorithm. Adaptive Multi-Level Splitting (AMLS) is a rare-event Monte Carlo method that estimates a small probability by decomposing it into a product of larger conditional probabilities (Guyader et al., 2011; Cérou et al., 2019; Webb et al., 2019). Let V∼GV G be a random variable, let s:ℝd→ℝs:R^d be a score function, and let the rare event be s(V)≥τ\s(V)≥τ\. For levels −∞=L0<L1<⋯<LK=τ-∞=L_0<L_1<·s<L_K=τ, Pr(s(V)≥τ)=∏k=1KPr(s(V)≥Lk∣s(V)≥Lk−1). (s(V)≥τ)= _k=1^K (s(V)≥ L_k s(V)≥ L_k-1). AMLS estimates these factors using a population of N particles. At each level, the threshold LkL_k is chosen adaptively from the empirical score distribution; particles below LkL_k are discarded, survivors are resampled, and the resulting duplicates are rejuvenated with an MCMC kernel targeting G(⋅∣s(V)≥Lk)G(· s(V)≥ L_k). The final estimate is q^AMLS=∏k=1Kp^k,p^k=1N∑i=1Ns(Vi)≥Lk. q_AMLS= _k=1^K p_k, p_k= 1N _i=1^N1\s(V_i)≥ L_k\. Thus, AMLS replaces one extremely rare event with a sequence of more frequent conditional events. MALA Proposal Kernel. The efficiency of AMLS depends on the MCMC kernel used to rejuvenate particles. We use the Metropolis-adjusted Langevin algorithm (MALA), which proposes moves using local gradient information. For a target density π, MALA proposes (Roberts and Tweedie, 1996; Srinivasan et al., 2025) x′=x+h∇xlogπ(x)+2hξ,ξ∼(0,I),x =x+h _x π(x)+ 2h\,ξ, ξ (0,I), and accepts or rejects the proposal using the standard Metropolis-Hastings correction. 3 Gradient Activation AMLS (GA-AMLS) for Estimating Rare Events We adapt the general AMLS algorithm to work in the activation space of language models, and use a MALA kernel to guide the search (Section 2.2). We call this method Gradient Activation AMLS (GA-AMLS), presented in Appendix C as Algorithm 1 and outlined below. Notation. Let a∈ℝda ^d denote the raw pre-LayerNorm pre-unembedding activation, and let u denote its whitened coordinate. Given mean μ and covariance Σ , choose a de-whitening matrix A such that AA⊤=ΣA = , and define u=(a−μ)A−⊤,a(u)=uA⊤+μ.u=(a-μ)A^- ,a(u)=uA +μ. GA-AMLS and the MALA kernel operate over u, while model logits are evaluated by inserting the corresponding unwhitened activation a(u)a(u) back into the model. Prepare the Activation Space: We use activations from the pre-LayerNorm pre-unembedding layer, and whiten them. Raw transformer activations a are characteristically anisotropic, with a few outlier dimensions whose scale, in magnitude and in across-distribution variance, is orders of magnitude larger than the rest. This outlier structure is documented in large language models (Sun et al., 2024; Dettmers et al., 2022); we verify empirically that it also holds at the smaller models we study in the Appendix in Figure 12. To improve mixing of MCMC kernels in this high-dimensional space, we operate in whitened coordinates u=(a−μ)A−⊤u=(a-μ)A^- . This global preconditioning makes isotropic Euclidean steps in u-space correspond to Mahalanobis-scaled steps in the original activation space. We sample pre-LayerNorm activations so that the evolved activations do not need to satisfy the LayerNorm constraint directly. Define the Score function: We define the score function ss that directly describes how “close” an activation is to an activation producing the target token. Given whitened pre-LayerNorm pre-unembedding activation u as input, A as the de-whitening matrix, μ as the mean, t as the target token index, the score function computes: s(u)=(LayerNorm(uA⊤+μ)WU)t−maxj≠t(LayerNorm(uA⊤+μ)WU)j.s(u)= (LayerNorm(uA +μ)\,W_U )_t- _j≠ t (LayerNorm(uA +μ)\,W_U )_j. That is, s(u)s(u) is the margin by which the target token’s logit exceeds the maximum logit among all other tokens. Using the general AMLS algorithm from Section 2.2 , we define the rare event as P(s(u)≥τ)P (s(u)≥τ ). In our setting τ=0τ=0, since this is when the logit for the target token t is the largest, and thus t will be the next token predicted under our assumption of argmax sampling. Determine the prior distribution: The prior π is defined over whitened activations u and is used to guide the MCMC kernel. In whitened space, we use a diagonal product Student-t prior. This captures heavier-than-Gaussian marginal tails. This choice is supported by the tuning ablations in Appendix D.1, where the Student-t prior outperforms a Gaussian prior. Choose the MCMC kernel K: To efficiently evolve the clones’ activations within the high-dimensional activation space, we use the Metropolis-adjusted Langevin algorithm (MALA) as outlined in Section 2.2, with the below correction to take into account the constraint induced by the score mechanism of AMLS. We must strictly enforce the level constraints s(u)≥Lis(u)≥ L_i. This is addressed through a Metropolis-Hastings filter. The proposal density q(u′∣u)q(u ) is Gaussian, while the prior distribution is Student’s t distribution. The acceptance probability r is computed in two stages to handle the hard constraint: 1. Hard score check: If s(u′)<Lis(u )<L_i, the proposal is invalid and immediately rejected (r=0r=0); 2. Ratio Computation: If valid, we compute the standard Metropolis Hastings (MH) acceptance ratio r: r=min(1,π(u′)q(u∣u′)π(u)q(u′∣u)).r= (1, π(u )\,q(u )π(u)\,q(u ) ). We accept the proposal u′u with probability r; otherwise, the activation remains at u. Standard Random Walk Metropolis-Hastings (RWMH) proposes isotropic steps blind to the geometry of the distribution, leading to slow mixing in this high dimensional activation space. MALA improves upon RWMH by using gradient information from the underlying prior distribution to guide the proposal towards the activation space’s typical set for the specific input domain D. In GA-AMLS this MALA kernel is applied to the restricted target πk(u)∝π(u) 1s(u)≥Lk, _k(u) π(u)\,1\s(u)≥ L_k\, so proposals below the current level LkL_k are rejected by the MH filter. Thus the Langevin drift improves exploration within the current constrained level set, while the adaptive thresholds LkL_k move the population toward the rare target-token regions. This choice is also supported by the tuning ablations in Appendix D.1, where MALA outperforms RWMH despite using half as many MH proposal steps. To ensure optimal mixing, we adapt the step size h (from MALA under 2.2) during a burn-in period of TburnT_burn steps. We employ a multiplicative update rule to tune h toward a target acceptance rate of r∗=0.57r^*=0.57, which is theoretically optimal for MALA in high dimensions (Roberts and Rosenthal, 1998): ht+1=ht⋅exp(ω(r¯t−r∗))h_t+1=h_t· (ω( r_t-r^*) ), where ω=0.1ω=0.1 is the adaptation rate and r¯t r_t is the empirical acceptance rate at step t. After TburnT_burn steps, h is frozen to ensure the Markov chain satisfies detailed balance condition during the sampling. 4 Shifted Power Bregman (SPB) Loss for Evaluation We introduce Shifted-Power Bregman (SPB) Loss, a proper scoring rule for evaluating rare-probability estimates. SPB remains finite when an estimator returns zero and provides tunable asymmetry between underestimation and overestimation. We define both the pointwise loss and a dataset-level aggregation that controls scale dependence across rare-event probabilities. Desiderata for an ideal loss for LPE. To evaluate an estimator for LPE, an ideal loss would satisfy the following requirements: Scale Invariance: As mentioned in Wu and Hilton (2025), it should penalize relative errors similarly across the scale of ground truth probabilities, as we care about performance across a wide range of rarities (e.g., 10−510^-5 to 10−910^-9). Asymmetric Sensitivity: It should support tunable asymmetry between underestimation and overestimation penalties, as different deployment contexts may prioritize avoiding underestimation over overestimation, or vice versa. Numerical Stability: It must be defined when the estimate is exactly zero, avoiding the need for calibration steps that require ground truth knowledge. Statistical Rigor: It should be the divergence of a proper scoring rule, so that an estimator minimizing expected loss is incentivized to report its unbiased best estimate rather than a systematically distorted one (Gneiting and Raftery, 2007). The Itakura–Saito (IS) loss used in Wu and Hilton (2025) satisfies most criteria but fails on numerical stability: it is undefined (infinite) when the estimate is 0. Wu and Hilton (2025) overcome this by fitting an affine transformation x↦axc+bx ax^c+b to the outputs, where parameters are chosen to minimize IS loss via leave-one-out-cross-validation (LOOCV) for each method, input, distribution pair. A significant drawback is that this transformation relies on knowing the ground truth probabilities to calculate the IS loss which is to be minimized during this calibration. In a realistic deployment setting, establishing this ground truth for rare tokens is computationally infeasible using naive sampling, as estimating these probabilities is the very problem we are trying to solve. SPB uses ground truth only as the reference value in the final score, as any evaluation metric does. By contrast, the affine calibration used in prior evaluation procedures assumes that ground-truth probabilities are available to transform each estimator’s outputs before scoring. This is a stronger requirement than ordinary evaluation against ground truth, since such calibration information would generally not be available in deployment. Pointwise SPB Loss. We propose pointwise Shifted-Power Bregman (SPB) Loss, derived from the framework of proper scoring rules. It overcomes this limitation by accepting a trade-off: if the estimate is 0, the loss will not be infinite, but rather capped by a large finite constant. We also introduce parameters α: to control the degree of asymmetric sensitivity, and γ: a dataset level rarity premium. Following Buja et al. (2005) , every smooth proper scoring rule for binary probability estimation induces a Bregman divergence determined by a nonnegative weight function ω(t)≥0ω(t)≥ 0 on (0,1)(0,1). If q denotes the true probability and p the forecast, the associated divergence can be written as B(q∣p)=∫qp(t−q)ω(t)t.B(q p)= _q^p(t-q)ω(t)\,dt. We define the shifted-power weight function as: ωε(t)=(t+ε)−α _ (t)=(t+ )^-α, where α>0,ε>0α>0, >0. The shift parameter ε ensures the penalty remains finite at p=0p=0. Geometrically, this integral accumulates cost along the one-dimensional probability axis between the truth q and the prediction p. At an intermediate probability level t, the local contribution is the distance from the truth, scaled by the weight ωε(t)=(t+ε)−α _ (t)=(t+ )^-α. Since this weight is largest near zero, errors that push the prediction toward smaller probabilities are penalized more strongly. For an overestimate p=mqp=mq, the loss integrates over t∈[q,mq]t∈[q,mq], where the weight decreases as t increases. For a reciprocal underestimate p=q/mp=q/m, the loss integrates over t∈[q/m,q]t∈[q/m,q], which includes smaller probabilities where ωε(t) _ (t) is larger. Increasing α steepens this growth near zero and therefore increases the relative penalty for underestimation compared with overestimation. Thus, α provides a tunable asymmetry parameter, as formalized in Theorem 2. Dataset-Level SPB Loss and the Rarity Premium γ. To evaluate an estimator across a diverse set of rare events, we aggregate the pointwise divergences into a single dataset-level loss. As derived in Appendix A, the unshifted pointwise divergence (ε=0 =0) scales exactly proportionally to q2−αq^2-α for a fixed relative error m=p/qm=p/q. While the shift parameter ε>0 >0 slightly perturbs this geometry, the shifted divergence Bε(q∣p)B_ (q p) remains dominated by this leading-order q2−αq^2-α scaling. Left uncorrected, an aggregate loss would be dominated by the absolute scale of the ground-truth probabilities rather than the relative accuracy of the estimators. To neutralize this and establish controlled scale invariance, we define the dataset-level SPB Loss as the arithmetic mean111Crucially, we divide by the sample size n rather than the sum of the weights ∑iw(qi) _iw(q_i). Since the weight w(q)w(q) is specifically designed to perform a pointwise cancellation of the q2−αq^2-α scale factor, applying a normalized weighted average would reintroduce a dependence on the empirical distribution of the ground truth probabilities q, undoing the cancellation. of the weighted pointwise divergences across n samples: ℒSPB=1n∑i=1nw(qi)Bε(qi∣pi).L_SPB\;=\; 1n _i=1^nw(q_i)\,B_ (q_i p_i). We define the weight function as: w(q)=qα−2−γw(q)\;=\;q^\,α-2-γ, where γ≥0γ≥ 0. The parameter γ allows us to toggle between two distinct evaluation modes: Scale Neutrality (γ=0γ=0): By setting γ=0γ=0, the weight becomes qα−2q^α-2. This term cancels the leading q2−αq^2-α scaling of the divergence. Consequently, the final metric approximates scale invariance, depending almost entirely on the relative multiplicative error (m=p/qm=p/q) rather than the absolute rarity of the event. The residual distortion due to ε is formalized and bounded in the Appendix in Theorem 1. The Rarity Premium (γ>0γ>0): In specific safety-critical deployments, evaluating performance symmetrically across scales is less important than severely punishing errors on the deepest tail events. Setting γ>0γ>0 intentionally breaks scale neutrality by introducing an extra q−γq^-γ factor. This acts as a tunable rarity premium, disproportionately upweighting the smallest-q events in the aggregate score. Relation to other Losses The unshifted power family (ε=0 =0) unifies standard losses: α=0α=0 recovers squared error and α=2α=2 is exactly the Itakura–Saito loss (Appendix B.8). Therefore SPB comes from a strict generalization of the metric used by Wu and Hilton (2025), adding finiteness at zero estimates (with ε>0 >0) and a tunable asymmetry knob. 4.1 Practical parameter selection (α, γ, ε ) The metric has three user-facing parameters: 1) α sets the FN-vs-FP asymmetry: α>1.5α>1.5 penalizes underestimation more than overestimation and α<1.5α<1.5 the reverse (Theorem 2, Eq. 20 in Appendix; fine-grained control of α is described in Appendix B.6.1). 2) γ≥0γ≥ 0 is an optional rarity premium: γ=0γ=0 gives scale invariance up to a tolerance η (Theorem 1 in Appendix), while γ>0γ>0 multiplies each example’s contribution to the aggregate loss by q−γq^-γ, so at a fixed relative error a 10×10× rarer event carries 10γ×10^γ× the weight. 3) ε is the floor that makes p=0p=0 finite at the cost of controlled distortion; a heuristic default is ε=0.01qmin/mmax =0.01\,q_ /m_ , and a guaranteed choice is any fraction of the bound given in Appendix by Corollary 1 (interpretation in Appendix B.5). Table 2 (Appendix B.6) suggests settings by evaluation use case. 5 Experimental Results This section empirically evaluates both components of our proposal. Section 5.1 tests GA-AMLS on the real-data LPE benchmark of Wu and Hilton (2025), comparing it against input-space Importance Sampling methods and QLD using both point-estimate plots and aggregate SPB Loss. Section 5.2 uses synthetic experiments to validate the behavior predicted by the SPB Loss theory, including approximate scale invariance, bounded distortion from the ε -shift, and tunable asymmetry between underestimation and overestimation. 5.1 GA-AMLS: Real Data Experiments Dataset: We use 8 distributions and the ground-truth probabilities as used in Wu and Hilton (2025). They generated the ground-truth probabilities for each of the 8 distributions by running forward passes on 2322^32 random samples. We select a random set of 256 tokens among those with ground-truth probabilities between 10−910^-9 and 10−510^-5, and test the methods on these tokens. Methodology: We follow the experimental setup of Wu and Hilton (2025). To prevent overfitting, the method was only run on the first four distributions during development, and finalized before testing on the last four distributions. We test the methods on 3 models: a 1-layer, a 2-layer, and a 4-layer transformer from Nanda and Bloom (2022). All models have a hidden dimension of d=512d=512, a vocabulary size of |V|=48262|V|=48262, GELU non-linearities Hendrycks and Gimpel (2023), and were trained on the C4 dataset Raffel et al. (2023) and CodeParrot Tunstall et al. (2022). We use the Shifted-Power Bregman (SPB) Loss as our evaluation loss. By varying the asymmetry parameter α and the dataset-level rarity premium γ , we demonstrate how different penalization profiles (detailed in Appendix in Table 2) alter the apparent ranking of the methods. For hyperparameter tuning details please refer to Appendix D.1. Figure 3: Multi-metric evaluation: Plotted side-by-side, we observe a rank inversion depending on the metric’s sensitivity to underestimation. Under symmetric penalties (SPB α=1.5α=1.5 and Squared Error in log space), GA-AMLS’s lower bias leads to outperformance. Under severe asymmetric penalties (SPB α=2α=2 and IS Loss), QLD’s strategy of systematic overestimation guards it against false-negative penalties, resulting in lower loss. SPB Loss safely evaluates input-space methods that collapse to zero, overcoming the numerical instability of uncalibrated IS loss and squared error loss in log space. (All plotted with log transformed y axis.) Baselines: We compare against QLD, ITGIS and MHIS as described in Section 2.1. Performance under Symmetric Penalties: Figure 3 compares the overall evaluation losses of all methods across different model sizes grouped by symmetric and asymmetric metrics, while Figure 16 (Appendix F) details the SPB loss for each of the 8 individual data distributions. Under evaluation metrics that penalize overestimation and underestimation symmetrically, such as log-space Squared Error and SPB Loss with α=1.5α=1.5, we observe that GA-AMLS significantly outperforms all baseline methods (Figure 3, first two panels). This outperformance is consistent across 7 of the 8 individual distributions (Figure 16). Figure 4: Examples of probability estimate vs ground truth on two input distributions and models. Input Space Methods (ITGIS/MHIS) frequently fail to find valid triggers, causing their estimates to collapse to zero (placed at the bottom of each graph for visibility). While QLD shows a systematic overestimation bias, GA-AMLS centers closest to the ground truth, at the cost of higher variance. Performance under Asymmetric Penalties: Focusing on the latter half of Figure 3, the figure illustrates the methods’ performance under asymmetric metrics that severely penalize underestimation (SPB Loss with α=2α=2 and IS Loss). When evaluating with these metrics, we observe a clear inversion in method rankings, where QLD outperforms GA-AMLS under these conditions. This occurs because QLD is biasing its estimates toward overestimation. While less accurate overall, this systematic positive bias acts as a natural safeguard against the severe false-negative penalties imposed by asymmetric losses. Deep-Tail Overestimation and Rarity Premium γ: Figure 4 plots the methods’ individual point estimates against the true probabilities to visualize estimator bias, and Figure 17 (Appendix F) shows how the aggregate SPB loss shifts across methods as the rarity premium γ increases. The scatter plots (Figure 4 and Appendix E) reveal that QLD’s overestimation bias is not uniform; it becomes significantly more pronounced at the deepest tail (the rarest events). Consequently, as seen in Figure 17, as we increase the dataset-level rarity premium (γ>0γ>0) to disproportionately upweight the rarest events, the performance gap between QLD and GA-AMLS narrows, demonstrating the cost of QLD’s deep-tail overbias. The Zero-Estimate Collapse and Bias-Variance Tradeoff: The scatter plots of method estimates (Figure 4 and Appendix E) also directly capture the distribution and structural failure modes of each estimator’s predictions. The distinct performance profiles of these methods stem from a fundamental bias-variance tradeoff, visually observed in these plots. The input-space Importance Sampling methods (MHIS/ITGIS) suffer from frequently collapsing to zero for the rarest events. QLD avoids zero-estimates but shifts the entire distribution of predictions upward. GA-AMLS shows lower bias and is more centred around the ground truth line, but exhibits higher variance across tokens than QLD (as seen visually in scatterplots in Appendix E). 5.2 SPB Loss: Synthetic Data Experiments We now isolate the behavior of SPB Loss itself, independently of any estimator. The goal of these synthetic experiments is to illustrate the loss geometry discussed in Section 4.1 and formalized in Appendix B: approximate scale invariance, controlled distortion from the finite shift ε , and tunable asymmetry between underestimation and overestimation. Setup. We evaluate rare probabilities q∈[10−9,10−5]q∈[10^-9,10^-5] on a log-spaced grid and we perturb each probability by fixed multiplicative factors. For scale-invariance experiments, predictions range from 0.1q0.1q to 10q10q. For asymmetry experiments, we compare paired reciprocal errors: an overestimate/False-Positive (FP) p=mqp=mq and an underestimate/False-Negative (FN) p=q/mp=q/m, with m ranging up to 10001000. We use the scale-neutral setting γ=0γ=0. For finite-shift ε>0 >0 plots, we compare two choices of ε : one chosen according to the sufficient bound in Corollary 1, and a simple heuristic choice ε=0.01(qmin/mmax) =0.01(q_ /m_ ). The tolerance for the scale-distortion plots is η=0.05η=0.05. (a) Unshifted scale invariance. (b) FP distortion. (c) FN distortion. Figure 5: Scale invariance and finite-shift distortion of SPB Loss. (a) With ε=0 =0, the weighted loss is flat across q for each fixed multiplicative error, showing exact scale neutrality. (b)–(c) Introducing ε>0 >0 makes the loss finite at zero estimates but slightly perturbs the unshifted geometry. The observed distortion remains below the target tolerance η=0.05η=0.05 for both overestimation (FP) and underestimation (FN). (a) Corollary ε . (b) Heuristic ε . (c) Comparison with standard losses. Figure 6: Asymmetry control via α. (a)–(b) FN/FP loss ratios for reciprocal multiplicative errors under the Corollary-based and heuristic choices of ε . Ratios above one indicate that underestimation is penalized more strongly than overestimation. (c) At m=10m=10, SPB Loss provides a tunable continuum between approximately symmetric behavior and strongly FN-averse behavior, compared with log-space squared error and IS Loss. Scale invariance and shift distortion. Figure 5 examines whether SPB Loss treats the same relative error similarly across different rarity levels. In Panel 5(a), the unshifted weighted loss is flat as a function of q: for a fixed multiplicative error, the loss is essentially independent of whether the true probability is 10−510^-5 or 10−910^-9. This is the desired scale-neutral behavior. Panels 5(b) and 5(c) then show what changes when the finite shift ε>0 >0 is introduced. The shift is needed to keep the loss finite when an estimator outputs zero, but it slightly perturbs the ideal unshifted geometry. The plots report this perturbation separately for overestimation and underestimation. For both the Corollary-based shift and the heuristic shift, the observed distortion remains below the target tolerance η=0.05η=0.05 over the displayed range. Thus, in this regime, the finite correction preserves the practical scale-neutral behavior of the loss. Asymmetry control via α. Figure 6 studies the relative penalty for FN versus FP. Panels 6(a) and 6(b) plot the FN/FP loss ratio for reciprocal errors: a value above one means that underestimating q by a factor m is penalized more than overestimating q by the same factor. The curves show that α controls this asymmetry. Values above the threshold α=1.5α=1.5, such as α=1.6α=1.6 and α=2.5α=2.5, produce FN/FP ratios above one across the plotted range, consistent with Theorem 2. The α=0.5α=0.5 curve is included as a contrasting setting outside this FN-averse regime. Panel 6(c) compares this behavior with standard losses at a fixed multiplicative error m=10m=10. Log-space squared error is approximately symmetric between underestimation and overestimation, while IS Loss is FN-averse. SPB Loss interpolates between these behaviors through α: α≈1.5α≈ 1.5 is close to symmetric, while larger α increasingly penalizes underestimation. This gives SPB Loss an explicit asymmetry knob while retaining finiteness at zero estimates. 6 Related Work Low Probability Estimation in LLMs. Low probability estimation for language models was formalized by Wu and Hilton (2025), who introduced input-space Importance Sampling methods and the activation-space QLD baseline used in our experiments. We adopt their benchmark setting, but replace discrete input-space search with gradient-guided sampling in activation space and introduce SPB Loss to handle zero estimates and asymmetric safety costs. In a complementary line of work, Jones et al. (2025) address rare behavior estimation from a forecasting perspective rather than direct probability estimation. Their approach leverages elicitation probabilities defined as the probability that a given query produces a target behavior under repeated sampling, and uses extreme value theory to forecast how deployment-scale risks grow from smaller evaluation sets. Their method is particularly suited for anticipating when rare failures might emerge at scale, whereas our approach provides precise probability estimates for individual rare events. The two approaches are complementary: GA-AMLS can provide accurate point estimates of elicitation probabilities that could serve as inputs to the forecasting framework of Jones et al. (2025), potentially improving the precision of deployment-scale risk predictions. Recent work by Dorman et al. (2026) develops an end-to-end framework for rare event analysis over full text completions in the discrete token space, using MCMC-based trajectory sampling from tilted completion distributions. This requires repeated full-sequence generation and mixing in sequence space, whereas GA-AMLS operates in continuous activation space and avoids generating or scoring full completions. Another complementary line of work estimates rare harmful behaviors in stochastic language-model output distributions. Angell et al. (2026) estimate, for a fixed query, the probability that repeated sampling from a target model produces a harmful completion. They construct unsafe proposal models using activation steering and use importance sampling to reweight generated completions back to the target model, yielding substantial sample savings relative to naive Monte Carlo. This setting differs from ours in both the source of randomness and the event definition: they estimate behavior-level probabilities over full stochastic completions for fixed prompts, whereas we estimate single-token argmax probabilities over random inputs under deterministic decoding. Beyond trained transformers, Wu et al. (2026) study sample-free mechanistic estimation for wide random MLPs with Gaussian inputs. Their cumulant-propagation algorithms approximate activation distributions through fixed randomly initialized networks using cumulants and Hermite expansions, and can outperform Monte Carlo sampling, including on low-probability threshold events. This is complementary to our approach: their guarantees and experiments apply to analytically tractable random MLPs and closed-form input distributions, whereas we study trained transformer language models and use adaptive rare-event sampling in activation space. Activation-Space Methods. A growing interpretability literature supports treating language-model activations as a structured domain for analysis and intervention. The linear representation hypothesis (Park et al., 2024), representation engineering (Zou et al., 2023a), and activation steering methods (Turner et al., 2024; Panickssery et al., 2024) show that model behaviors can often be manipulated through directions in residual-stream space. Recent work further suggests that such behavioral structure need not be captured by a single global direction: heterogeneity-aware steering methods model activations as clustered distributions and use optimal transport to construct input-dependent interventions (Abdullaev et al., 2026). Related work finds low-dimensional structure for refusal behavior (Arditi et al., 2024), while sparse autoencoders reveal richer monosemantic feature structure in activations (Bricken et al., 2023; Templeton et al., 2024; Gao et al., 2024). The Tuned Lens (Belrose et al., 2025) further motivates operating near the pre-unembedding layer, where activations are predictive of output tokens. GA-AMLS builds on this view of activation space, but treats it as a probability space: instead of identifying causal directions, interventions, or features, we estimate the probability mass of rare-event regions. Rare-Event Sampling. Our method adapts Adaptive Multi-Level Splitting (AMLS), a rare-event Monte Carlo technique from statistical physics and particle simulation (Cérou et al., 2019). Webb et al. (2019) applied AMLS to neural-network robustness verification in computer vision using a random-walk Metropolis-Hastings kernel in input space. Directly applying this approach to language models is difficult because text is discrete. GA-AMLS instead moves the estimator to continuous activation space, where gradient-based MCMC kernels such as MALA can be used efficiently. Adversarial Search and Defenses. Red-teaming and jailbreaking methods search for individual inputs that trigger undesired behavior, using methods ranging from discrete gradient-based attacks such as GCG (Zou et al., 2023b) to activation-space attacks such as Subspace Rerouting (Winninger et al., 2025). These methods are useful for finding failures and generating adversarial training data (Goodfellow et al., 2015; Madry et al., 2019), including in latent adversarial training settings (Casper et al., 2024; Sheshadri et al., 2025). However, finding one failure is not the same as estimating its probability, and transfer across search methods can be unreliable (Kang et al., 2019; Wei et al., 2023). GA-AMLS is therefore diagnostic rather than defensive: it estimates how much probability mass lies in the rare-event region. 7 Discussion and Limitations While GA-AMLS provides a principled activation-space estimator for rare model behaviours, several limitations remain. Residual bias and activation-space approximation. GA-AMLS is designed as an unbiased rare-event estimator under the assumed activation prior and exact MCMC sampling. In practice, residual bias can arise from three sources. First, we approximate the true activation distribution with a fitted Student’s t distribution, which introduces distributional mismatch. Second, finite MCMC budgets may prevent full mixing between AMLS levels, leaving residual correlations among particles. Third, because GA-AMLS operates in continuous activation space, extreme activations found by the sampler may not correspond to any valid discrete token sequence. We partially mitigate this activation pre-image issue by fitting the prior to activations induced by samples from the input distribution D, encouraging samples to remain near the empirical activation manifold. However, the gap between continuous activations and discrete inputs remains a fundamental limitation of activation-space methods, including QLD. Computational cost. GA-AMLS has a different computational profile from input-space importance sampling. It avoids full-model backward passes and operates from cached activations, but it requires many sequential activation-space scorer evaluations across AMLS levels. GA-AMLS is computationally heavier than QLD: QLD reduces estimation to fast closed-form matrix operations, whereas GA-AMLS requires sequential Markov-chain evolution across adaptive levels. This additional cost is the price of replacing QLD’s independence-based recombination approximation with conditional sampling under a fitted activation prior. It may therefore limit very large sweeps over many target tokens, layers, or models. Evaluation scope. For comparability with prior low-probability estimation work, our experiments follow the benchmark setting of Wu and Hilton (2025): the input distribution D consists of independent tokens, and rare events are defined by a single target token under deterministic argmax conditions. This controlled setting enables direct comparison with ITGIS, MHIS, and QLD, but it does not capture correlated natural prompts, instruction-following contexts, or multi-step autoregressive failures. Extending activation-space rare-event estimation to realistic prompt distributions and trajectory-level events is an important direction for future work. Finally, all of our evidence comes from the 11–44 layer, d=512d=512 transformers of this benchmark; whether the bias–variance profile of GA-AMLS, and the adequacy of a diagonal Student-t prior in whitened coordinates, persist at modern scale is untested. 8 Conclusion We introduced GA-AMLS, an activation-space adaptive multilevel splitting method for estimating rare LLM behaviors that are infeasible to measure accurately with naive sampling. Empirically, GA-AMLS reduces zero-estimate failures and under symmetric penalties improves rare-probability estimation relative to baselines. We also proposed SPB Loss to evaluate rare-event estimates under asymmetric costs for underestimation and overestimation. While the method depends on the fidelity of the activation-space model, the results support activation-space rare-event simulation as a useful tool for measuring low-probability LLM behavior. Acknowledgement This project is partially supported by Coefficient Giving. References L. U. Abdullaev, N. Y. L. Wong, R. T. Z. Lee, S. Jiang, K. N. M. Nguyen, and T. M. Nguyen (2026) Concept heterogeneity-aware representation steering. External Links: 2603.02237, Link Cited by: §6. R. Angell, R. Singhal, Z. Horvitz, Z. Yu, R. Ranganath, K. McKeown, and H. He (2026) Estimating tail risks in language model output distributions. External Links: 2604.22167, Link Cited by: §6. A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda (2024) Refusal in language models is mediated by a single direction. External Links: 2406.11717, Link Cited by: §6. N. Belrose, I. Ostrovsky, L. McKinney, Z. Furman, L. Smith, D. Halawi, S. Biderman, and J. Steinhardt (2025) Eliciting latent predictions from transformers with the tuned lens. External Links: 2303.08112, Link Cited by: §6. T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. L. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah (2023) Towards monosemanticity: decomposing language models with dictionary learning. Note: Transformer Circuits Thread External Links: Link Cited by: §6. A. Buja, W. Stuetzle, and Y. Shen (2005) Loss functions for binary class probability estimation and classification: structure and applications. Working draft University of Pennsylvania, The Wharton School. Cited by: §4. S. Casper, L. Schulze, O. Patel, and D. Hadfield-Menell (2024) Defending Against Unforeseen Failure Modes with Latent Adversarial Training. arXiv. Note: arXiv:2403.05030 [cs] External Links: Link, Document Cited by: §6. F. Cérou, A. Guyader, and M. Rousset (2019) Adaptive multilevel splitting: Historical perspective and recent results. Chaos: An Interdisciplinary Journal of Nonlinear Science 29 (4), p. 043108 (en). External Links: ISSN 1054-1500, 1089-7682, Link, Document Cited by: §1, §2.2, §6. T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer (2022) LLM.int8(): 8-bit matrix multiplication for transformers at scale. External Links: 2208.07339, Link Cited by: §1, §3. J. M. Dorman, E. Gillman, D. C. Rose, J. F. Mair, and J. P. Garrahan (2026) Rare event analysis of large language models. External Links: 2602.06791, Link Cited by: §6. L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu (2024) Scaling and evaluating sparse autoencoders. External Links: 2406.04093, Link Cited by: §6. T. Gneiting and A. E. Raftery (2007) Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102 (477), p. 359–378 (en). External Links: ISSN 0162-1459, 1537-274X, Link, Document Cited by: §4. I. J. Goodfellow, J. Shlens, and C. Szegedy (2015) Explaining and harnessing adversarial examples. External Links: 1412.6572, Link Cited by: §6. A. Guyader, N. Hengartner, and E. Matzner-Løber (2011) Simulation and estimation of extreme quantiles and extreme probabilities. Applied Mathematics & Optimization 64, p. 171–196. External Links: Document Cited by: §1, §2.2. D. Hendrycks and K. Gimpel (2023) Gaussian error linear units (gelus). External Links: 1606.08415, Link Cited by: §5.1. E. Jones, M. Tong, J. Mu, M. Mahfoud, J. Leike, R. Grosse, J. Kaplan, W. Fithian, E. Perez, and M. Sharma (2025) Forecasting rare language model behaviors. External Links: 2502.16797, Link Cited by: §6. D. Kang, Y. Sun, T. Brown, D. Hendrycks, and J. Steinhardt (2019) Transfer of adversarial robustness between perturbation types. External Links: 1905.01034, Link Cited by: §6. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu (2019) Towards deep learning models resistant to adversarial attacks. External Links: 1706.06083, Link Cited by: §6. N. Nanda and J. Bloom (2022) TransformerLens. Note: https://github.com/TransformerLensOrg/TransformerLens Cited by: §5.1. N. Panickssery, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner (2024) Steering llama 2 via contrastive activation addition. External Links: 2312.06681, Link Cited by: §6. K. Park, Y. J. Choe, and V. Veitch (2024) The linear representation hypothesis and the geometry of large language models. External Links: 2311.03658, Link Cited by: §6. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2023) Exploring the limits of transfer learning with a unified text-to-text transformer. External Links: 1910.10683, Link Cited by: §5.1. G. O. Roberts and J. S. Rosenthal (1998) Optimal Scaling of Discrete Approximations to Langevin Diffusions. Journal of the Royal Statistical Society Series B: Statistical Methodology 60 (1), p. 255–268 (en). External Links: ISSN 1369-7412, 1467-9868, Link, Document Cited by: §3. G. O. Roberts and R. L. Tweedie (1996) Exponential convergence of langevin distributions and their discrete approximations. Bernoulli 2 (4), p. 341–363. External Links: ISSN 13507265, Link Cited by: §1, §2.2. R. Shah, V. Varma, R. Kumar, M. Phuong, V. Krakovna, J. Uesato, and Z. Kenton (2022) Goal misgeneralization: why correct specifications aren’t enough for correct goals. External Links: 2210.01790, Link Cited by: §1. A. Sheshadri, A. Ewart, P. Guo, A. Lynch, C. Wu, V. Hebbar, H. Sleight, A. C. Stickland, E. Perez, D. Hadfield-Menell, and S. Casper (2025) Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs. arXiv. Note: arXiv:2407.15549 [cs] External Links: Link, Document Cited by: §6. V. Srinivasan, A. Wibisono, and A. Wilson (2025) High-accuracy sampling from constrained spaces with the Metropolis-adjusted Preconditioned Langevin Algorithm. arXiv. Note: arXiv:2412.18701 [stat] External Links: Link, Document Cited by: §D.1, §1, §2.2. M. Sun, X. Chen, J. Z. Kolter, and Z. Liu (2024) Massive activations in large language models. External Links: 2402.17762, Link Cited by: §1, §3. A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, C. McDougall, M. MacDiarmid, A. Tamkin, E. Durmus, T. Hume, F. Mosconi, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan (2024) Scaling monosemanticity: extracting interpretable features from claude 3 sonnet. Note: Transformer Circuits Thread External Links: Link Cited by: §6. L. Tunstall, L. von Werra, and T. Wolf (2022) Natural language processing with transformers: building language applications with hugging face. O’Reilly Media. External Links: ISBN 9781098103248, LCCN 2023275986, Link Cited by: §5.1. A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid (2024) Steering language models with activation engineering. External Links: 2308.10248, Link Cited by: §6. S. Webb, T. Rainforth, Y. W. Teh, and M. P. Kumar (2019) A Statistical Approach to Assessing Neural Network Robustness. arXiv. Note: arXiv:1811.07209 [stat] External Links: Link, Document Cited by: §1, §2.2, §6. A. Wei, N. Haghtalab, and J. Steinhardt (2023) Jailbroken: how does llm safety training fail?. External Links: 2307.02483, Link Cited by: §6. T. Winninger, B. Addad, and K. Kapusta (2025) Using mechanistic interpretability to craft adversarial attacks against large language models. External Links: 2503.06269, Link Cited by: §6. G. Wu and J. Hilton (2025) Estimating the probabilities of rare outputs in language models. External Links: 2410.13211, Link Cited by: §D.1, §D.1, §1, §1, §1, §2.1, §2.1, §4, §4, §4, §5.1, §5.1, §5, §6, §7. W. Wu, V. Lecomte, M. Winer, G. Robinson, J. Hilton, and P. Christiano (2026) Estimating the expected output of wide random mlps more efficiently than sampling. External Links: 2605.05179, Link Cited by: §6. A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks (2023a) Representation engineering: a top-down approach to ai transparency. arXiv preprint arXiv:2310.01405. External Links: 2310.01405, Link Cited by: §6. A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson (2023b) Universal and transferable adversarial attacks on aligned language models. External Links: 2307.15043, Link Cited by: §6. Appendix outline. Appendix A develops the SPB loss theory from its Bregman-divergence representation, derives the unshifted power family, and analyzes the resulting false-negative/false-positive asymmetry. Appendix B gives the shifted-power parameterization used in the paper, proves bounded scale distortion, explains how to choose ε , α, and γ, provides a closed-form computation, connects SPB to standard losses, and empirically verifies the intended scaling behavior. Appendix C gives the full GA-AMLS procedure in Algorithm 1. Appendix D reports experimental and implementation details, including hyperparameters, baseline implementation notes, hyperparameter sweeps and ablations, and the activation-whitening diagnostic. Appendix E provides the full set of estimate versus ground-truth scatterplots across model sizes and input distributions. Appendix F breaks down SPB losses by distribution, model size, method, and evaluation profile. Code Availability. The code is available upon request currently and will be made public soon. Appendix A SPB Loss Theory A.1 Bregman integral representation Let q∈(0,1)q∈(0,1) denote the true probability and p∈[0,1]p∈[0,1] the predicted probability. A one-dimensional Bregman divergence generated by a twice differentiable convex potential with second derivative ω(t)>0ω(t)>0 can be written as Bω(q∣p)=∫qp(t−q)ω(t)t.B_ω(q p)= _q^p(t-q)ω(t)\,dt. (1) This is an oriented integral. Equivalently, Bω(q∣p)=∫qp(t−q)ω(t)t,p≥q,∫pq(q−t)ω(t)t,p<q.B_ω(q p)= cases _q^p(t-q)ω(t)\,dt,&p≥ q,\\[8.0pt] _p^q(q-t)ω(t)\,dt,&p<q. cases Thus the divergence is nonnegative, and if ω(t)>0ω(t)>0 almost everywhere, then Bω(q∣p)=0B_ω(q p)=0 if and only if p=qp=q, whenever the integral is finite. A.2 The unshifted power family Consider the unshifted power weight ω0(t)=t−α,α>0. _0(t)=t^-α, α>0. (2) For a multiplicative overestimate p=mqp=mq, with m>1m>1, the change of variables t=qut=qu gives B0(q∣mq) B_0(q mq) =∫qmq(t−q)t−αt = _q^mq(t-q)t^-α\,dt =q2−α∫1m(u−1)u−αu =q^2-α _1^m(u-1)u^-α\,du =q2−αIFP(α,m), =q^2-αI_FP(α,m), (3) where IFP(α,m):=∫1m(u−1)u−αu.I_FP(α,m):= _1^m(u-1)u^-α\,du. (4) Similarly, for a multiplicative underestimate p=q/mp=q/m, B0(q∣q/m) B_0(q q/m) =∫q/mq(q−t)t−αt = _q/m^q(q-t)t^-α\,dt =q2−α∫1/m1(1−u)u−αu =q^2-α _1/m^1(1-u)u^-α\,du =q2−αIFN(α,m), =q^2-αI_FN(α,m), (5) where IFN(α,m):=∫1/m1(1−u)u−αu.I_FN(α,m):= _1/m^1(1-u)u^-α\,du. (6) Therefore both false-positive and false-negative multiplicative errors factor into a ground-truth scale term q2−αq^2-α and a relative-error term. The dataset-level weight w(q)=qα−2w(q)=q^α-2 (7) cancels the q2−αq^2-α prefactor exactly: w(q)B0(q∣mq)=IFP(α,m),w(q)B0(q∣q/m)=IFN(α,m).w(q)B_0(q mq)=I_FP(α,m), w(q)B_0(q q/m)=I_FN(α,m). Thus the weighted unshifted power family is exactly multiplicatively scale-invariant: for fixed relative error m, the weighted loss is independent of the absolute scale q. More generally, if one wants to add an explicit rarity premium γ≥0γ≥ 0, one can use wγ(q)=qα−2−γ,w_γ(q)=q^α-2-γ, in which case the weighted unshifted loss scales as q−γq^-γ times the relative-error term. The formal scale-invariance results below correspond to the scale-neutral case γ=0γ=0. A.3 Unshifted FN/FP asymmetry Define the unshifted FN-to-FP ratio Rα(m):=IFN(α,m)IFP(α,m).R_α(m):= I_FN(α,m)I_FP(α,m). (8) The following lemma shows that for α>3/2α>3/2, underestimation is penalized more than overestimation, and the asymmetry increases with the multiplicative error size. Lemma 1. For α>3/2α>3/2, Rα(m)R_α(m) is strictly increasing on (1,∞)(1,∞). Moreover, Rα(m)>1R_α(m)>1 for all m>1m>1. Proof. Starting from the definition of IFNI_FN, apply the substitution u=1/vu=1/v. Then du=−v−2dvdu=-v^-2\,dv, and as u goes from 1/m1/m to 11, v goes from m to 11. Hence IFN(α,m) I_FN(α,m) =∫1/m1(1−u)u−αu = _1/m^1(1-u)u^-α\,du =∫m1(1−1v)vα(−v−2)v = _m^1 (1- 1v )v^α(-v^-2)\,dv =∫1m(1−1v)vα−2v = _1^m (1- 1v )v^α-2\,dv =∫1m(v−1)vα−3v. = _1^m(v-1)v^α-3\,dv. Renaming v as u, IFN(α,m)=∫1m(u−1)uα−3u=∫1m(u−1)u−αu2α−3u.I_FN(α,m)= _1^m(u-1)u^α-3\,du= _1^m(u-1)u^-αu^2α-3\,du. Let f(u):=(u−1)u−α,g(u):=u2α−3.f(u):=(u-1)u^-α, g(u):=u^2α-3. Then f(u)>0f(u)>0 for u>1u>1, and because α>3/2α>3/2, g(u)g(u) is strictly increasing on (1,∞)(1,∞). Moreover, IFP(α,m)=∫1mf(u)u,IFN(α,m)=∫1mf(u)g(u)u.I_FP(α,m)= _1^mf(u)\,du, I_FN(α,m)= _1^mf(u)g(u)\,du. Thus Rα(m)=∫1mf(u)g(u)u∫1mf(u)u,R_α(m)= _1^mf(u)g(u)\,du _1^mf(u)\,du, so Rα(m)R_α(m) is the f-weighted average of the increasing function g over [1,m][1,m]. To prove strict monotonicity, define A(m):=∫1mf(u)g(u)u,C(m):=∫1mf(u)u.A(m):= _1^mf(u)g(u)\,du, C(m):= _1^mf(u)\,du. Then Rα(m)=A(m)/C(m)R_α(m)=A(m)/C(m), and A′(m)=f(m)g(m),C′(m)=f(m).A (m)=f(m)g(m), C (m)=f(m). Therefore Rα′(m) R_α (m) =A′(m)C(m)−A(m)C′(m)C(m)2 = A (m)C(m)-A(m)C (m)C(m)^2 =f(m)C(m)[g(m)−A(m)C(m)] = f(m)C(m) [g(m)- A(m)C(m) ] =f(m)C(m)[g(m)−Rα(m)]. = f(m)C(m) [g(m)-R_α(m) ]. Since g is strictly increasing, its weighted average over [1,m][1,m] is strictly smaller than its endpoint value g(m)g(m). Hence g(m)−Rα(m)>0.g(m)-R_α(m)>0. Also f(m)>0f(m)>0 and C(m)>0C(m)>0 for m>1m>1. Therefore Rα′(m)>0.R_α (m)>0. So RαR_α is strictly increasing on (1,∞)(1,∞). Finally, because g(u)=u2α−3>1g(u)=u^2α-3>1 for all u>1u>1, its weighted average is also greater than 11. Hence Rα(m)>1R_α(m)>1 for all m>1m>1. ∎ At the boundary α=3/2α=3/2, we have g(u)≡1g(u)≡ 1, so R3/2(m)=1R_3/2(m)=1 for all m>1m>1. Thus α=3/2α=3/2 is the symmetric point, while α>3/2α>3/2 makes false negatives more costly than false positives; the mirrored argument, with g strictly decreasing, shows that α<3/2α<3/2 makes false positives more costly. Appendix B SPB Loss Guarantees and Parameterization B.1 The shifted power family The unshifted power weight t−αt^-α gives exact scale invariance after dataset weighting, but it is singular at t=0t=0. In particular, for α≥1α≥ 1, predicting p=0p=0 can produce an infinite penalty. To make the loss finite at zero, define the shifted power weight ωε(t):=(t+ε)−α,α>0,ε>0. _ (t):=(t+ )^-α, α>0, >0. (9) The corresponding shifted divergence is Bε(q∣p)=∫qp(t−q)(t+ε)−αt.B_ (q p)= _q^p(t-q)(t+ )^-α\,dt. (10) Because (t+ε)−α≤ε−α<∞(t+ )^-α≤ ^-α<∞ on [0,1][0,1], the shifted divergence is finite even when p=0p=0. The price of this shift is that exact multiplicative scale invariance is lost: after the change of variables t=qut=qu, the ratio x:=εqx:= q remains inside the integral. The following results show that the desired properties are preserved on a finite evaluation box q∈[qmin,qmax],m∈(1,mmax],q∈[q_ ,q_ ], m∈(1,m_ ], provided ε is sufficiently small relative to qminq_ . B.2 Bounded scale distortion Theorem 1 (Bounded Scale Distortion). Fix α>0α>0, η∈(0,1)η∈(0,1), and mmax>1m_ >1. Let w(q)=qα−2w(q)=q^α-2. If ε≤qminmmax((1−η)−1/α−1), ≤ q_ m_ ((1-η)^-1/α-1 ), (11) then, uniformly over q∈[qmin,qmax]q∈[q_ ,q_ ] and m∈(1,mmax]m∈(1,m_ ], 0≤IFP(α,m)−w(q)Bε(q∣mq)IFP(α,m) 0≤ I_FP(α,m)-w(q)B_ (q mq)I_FP(α,m) ≤η, ≤η, (12) 0≤IFN(α,m)−w(q)Bε(q∣q/m)IFN(α,m) 0≤ I_FN(α,m)-w(q)B_ (q q/m)I_FN(α,m) ≤η. ≤η. (13) Proof. Fix q∈[qmin,qmax]q∈[q_ ,q_ ] and m∈(1,mmax]m∈(1,m_ ], and define x:=εq.x:= q. First consider the overestimate p=mqp=mq. Using t=qut=qu, Bε(q∣mq) B_ (q mq) =∫qmq(t−q)(t+ε)−αt = _q^mq(t-q)(t+ )^-α\,dt =q2−α∫1m(u−1)(u+x)−αu =q^2-α _1^m(u-1)(u+x)^-α\,du =q2−αIFP,ε(α,m;x), =q^2-αI_FP, (α,m;x), (14) where IFP,ε(α,m;x):=∫1m(u−1)(u+x)−αu.I_FP, (α,m;x):= _1^m(u-1)(u+x)^-α\,du. For u∈[1,m]u∈[1,m], u≤u+x≤u(1+x).u≤ u+x≤ u(1+x). Since α>0α>0, raising to the power −α-α reverses the inequalities: u−α≥(u+x)−α≥u−α(1+x)−α.u^-α≥(u+x)^-α≥ u^-α(1+x)^-α. Multiplying by (u−1)≥0(u-1)≥ 0 and integrating gives (1+x)−αIFP(α,m)≤IFP,ε(α,m;x)≤IFP(α,m).(1+x)^-αI_FP(α,m)≤ I_FP, (α,m;x)≤ I_FP(α,m). (15) Therefore 0≤IFP(α,m)−IFP,ε(α,m;x)IFP(α,m)≤1−(1+x)−α.0≤ I_FP(α,m)-I_FP, (α,m;x)I_FP(α,m)≤ 1-(1+x)^-α. (16) Now consider the underestimate p=q/mp=q/m. Again using t=qut=qu, Bε(q∣q/m) B_ (q q/m) =∫q/mq(q−t)(t+ε)−αt = _q/m^q(q-t)(t+ )^-α\,dt =q2−α∫1/m1(1−u)(u+x)−αu =q^2-α _1/m^1(1-u)(u+x)^-α\,du =q2−αIFN,ε(α,m;x), =q^2-αI_FN, (α,m;x), (17) where IFN,ε(α,m;x):=∫1/m1(1−u)(u+x)−αu.I_FN, (α,m;x):= _1/m^1(1-u)(u+x)^-α\,du. For u∈[1/m,1]u∈[1/m,1], we have 1/u≤m1/u≤ m. Hence u+x=u(1+xu)≤u(1+mx),u+x=u (1+ xu )≤ u(1+mx), and also u+x≥u+x≥ u. Therefore u≤u+x≤u(1+mx).u≤ u+x≤ u(1+mx). Raising to the power −α-α gives u−α≥(u+x)−α≥u−α(1+mx)−α.u^-α≥(u+x)^-α≥ u^-α(1+mx)^-α. Multiplying by (1−u)≥0(1-u)≥ 0 and integrating gives (1+mx)−αIFN(α,m)≤IFN,ε(α,m;x)≤IFN(α,m).(1+mx)^-αI_FN(α,m)≤ I_FN, (α,m;x)≤ I_FN(α,m). (18) Therefore 0≤IFN(α,m)−IFN,ε(α,m;x)IFN(α,m)≤1−(1+mx)−α.0≤ I_FN(α,m)-I_FN, (α,m;x)I_FN(α,m)≤ 1-(1+mx)^-α. (19) Now assume (11). Since q≥qminq≥ q_ and m≤mmaxm≤ m_ , mx=mεq≤mmaxεqmin≤(1−η)−1/α−1.mx= m q≤ m_ q_ ≤(1-η)^-1/α-1. Thus 1+mx≤(1−η)−1/α.1+mx≤(1-η)^-1/α. Equivalently, (1+mx)−α≥1−η,(1+mx)^-α≥ 1-η, so 1−(1+mx)−α≤η.1-(1+mx)^-α≤η. This proves the FN distortion bound. Since x≤mx≤ mx, the same condition also implies 1−(1+x)−α≤η,1-(1+x)^-α≤η, and therefore proves the FP distortion bound. Finally, because w(q)=qα−2w(q)=q^α-2, w(q)Bε(q∣mq)=IFP,ε(α,m;x),w(q)B_ (q mq)=I_FP, (α,m;x), and w(q)Bε(q∣q/m)=IFN,ε(α,m;x).w(q)B_ (q q/m)=I_FN, (α,m;x). Substituting these identities into the relative distortion bounds proves the claim. ∎ B.3 Asymmetric sensitivity Theorem 2 (Asymmetric Sensitivity). Fix α>3/2α>3/2 and 1<m0≤mmax1<m_0≤ m_ . Let Rα(m):=IFN(α,m)IFP(α,m).R_α(m):= I_FN(α,m)I_FP(α,m). If ε<qminmmax[Rα(m0)1/α−1], < q_ m_ [R_α(m_0)^1/α-1 ], (20) then Bε(q∣q/m)>Bε(q∣mq)B_ (q q/m)>B_ (q mq) (21) for all q∈[qmin,qmax]q∈[q_ ,q_ ] and all m∈[m0,mmax]m∈[m_0,m_ ]. Proof. Fix q∈[qmin,qmax]q∈[q_ ,q_ ] and m∈[m0,mmax]m∈[m_0,m_ ], and define x:=εq.x:= q. For the overestimate p=mqp=mq, Bε(q∣mq)=q2−αIFP,ε(α,m;x),B_ (q mq)=q^2-αI_FP, (α,m;x), where IFP,ε(α,m;x)=∫1m(u−1)(u+x)−αu.I_FP, (α,m;x)= _1^m(u-1)(u+x)^-α\,du. Since x≥0x≥ 0 and α>0α>0, (u+x)−α≤u−α.(u+x)^-α≤ u^-α. Therefore IFP,ε(α,m;x)≤IFP(α,m).I_FP, (α,m;x)≤ I_FP(α,m). (22) For the underestimate p=q/mp=q/m, Bε(q∣q/m)=q2−αIFN,ε(α,m;x),B_ (q q/m)=q^2-αI_FN, (α,m;x), where IFN,ε(α,m;x)=∫1/m1(1−u)(u+x)−αu.I_FN, (α,m;x)= _1/m^1(1-u)(u+x)^-α\,du. For u∈[1/m,1]u∈[1/m,1], we have u+x=u(1+xu)≤u(1+mx).u+x=u (1+ xu )≤ u(1+mx). Hence (u+x)−α≥u−α(1+mx)−α.(u+x)^-α≥ u^-α(1+mx)^-α. Multiplying by (1−u)≥0(1-u)≥ 0 and integrating gives IFN,ε(α,m;x)≥(1+mx)−αIFN(α,m).I_FN, (α,m;x)≥(1+mx)^-αI_FN(α,m). (23) It is therefore enough to ensure (1+mx)−αIFN(α,m)>IFP(α,m).(1+mx)^-αI_FN(α,m)>I_FP(α,m). Equivalently, using Rα(m)=IFN(α,m)IFP(α,m),R_α(m)= I_FN(α,m)I_FP(α,m), it is enough to have (1+mx)α<Rα(m),(1+mx)^α<R_α(m), or 1+mx<Rα(m)1/α.1+mx<R_α(m)^1/α. By Lemma 1, Rα(m)R_α(m) is strictly increasing in m for α>3/2α>3/2. Since m≥m0m≥ m_0, Rα(m)≥Rα(m0).R_α(m)≥ R_α(m_0). Also, since m≤mmaxm≤ m_ and q≥qminq≥ q_ , mx=mεq≤mmaxεqmin.mx= m q≤ m_ q_ . The assumed condition (20) implies 1+mmaxεqmin<Rα(m0)1/α.1+ m_ q_ <R_α(m_0)^1/α. Therefore 1+mx<Rα(m0)1/α≤Rα(m)1/α.1+mx<R_α(m_0)^1/α≤ R_α(m)^1/α. Hence (1+mx)−αIFN(α,m)>IFP(α,m).(1+mx)^-αI_FN(α,m)>I_FP(α,m). Combining this strict inequality with (22) and (23), we obtain IFN,ε(α,m;x)>IFP,ε(α,m;x).I_FN, (α,m;x)>I_FP, (α,m;x). Multiplying by the positive factor q2−αq^2-α gives Bε(q∣q/m)>Bε(q∣mq).B_ (q q/m)>B_ (q mq). This proves the theorem. ∎ B.4 Simultaneous finite-error guarantee Corollary 1 (Simultaneous finite-error guarantee). Fix α>3/2α>3/2, η∈(0,1)η∈(0,1), and 1<m0≤m≤mmax.1<m_0≤ m≤ m_ . Let Rα(m):=IFN(α,m)IFP(α,m).R_α(m):= I_FN(α,m)I_FP(α,m). If 0≤ε<minqminmmax((1−η)−1/α−1),qminmmax[Rα(m0)1/α−1],0≤ < \ q_ m_ ((1-η)^-1/α-1 ),\; q_ m_ [R_α(m_0)^1/α-1 ] \, (24) then the shifted weighted loss simultaneously satisfies: 1. approximate scale invariance within relative tolerance η: 0≤IFP(α,m)−w(q)Bε(q∣mq)IFP(α,m)≤η,0≤ I_FP(α,m)-w(q)B_ (q mq)I_FP(α,m)≤η, and 0≤IFN(α,m)−w(q)Bε(q∣q/m)IFN(α,m)≤η;0≤ I_FN(α,m)-w(q)B_ (q q/m)I_FN(α,m)≤η; 2. finite-error asymmetric sensitivity: Bε(q∣q/m)>Bε(q∣mq).B_ (q q/m)>B_ (q mq). Both guarantees hold uniformly for q∈[qmin,qmax]q∈[q_ ,q_ ] and m∈[m0,mmax]m∈[m_0,m_ ]. Proof. The first term in the minimum in (24) is exactly the sufficient condition of Theorem 1. The second term is exactly the sufficient condition of Theorem 2. Therefore both conclusions hold simultaneously. ∎ B.5 Interpretation of the shift bound Setting ε=0 =0 gives exact multiplicative scale invariance and, for α>3/2α>3/2, strict FN-over-FP asymmetry. However, the unshifted loss can be infinite when a model predicts p=0p=0. Introducing ε>0 >0 removes this singularity, but it also introduces an additive floor into the local weight (t+ε)−α(t+ )^-α. If ε is too large relative to qminq_ , this floor overwhelms the relevant probability scale and destroys both approximate scale invariance and asymmetric sensitivity. The bounds above quantify the permissible size of ε . They are sufficient rather than necessary, and are intentionally conservative. A simple practical choice is to set ε=cqminmmax,c≪1, =c q_ m_ , c 1, and then verify that this value lies below the theorem-specific upper bounds. B.6 Parameter Guidance Table 2 summarizes suggested (α,γ)(α,γ) settings for the evaluation use cases referenced in Section 4.1. The parameter γ controls any additional dataset-level rarity premium, while α controls the asymmetry between reciprocal overestimation and underestimation. We now describe a more fine-grained way to choose α by specifying a desired worst-case FN/FP ratio. B.6.1 Choosing α via a worst-case FN/FP ratio A natural way to calibrate the asymmetry parameter α is to specify a desired lower bound r⋆>1r_ >1 on the cost of reciprocal underestimation relative to reciprocal overestimation. In the unshifted geometry, B0(q∣q/m)B0(q∣mq)=IFN(α,m)IFP(α,m)=Rα(m), B_0(q q/m)B_0(q mq)= I_FN(α,m)I_FP(α,m)=R_α(m), so this ratio is independent of the base probability q. The same is true after multiplying both losses by any dataset-level weight depending only on q, since the weight cancels in the ratio. Such a guarantee cannot hold uniformly as m↓1m 1. Indeed, for every fixed α, limm↓1Rα(m)=1. _m 1R_α(m)=1. Thus no finite α can guarantee Rα(m)≥r⋆>1R_α(m)≥ r_ >1 for arbitrarily small multiplicative errors. A meaningful worst-case calibration must therefore specify a minimum multiplicative error m0>1m_0>1. Fix m0>1m_0>1 and mmax≥m0m_ ≥ m_0. We require Rα(m)≥r⋆∀m∈[m0,mmax].R_α(m)≥ r_ ∀ m∈[m_0,m_ ]. For r⋆>1r_ >1, any feasible choice must have α>3/2α>3/2. By Lemma 1, Rα(m)R_α(m) is then increasing in m, so the worst case occurs at m=m0m=m_0. Therefore, in the unshifted geometry, the uniform calibration condition over m∈[m0,mmax]m∈[m_0,m_ ] is equivalent to Rα(m0)≥r⋆.R_α(m_0)≥ r_ . Using the identity IFN(α,m)=IFP(3−α,m)I_FN(α,m)=I_FP(3-α,m), we can write Rα(m)=IFP(3−α,m)IFP(α,m).R_α(m)= I_FP(3-α,m)I_FP(α,m). Here IFP(β,m)=∫1m(u−1)u−βu=m2−β−12−β−m1−β−11−β,β∉1,2,m−1−logm,β=1,logm+1m−1,β=2.I_FP(β,m)= _1^m(u-1)u^-β\,du= cases m^2-β-12-β- m^1-β-11-β,&β∉\1,2\,\\[12.50002pt] m-1- m,&β=1,\\[5.0pt] m+ 1m-1,&β=2. cases For fixed m0>1m_0>1, the map α↦Rα(m0)α R_α(m_0) is strictly increasing. Indeed, defining F(β,m)=IFP(β,m),F(β,m)=I_FP(β,m), we have ∂βF(β,m)=−∫1m(u−1)u−βlogudu<0. _βF(β,m)=- _1^m(u-1)u^-β u\,du<0. Thus F(β,m)F(β,m) is strictly decreasing in β, and Rα(m0)=F(3−α,m0)F(α,m0)R_α(m_0)= F(3-α,m_0)F(α,m_0) is strictly increasing in α. Moreover, R3/2(m0)=1,limα→∞Rα(m0)=∞.R_3/2(m_0)=1, _α→∞R_α(m_0)=∞. Hence for every r⋆>1r_ >1 there is a unique threshold α⋆=α⋆(m0,r⋆)>3/2 _ = _ (m_0,r_ )>3/2 satisfying Rα⋆(m0)=r⋆.R_ _ (m_0)=r_ . Choosing α≥α⋆α≥ _ guarantees minm∈[m0,mmax]Rα(m)≥r⋆ _m∈[m_0,m_ ]R_α(m)≥ r_ for the unshifted loss. Computing α⋆ _ . The threshold α⋆ _ generally has no closed-form inverse, but it can be computed by bisection because α↦Rα(m0)α R_α(m_0) is continuous and strictly increasing. A practical procedure is: 1. Choose the minimum relevant multiplicative error m0>1m_0>1, the desired worst-case ratio r⋆>1r_ >1, and a numerical tolerance τ>0τ>0. 2. Set αlo=32. _lo= 32. Then Rαlo(m0)=1<r⋆.R_ _lo(m_0)=1<r_ . 3. Choose an initial upper bracket αhi>3/2 _hi>3/2. Increase it, for example by repeatedly doubling,αhi←2αhi, _hi← 2 _hi, until Rαhi(m0)≥r⋆.R_ _hi(m_0)≥ r_ . 4. While αhi−αlo>τ, _hi- _lo>τ, set αmid=αlo+αhi2. _mid= _lo+ _hi2. If Rαmid(m0)<r⋆,R_ _mid(m_0)<r_ , set αlo←αmid. _lo← _mid. Otherwise set αhi←αmid. _hi← _mid. 5. Return αhi _hi. This gives a conservative numerical choice satisfying Rαhi(m0)≥r⋆.R_ _hi(m_0)≥ r_ . Effect of the shift. For the shifted loss, the exact unshifted ratio no longer holds. However, the proof of Theorem 2 gives the conservative lower bound Bε(q∣q/m)Bε(q∣mq)≥(1+mε/q)−αRα(m). B_ (q q/m)B_ (q mq)≥(1+m /q)^-αR_α(m). Therefore, uniformly over q∈[qmin,qmax]q∈[q_ ,q_ ] and m∈[m0,mmax]m∈[m_0,m_ ], Bε(q∣q/m)Bε(q∣mq)≥(1+mmaxεqmin)−αRα(m0). B_ (q q/m)B_ (q mq)≥ (1+ m_ q_ )^-αR_α(m_0). Thus a sufficient condition for the shifted loss to achieve worst-case ratio at least r⋆r_ is (1+mmaxεqmin)−αRα(m0)≥r⋆. (1+ m_ q_ )^-αR_α(m_0)≥ r_ . Equivalently, after choosing α, it is sufficient to choose ε≤qminmmax[(Rα(m0)r⋆)1/α−1]. ≤ q_ m_ [ ( R_α(m_0)r_ )^1/α-1 ]. If a strict ratio >r⋆>r_ is desired, use a strict inequality. This bound is meaningful for positive ε only when Rα(m0)>r⋆.R_α(m_0)>r_ . Consequently, if one intends to use a positive shift ε>0 >0, one should choose α with some margin above the unshifted threshold α⋆ _ , rather than choosing exactly α=α⋆α= _ . Theorem 2 corresponds to the special case r⋆=1r_ =1. Use case What you want the metric to emphasize α γ Unbiased method benchmarking Scale-neutral comparison across a wide ground truth range where FN and FP treated approximately symmetrically. 1.51.5 0 Product safety oversight Mild FN aversion (similar to Itakura-Saito Loss) but avoid excessive false alarms; performance should still matter across the full ground truth range. 1.71.7–2.22.2 0 Deep-tail stress testing (tail-focused but not fully worst-case) Put substantially more weight on the smallest q examples (rarest events), while still comparing methods on estimation quality (not just conservatism). 1.51.5–2.52.5 0.50.5–11 Catastrophic-risk audit Extreme-tail priority and willing to tolerate many false alarms if it reduces underestimation in the deepest tail. 33–66 11–22 Table 2: Suggested shifted-power metric parameters by evaluation use case. Here α controls FN-vs-FP asymmetry (higher α penalizes underestimation more), and γ controls dataset-level rarity premium (higher γ upweights smaller ground-truth probabilities q). B.7 Computation: closed form Closed-form Bε(q∣p)B_ (q p). For α≠1,2α≠ 1,2, the shifted power family admits the closed form Bε(q∣p)=(p+ε)2−α−(q+ε)2−α2−α−(ε+q)[(p+ε)1−α−(q+ε)1−α]1−α.B_ (q p)= (p+ )^2-α-(q+ )^2-α2-α- ( +q) [(p+ )^1-α-(q+ )^1-α ]1-α. (25) For α=1α=1: Bε(q∣p)=(p−q)−(ε+q)log(p+εq+ε).B_ (q p)=(p-q)-( +q) \! ( p+ q+ ). (26) For α=2α=2: Bε(q∣p)=log(p+εq+ε)+(ε+q)(1p+ε−1q+ε).B_ (q p)= \! ( p+ q+ )+( +q) ( 1p+ - 1q+ ). (27) B.8 Connections to standard losses For the unshifted family ω0(t)=t−α _0(t)=t^-α, the divergence is B0(q∣p)=∫qp(t−q)t−αt.B_0(q p)= _q^p(t-q)t^-α\,dt. For α=0α=0, B0(q∣p)=∫qp(t−q)t=(p−q)22,B_0(q p)= _q^p(t-q)\,dt= (p-q)^22, so the family recovers squared error up to a constant factor 1/21/2. For α=2α=2, B0(q∣p)=∫qp(1t−qt2)t=logpq+qp−1=qp−logqp−1,B_0(q p)= _q^p ( 1t- qt^2 )\,dt= pq+ qp-1= qp- qp-1, which is exactly the Itakura–Saito divergence DIS(q,p)D_IS(q,p). B.9 Empirical verification of SPB scaling behavior Figure 7 contrasts the scale behavior of the metrics for a fixed relative error across the ground-truth range. Figure 7: Scale dependence of each metric for a fixed relative error (overestimate p=10qp=10q, left; underestimate p=q/10p=q/10, right) across ground-truth probabilities q∈[10−9,10−5]q∈[10^-9,10^-5]. Squared error varies by orders of magnitude with q, while IS loss, log-space squared error, and the weighted shifted-power loss are flat (scale invariant). Appendix C Pseudocode Algorithm 1 gives the full GA-AMLS procedure described in Section 3. Algorithm 1 GA-AMLS with MALA moves and a Student-t prior 1:Input distribution D, model M, target token t, population size N, quantile level ρ, target score τ, MALA step size h, MCMC steps T 2:Probability estimate P P 3:Draw calibration inputs from D and compute raw activations ajj=1ncal\a_j\_j=1^n_cal at the chosen site. 4:Estimate μ,Σμ, , choose A such that AA⊤=ΣA = , and transform activations by uj←(aj−μ)A−⊤.u_j←(a_j-μ)A^- . 5:Fit the prior in whitened coordinates as an independent Student-t distribution π(u)=∏d=1Dtν=5(ud;0,1).π(u)= _d=1^Dt_ν=5(u_d;0,1). 6:Define the score s(u)s(u) by inserting the unwhitened activation a(u)=uA⊤+μa(u)=uA +μ into the model. 7:Initialize particles uii=1N\u_i\_i=1^N by sampling from the whitened calibration activations. 8:k←0k← 0, L0←−∞L_0←-∞, P^←1 P← 1 9:while Lk<τL_k<τ do 10: Compute scores s(ui)i=1N\s(u_i)\_i=1^N. 11: Set the next level Lk+1←minτ,Quantileρ(s(ui)i=1N).L_k+1← \τ,Quantile_ρ (\s(u_i)\_i=1^N ) \. 12: Estimate the conditional survival probability p^k←1N∑i=1Ns(ui)≥Lk+1,P^←P^p^k. p_k← 1N _i=1^N1\s(u_i)≥ L_k+1\, P← P p_k. 13: Keep the survivors ←ui:s(ui)≥Lk+1S←\u_i:s(u_i)≥ L_k+1\ and resample with replacement from S until the population again has size N. 14: for r=1,…,Tr=1,…,T do 15: for each particle uiu_i do 16: Compute the prior score gradient gi←∇ulogπ(ui).g_i← _u π(u_i). 17: Propose a MALA move ui′←ui+hgi+2hξi,ξi∼(0,I).u_i ← u_i+hg_i+ 2h\, _i, _i (0,I). 18: if s(ui′)<Lk+1s(u_i )<L_k+1 then 19: Reject the proposal. 20: else 21: Compute gi′←∇ulogπ(ui′)g_i ← _u π(u_i ) and proposal densities q(ui′∣ui)=(ui′;ui+hgi, 2hI),q(ui∣ui′)=(ui;ui′+hgi′, 2hI).q(u_i u_i)=N(u_i ;\,u_i+hg_i,\,2hI), q(u_i u_i )=N(u_i;\,u_i +hg_i ,\,2hI). 22: Accept ui′u_i with probability ai=min1,π(ui′)q(ui∣ui′)π(ui)q(ui′∣ui).a_i= \1,\, π(u_i )q(u_i u_i )π(u_i)q(u_i u_i) \. 23: end if 24: end for 25: Adapt h during burn-in toward the MALA target acceptance rate. 26: end for 27: k←k+1k← k+1 28:end while 29:return P P Appendix D Experimental and Implementation Details D.1 Hyperparameters MAPLA tuning candidate. In the tuning sweep we also considered MAPLA, the Metropolis-adjusted Preconditioned Langevin Algorithm (Srinivasan et al., 2025). MAPLA generalizes MALA by replacing the isotropic proposal covariance with a position-dependent metric. For a target density π, its proposal has the form u′∼(u+hG(u)−1∇ulogπ(u),2hG(u)−1),u \! (u+hG(u)^-1 _u π(u),2hG(u)^-1 ), followed by the usual Metropolis–Hastings correction. In our AMLS setting this kernel is applied to the constrained target πk(u)∝π(u)s(u)≥Lk. _k(u) π(u)1\s(u)≥ L_k\. We used MAPLA only as a hyperparameter-tuning candidate: although it can reduce rejections near level-set boundaries by adapting proposal geometry, its extra metric computations did not yield a sufficient improvement over MALA under a matched compute budget. Therefore all main GA-AMLS results use MALA. Tuning protocol. Like Wu and Hilton (2025), to prevent overfitting our method was only run on the first four distributions during development, and architectural choices were finalized before testing on the last four distributions. We chose the MH kernel and the prior as those which minimized squared error loss and IS loss on 100 randomly chosen tokens with ground-truth probabilities in the range [10−5,10−3][10^-5,10^-3]; probabilities in this range are directly estimable by naive Monte Carlo with a modest budget, so tuning does not require privileged knowledge of the extreme tail. The selected prior and kernel were consistent across metrics, distributions D, and model sizes. Both MALA and MAPLA outperformed the random-walk kernel (RWMH), and the Student-t prior outperformed a Gaussian prior; since MAPLA’s higher computational cost did not yield a commensurate performance gain over MALA, we chose the MALA kernel. To ensure a fair comparison across MH kernels, we equalized the total computational budget per rejuvenation step: since MALA and MAPLA require gradient computations, we allocate 1500 MH steps to RWMH and 750 steps to the gradient-based kernels. GA-AMLS configuration All GA-AMLS estimates are from a single run per (token, distribution, model). We use a population of N=2000N=2000 particles, threshold quantile ρ=0.6ρ=0.6, T=900T=900 MALA steps per level with a burn-in of Tburn=126T_burn=126 steps and initial step size h0=10−3h_0=10^-3, and a diagonal Student-t prior with ν=5ν=5 degrees of freedom fit on ncal=216=65,536n_cal=2^16=65,536 calibration activations drawn from D. The calibration activations are generated once per (model,D) pair and reused across target tokens. Baselines. For the importance-sampling baselines ITGIS and MHIS we used the temperatures tuned by Wu and Hilton (2025) .We identified and fixed a numerical-tolerance bug in the public QLD implementation: in the randomized projection routine for finding the shortest accepting vector, the tolerance was meant to ignore only slightly negative constraint violations Px+b≥−tolPx+b≥- tol, but the original code used Px+b<tolPx+b< tol, incorrectly marking feasible near-boundary constraints as violated.; all QLD results in this paper use the corrected implementation, which improves QLD’s performance relative to the numbers reported by Wu and Hilton (2025). Ablation studies. Figures 8–11 report the per-distribution SPB loss for every kernel–prior–hyperparameter configuration explored during tuning, under each of the four evaluation profiles of Table 2. Two patterns hold consistently across distributions and model sizes: (i) the MALA kernel outperforms RWMH despite using half as many MH proposal steps, and (i) the Student-t prior outperforms a Gaussian prior. Figure 8: Hyperparameter sweep under the product-monitoring evaluation profile (α=2α=2, γ=0γ=0, ε=2.073×10−12 =2.073× 10^-12). Bars show average SPB loss (±1.28± 1.28 SEM, log scale) for each kernel–prior–hyperparameter configuration, per input distribution (rows) and model size (columns). Configuration names encode kernel, prior, activation site, and tuning parameters. Figure 9: Hyperparameter sweep under the deep-tail stress-testing profile (α=2α=2, γ=1γ=1); format as in Figure 8. Figure 10: Hyperparameter sweep under the unbiased method benchmarking profile (α=1.5α=1.5, γ=0γ=0); format as in Figure 8. Figure 11: Hyperparameter sweep under the catastrophic-risk audit profile (α=4α=4, γ=2γ=2); format as in Figure 8. D.2 Activation Whitening Diagnostic Figure 12 plots the held-out marginal variance of each pre-LayerNorm, pre-unembedding activation dimension, sorted by variance, before and after whitening; the whitening transform used by GA-AMLS substantially flattens this marginal variance profile. Figure 12: Marginal activation variances before and after whitening. Let ai∈ℝda_i ^d denote held-out pre-LayerNorm, pre-unembedding activations. For each model size and input distribution, we compute the empirical marginal variance Vari(aij)Var_i(a_ij) of every activation dimension j, and repeat the calculation after applying the whitening transform used by GA-AMLS. For visualization, dimensions are sorted from largest to smallest marginal variance within each distribution; the x-axis is therefore the sorted dimension rank. Each panel reports the median sorted curve across the eight input distributions, with shaded bands showing the interquartile range. Raw activations exhibit strong marginal anisotropy, while whitening produces a much flatter variance profile, making isotropic MALA proposals in whitened coordinates more reasonable. Appendix E Scatterplots Figures 13–15 show estimates against ground truth for all four methods on all 8 input distributions, for each model size, complementing the two examples in Figure 4. Figure 13: Estimates vs. ground truth for all methods across all 8 input distributions (1-layer model). MHIS/ITGIS zero estimates are plotted as crosses at the bottom of each panel for visibility. Figure 14: As Figure 13, for the 2-layer model. Figure 15: As Figure 13, for the 4-layer model. Appendix F All Methods Loss by Distributions Figure 16 breaks the aggregate SPB results down by input distribution, Figure 17 reports the aggregate SPB loss across model sizes for all four (α,γ)(α,γ) evaluation profiles, and Tables 3 and 4 report per-distribution SPB loss. Figure 16: SPB loss across individual distributions and parameter configurations. GA-AMLS (blue) consistently outperforms baseline methods across 7 of the 8 text distributions under a symmetric penalty (α=1.5α=1.5), demonstrating that aggregate improvements are not skewed by outliers. As the penalty for underestimation increases (α>1.5α>1.5 ), QLD (orange) gains an advantage due to its systematic overestimation bias. Lower is better; error bars denote standard error across the three model sizes. Figure 17: The SPB loss of all methods across different model sizes and parameter combinations. The solid lines indicate the loss of each method averaged over all 8 distributions, with bands showing standard error. The colored points indicate the loss on individual distributions, with horizontal jitter added for visibility. Lower is better. We see that GA-AMLS (blue) outperforms when the penalty for FN is approximately the same as FP, and when the weight on the rarest events increases. Table 3: SPB loss comparison across model sizes (α=1.5α=1.5, γ=0γ=0, ε=εcorr = _corr). Lower is better. (a) 1-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 117.64 538.82 53.26 35.44 camel 660.47 148.67 125.83 14.62 colon 127.09 278.54 58.18 6.24 if 362.68 211.57 46.96 15.16 caps 1489.97 421.47 83.47 17.05 english 190.48 58.75 27.49 16.16 spanish 480.38 106.99 78.54 13.67 icl 218.55 37.67 64.68 20.05 Avg. 455.91 225.31 67.30 17.30 (b) 2-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 136.82 1141.36 66.22 272.59 camel 947.94 725.50 93.04 27.75 colon 361.36 555.99 48.02 12.63 if 270.16 146.68 48.44 7.31 caps 463.91 392.87 56.80 20.55 english 708.88 295.81 17.51 21.46 spanish 637.53 259.79 61.24 15.04 icl 97.04 26.26 38.06 28.94 Avg. 452.96 443.03 53.67 50.78 (c) 4-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 308.79 1520.17 45.92 133.83 camel 1167.25 561.11 90.86 12.81 colon 151.61 767.77 34.08 11.06 if 119.23 615.02 34.02 6.41 caps 205.58 609.63 56.90 17.07 english 306.29 327.72 19.07 22.30 spanish 743.64 312.04 66.29 7.39 icl 38.85 940.25 28.98 10.07 Avg. 380.15 706.71 47.01 27.62 Table 4: SPB loss comparison across model sizes (α=2α=2, γ=0γ=0, ε=εcorr = _corr). Lower is better. (a) 1-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 3.62e+04 2.70e+05 26.99 2053.05 camel 4.34e+05 3.84e+04 6.15 318.50 colon 1.43e+04 6.20e+04 5.32 11.35 if 1.02e+05 7.99e+04 4.72 935.19 caps 1.62e+06 1.90e+05 5.34 15.28 english 5.57e+04 1.63e+04 3.79 1183.62 spanish 1.79e+05 9693.76 5.55 289.98 icl 5.00e+04 3564.92 5.13 556.86 Avg. 3.12e+05 8.38e+04 7.87 670.48 (b) 2-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 2.96e+04 1.02e+06 1.46e+04 1.52e+05 camel 9.71e+05 7.79e+05 644.61 5066.22 colon 1.35e+05 2.76e+05 4.88 62.13 if 7.62e+04 4.30e+04 4.92 27.54 caps 2.89e+05 3.60e+05 42.00 1379.45 english 4.02e+05 1.12e+05 3.12 2853.15 spanish 3.06e+05 1.14e+05 4.99 816.30 icl 7.31e+04 1545.28 798.60 1858.51 Avg. 2.85e+05 3.38e+05 2017.40 2.05e+04 (c) 4-layer model Dist. MHIS ITGIS QLD GA-AMLS hex 1.64e+05 1.38e+06 6749.56 2.81e+04 camel 1.30e+06 5.27e+05 4.90 287.16 colon 4.77e+04 5.83e+05 4.14 11.17 if 2.03e+04 3.53e+05 4.16 97.97 caps 5.92e+04 5.17e+05 4.42 41.13 english 7.00e+04 8.72e+04 68.21 7759.93 spanish 5.58e+05 1.18e+05 5.17 33.23 icl 8595.27 7.34e+05 12.28 19.18 Avg. 2.79e+05 5.38e+05 856.60 4547.43