Paper deep dive
Logit Leakage through Floating-Point Bias Addition
Shaomin Chen, Xiaorui Gong
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Language model APIs can expose token-specific biases while withholding logits and token probabilities. Existing bias-based extraction recovers relative logits, leaving a common additive component undetermined in ideal arithmetic. We identify a numerical side channel in floating-point bias addition that exposes information about this component through token identities alone. The channel distinguishes score vectors with identical complete unperturbed log probabilities. Our method exploits changes in representable spacing at binary exponent boundaries. Under the stated binary32 arithmetic assumptions and a known reachable score interval, it recovers a logit pair and its unknown tie-breaking priority exactly in at most 52 token queries. Given complete relative logits, one recovered anchor determines the original logit vector. We evaluate recovery across 11 public checkpoints from seven model families and extend the local method with adaptive interval search and anchor selection. In end-to-end experiments with vLLM, SGLang, and this http URL, adaptive recovery succeeds on all 128 Qwen3-0.6B inputs per system, including 64 held-out inputs. A separate token-only experiment extracts a pretrained small model's complete output-head representation up to an invertible hidden-basis transformation. We also establish a sufficient condition for suppressing common-shift disclosure by subtracting the maximum before bias addition. In controlled experiments, this mitigation eliminates all observed distinctions between common-shift variants and preserves token decisions under the tested random biases.
Tags
Links
- Source: https://arxiv.org/abs/2610.16790v1
- Canonical: https://arxiv.org/abs/2610.16790v1
Trouble viewing inline? Open PDF directly →
Full Text
100,115 characters extracted from source content.
Expand or collapse full text
Logit Leakage through Floating-Point Bias Addition Shaomin Chen Xiaorui Gong Abstract Language model APIs can expose token-specific biases while withholding logits and token probabilities. Existing bias-based extraction recovers relative logits, leaving a common additive component undetermined in ideal arithmetic. We identify a numerical side channel in floating-point bias addition that exposes information about this component through token identities alone. The channel distinguishes score vectors with identical complete unperturbed log probabilities. Our method exploits changes in representable spacing at binary exponent boundaries. Under the stated binary32 arithmetic assumptions and a known reachable score interval, it recovers a logit pair and its unknown tie-breaking priority exactly in at most 52 token queries. Given complete relative logits, one recovered anchor determines the original logit vector. We evaluate recovery across 11 public checkpoints from seven model families and extend the local method with adaptive interval search and anchor selection. In end-to-end experiments with vLLM, SGLang, and llama.cpp, adaptive recovery succeeds on all 128 Qwen3-0.6B inputs per system, including 64 held-out inputs. A separate token-only experiment extracts a pretrained small model’s complete output-head representation up to an invertible hidden-basis transformation. We also establish a sufficient condition for suppressing common-shift disclosure by subtracting the maximum before bias addition. In controlled experiments, this mitigation eliminates all observed distinctions between common-shift variants and preserves token decisions under the tested random biases. I Introduction Language-model services expose configurable decoding interfaces while withholding internal logits and model parameters. Logit bias lets users steer generated text by increasing or decreasing the likelihood of specified tokens, even when the interface returns only the selected token [1]. Understanding what numerical information can be recovered through these controls is part of assessing the information exposed by a prediction interface. Prediction interfaces support model extraction [2], including recovery of language-model output projections and their dimensions [3, 4]. Morris et al. and Carlini et al. recover relative logits through token-specific biases and observed token choices [5, 3]. In exact real arithmetic, both softmax probabilities and biased-argmax responses are invariant to a common additive shift of the logit vector. These observations determine logit differences while leaving one additive component unidentified for each input. We show that floating-point bias addition can expose this component through token identities alone. Floating-point spacing varies with magnitude, so adding a chosen bias can produce different rounded comparisons for logit vectors related by a common shift. We construct queries that distinguish such vectors even when their complete unperturbed log probabilities are identical. Under explicit arithmetic and range conditions, we use these differences to recover original-logit anchors. Combined with complete relative logits, an anchor determines the original logit vector, including its common additive component. Figure 1 summarizes the mechanism and recovery. We address three technical challenges: • Identifying leakage beyond relative logits. Common-shift score vectors produce identical ideal query responses. Establishing additional disclosure requires floating-point queries that distinguish these vectors while their complete unperturbed probability outputs remain identical. • Recovering original logits from token responses. A token comparison combines score gaps, rounding, and an unknown tie-breaking order. Bounded biases restrict the accessible numerical regions, so recovery requires queries that separate these effects. • Suppressing leakage while preserving bias control. Removing dependence on the common additive component can change rounded comparisons and token choices. A mitigation must retain token-specific bias control and be evaluated for both channel suppression and preservation of decoding behavior. Fig. 1: Overview of the numerical channel and recovery. (a) The observer chooses token biases b for fixed input x and observes the selected token; RN32RN_32 denotes binary32 rounding. (b) At the positive boundary P=2eP=2^e, spacing increases from Δ to 2Δ2 . At the illustrated point below P, the increment is exact. At P, the exact sum P+ΔP+ lies halfway between P and P+2ΔP+2 and rounds to P under ties-to-even. (c) Gap and priority recovery followed by boundary probes recovers the score pair under the stated arithmetic and range assumptions. (d) Subtracting the maximum before bias suppresses common-shift disclosure when stored variants yield identical centered vectors. For positive-score recovery, we determine the logit gap and the token tie-breaking priority, then probe the original anchor using the spacing change at an exponent boundary. The probe responses are periodic over part of the candidate range. We first use a small number of queries to determine the anchor’s position within this repeating pattern, then restrict the candidate values accordingly to obtain a monotone comparison for binary search. Binary search then recovers the anchor exactly within the stated interval, and the recovered gap determines the second original logit. Our central result assumes stable scores, independently converted biases, one correctly rounded addition per coordinate, greedy selection, and a fixed but unknown token tie-breaking order. Given a known reachable score interval, recovery takes at most 50 binary queries for a positive binary32 score pair and its priority, or 47 for negative scores. Selecting the two tokens takes two additional queries. We validate recovery across 11 public checkpoints from seven model families and extend the local method with interval search and anchor selection. End-to-end adaptive recovery through vLLM, SGLang, and llama.cpp succeeds on all 128 Qwen3-0.6B inputs per system, including 64 held-out inputs. The recovered component has a direct role in output-head extraction. For a bias-free linear head under the rank conditions in Appendix G, centered logits determine CWGCWG, where C subtracts the vocabulary mean, W is the output head, and G is an invertible hidden-basis transformation. Original-logit anchors supply the missing shared row component and permit recovery of WGWG. We demonstrate this composition by extracting a pretrained model’s complete 512×64512× 64 output-head representation from token responses across three independent input sets. On vocabulary rows excluded from basis alignment, weight RMSE ranges from 3.60×10−73.60× 10^-7 to 1.20×10−61.20× 10^-6 (Section V-C). Finally, we analyze subtracting the maximum before bias addition as a mitigation. When common-shift variants yield identical preprocessed vectors, subsequent bias queries disclose no difference between them. Controlled experiments verify this condition and measure output preservation under independently generated sparse biases. We make three contributions: 1. Logit leakage through floating-point bias addition. We characterize how rounding exposes a common additive component that remains unidentified by ideal probabilities and biased-argmax responses. 2. Constructive recovery and output-head extraction. We give local exact-recovery algorithms and query bounds, validate them through generation and serving interfaces, and combine recovered anchors with relative logits to extract an output-head representation. 3. Processing-order analysis and mitigation. We establish a sufficient condition for suppressing common-shift disclosure through pre-bias max subtraction and measure its effect on token decisions. I Background and Problem Formulation I-A Logits, Probabilities, and Common Shifts At one generation position, input x determines the vector q(x)∈ℝVq(x) ^V entering bias addition, where V is the vocabulary size. We use scores as a general term for numerical values used in token selection. A linear output head produces logits Wh(x)+βWh(x)+β before softmax [6, 3]. When bias addition acts directly on these outputs, we distinguish the original logits from the resulting biased logits. If temperature scaling or clipping precedes bias addition, our numerical analysis instead applies to the processed scores entering that operation [7]. For a fixed input, write q=q(x)q=q(x) and let ∈ℝV1 ^V denote the all-ones vector. For any scalar s∈ℝs , softmax(q+s)=softmax(q).softmax(q+s1)=softmax(q). (1) Writing the mean-centering projection as C=I−⊤/VC=I-11 /V gives q=Cq+μ,μ=⊤q/V.q=Cq+ 1, μ=1 q/V. (2) Here, μ is the mean score, and CqCq contains the deviations from that mean. Probabilities determine the centered component CqCq in exact arithmetic, but not μ. An ideal biased-token interface Oqideal(b)=argmaxi(qi+bi)O_q^ideal(b)= *arg\,max_i(q_i+b_i) (3) has the same ambiguity. This is the common freedom remaining in the logprob-free extraction setting of Carlini et al. [3]. Let A be a selected token and a=qAa=q_A its original score, which we call the anchor. Any recovered anchor fixes the common additive component: if an earlier extractor uses reference token R and returns di=qi−qRd_i=q_i-q_R, set ri=di−dAr_i=d_i-d_A. Then qi=a+riq_i=a+r_i and μ=a+∑iri/Vμ=a+ _ir_i/V. The local protocol below recovers a score pair and its tie-breaking priority; Section I-G extends it to selecting the anchor. The ambiguity also has an architectural interpretation. Replacing a linear output head W by W+v⊤W+1v changes the scores by (v⊤h(x))(v h(x))1 without changing ideal probabilities. The lost scalar can therefore vary with the input and depend on a shared head component. Softmax-preserving output-head reparameterizations and their quantization effects have been studied for output-head optimization [8]. We ask whether token responses to chosen biases can recover an original logit and thereby determine the missing mean μ when the complete relative logits are known. I-B The Numerical Interface We use the IEEE 754-2019 binary32 format (FP32), which has 24-bit significand precision [9]. For a fixed input, let q be a vector of finite binary32 values. Let RN32RN_32 denote round-to-nearest, ties-to-even: halfway cases select the adjacent binary32 value with an even least significant significand bit [10]. The observed interface is Oq(b)=argmaxiRN32(qi+RN32(bi)).O_q(b)= *arg\,max_iRN_32\! (q_i+RN_32(b_i) ). (4) Each bias is converted independently before one binary32 addition. Scores remain fixed across queries for the same input. A fixed, initially unknown global tie-breaking order selects among tokens with equal maximum rounded scores. This selection is distinct from midpoint rounding; only the selected token identity is returned. The recovery proofs use the separate bias conversion, rounded addition, and greedy selection in (4). Conversion precision, inserted quantization, and fused operations can alter the comparisons [11]. Section V-D extends the comparison protocol to specified errors in the returned tokens while keeping the underlying scores fixed. A binary interval [2e−1,2e)[2^e-1,2^e) is a binade. Binary32 values in its normal range have spacing Δ=2e−24 =2^e-24. At P=2eP=2^e, spacing doubles. We use this change to produce score-dependent rounding responses. The experiments use normal values far from overflow and underflow. The abstract extension to very small exponents additionally requires gradual underflow rather than flush-to-zero. I-C Threat Model Protected information and adversary objective. The provider withholds the logits for each input. An external client seeks their original numerical values, including the common additive component left unidentified by ideal probabilities and biased argmax queries. Recovering one original anchor fixes this component when relative logits are known. Our local algorithm additionally recovers the target score t and the pairwise tie-breaking priority; its exact-recovery criterion requires agreement with all three values at the specified decoder stage. Figure 1(a) separates the client-visible interface from the private computation. Capabilities and observations. The client can repeat its own fixed input, request one greedy output token, and assign independent additive biases to selected token IDs within a public magnitude limit H. It adapts each bias to the preceding token responses. The selected token identity and tokenization are assumed observable. No returned probabilities, scores, score gaps, weights, hidden states, execution timings, co-location, model modification, or access to another user’s request is required. The client does not choose the service’s tie-breaking order, arithmetic implementation, or processing order. Numerical assumptions and prior knowledge. The service follows the stable-score operator in (4), with an unknown fixed global tie-breaking order. Greedy selection alone does not establish score stability; Section V-D evaluates execution changes and extensions for noisy responses. The client knows this arithmetic model and a stated reachable score interval. The stated interval is required side information for the recovery theorem. In the single-window generation evaluation, separate calibration inputs select a public interval before testing; the test inputs do not supply scores to the observer. Precision, range, and stability are checked privately after predictions are frozen, and all fixed test inputs remain in the accounting. The experiment evaluator can inspect private scores to measure success; the recovery controller cannot. In a search over multiple public intervals, uniqueness is defined relative to the prespecified search domain. I Logit Leakage and Recovery I-A Leakage Mechanism Equation (3) makes every ideal query invariant under q↦q+sq q+s1. Common-shift variants therefore yield identical transcripts even for adaptive, randomized observers: conditioned on the observer’s randomness, identical responses lead to identical subsequent queries. Floating-point addition need not preserve this equivalence. We compare complete unperturbed probability outputs with active numerical queries. An interface that returns probabilities after the same bias operations may also expose the additional information. A distinguishing query. Consider two binary32 vectors q=(30,28),q′=(29.875,27.875).q=(30,28), q =(29.875,27.875). (5) Their difference is a common shift. Their max-centered vectors are both (0,−2)(0,-2), and their unperturbed probabilities are equal. Give the anchor bias 22 and the target bias 4+3⋅2−214+3· 2^-21. All supplied values are representable in binary32. For q, the anchor becomes 3232. The target’s exact sum is 32+3⋅2−2132+3· 2^-21, which rounds to 3232 because binary32 spacing above 3232 is 2−182^-18. If the anchor has priority, it wins. For q′q , the anchor becomes 31.87531.875. The target rounds to 31.875+2−1931.875+2^-19, because spacing below 3232 is 2−192^-19, and wins strictly. Thus one numerical query distinguishes the two score vectors although their passive outputs are the same. Fig. 2: Token responses for q=(30,28)q=(30,28) and q′=q−18q =q- 181. The anchor bias is 22; the target bias is 4+u2−214+u2^-21 for integer offsets u. The vectors have identical unperturbed probabilities. Shading marks offsets with different winners. The anchor wins ties between rounded scores; connecting lines guide the eye. This example isolates the source of information. In exact arithmetic, the target would win for both vectors at the stated bias. Rounding distinguishes the two vectors, whose original relative scores are identical. Section VI-B evaluates how preprocessing affects these differences over complete vocabularies. I-B Recovery Setting Pair discovery and binary responses. An initial unbiased query selects the anchor, and a second query suppresses it by an allowed negative bias to select a target. The anchor is the original greedy winner, so a≥ta≥ t. Local recovery gives the anchor only nonnegative biases. No unmodified third token can then displace it under the fixed global tie-breaking order, making the subsequent response binary. Pair discovery is counted separately from the recovery bound. Our recovery construction requires enough bias to move the two scores across a chosen binary exponent boundary P=2eP=2^e, where the spacing between adjacent binary32 values increases from Δ just below P to 2Δ2 just above it. Queries must also resolve the unknown tie-breaking priority: which token wins when the rounded scores are equal. We first specify a reachable score interval; Section I-F addresses searches over multiple intervals and bounded-bias nonidentifiability. Let P=2eP=2^e with −125≤e≤127-125≤ e≤ 127, assume gradual underflow and no overflow in probe arithmetic, and set Δ=P/224 =P/2^24. Define K=⌊min(H,P/2)Δ⌋,H0=KΔ,F=P−H0.K= (H,P/2) , H_0=K , F=P-H_0. (6) For the anchor score a and target score t, assume K≥4K≥ 4 and a≥t>F,a∈[ℓ,u]⊂(F,P−2Δ],a≥ t>F, a∈[ ,u]⊂(F,P-2 ], (7) where ℓ,u ,u are Δ grid points. Both scores lie in the same binade, where binary32 values have spacing Δ . Their gap d=a−td=a-t is therefore a multiple of Δ . The interval in (7) provides enough bias to move the scores across P while keeping every recovery bias between 00 and H0H_0. The margin below P also permits the adjacent probes used to resolve the priority and anchor residue. I-C Recovery Algorithm Algorithm I-C implements the positive-score recovery sequence in Figure 1(c). FirstWinningIndexFirstWinningIndex finds the first target-winning bias index; ResolveTieResolveTie identifies the gap and priority. ResidueCandidatesResidueCandidates selects anchors with the observed residue for BinarySearchBinarySearch, using the monotone response Probed,τProbe_d,τ. Write O(b0,b1)O(b_0,b_1) for biases on tokens 0 and 1, with zero bias elsewhere. All subroutines use the public range parameters. Algorithm 1. Exact recovery of a positive score pair. Input: Token oracle O, original winner 0, target 1, and public P,H,ℓ,uP,H, ,u satisfying (7); Δ,K,F ,K,F follow (6). Output: Scores (a,t)(a,t) before bias addition and tie-breaking priority τ. 1 j←FirstWinningIndex(O,Δ,K)j (O, ,K) 2 d0←(j−1)Δd_0←(j-1) 3 τ←ResolveTie(O,d0)τ (O,d_0) 4 d←d0+τΔd← d_0+τ 5 p0←maxℓ,F+d+Δp_0← \ ,F+d+ \ 6 ←ResidueCandidates(O,d,τ,p0,u)S (O,d,τ,p_0,u) 7 a←BinarySearch(,Probed,τ)a (S,Probe_d,τ) 8 return(a,a−d,τ)return\ (a,a-d,τ) Recovering the gap and tie-breaking priority. Label the original winner as token 0 and the target as token 1. Their pairwise tie-breaking priority is τ=0τ=0 if token 0 wins when their rounded scores are equal, and τ=1τ=1 otherwise. When only the target receives bias kΔk , the first winning grid point j determines exactly two possibilities: τ=0:d0=(j−1)Δ,τ=1:d1=jΔ=d0+Δ. array[]lτ=0:&d_0=(j-1) ,\\ τ=1:&d_1=j =d_0+ . array (8) The target loses at k=0k=0 by the original-anchor definition and wins at k=Kk=K because t+H0>P>at+H_0>P>a. FirstWinningIndexFirstWinningIndex finds j by integer binary search in at most ⌈log2K⌉ _2K queries; the endpoint responses require no calls. Around the comparison threshold, the sums are representable grid values, so the search determines the two gap hypotheses exactly. Three rounding probes identify the implementation’s tie-breaking priority without assuming a token-ID order. Define p=maxℓ,F+d0+Δ.p= \ ,F+d_0+ \. (9) For r=p,p+Δ,p+2Δr=p,p+ ,p+2 , submit b0=P−r,b1=P−r+d0+Δ.b_0=P-r, b_1=P-r+d_0+ . (10) Under the second hypothesis in (8), the two exact sums coincide and the target has priority, so all three replies are token 1. Under the first hypothesis, the responses follow the rounding pattern derived next and cannot all be token 1. ResolveTieResolveTie returns τ=1τ=1 exactly when all three replies are token 1, thereby resolving the gap ambiguity. The equality case a=ta=t is included. It implies τ=0τ=0 because token 0 was the original winner, and the first winning target bias is Δ . Identifying the anchor residue. Suppose d and τ are known. For a pivot p∈Δℤp∈ with F+d+Δ≤p≤PF+d+ ≤ p≤ P, give the anchor bias P−pP-p and the target bias P−p+dP-p+d. Their exact biased scores are equal: x=P+(a−p)=P+kΔ,k=(a−p)/Δ.x=P+(a-p)=P+k , k=(a-p)/ . (11) Add one further Δ to whichever token has lower priority. After incorporating the increment, convert each bias independently to binary32 and add it to its score with one rounded addition. Define Probed,τ(p)Probe_d,τ(p) as the indicator that token 1−τ1-τ wins this query, and write f(k)f(k) for its response at k=(a−p)/Δk=(a-p)/ . Each evaluation makes one oracle call. Below P, both x and x+Δx+ are representable and f(k)=1f(k)=1. At or above P, ties-to-even rounding operates on a grid of spacing 2Δ2 , yielding f(k)=1,k<0,0,k≥0,kmod4∈0,3,1,k≥0,kmod4∈1,2.f(k)= cases1,&k<0,\\ 0,&k≥ 0,\ k 4∈\0,3\,\\ 1,&k≥ 0,\ k 4∈\1,2\. cases (12) The nonnegative pattern is 0,1,1,00,1,1,0 (Figure 3(a)). In particular, it contains no run of three ones. This establishes the priority test above because its initial pivot satisfies p≤ap≤ a under either gap hypothesis. Set p0=maxℓ,F+d+Δp_0= \ ,F+d+ \. Using at most three queries, ResidueCandidatesResidueCandidates either identifies a=p0a=p_0 or decodes the residue ρ=((a−p0)/Δ)mod4ρ=((a-p_0)/ ) 4. It returns =p0S=\p_0\ in the former case and =[p0,u]∩(p0+ρΔ+4Δℤ)S=[p_0,u]∩(p_0+ρ +4 ) otherwise. A singleton interval needs no query. Appendix A gives the response table, the zero-offset exception, and the payload bounds. Fig. 3: Eliminating the periodic response. (a) Full-grid probes are periodic for p≤ap≤ a and constant for p>ap>a. (b) The recovered residue class (open markers) gives a monotone comparison; gray points are excluded. The change at p=ap=a corresponds to floating-point spacing changing from 2Δ2 to Δ . Lines connect discrete queries. The horizontal origin uses the unknown a for illustration. Once the residue is known, restrict candidate anchors to that residue class, spaced 4Δ4 apart. Both a and a candidate pivot p lie in this class, so (a−p)/Δ∈4ℤ(a-p)/ ∈ 4Z. Equation (12) therefore reduces to f((a−p)/Δ)=1⟺a<p.f((a-p)/ )=1 a<p. (13) Figure 3 illustrates this restriction to a monotone comparison. BinarySearchBinarySearch queries Probed,τProbe_d,τ on S, retaining candidates below the pivot after response 1 and those at or above it after response 0. It recovers a in at most ⌈log2||⌉ _2|S| queries, including zero for a singleton; then t=a−dt=a-d. I-D Correctness and Query Complexity Theorem 1 (Local pair recovery). Under (4) and (7), with token 0 the original winner and a fixed global tie-breaking order, Algorithm I-C recovers (a,t,τ)(a,t,τ) exactly. If the legal anchor interval after determining the gap contains N=(u−p0)/Δ+1N=(u-p_0)/ +1 grid points, it uses at most ⌈log2K⌉+6+⌈log2⌈N/4⌉ _2K +6+ _2 N/4 (14) binary recovery queries. For binary32 this is at most 50. Proof. The target-only search gives the two gap hypotheses exactly. The three probes in (10) identify the correct hypothesis by (12). At most three more probes identify the anchor residue, including the k=0k=0 exception. The remaining class contains at most ⌈N/4⌉ N/4 points, and (13) permits ordinary binary search. The special case of a singleton interval terminates earlier. Every pivot keeps the anchor bias nonnegative and all payloads within H0H_0; the original-anchor property excludes third-token responses. Finally, K≤223K≤ 2^23 and N≤KN≤ K, giving 23+3+3+21=5023+3+3+21=50. ∎ For binary precision p≥3p≥ 3, the same full-binade construction gives a worst-case upper bound 2p+22p+2, under the corresponding representability assumptions. The full-binade domain in (7) contains (K−2)2(K-2)^2 admissible pair-and-priority states. For a=F+iΔa=F+i , with 1≤i≤K−21≤ i≤ K-2, targets t=F+jΔt=F+j , 1≤j≤i1≤ j≤ i, admit 2i−12i-1 states: two priorities for each j<ij<i and only the original-anchor priority for j=ij=i. A binary decision tree of depth T has at most 2T2^T leaves. Thus any deterministic algorithm that always recovers both scores and their priority has worst-case binary query cost at least ⌈log2(K−2)2⌉, _2(K-2)^2 , (15) where K=2p−1K=2^p-1; for binary32, the bound is 46. Existing multiclass extraction methods use more informative responses to recover many relative coordinates [3], with costs measured per coordinate. I-E Negative Scores and Mixed Precision Logits can be negative. A negative exponent boundary reverses the direction of the spacing change and admits a separate construction without shifting the service’s scores. Let S=2eS=2^e, δ=S/223δ=S/2^23, K=⌊min(H,S)/δ⌋K= (H,S)/δ , H0=KδH_0=Kδ, and F=−S−H0F=-S-H_0. Assume K≥2K≥ 2, representable δ/2δ/2, no overflow or flushing of required subnormals, and a≥t>F,a∈[ℓ,u]⊂(F,−S−δ].a≥ t>F, a∈[ ,u]⊂(F,-S-δ]. (16) The public endpoints lie on the δ grid. Theorem 2 (Negative-score recovery). Under (4) and (16), the score pair and unknown fixed tie-breaking priority can be recovered in at most ⌈log2K⌉+2+⌈log2⌈N/2⌉≤47 _2K +2+ _2 N/2 ≤ 47 (17) binary recovery queries, where N is the remaining number of legal anchor grid points after gap recovery. The worst-case binary-query lower bound for the full-binade pair-and-priority problem is 46. The gap search is unchanged. One half-step query in the finer region resolves the tie-breaking priority. A further query at a pivot above every legal anchor identifies the anchor’s parity. Restricting pivots to that parity turns the rounding response into a monotone comparison. Appendix C proves the construction and its payload bounds. Changing to another known IEEE rounding direction permits modified probes with the same uniform upper bound; a mode change alone is therefore not a general suppression argument. Known score precision restricts both searches to the storage grid. Storage-aligned probes also remove the fine-grid residue step, giving at most 2s−12s-1 recovery queries for s-bit stored scores when the bias and addition precision is at least s+2s+2 and the local range and representability conditions hold: 15 for BF16 and 21 for binary16 with binary32 addition (Appendix D). The required grid is that of the scores entering bias addition; weight or forward-pass precision alone does not establish it. Lower-precision storage retains the channel when promoted scores accept finer biases. I-F Consistency Checks and Abstention Outside a supplied local interval, we use local recovery to generate candidates over a fixed collection of windows. Each candidate must reproduce the combined transcript under the declared operator; zero or multiple surviving candidates cause abstention. A unique candidate is accepted relative to the specified search domain. Bounded biases can leave distinct score vectors observationally equivalent (Appendix B). Thus, transcript consistency establishes uniqueness within the searched domain; exact recovery additionally requires the local arithmetic and range assumptions. I-G Adaptive Anchor Selection If no unique candidate survives the current pair’s checks, discard its anchor and retry. Every query fixes bias −H-H on previously discarded anchors S. A query with zero bias elsewhere selects anchor A; additionally suppressing A selects target T. A response from S, or failure to select a distinct target, stops the search. Preservation of local recovery. The fixed mask defines a baseline vector with entries RN32(qi−H)RN_32(q_i-H) on S and qiq_i elsewhere. Its greedy winner is A, so nonnegative anchor biases preserve binary responses under the fixed tie-breaking order. For A,T∉SA,T∉ S, the service computes RN32(qA+bA)RN_32(q_A+b_A) and RN32(qT+bT)RN_32(q_T+b_T) directly from the original logits. The local theorems therefore apply whenever the selected pair satisfies their range assumptions. The search returns a unique candidate passing all checks or stops after R pair attempts, biasing at most R+1R+1 tokens per query. Window searches, checks, and retries contribute to the complete recovery cost. I-H Output-Head Extraction Building on the output-head factorization of Carlini et al. [3], we combine complete relative logits with recovered anchors to determine a bias-free linear head up to an invertible hidden-basis transformation. Bias bisection against the unbiased winner estimates each relative coordinate (Appendix G). Centering these estimates across the vocabulary gives a matrix that we factor as B^X B X. The anchors determine the vector μ of mean logits over extraction inputs. Fitting =uX μ=u X gives the shared row u, and adding it to every vocabulary row completes the head as W^=B^+u W= B+1u. If the centered head and hidden-state matrix have full hidden rank, these factors represent CWGCWG and WGWG for an invertible G; Appendix G derives the identity and handles rank loss under centering. Relative-coordinate and floating-point errors make the numerical head estimates approximate even when the anchors are exact. For a held-out input, queried relative coordinates determine its coordinates in the learned basis; u predicts the mean without using that input’s anchor. IV Experimental Setup IV-A Models and Interfaces We evaluate 11 checkpoints from seven families, ranging from 14M to 4B parameters: Pythia [12], GPT-2 [13], GPT-Neo [14], OPT [15], BLOOM [16], SmolLM2 [17], and Qwen3 [18]. Generation experiments invoke unmodified model.generate for one greedy token with the SequenceBiasLogitsProcessor in Transformers 4.57.1 [19, 7]. Forward computation uses FP32 or BF16 as specified below; bias addition uses FP32. Serving experiments use Qwen3-0.6B and SmolLM2-360M on research-controlled vLLM 0.10.2 [20], SGLang 0.5.3.post2 [21], and llama-server at commit 7f2d88b0ac3 [22]. GPU servers use BF16 weights; llama.cpp uses CPU FP32 GGUF. All add binary32 biases. Output-head extraction uses pretrained TinyStories-260K [23], whose bias-free head has a complete 512-token vocabulary, through a research-controlled CPU FP32 HTTP service. IV-B Observation and Validation Protocol The observer receives token IDs; evaluator-only hooks record original logits as ground truth. End-to-end runs obtain every response through generation calls, while cached evaluations apply the decoder to saved full-vocabulary vectors behind the same token-only interface. Predictions are frozen before private scoring. Independent replay checks recorded responses, and selected serving traces are repeated without instrumentation. Calibration uses separate inputs; precision, budget, and framework comparisons reuse the specified input sets and count their model–input conditions separately. Each study below gives its input split and comparison protocol. IV-C Evaluation Metrics For true scores and tie-breaking priority (a,t,τ)(a,t,τ) and a returned estimate (a^,t^,τ^)( a, t, τ), we define the maximum absolute score error and exact-recovery criterion as Emax E_ =max|a^−a|,|t^−t|, = \| a-a|,\ | t-t|\, (18) Exact recovery recovery ⇔Emax=0∧τ^=τ. E_ =0\ \ τ=τ. For the finite, nonzero scores in our recovery domains, score equality is bitwise equality of the binary32 representations. Even a one-ULP discrepancy is counted as incorrect. We report recovered, incorrect, and unresolved cases over all attempted inputs, as well as exact recovery conditional on the stated range assumptions. Adaptive anchor recovery requires a^=qA a=q_A for the returned token A; we also evaluate the selected pair under (18), retaining all attempted inputs. A recovery query is one call used after token-pair selection; the two selection calls are counted separately. A generation call recomputes the model and applies its decoding operations. A cached decoder evaluation applies those operations to a saved score vector. We report these computational costs separately. V Evaluation We address four research questions: • RQ1: Can token responses recover original logits across models and numerical precisions, and how does range coverage affect recovery? (Section V-A) • RQ2: Can the adaptive procedure recover logits through serving APIs, and what is its complete query cost? (Section V-B) • RQ3: Can recovered anchors extend token-only output-head extraction from CWGCWG to WGWG? (Section V-C) • RQ4: How do execution changes and token-response errors affect recovery? (Section V-D) V-A RQ1: Recovery across Models and Precisions The FP32 evaluation uses the same 64 fixed test inputs per checkpoint. Eight prefixes, including Chinese, JSON, and always-A instructions, are balanced across arithmetic and word-continuation text to exercise the numerical interface. The single-window baseline decodes under one evaluator-calibrated interval; the adaptive controller searches windows, checks the combined transcript, and reselects anchors as defined in Sections I-F–I-G. Before testing, eight separate calibration inputs select one public positive or negative interval by maximum calibration coverage, breaking coverage ties by ascending exponent and then positive before negative. The candidate boundaries are ±2e± 2^e for e=−6,…,10e=-6,…,10, with the theorem’s endpoint margins and primary bias limit H=100H=100. Calibration uses evaluator score access to supply the assumed interval; it is not a token-only range-estimation procedure. No test score reaches the observer, and every fixed input is retained. A post-hoc cached evaluation compares all 34 signed windows, first with the original pair and then with selection capped at eight pairs. It also evaluates larger bias budgets and generic versus known-precision recovery on the same vectors (Appendix E). End-to-end adaptive runs use 64 FP32 GPT-Neo inputs and 32 BF16 inputs each for OPT and Qwen3 at H=100H=100. Each query recomputes the model and returns only the selected token. TABLE I: FP32 recovery with a calibrated interval and adaptive anchor selection. Generation Cached multi-window search Model H Boundary Exact Calls Boundaries Fixed Selected Pythia-14M 100 1616 64 3,284 1616 64 64 Pythia-160M 100 – 0 1,600 – 0 0 Pythia-1.4B 100 1616 64 3,269 1616 64 64 GPT-2 Medium 100 −64-64 54 3,136 −64,−32-64,-32 62 64 GPT-Neo-125M 100 −1-1 13 2,363 −2,−1,−0.5-2,-1,-0.5 19 64 OPT-1.3B 100 1616 58 3,273 8,168,16 59 64 BLOOM-560M 100 512512 56 3,095 512512 56 56 SmolLM2-360M 100 3232 64 3,288 3232 64 64 SmolLM2-1.7B 100 1616 32 3,279 16,3216,32 64 64 Qwen3-0.6B 100 3232 63 3,254 3232 63 64 Qwen3-4B-Instruct 100 6464 36 3,143 32,6432,64 50 64 Larger-budget ablation on the same inputs Pythia-160M 256 10241024 64 3,156 10241024 64 64 BLOOM-560M 160 512512 64 3,198 512512 64 64 Notes. Counts are out of 64 inputs. Generation uses one evaluator-calibrated interval; Calls exclude calibration. Its 704 primary cases include 103 incorrect and 97 unresolved outcomes. Cached Boundaries list accepted windows (–: none), from ±2e± 2^e, e=−6,…,10e=-6,…,10. Fixed uses one pair; Selected tries up to eight. Accepted anchors meet exact pair-and-priority recovery; other cached outcomes are unresolved. Pythia-160M at H=100H=100 has zero calibration coverage (–); its 1,600 calls use the fallback P=2−6P=2^-6. Across 32,984 generation calls on 704 FP32 cases, all 504 covered pairs and priorities recover exactly: 504/704 (71.6%) overall and 504/504 within the calibrated intervals (Table I). GPT-2 and GPT-Neo exercise negative recovery on 54 and 13 covered inputs. Every response matches full-vocabulary binary32 replay, and original-logit hashes remain stable within each input, validating the tested generation path in (4). Outside the calibrated interval, the single-window baseline returns 103 incorrect candidates and leaves 97 unresolved. Cached window search recovers 565/704 inputs with the original pair; adaptive selection increases this to 632/704 (89.8%) with zero incorrect accepted candidates. Nine checkpoints reach 64/64, using at most three pairs. All 72 remaining inputs exhaust eight attempts outside the tested local domains. The larger-budget ablation reaches 64/64 for Pythia-160M at H=256H=256 and BLOOM at H=160H=160, bringing cached adaptive coverage to 704/704. BLOOM’s eight unresolved H=100H=100 targets require 512−t≈106.77512-t≈ 106.77–143.17143.17 bias, explaining their exclusion by that budget. Table I retains both budgets; the artifact contains the full budget sweep. With all 34 windows and anchor selection, end-to-end GPT-Neo recovery reaches 64/64 at H=100H=100; 45 inputs require another anchor. The complete procedure averages 2,428 generation calls per input, including selection, window searches, checks, and retries. This cost measures the broad search beyond the local recovery bound. BF16 logits. Four checkpoints additionally use BF16 forward computation: Pythia-160M, OPT-1.3B, BLOOM-560M, and Qwen3-0.6B. For each, the same 32 inputs are evaluated with the binary32 recovery algorithm and the algorithm using the known logit precision; the generation processor still adds FP32 biases. We verify the logit lattice privately. A separately specified budget ablation repeats configurations with zero calibration coverage at H=100H=100 using H=256H=256. Selection depends on calibration, not test success. It uses the same test inputs and keeps the original-budget outcomes. The channel persists with BF16 logits and FP32 bias processing in the tested configurations. At H=100H=100, both controllers recover all 88 covered inputs out of 128; the larger-budget Pythia-160M intervention adds 32 covered paired cases. Complete original logit vectors, token pairs, and rendered inputs match bit for bit between controllers. The original logits lie on a BF16 lattice, as required by Proposition 3. TABLE I: BF16 recovery with FP32 bias processing. Generation Cached Model H Exact Generic Known Selected Pythia-160M 100 0 – – 0 Pythia-160M 256 32 48 13 32 OPT-1.3B 100 30 50 14–15 32 BLOOM-560M 100 28 46–48 11–13 28 BLOOM-560M 160 NR NR NR 32 Qwen3-0.6B 100 30 49–50 14–15 32 Notes. Counts are out of 32; both algorithms agree. Generic and Known columns give generic and BF16-aware recovery queries, excluding pair selection. Both use FP32 bias addition. Cached selection uses multiple windows and up to eight pairs, with zero incorrect returns. –: no covered case; NR: generation not run. Table I shows 11–15 local queries with known BF16 precision versus 46–50 with generic recovery. Cached anchor selection covers 128/128 inputs at the displayed larger budgets. In end-to-end adaptive runs at H=100H=100, OPT reaches 32/32 with both algorithms on bitwise-identical original logits. Mean complete costs are 439 calls with known precision and 1,265 with generic recovery. Qwen3 reaches 32/32 with generic recovery at 1,060 calls per input. Each model needs a second pair on two inputs. V-B RQ2: Recovery in Serving Systems Each model–framework configuration uses 64 fixed inputs, H=100H=100, and one interval calibrated on eight separate inputs with evaluator-only scores. An independent client submits biases through the generation API under fixed execution settings. Read-only probes record evaluation scores; every eighth input’s complete trace is rerun on a fresh uninstrumented server. Appendix H specifies the interface fields. Cached ablations on these 384 vectors compare P=16P=16, P=32P=32, and both, with up to eight pairs and six candidate checks. We freeze the two-window controller before evaluating it end to end on Qwen3 through each API, using the original 64 inputs and 64 disjoint held-out inputs per framework. All payloads, including repeats, are submitted to the server. The single-window baseline at P=32P=32 recovers exactly 301/384 inputs: 37/36/36 Qwen3 cases across vLLM/SGLang/llama.cpp and all 192 SmolLM2 cases (Table I). The other 83 return incorrect candidates outside the calibrated interval; none is unresolved (Section VIII-A). Covered cases use 49–50 recovery queries plus two selection calls. All 19,686 responses match binary32 replay, with stable original logits; 2,447 uninstrumented-server queries reproduce their tokens. TABLE I: End-to-end recovery in LLM serving systems. Single window Adaptive Qwen3 Framework Qwen3 SmolLM2 Dev. Test Calls vLLM 37 64 64 64 119.6 SGLang 36 64 64 64 120.4 llama.cpp 36 64 64 64 115.4 Notes. Recovery counts are out of 64; H=100H=100. Models: Qwen3-0.6B and SmolLM2-360M. Single window uses P=32P=32. Adaptive uses P=16,32P=16,32, checks, and up to eight pairs. Dev./Test denote the development and disjoint held-out inputs. Calls are mean complete API calls across both adaptive sets, including selection, searches, checks, and retries; calibration and independent replay are excluded. Single-window checks still accept seven incorrect Qwen3 candidates on llama.cpp: four at P=16P=16 and three at P=32P=32. All lie outside the assumed local domain and contradict the other window’s transcript. Combining both windows recovers 58/57/56 of the 64 development inputs across the three frameworks with the original pair; anchor selection raises each to 64/64. The frozen two-window controller recovers 64/64 development and 64/64 held-out inputs per framework: 384/384 exact pairs and priorities, with no incorrect or unresolved returns. Another anchor is needed in 39 cases. Complete cost averages 118.5 API calls per input condition; per-framework medians are 109/109/108 and maxima are 310/312/215. All 45,497 responses match binary32 replay, and 6,271 uninstrumented-server queries reproduce their tokens. Appendix E gives the search-cost breakdown. V-C RQ3: Output-Head Extraction We evaluate the completion in Section I-H on three independently generated input sets for TinyStories-260K. Sets 1 and 3 contain 96 extraction inputs and 16 held-out inputs; set 2 contains 192 and 32. Random token prefixes have lengths 4, 8, and 16. Each HTTP request recomputes 64 copies of the input, applies independent biases, and returns token IDs; padding fixes the batch shape. The observer uses H=100H=100, boundaries P∈1,2,4,8,16,32P∈\1,2,4,8,16,32\, and at most four pair attempts. The observer receives no hidden width. Rank selection uses the largest adjacent singular-value ratio, requiring a ratio of at least 10, a preceding singular value above 10−710^-7 of the largest, and two input columns beyond the selected rank. Separate programs collect observations, extract factors, and validate frozen results. The centered and completed factors share one basis alignment fitted on 256 vocabulary rows, centered using those rows; head errors are evaluated on the remaining 256. Held-out predictions use queried relative coordinates and u; separately recovered test anchors are used only for evaluation. A post-hoc constant baseline predicts the average of the extraction inputs’ recovered means. An input-budget analysis refactors nested extraction prefixes with the held-out inputs fixed. On TinyStories-260K, all three random-prefix input sets yield complete relative-coordinate estimates and exact anchors, pairs, and priorities for every extraction and held-out input. Each full extraction identifies rank 64, matching the hidden width. Table IV reports head reconstruction and prediction of held-out mean logits from their queried relative coordinates. TABLE IV: Token-only extraction of TinyStories-260K’s complete 512×64512× 64 output head and held-out mean prediction. Input set Extraction/test Exact anchors Relative head error Mean RMSE Token decisions HTTP batches 1 96/16 96/96 3.96×10−63.96× 10^-6 3.11×10−73.11× 10^-7 1,370,786 45,873 2 192/32 192/192 1.19×10−61.19× 10^-6 4.59×10−74.59× 10^-7 2,740,717 91,584 3 96/16 96/96 3.35×10−63.35× 10^-6 7.21×10−77.21× 10^-7 1,372,446 46,928 Notes. Independent input sets use the same checkpoint. Head error is relative Frobenius error on 256 vocabulary rows excluded from basis alignment. Mean-logit RMSE evaluates prediction from held-out relative coordinates without their anchors. Anchor counts and costs cover extraction inputs only; all 64 separately evaluated test anchors are also exact. HTTP requests batch independently biased decisions. Probability queries: zero. Recovery of the shared component. Under the shared basis alignment, the completed head has relative Frobenius error 1.19×10−61.19× 10^-6–3.96×10−63.96× 10^-6. The relative branch recovers its own target CWCW with RMSE 3.50×10−73.50× 10^-7–1.10×10−61.10× 10^-6, but differs from the original W by RMSE 0.1890.189 because it omits the shared row. The recovered shared row has RMSE 8.25×10−88.25× 10^-8–4.32×10−74.32× 10^-7. Figure 4(a) shows the entrywise absolute errors before and after this completion. The extracted representation is WGWG, retaining the invertible hidden-basis ambiguity. For the 64 held-out inputs, the learned map predicts the mean from relative coordinates with RMSE 3.11×10−73.11× 10^-7–7.21×10−77.21× 10^-7. Predicting the average extraction-input mean instead gives RMSE 0.8950.895–1.1621.162, showing the variation captured by the map. Combining the predicted means with relative coordinates gives complete original-logit RMSE 2.15×10−62.15× 10^-6–3.60×10−63.60× 10^-6. Appendix G reports numerical diagnostics. Fig. 4: Token-only output-head extraction from TinyStories-260K across three independently generated input sets. (a) Empirical distributions of absolute entry errors against the original W, after the same basis alignment. Solid curves include the recovered shared component; dashed curves omit it. The relative branch’s own target is CWCW. (b) Output-head relative error as the extraction input count increases, with each set’s test inputs fixed. Crosses mark inferred ranks below 64; set 2 is unresolved at 32–64 inputs and has no corresponding error points. Lines connect measured counts. Input budget and query cost. At 32–64 inputs, the rank rule selects rank 1 or remains unresolved: its full-rank candidate needs more than 64 input columns. All three sets select rank 64 at 72 inputs. For set 2, increasing the budget from 96 to 192 reduces relative head error from 2.75×10−62.75× 10^-6 to 1.19×10−61.19× 10^-6 (Figure 4(b)). Across the three sets, mean cost per extraction input is about 14,300 token decisions: 255–267 for the anchor and 14,019–14,029 for relative coordinates. Anchors account for about 1.8% of decisions; their sequential queries and the batched relative-coordinate probes have different latency costs. Table IV separates extraction cost from held-out evaluation. V-D RQ4: Execution Stability and Response Noise Stability of the numerical target. Greedy decoding can produce execution-dependent floating-point scores [24]. We test Pythia-14M and Pythia-160M with eight new inputs each, CPU FP32 inference, and batch sizes 1,2,4,81,2,4,8. Each input–batch condition repeats six probes ten times: no bias, early and final gap search, priority identification, residue identification, and final anchor search. We also repeat complete recovery at fixed batch sizes 1 and 4. Across 3,840 generation calls, unmodified greedy tokens remain unchanged, but the selected Pythia-14M and Pythia-160M scores move by up to 5 and 2 ULPs across batch conditions. Scores remain bitwise stable within each fixed input–batch condition. Complete recovery succeeds on 16/16 inputs at each of batch sizes 1 and 4, although 15/16 recovered pairs differ between conditions. Fixing execution conditions defines a stable recovery target in this study. Errors in responses about a fixed target. To isolate response noise from score changes, we inject independent token-response flips with probability pflip∈0,0.01,0.05,0.1,0.2,0.3,0.4p_flip∈\0,0.01,0.05,0.1,0.2,0.3,0.4\ into the 504 previously covered FP32 cases. Five noise seeds per case give 2,520 simulated recoveries per method and noise level. Token pairs and public intervals are fixed before injection; the 200 out-of-range cases and pair discovery are outside this conditional study. We compare four binary-response rules. Single uses one response; Majority-9 votes over nine. The sequential likelihood-ratio rule (LR) uses error bound p¯flip=max(0.1,pflip) p_flip= (0.1,p_flip) [25]. The confidence-interval rule (CI) samples until an interval excludes 1/21/2, without a noise-rate bound, allocating error across predetermined sample sizes [26]. Both use recovery-wide error budget δ=0.01δ=0.01 for at most M=50M=50 logical comparisons. Each comparison is capped at 4,096 responses; an undecided comparison stops recovery without an estimate. Appendix F specifies the assumptions and tests. We report exact recovery and mean token responses after pair discovery. Fig. 5: Exact recovery and cost under independent response flips. Single uses one response per comparison; Majority-9 uses nine-response voting; LR and CI use sequential likelihood-ratio and confidence-interval tests, respectively. Each point aggregates 504 fixed in-range pairs and five noise seeds. Cost counts responses after pair discovery. The inset enlarges LR and CI near 100%; the cost axis is logarithmic. Lines connect measured conditions. Figure 5 and Table V evaluate independent flips of responses from fixed in-range pairs. At 10% flips, LR achieves higher recovery and lower mean cost than nine-response voting. LR uses a noise-rate upper bound; CI recovers all simulated cases at higher cost without that bound. The 2,520 attempts comprise 504 numerical cases and five noise seeds. TABLE V: Recovery at 10% independent response flips. Rule Exact recovery Mean responses Single 17/2,520 47.8 Majority-9 2,411/2,520 437.4 LR 2,502/2,520 242.7 CI 2,520/2,520 2,944.2 Notes. Rules are defined in Section V-D. Exact recovery requires both scores and priority; all 2,520 attempts per rule are counted. Responses include repetitions after pair discovery. No attempt is unresolved at this noise level. A separate condition selects uniformly between tokens with equal maximum rounded scores; strict comparisons remain exact. We compare CI with a rule that emulates a chosen fixed priority: return token 1 only when all 13 repeated responses are token 1, and token 0 otherwise. Its guarantee and score-pair recovery objective are given in Appendix F. Under uniform random token tie breaking, CI leaves all 2,520 cases unresolved. Emulating fixed priority instead recovers 2,516/2,5202,516/2,520 score pairs at mean cost 632.1. These rules estimate comparisons for fixed scores; they do not undo the score changes measured in the batch experiment. VI Defense VI-A Subtracting the Maximum before Bias Addition Subtract the maximum score before adding any bias (Figure 1(d)): Ai(q)=RN32(qi−maxjqj)A_i(q)=RN_32(q_i- _jq_j), computed coordinatewise. Proposition 1 (Max subtraction before bias addition). For every allowed input x, let the finite stored binary32 vectors satisfy the exact real relation q′(x)=q(x)+c(x)q (x)=q(x)+c(x)1. Both services apply correctly rounded max subtraction A with consistent signed-zero handling. Suppose subsequent observable responses depend on the original scores only through A(q(x))A(q(x)), under the same conditional response kernel given the public history, input, and controls, including tie breaking and internal randomness. Then every adaptive observer obtains identical transcript distributions for any finite query budget or almost-surely finite stopping rule. Proof. For each input, qi′−maxjqj′=qi−maxjqjq _i- _jq _j=q_i- _jq_j. Correct rounding therefore gives A(q′)=A(q)A(q )=A(q) bit for bit. Couple the observer’s randomness and the services’ responses. Equal histories give the same next query and response distribution, so induction gives identical histories at every finite round. A common stopping rule yields identical stopped transcripts. ∎ The exact-shift condition applies to the stored vectors. If a common shift is rounded earlier, it can already change their relative scores. For example, binary32 q=(1,1−2−24)q=(1,1-2^-24) centers to (0,−2−24)(0,-2^-24), whereas componentwise rounding of q+224q+2^241 gives (224,224)(2^24,2^24), which centers to (0,0)(0,0). The prior rounding has changed the relative scores, so the proposition’s exact-shift premise no longer holds. When the maximum is subtracted after bias addition, rounding has already mapped the candidates to potentially different vectors. A later operation that preserves the winning token retains the distinguishing response. Stable softmax implementations already subtract a maximum to avoid overflow [10, 27]; the relevant question is its position relative to the exposed control. VI-B Effect on Distinguishing Queries The suppression experiment compares the original processor with subtraction of the maximum score either before or after bias addition. We take eight cached positions from each of two FP32 cohorts, Pythia-14M and Qwen3-0.6B, round their complete score vectors to grids 2−82^-8 and 2−62^-6, and apply shifts −0.25,−0.125,0,0.125,0.25-0.25,-0.125,0,0.125,0.25. This gives 160 conditions in 32 groups, with bitwise-identical complete max-centered and FP32 log-softmax vectors within each group. We freeze recovery predictions and replay the union of observed bias payloads across shifts. Figure 6 compares cross-shift winner disagreements for the original pipeline (Original), max subtraction before bias addition (Pre), and max subtraction afterward (Post). Applying 9,742 union payloads to the five shifts in each group gives 48,710 condition–payload evaluations and 38,968 cross-shift comparisons. Pre removes all 2,578 original disagreements; Post retains them. Fig. 6: Cross-shift winner disagreements under three processing orders. Pre and Post apply max subtraction before and after bias addition, respectively. Rates use 38,968 comparisons: 2,578 disagreements for Original and Post, and 0 for Pre. The experiment uses fixed-grid decoder groups; unperturbed probabilities and argmax decisions remain bitwise unchanged in all 160 conditions. Preprocessing changes 10,038 of the 48,710 original biased outputs. This is expected for payloads selected to expose fine rounding behavior: suppressing the channel changes those responses. In contrast, all 160 unperturbed argmax, softmax, and log-softmax outputs remain bitwise identical. VI-C Bias-Control Preservation To measure bias-control preservation, we use 16 evenly spaced inputs from each of the six serving configurations: 96 original cached vectors. Each receives 96 random sparse biases, with eight tokens from the full vocabulary or top 64 and 16 draws per support and amplitude 0.5,2,100.5,2,10. It also receives 33 target-only biases RN32(d+kU/8)RN_32(d+kU/8), k=−16,…,16k=-16,…,16, where d is the top-two gap and U is the binary32 spacing at their maximum absolute score. We compare processing orders, complete FP32 probabilities, and two fixed offline tie-breaking rules: smallest or largest token ID. All payloads satisfy H=100H=100. The inputs, seed, and bias-generation rules are fixed before execution; Appendix H gives the details. Pre-bias max subtraction preserves all 96 unperturbed greedy decisions and complete probability vectors bit for bit. It changes no greedy decision across 9,216 random biases; the mean complete-probability L1L_1 difference is 6.08×10−86.08× 10^-8 and its maximum is 9.41×10−79.41× 10^-7. The biases themselves change the unperturbed winner in 1,776 cases, including 1,737 of 4,608 top-64 controls. Among 3,168 biases near the comparison threshold, preprocessing changes 738 maximizing sets and 369 greedy decisions under each fixed tie-breaking rule. Every change separates an original rounded tie; no strict-winner reversal occurs. Post-bias subtraction preserves all decisions and complete probabilities. Independent NumPy replay reproduces every FP32 processing vector. Thus, fine control near a decision boundary can change token selection even when the probability difference is small. Processing order also matters for other controls. Repetition penalties can depend on the common additive component, so moving max subtraction across such a penalty can change its effect. VII Related Work VII-A Prediction-Interface and Language Model Extraction Tramèr et al. established extraction from prediction interfaces [2]; subsequent work studies high-fidelity extraction and its relationship to recovered parameters [28, 29]. For language models, Morris et al. use biases and token responses to recover probability information [5]. Carlini et al. recover output-projection information and develop efficient logprob-free score-difference attacks [3]. Finlayson et al. analyze the low-dimensional linear structure exposed by API logits [4]. The softmax bottleneck is established [30]. Our contribution concerns the common additive component left unresolved by ideal normalized or biased-argmax observations. Geometric output signatures use constraints shared across model inputs. Finlayson et al. exploit an ellipse associated with the output projection for source verification [31]. We study a per-input observation that recovers original logit values. VII-B Finite Precision and Numerical Side Channels Numerical analysis studies rounding and evaluation order [10, 11, 9], including stable softmax [27]. Andrysco et al. demonstrate operand-dependent floating-point timing leakage, including subnormal arithmetic [32]. We instead observe token selection under chosen biases near a normal exponent boundary. Machine-learning implementations also expose execution-dependent signatures. iNNformant uses numerical differences in inference for device identification [33], and HSPI studies hardware signatures in model outputs [34]. Our setting fixes the implementation and recovers a score component through chosen decoder controls. Quantamination exploits shared dynamic-quantization parameters to leak data across a batch [35], while FloatDoor modifies a model to turn platform-dependent numerical differences into a backdoor trigger [36]. Our mechanism requires neither a co-batched victim input nor modified model weights; it recovers the numerical values entering the exposed bias operation. Kadav et al. study softmax-preserving output-head reparameterizations to improve output-head quantization [8]. Their work establishes that ideal-equivalent representatives can behave differently after quantization. Our direction is observational: determine which representative-dependent information can be identified externally from token responses, and provide a local recovery construction. VII-C Decoder Controls and Interface Semantics Decoding controls operate on model logits. Repetition penalties originate in work such as CTRL [37]; Baichuan 2 discusses their dependence on logit values beyond relative differences [38]. This dependence on the common additive component is therefore established. Our recovery construction instead concerns the rounding of an additive bias, whose ideal real-arithmetic counterpart preserves the common-additive-shift ambiguity. Transformers and vLLM expose implementations of logit processing and token selection [19, 39]. We analyze the rounded addition used by such processors and evaluate recovery end to end in LLM serving systems, linking the numerical operator to observable token responses. VIII Discussion VIII-A Range Identification and Future Work Opaque endpoints require checks of bias resolution, operation order, score stability, and reachable ranges. Controlled passive-equality experiments isolate the mechanism; its prevalence in deployments requires separate measurement. Range selection affects both coverage and cost. Outside the calibrated intervals, erroneous candidates often preserve the score gap while shifting both logits toward a window boundary. The seven single-window false acceptances in Section V-B illustrate this ambiguity; the second window rejects those candidates by providing a contradictory response. Globally justified range tests and recovery across binades remain open problems under bounded biases. Extending complete output-head extraction to larger vocabularies requires measuring the relative-coordinate cost, which dominates the present study. Variable execution conditions require a target definition that accounts for changing scores; independent response-noise guarantees apply to a fixed target. VIII-B Research Ethics and Reproducibility We use public checkpoints, synthetic inputs, research-controlled CPU/GPU resources, and one output token per bias setting; the output-head experiment batches independent settings. Controlled variants modify locally held vectors. All fixed cases remain in the records, including unmet preconditions. The study collects no other users’ data and involves no access-control bypass. Our experimental records include input generators, pinned model revisions, query transcripts, frozen predictions, and CPU replay checks (Appendix H). We are preparing disclosure materials for the vLLM, SGLang, and llama.cpp maintainers, including minimal reproductions, tested configurations, and the pre-bias centering analysis. We plan to coordinate validation and mitigation with the maintainers. Use of generative AI. We used ChatGPT and Codex to assist literature review, theoretical development, experimental design, implementation, data analysis, and manuscript drafting, revision, and translation. The authors are responsible for reviewing and validating AI-assisted material and for the accuracy and originality of the final work. IX Conclusion Floating-point bias addition can disclose the common additive component of logits that ideal probabilities and ideal bias comparisons leave unidentified. We characterize this difference, provide a local binary32 recovery method with explicit query bounds, and verify exact recovery through complete generation calls. A token-only output-head experiment verifies the recovery of the shared row component under an invertible change of hidden basis. Preprocessing-order experiments identify a condition that suppresses disclosure. The results show that the numerical implementation of a token-only interface is part of its information-disclosure behavior. References [1] OpenRouter (2026) API parameters. Note: Accessed 2026-10-01 External Links: Link Cited by: §I. [2] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart (2016) Stealing machine learning models via prediction APIs. In 25th USENIX Security Symposium, p. 601–618. Cited by: §I, §VII-A. [3] N. Carlini, D. Paleka, K. D. Dvijotham, T. Steinke, J. Hayase, A. F. Cooper, K. Lee, M. Jagielski, M. Nasr, A. Conmy, E. Wallace, D. Rolnick, and F. Tramèr (2024) Stealing part of a production language model. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 5680–5705. Cited by: Appendix G, §I, §I-A, §I-A, §I-D, §I-H, §VII-A. [4] M. Finlayson, X. Ren, and S. Swayamdipta (2024) Logits of API-protected LLMs leak proprietary information. In Conference on Language Modeling, Cited by: §I, §VII-A. [5] J. X. Morris, W. Zhao, J. T. Chiu, V. Shmatikov, and A. M. Rush (2024) Language model inversion. In International Conference on Learning Representations, Cited by: §I, §VII-A. [6] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §I-A. [7] Hugging Face (2026) Utilities for generation. Note: Transformers 4.57.1 documentationAccessed 2026-10-02 External Links: Link Cited by: §I-A, §IV-A. [8] A. Kadav, C. Flores, C. Arora, V. Kotte, H. Zheng, L. Yan, P. Shanmugasundaram, and T. H. King (2026) Softmax reparameterization for output-head quantization. Note: arXiv:2609.31291Version 2 Cited by: §I-A, §VII-B. [9] IEEE (2019) IEEE standard for floating-point arithmetic. Note: IEEE Std 754-2019 External Links: Document, Link Cited by: §I-B, §VII-B. [10] D. Goldberg (1991) What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys 23 (1), p. 5–48. External Links: Document Cited by: §I-B, §VI-A, §VII-B. [11] N. J. Higham (2002) Accuracy and stability of numerical algorithms. 2 edition, Society for Industrial and Applied Mathematics. External Links: Document Cited by: §I-B, §VII-B. [12] S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, A. Skowron, L. Sutawika, and O. Van Der Wal (2023) Pythia: a suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 2397–2430. Cited by: §IV-A. [13] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever (2019) Language models are unsupervised multitask learners. Note: OpenAI technical report External Links: Link Cited by: §IV-A. [14] S. Black, L. Gao, P. Wang, C. Leahy, and S. Biderman (2021) GPT-Neo: large scale autoregressive language modeling with mesh-tensorflow. Note: Zenodo External Links: Document, Link Cited by: §IV-A. [15] S. Zhang, S. Roller, N. Goyal, et al. (2022) OPT: open pre-trained transformer language models. Note: arXiv:2205.01068 Cited by: §IV-A. [16] BigScience Workshop (2022) BLOOM: a 176b-parameter open-access multilingual language model. Note: arXiv:2211.05100 Cited by: §IV-A. [17] L. Ben Allal, A. Lozhkov, E. Bakouch, et al. (2025) SmolLM2: when smol goes big—data-centric training of a fully open small language model. In Conference on Language Modeling, External Links: Link Cited by: §IV-A. [18] A. Yang, A. Li, B. Yang, et al. (2025) Qwen3 technical report. Note: arXiv:2505.09388 Cited by: §IV-A. [19] T. Wolf, L. Debut, V. Sanh, et al. (2020) Transformers: state-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, p. 38–45. External Links: Document Cited by: §IV-A, §VII-C. [20] W. Kwon, Z. Li, S. Zhuang, et al. (2023) Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, p. 611–626. External Links: Document Cited by: §IV-A. [21] L. Zheng, L. Yin, Z. Xie, et al. (2024) SGLang: efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: §IV-A. [22] ggml-org Contributors (2026) Llama.cpp server. Note: Commit 7f2d88b0ac393357ae6a9e1992185a48c20b13b External Links: Link Cited by: §IV-A. [23] A. Karpathy TinyLlamas: TinyStories-260K Checkpoint. Note: Hugging Face model repositoryRevision 0bd21da7698eaf29a0d7de3992de8a46ef624add, stories260K; accessed October 9, 2026 External Links: Link Cited by: §IV-A. [24] H. He (2025) Defeating nondeterminism in LLM inference. Note: Thinking Machines Lab External Links: Link Cited by: §V-D. [25] A. Wald (1945) Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics 16 (2), p. 117–186. External Links: Document Cited by: §F-A, §V-D. [26] S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon (2021) Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics 49 (2), p. 1055–1080. External Links: Document Cited by: §F-B, §V-D. [27] P. Blanchard, D. J. Higham, and N. J. Higham (2021) Accurately computing the log-sum-exp and softmax functions. IMA Journal of Numerical Analysis 41 (4), p. 2311–2330. External Links: Document Cited by: §VI-A, §VII-B. [28] M. Jagielski, N. Carlini, D. Berthelot, A. Kurakin, and N. Papernot (2020) High accuracy and high fidelity extraction of neural networks. In 29th USENIX Security Symposium, p. 1345–1362. Cited by: §VII-A. [29] N. Carlini, M. Jagielski, and I. Mironov (2020) Cryptanalytic extraction of neural network models. In Advances in Cryptology—CRYPTO 2020, Lecture Notes in Computer Science, Vol. 12172, p. 189–218. External Links: Document Cited by: §VII-A. [30] Z. Yang, Z. Dai, R. Salakhutdinov, and W. W. Cohen (2018) Breaking the softmax bottleneck: a high-rank RNN language model. In International Conference on Learning Representations, Cited by: §VII-A. [31] M. Finlayson, X. Ren, and S. Swayamdipta (2026) Every language model has a forgery-resistant signature. In International Conference on Learning Representations, Cited by: §VII-A. [32] M. Andrysco, D. Kohlbrenner, K. Mowery, R. Jhala, S. Lerner, and H. Shacham (2015) On subnormal floating point and abnormal timing. In 2015 IEEE Symposium on Security and Privacy, p. 623–639. Cited by: §VII-B. [33] A. Schlögl, T. Kupek, and R. Böhme (2021) iNNformant: boundary samples as telltale watermarks. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, p. 81–86. External Links: Document, Link Cited by: §VII-B. [34] C. Zhang, H. Foerster, R. D. Mullins, Y. Zhao, and I. Shumailov (2025) Hardware and software platform inference. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 74882–74895. Cited by: §VII-B. [35] H. Foerster, I. Shumailov, C. Zhang, Y. Zhao, J. Hayes, and R. Mullins (2026) Quantamination: dynamic quantization leaks your data across the batch. Note: arXiv:2604.26505 Cited by: §VII-B. [36] N. Loose, J. Sander, F. Mächtle, and T. Eisenbarth (2026) FloatDoor: platform-triggered backdoors in LLMs. Note: arXiv:2606.19535 Cited by: §VII-B. [37] N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher (2019) CTRL: a conditional transformer language model for controllable generation. Note: arXiv:1909.05858 Cited by: §VII-C. [38] A. Yang, B. Xiao, B. Wang, et al. (2023) Baichuan 2: open large-scale language models. Note: arXiv:2309.10305 Cited by: §VII-C. [39] vLLM Contributors (2026) vLLM V1 GPU sampling implementation. Note: Commit cb4f016c5d02c9d483e6e3f1702d51da239b1a9b External Links: Link Cited by: §VII-C. [40] D. Kalamkar, D. Mudigere, N. Mellempudi, D. Das, K. Banerjee, S. Avancha, D. T. Vooturi, N. Jammalamadaka, J. Huang, H. Yuen, J. Yang, J. Park, A. Heinecke, E. Georganas, S. Srinivasan, A. Kundu, M. Smelyanskiy, B. Kaul, and P. Dubey (2019) A study of BFLOAT16 for deep learning training. Note: arXiv:1905.12322 Cited by: Appendix D. Appendix A Boundary Cases in Local Recovery The recovery proof uses independently formed payloads. If a processor instead evaluates RN32(RN32(qi+B)+d)RN_32(RN_32(q_i+B)+d), it can have a different rounding history from RN32(qi+RN32(B+d))RN_32(q_i+RN_32(B+d)). The artifact checks the latter operation order and rejects payloads that are not exactly representable or exceed the allowed budget. This distinction matters most near the exponent boundary used by the probes. For the tie-breaking priority test, p=maxℓ,F+d0+Δp= \ ,F+d_0+ \ satisfies p≤ap≤ a under both hypotheses because t>Ft>F and the scores lie on the Δ grid. The last pivot is at most u+2Δ≤Pu+2 ≤ P. Its anchor bias is therefore nonnegative, and the first target bias is at most H0H_0; later target biases are smaller. When the true scores are equal, the original-anchor definition enforces anchor tie-breaking priority and the same test applies. For residue identification, the two responses at p0,p0+Δp_0,p_0+ are 00,10,11,0100,10,11,01 for residues 0,1,2,30,1,2,3, apart from k=0k=0, which also produces 0101. The query at p0+3Δp_0+3 produces one for k=0k=0 and zero for nonnegative residue-three candidates. If p0<up_0<u, then p0≤u−Δ≤P−3Δp_0≤ u- ≤ P-3 , so that extra pivot is valid. If p0=up_0=u, the algorithm returns before making it. The lower-bound state count includes the unknown pairwise tie-breaking priority. For each anchor index i, the i−1i-1 strictly lower targets admit two priorities, while the equal target admits only the original-anchor priority. Summing 2(i−1)+12(i-1)+1 over i=1,…,K−2i=1,…,K-2 gives (K−2)2(K-2)^2. If only the score pair must be returned and tie-breaking priority is a nuisance variable, the number of distinct pairs is (K−2)(K−1)/2(K-2)(K-1)/2, giving a 45-query counting lower bound for binary32. The 46-query comparison in the main text therefore explicitly includes tie-breaking priority. Allowing nonbinary responses also changes the argument. Appendix B A Bounded-Bias Nonidentifiability Region Proposition 2 (An explicit BF16 nonidentifiability region). Let q be a finite BF16 vector, let M=maxiqi∈360,362,…,410M= _iq_i∈\360,362,…,410\, and define qi′=RNBF16(qi−2)q _i=RN_BF16(q_i-2). Under (4), allowing arbitrary simultaneously biased tokens with every payload in [−100,100][-100,100], the complete sets of maximizing indices are equal for q and q′q for every query. They therefore have identical adaptive winner transcripts under the same fixed tie-breaking order. Proof. Let S=i:qi≥M−200S=\i:q_i≥ M-200\. For i∈Si∈ S, 160≤qi≤410160≤ q_i≤ 410 and subtracting two is exact in BF16, including across 256256. Hence qi′=qi−2q _i=q_i-2. Every other score is at most M−201M-201, and its shifted-and-rounded value is at most M−203M-203. With biases bounded by 100100, such a token remains strictly below a maximum-score token in both worlds and cannot win. For i∈Si∈ S, put βi=RN32(bi) _i=RN_32(b_i) and xi=qi+βix_i=q_i+ _i, where the latter denotes the exact real sum. Its maximum x∗x_* lies in [M−100,M+100]⊂[260,510][M-100,M+100]⊂[260,510]. The translated maximum lies in [258,508][258,508]. Both maxima and their rounding cells are strictly inside the binary32 binade [256,512)[256,512), with spacing δ=2−15δ=2^-15. Translation by −2=−65536δ-2=-65536δ shifts each relevant rounding cell by an even number of grid steps, preserving ties-to-even endpoint membership. Therefore RN32(xi)=RN32(x∗)⇔RN32(xi−2)=RN32(x∗−2). gatheredRN_32(x_i)=RN_32(x_*)\\ _32(x_i-2)=RN_32(x_*-2). gathered This equivalence also holds for xix_i outside the maximum’s binade because membership is in the translated maximum rounding cell. Thus the complete maximizing sets agree. The same tie-breaking order gives equal responses to every query; induction on the resulting histories gives identical adaptive transcripts. Rounding the most negative finite BF16 value after subtracting two remains finite, so q′q is a valid score vector. ∎ Appendix C Negative-Binade Construction The sign of the scores affects which side of an exponent boundary has finer spacing. This distinction permits a separate construction for negative model scores; no common score shift is inserted into the service. Let the addition format have binary precision p≥3p≥ 3, and let S=2eS=2^e be public. Set δ=S/2p−1δ=S/2^p-1, K=⌊min(H,S)/δ⌋K= (H,S)/δ , H0=KδH_0=Kδ, and F=−S−H0F=-S-H_0. Assume K≥2K≥ 2, that the original greedy winner has score a, and that the selected target has score t, with a≥t>F,a∈[ℓ,u]⊂(F,−S−δ].a≥ t>F, a∈[ ,u]⊂(F,-S-δ]. The endpoints are on the δ lattice. Both scores therefore lie in the negative binade (−2S,−S)(-2S,-S). The interface uses separately cast biases, a single correctly rounded addition, and an unknown fixed global tie-breaking order. All constructed biases are nonnegative and at most H0H_0. The format must represent δ/2δ/2; we exclude overflow and flushing of the required subnormal operands. The following proves Theorem 2 and its precision-dependent extension. Proof. Write d=a−td=a-t. Adding jδjδ to the target gives a monotone threshold because all score sums near the comparison remain exactly on the original lattice. Let j be the first winning integer. Binary search costs at most ⌈log2K⌉ _2K queries and leaves two hypotheses: d=d0=(j−1)δd=d_0=(j-1)δ with anchor tie-breaking priority, or d=d0+δd=d_0+δ with target tie-breaking priority. The search endpoints follow from the original winner and t+H0>−S>at+H_0>-S>a. Set r=F+d0+δr=F+d_0+δ and submit biases ba=−S−r,bt=−S−r+d0+δ/2.b_a=-S-r, b_t=-S-r+d_0+δ/2. These are legal: bt=H0−δ/2b_t=H_0-δ/2 and 0≤ba≤H00≤ b_a≤ H_0. Under anchor priority, a−r=t−F−δ≥0a-r=t-F-δ≥ 0, so the two exact sums are x and x+δ/2x+δ/2, with x=−S+(a−r)≥−Sx=-S+(a-r)≥-S. Both are representable in the finer region, and the target wins. Under target priority, a−r=t−F≥δa-r=t-F≥δ, and the exact sums are x and x−δ/2x-δ/2, both strictly in the finer region; the anchor wins. Thus one query determines the true gap and tie-breaking priority. For a pivot r in the remaining legal range, first align the two exact sums to x=−S+(a−r)x=-S+(a-r) using biases −S−r-S-r and −S−r+d-S-r+d. Increase the bias of the lower-priority token by δ/2δ/2. With k=(a−r)/δk=(a-r)/δ, its winning indicator is g(k)=1,k≥0,1,k<0 and k odd,0,k<0 and k even.g(k)= cases1,&k≥ 0,\\ 1,&k<0 and k odd,\\ 0,&k<0 and k even. cases For k≥0k≥ 0, the half-step is exactly representable. For k<0k<0, it is a midpoint in the coarser binade; ties-to-even rounds upward precisely for odd k. Choose the initial residue-test pivot r=u+δr=u+δ. It satisfies r>ar>a and r≤−Sr≤-S, so the response identifies the parity of (a−r)/δ(a-r)/δ without a boundary ambiguity. Restrict subsequent candidate pivots to the inferred parity class. Then k is even, and the response is exactly [a≥r]1[a≥ r]. Binary search on at most ⌈N/2⌉ N/2 candidates completes recovery. Every parity-test or search pivot is at least F+d+δF+d+δ. Thus its target payload, even after the half-step increment, is at most H0−δ/2H_0-δ/2; the other payload is nonnegative and no larger. All payloads are exact multiples of δ/2δ/2 within [0,S][0,S], hence representable. The anchor is never decreased, so no unmodified token can displace the original winner under the fixed global tie order. Each response is consequently binary. Since K≤2p−1K≤ 2^p-1 and N≤K−1N≤ K-1, the upper bound is 2p−12p-1. For the lower bound, take H0=SH_0=S and scores a=F+iδa=F+iδ, t=F+jδt=F+jδ with 1≤j≤i≤K−11≤ j≤ i≤ K-1. There are (K−1)2(K-1)^2 allowed score/priority triples. A binary decision tree requires at least ⌈log2(K−1)2⌉=2p−2 _2(K-1)^2 =2p-2 queries to identify all of them. This count concerns triple recovery, not score-only recovery or an oracle with more than two possible responses. ∎ Other known rounding directions. The finer-region gap-disambiguation query is exact and does not depend on the rounding direction. The final comparison is adapted to the specified rounding direction. Under rounding toward negative infinity, or nearest with ties away from zero, the lower-priority half-step increment wins exactly when a≥ra≥ r. Under rounding toward positive infinity, or toward zero (all constructed sums are negative), instead decrease the higher-priority token’s aligned bias by δ/2δ/2. The lower-priority token then wins exactly when a>ra>r. The latter probe is legal for search pivots r≤u≤−S−δr≤ u≤-S-δ. Ordinary binary search gives the same 2p−12p-1 uniform bound. These are algorithms for a known rounding direction, not an identification theorem for an unknown runtime mode. Nearest-away is analyzed as an IEEE-defined direction; its support is not assumed for binary hardware. Appendix D Storage Precision and Control Resolution Stored-score precision can also reduce the candidate set. Let s be the significand precision of a known score-storage format and let p≥s+2p≥ s+2 be the bias and addition precision. The scores lie in gℤgZ, where g=P2−sg=P2^-s below a positive boundary P and g=S21−sg=S2^1-s below a negative boundary −S-S. The appropriate local range and representability conditions still apply. Storage precision and arithmetic precision are distinct interface properties [40]. Proposition 3 (Recovery with known score precision). Let B=PB=P or B=−SB=-S be the chosen boundary and g the corresponding storage spacing defined above. Set H0=Kg≤H_0=Kg≤ H and F=B−H0F=B-H_0, where K is an integer and 2≤K≤2s−12≤ K≤ 2^s-1. Suppose a,t∈gℤa,t∈ gZ, a≥t>Fa≥ t>F, and a∈[ℓ,u]⊂(F,B−g]a∈[ ,u]⊂(F,B-g], with endpoints on the storage grid. Assume the required bias increments are representable and the arithmetic neither overflows nor flushes required subnormals. With the original anchor, unknown fixed tie-breaking priority, and round-to-nearest, ties-to-even addition, exact pair-and-priority recovery requires at most ⌈log2K⌉+1+⌈log2N⌉≤2s−1 _2K +1+ _2N ≤ 2s-1 (19) binary recovery queries, where N is the number of remaining anchor grid points after recovering the gap. BF16-to-binary32 and binary16-to-binary32 interfaces admit bounds of 15 and 21, respectively. Proof. Search the target-bias threshold on the coarse storage grid. If the first winning bias is jgjg, the remaining hypotheses are d=(j−1)gd=(j-1)g with anchor tie-breaking priority and d=jgd=jg with target tie-breaking priority. A single bias (j−1/2)g(j-1/2)g on the target gives score a+g/2a+g/2 under the former and a−g/2a-g/2 under the latter. Both are exactly representable in the arithmetic format, so this query identifies the gap and tie-breaking priority. For a positive boundary P, let Δ=P/2p =P/2^p be the arithmetic spacing below P. The storage spacing satisfies g=2p−sΔg=2^p-s . Choose pivots on the storage grid, align the scores at P+(a−r)P+(a-r), and increase the lower-priority token’s bias by Δ . Since (a−r)/Δ(a-r)/ is a multiple of four, the nearest-even response rule reduces directly to [a<r]1[a<r]; no residue-identification queries are needed. For a negative boundary −S-S, let δ=S/2p−1δ=S/2^p-1. Align at −S+(a−r)-S+(a-r) and use an increment δ/2δ/2 instead. Here (a−r)/δ(a-r)/δ is even, so the winning indicator is [a≥r]1[a≥ r]. Each case yields ordinary binary search on the storage candidates. After finding the gap, intersect the public interval with [F+d+g,boundary−g][F+d+g,\,boundary-g]. All search biases are nonnegative and bounded by H0H_0, including the small increment, because g exceeds that increment. They are exactly representable in the finer arithmetic format. Monotonicity and the fixed global tie-breaking order exclude untouched tokens, as in the binary32 construction. The gap and anchor searches each cost at most s−1s-1 queries. ∎ The lattice condition applies to scores entering bias addition; advertised weight or forward precision alone does not establish it. The proposition also requires biases to retain the finer arithmetic precision. A finite-alphabet control. For q=(30,28)q=(30,28) and q′=q−18q =q- 181, fine binary32 biases produce a winner difference. When biases are instead restricted to all finite BF16 values in [−100,100][-100,100], and addition remains binary32, the two pair oracles are identical for both tie-breaking priorities. The same holds for the binary16 bias alphabet. We verified this finite statement by comparing the first-winning target-bias index for every anchor bias: 34,193 BF16 values and 44,161 binary16 values. Monotonicity extends the threshold comparison to every bias pair in each specified Cartesian alphabet for these two vectors. If both the bias and addition precision are lowered, format-scaled distinguishing probes exist again. A sufficient condition for exact addition. Let g be a representable power of two and p the addition precision. Suppose every hidden score and accepted bias is a multiple of g, and q′=q+cq =q+c1 with c∈gℤc∈ gZ. If every possible biased sum in both vectors has magnitude at most 2pg2^pg, with no overflow, each sum is an exactly representable ngng with |n|≤2p|n|≤ 2^p. All additions are therefore exact. The biased vectors differ by cc1, so their maximizer sets and winners coincide under the same fixed tie-breaking order. Identical responses at each round give identical adaptive transcripts. This representability condition identifies when control quantization removes the nonlinearity used by the channel. Coarse biases alone are insufficient: with binary32 scores q=(224,224−2)q=(2^24,2^24-2), q′=q−2q =q-21, and integer biases (0,3)(0,3), the first vector yields equal rounded scores; the second has a strict target winner. Here the exact-lattice magnitude condition is violated. Appendix E Adaptive Selection Protocol For Table I, enumerate B=±2eB=± 2^e, e=−6,…,10e=-6,…,10, in ascending exponent order, positive before negative. Use each full local interval and require candidates to reproduce the pair’s combined transcript. Set η=|B|/224η=|B|/2^24 for B>0B>0 and η=|B|/223η=|B|/2^23 otherwise. Probe each candidate with b0=RN32(B−a^+sη),b1=RN32(B−t^+(s+u)η).b_0=RN_32(B- a+sη), b_1=RN_32(B- t+(s+u)η). Enumerate s=0,±1,±2,±3,±4s=0,± 1,± 2,± 3,± 4 and u=0,±1/4,±1/2,±3/4,±1u=0,± 1/4,± 1/2,± 3/4,± 1, negative before positive within each pair. Retain only legal payloads with b0≥0b_0≥ 0 and reject at the first contradiction. At most 81 checks per candidate precede the unique-candidate rule and eight-pair limit in Section I-G. The two-window controller uses P∈16,32P∈\16,32\, η=P/224η=P/2^24, s∈0,−1,1s∈\0,-1,1\, and u∈−3/4,3/4u∈\-3/4,3/4\. The cached ablations freeze predictions before scoring; the two BF16 controllers return identical selected pairs, scores, and priorities. Their full replay counts and input hashes are retained in the artifact. Complete query costs. The 34-window generation controller uses 155,390 calls on 64 GPT-Neo inputs; its median/maximum is 2,245/3,927. On 32 BF16 OPT inputs, generic and BF16-aware recovery use 40,486 and 14,048 calls, with medians 1,184.5 and 413. Generic Qwen3 recovery uses 33,915 calls on 32 inputs, with median 994.5. Native Qwen3 totals are 15,312/15,417/14,768 for vLLM/SGLang/llama.cpp across 128 inputs each. Mean costs decompose into 2.2/2.2/2.2 selection calls, 111.4/112.2/107.0 window-search calls, and 6.0/6.0/6.2 checks. Repeated payloads and retries are included; calibration and independent replay are excluded. Single-window ablation. With eight-pair selection, H=100H=100, and unchanged checks, P=16P=16 yields 64/64/60 Qwen3 recoveries across vLLM/SGLang/llama.cpp; P=32P=32 yields 37/36/36. SmolLM2 yields 1/64 and 64/64 per framework, respectively. Qwen3 on llama.cpp returns four incorrect candidates at P=16P=16 and three at P=32P=32; other failures are unresolved. All seven pass single-window checks outside the local domain but contradict the two-window transcript for the same pair and mask. All 163,242 unique queries match full-vocabulary replay. Appendix F Recovery with Noisy Binary Responses These results condition on a correctly isolated token pair, a fixed score vector in the public interval, and the original arithmetic assumptions. Let v∈0,1v∈\0,1\ be the ideal answer to a logical query. The controller uses at most M such queries on its correct path. Repetitions count as separate token responses. An undecided test stops recovery without a score estimate; the error bounds below concern returning an incorrect estimate, not the probability of abstention. F-A Known Response-Error Bound Assume each response, conditional on the complete preceding history, disagrees with v with probability at most 0<p¯flip<1/20< p_flip<1/2. Set c=1−p¯flipp¯flip,h=⌈log(M/δ)logc⌉.c= 1- p_flip p_flip, h= (M/δ) c . (20) For repeated responses Yj∈0,1Y_j∈\0,1\, accumulate Sn=∑j=1n(2Yj−1)S_n= _j=1^n(2Y_j-1). Return 1 at Sn≥hS_n≥ h and 0 at Sn≤−hS_n≤-h. If neither boundary is reached by the response cap, abstain. This is a sequential likelihood-ratio rule [25] with a threshold chosen for a nonasymptotic, recovery-wide guarantee. Proposition 4 (Error control for adaptive recovery). Under these assumptions, substituting the sequential rule for every logical comparison gives probability at most δ of returning an incorrect score pair or tie-breaking priority. Proof. For v=0v=0, the process cSnc^S_n is a nonnegative supermartingale starting at 1, since [c2Yj−1∣ℱj−1]≤p¯flipc+(1−p¯flip)c−1=1.E[c^2Y_j-1 _j-1]≤ p_flipc+(1- p_flip)c^-1=1. The maximal inequality bounds the probability of reaching the wrong boundary by c−h≤δ/Mc^-h≤δ/M. For v=1v=1, apply the same argument to c−Snc^-S_n. Condition on the correct prefix of the adaptive controller and sum over the at most M possible first erroneous comparisons. If none occurs, the deterministic recovery proof applies; stopping at the cap produces no incorrect estimate. ∎ For independent flips with actual probability 0<pflip<1/20<p_flip<1/2, the uncapped test has error probability and expected response count eh(pflip) e_h(p_flip) =11+((1−pflip)/pflip)h, = 11+((1-p_flip)/p_flip)^h, (21) [N] [N] =h1−2pflip(1−2eh(pflip)). = h1-2p_flip(1-2e_h(p_flip)). The absorbing random walk on −h,…,h\-h,…,h\ gives these expressions. At M=50M=50, δ=0.01δ=0.01, and p¯flip=pflip=0.1 p_flip=p_flip=0.1, h=4h=4 gives about five responses per comparison. Odd-r independent majority voting has error at most Mexp[−2r(1/2−p¯flip)2]M [-2r(1/2- p_flip)^2]; r=27r=27 suffices, costing at most 1,350 responses. F-B Unknown Response-Error Rate Suppose repetitions of a query are independent Bernoulli observations with fixed mean p. At sample sizes nj=2jn_j=2^j, use αj _j =δMj(j+1), = δMj(j+1), rj r_j =log(2/αj)2nj, = (2/ _j)2n_j, (22) Ij I_j =[p^j−rj,p^j+rj]∩[0,1]. =[ p_j-r_j, p_j+r_j]∩[0,1]. Hoeffding’s inequality and ∑j≥1αj=δ/M _j≥ 1 _j=δ/M give simultaneous coverage at all scheduled looks. Return 1 only when infIj>1/2 I_j>1/2, or 0 only when supIj<1/2 I_j<1/2. This implements the time-uniform coverage principle [26] through an elementary allocation of error probabilities. If the majority direction equals v for each logical query, the same first-error argument bounds recovery error by δ. Our implementation uses j=1,…,12j=1,…,12 and otherwise abstains. At a uniformly randomized token tie, p=1/2p=1/2 provides no majority direction; increasing the cap cannot justify a fixed-priority conclusion. F-C Random Token Tie Breaking and Changing Scores Assume fixed scores, independent uniform selection between tied maximizers, exact strict comparisons, and no third-token winner or tie. Repeat each query r times, returning 1 only if all responses are 1, and returning 0 otherwise. This emulates token-0 priority, with exact strict comparisons and tie error probability 2−r2^-r. Thus r=⌈log2(M/δ)⌉r= _2(M/δ) bounds recovery error by δ. For M=50M=50 and δ=0.01δ=0.01, r=13r=13 gives at most 650 responses. The recovered priority is the chosen emulation convention, not an intrinsic priority of the randomized service. These guarantees do not follow from a marginal disagreement rate alone. An error shared by all repetitions can make every response support the same wrong direction. Nor is an unspecified changing score vector equivalent to response flips: if observations use q+ξq+ξ, the representation (q+c,ξ−c)(q+c,ξ-c) gives identical observations for any fixed vector c. Identifying a particular unperturbed score therefore requires a restriction on the perturbation or a stable execution condition. The controlled batch experiment in Section V-D tests the latter condition. Appendix G Logit Anchors and Output-Head Extraction Following the factorization argument of Carlini et al. [3], let W∈ℝV×dW ^V× d be a bias-free linear output head and X∈ℝd×nX ^d× n its hidden states. Complete relative logits give D=CWXD=CWX, with C from (2). If CWCW and X have rank d, a rank factorization D=B^X^D= B X satisfies B^=CWG B=CWG and X^=G−1X X=G^-1X for an invertible G. Write =⊤WX/V∈ℝ1×n μ=1 WX/V ^1× n for the row of per-input mean logits. Each entry is determined by an original-logit anchor and its relative coordinates using (2). Then u u =X^⊤(X^X^⊤)−1, = μ X ( X X )^-1, (23) W W =B^+u=WG. = B+1u=WG. Since =(⊤WG/V)X μ=(1 WG/V) X and X X has full row rank, u=⊤WG/Vu=1 WG/V. The identity C+⊤/V=IC+11 /V=I then gives W^=WG W=WG. Anchors may vary by input: each determines its corresponding mean. If centering reduces rank, factor the completed matrix WX=D+WX=D+1 μ instead. These identities assume exact linear outputs; floating-point residuals, relative-coordinate errors, and conditioning affect their numerical accuracy. For held-out inputs, centered coordinates determine hidden coordinates and u predicts the mean without their anchors. Token-only numerical reconstruction. For each non-winning token, bias bisection retains losing and winning endpoints, stopping at width 2−212^-21 or adjacent binary32 values. The negative midpoint estimates its score relative to the unbiased winner. Rounding contributes additional error. If a token cannot be reached within the bias limit, we mark the input incomplete. The artifact includes per-coordinate rounding bounds and independent checks. Numerical diagnostics and evaluation cost. Across input sets 1–3, the rank-64 spectral gaps are 3.09×1043.09× 10^4, 1.07×1051.07× 10^5, and 4.05×1044.05× 10^4; relative singular-value thresholds from 10−410^-4 to 10−710^-7 all retain rank 64. Effective condition numbers are 1,340, 441, and 1,048. Recomputing private WhWh in FP64 gives RMSE 1.40×10−61.40× 10^-6–1.45×10−61.45× 10^-6 against the recorded FP32 logits, a numerical reference for the prediction errors. The held-out mean standard deviations are 0.831, 1.038, and 1.132; the constant-mean baseline uses only recovered extraction means. Held-out evaluation, including its independently scored anchors, adds 228,475/456,905/228,494 decisions and 7,564/15,858/7,500 HTTP requests. Those queries are excluded from the extraction costs in Table IV. Appendix H Reproducibility Details Paired bias-control experiment. The behavior-preservation study uses input indices 0,4,8,12,16,21,25,29,33,37,42,46,50,54,58,630,4,8,12,16,21,25,29,33,37,42,46,50,54,58,63 from each serving configuration and PCG64 seed 2026101201. Random controls choose eight distinct tokens from the complete vocabulary or top 64, then draw independent biases uniformly from [−A,A][-A,A], for A=0.5,2,10A=0.5,2,10. Top tokens are ordered by decreasing original score, then increasing ID. For the two leading tokens A,TA,T, set d=qA−qTd=q_A-q_T and U=nextafter32(m,+∞)−mU=nextafter_32(m,+∞)-m, where m=max(|qA|,|qT|)m= (|q_A|,|q_T|). The targeted controls set only bT=RN32(d+kU/8)b_T=RN_32(d+kU/8), k=−16,…,16k=-16,…,16. All 12,384 payloads are legal under H=100H=100 and distinct within each input. The two tie-breaking rules are fixed offline comparison conventions. Complete probabilities use CPU FP32 softmax, with L1L_1 differences accumulated in FP64. Among threshold-adjacent controls, the original rounded top-two margin is zero in 834 cases, at most one ULP in 1,404 positive-margin cases, and between one and four ULPs in 930 cases. All 369 greedy changes occur in the zero-margin group. Random controls have 299 zero-margin cases and 8,917 cases above 16 ULPs; neither group changes. Serving API configuration. The client sends HTTP requests to each server’s generation endpoint. vLLM: /v1/completions, logit_bias, choices[0].token_ids; SGLang: /generate, sampling_params.logit_bias, output_ids; llama.cpp: /completion, logit_bias, tokens. Each triple gives the endpoint, bias field, and returned IDs. Decimal token IDs map to exact binary32 bias values within [−100,100][-100,100]; probabilities are disabled. The artifact specifies payloads, launch flags, source hashes, and intervals. Artifact availability. The artifact is not publicly available at the time of this preprint. We plan to release these materials in a future revision.