Paper deep dive
Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models
Linkai Peng, Baorian Nuchged
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/10/2026, 2:01:54 AM
Summary
This paper introduces a generation-aligned diagnostic ladder to decompose performance failures in speech language models (SLMs) for paralinguistic tasks into endpoint validity, decision-rule misalignment, and readout-coverage gaps. Using five systems and two emotion corpora, the authors find that state decoding significantly outperforms generation (27.8 accuracy points higher). They demonstrate that a label-free logit correction can improve accuracy by addressing the decision-rule gap, while readout-external information exists but has limited causal influence on emitted answers, distinguishing information availability from behavioral use.
Entities (8)
Relation Signals (5)
state decoding → outperforms → generation
confidence 95% · state decoding exceeds generation by 27.8 accuracy points on average
generation-aligned diagnostic ladder → decomposes → performance failures
confidence 90% · We introduce a generation-aligned diagnostic ladder that compares the emitted answer... Successive differences separate endpoint, decision-rule, and readout-coverage gaps.
Phi-4-MM → evaluatedon → CREMA-D
confidence 90% · It is most pronounced in relative terms for Phi-4-MM on CREMA-D
decision-rule misalignment → isaddressedby → label-free logit correction
confidence 90% · A label-free logit correction improves generated accuracy in every condition, showing that part of the decision-rule gap is actionable.
readout-coverage gap → indicates → Information Availability
confidence 85% · These results distinguish information availability from behavioral use and localize performance losses across the decision rule and the state-to-answer readout.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation. We introduce a generation-aligned diagnostic ladder that compares the emitted answer, the option logits, an affine readout of those logits, and a linear readout of the hidden state at the same answer token. Successive differences separate endpoint, decision-rule, and readout-coverage gaps. Across five systems and two emotion corpora, state decoding exceeds generation by 27.8 accuracy points on average, and both the decision-rule and readout-coverage gaps are positive in all ten conditions. A label-free logit correction improves generated accuracy in every condition, showing that part of the decision-rule gap is actionable. In rank-matched comparisons, emotion information outside the native readout generalizes to held-out speakers and survives controls for measured acoustic descriptors, but replacing the selected readout-external directions usually has little effect on emitted answers. These results distinguish information availability from behavioral use and localize performance losses across the decision rule and the state-to-answer readout.
Tags
Links
- Source: https://arxiv.org/abs/2608.06409v1
- Canonical: https://arxiv.org/abs/2608.06409v1
Trouble viewing inline? Open PDF directly →
Full Text
112,565 characters extracted from source content.
Expand or collapse full text
Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models Linkai Peng and Baorian Nuchged Linkai Peng is with the Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT 06269, USA (e-mail: linkai.peng@uconn.edu). Baorian Nuchged is with the Department of Linguistics, The University of Texas at Austin, Austin, TX 78712, USA (e-mail: baorian@utexas.edu).Preprint. Abstract Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation. We introduce a generation-aligned diagnostic ladder that compares the emitted answer, the option logits, an affine readout of those logits, and a linear readout of the hidden state at the same answer token. Successive differences separate endpoint, decision-rule, and readout-coverage gaps. Across five systems and two emotion corpora, state decoding exceeds generation by 27.8 accuracy points on average, and both the decision-rule and readout-coverage gaps are positive in all ten conditions. A label-free logit correction improves generated accuracy in every condition, showing that part of the decision-rule gap is actionable. In rank-matched comparisons, emotion information outside the native readout generalizes to held-out speakers and survives controls for measured acoustic descriptors, but replacing the selected readout-external directions usually has little effect on emitted answers. These results distinguish information availability from behavioral use and localize performance losses across the decision rule and the state-to-answer readout. I Introduction Speech language models couple audio interfaces with generative language backbones, allowing a single system to process speech and respond through natural-language generation [1, 2, 3, 4, 5]. Their evaluation has expanded beyond transcription to emotion, prosody, and other paralinguistic judgments, typically through prompted multiple-choice or free-form responses [6, 7, 8]. Generation accuracy provides a convenient summary of behavioral performance, but it conflates at least three distinct failure locations: the relevant evidence may never reach the language-model component; it may be retained in the state without being exposed through the answer readout; or it may reach the answer logits yet be misread by a mismatched decision rule. These possibilities demand different interventions, yet behavioral accuracy alone cannot distinguish among them. We ground this localization problem in four-class speech emotion recognition, whose acoustic correlates are well characterized [9, 10] and whose answer can be elicited as a single token. Across ten system–corpus conditions, generated accuracy averages 0.4740.474, whereas linear decoding from the hidden state at the same answer position averages 0.7520.752; this deficit of 0.2780.278 in absolute accuracy persists under speaker-disjoint evaluation and appears in every condition. It is most pronounced in relative terms for Phi-4-M on CREMA-D [11], where generation reaches 0.2790.279 against a full-state probe at 0.7220.722. These results reveal a substantial state-to-answer loss even when emotion remains linearly decodable at the answer position. This discrepancy points to two distinct failure locations within the language model, which we introduce in the order encountered when tracing back from the emitted answer. First, closest to the answer, emotion evidence may be present in the option logits while the default decision rule uses those scores inefficiently. We call this decision-rule misalignment. Second, deeper in the state, additional evidence may remain accessible at the answer position without being expressed through the option-specific coordinates that determine the generated label. We call this a readout-coverage gap. The practical consequence is that decision-rule misalignment is amenable to logit correction, whereas a coverage gap cannot be recovered by reweighting the same option logits and requires moving beyond the native readout. Phi-4-M on CREMA-D makes this concrete. The best affine rule over its option logits reaches an accuracy of only 0.3540.354, far below the 0.7220.722 supported by the full state, so most of its deficit cannot be repaired at the option logits. To localize these failures, we construct a generation-aligned diagnostic ladder that compares actual generation, the default option-logit decision, an optimized affine decision over the same logits, and regularized linear decoding from the full state. Anchoring all four levels to one verified answer token ensures that behavior, logits, and states measure the same event. Their successive differences split the distance between generated accuracy and full-state decodability into three terms that sum exactly: endpoint validity (agreement between the emitted answer and the option favored by the model’s own logits), the decision-rule gap, and the readout-coverage gap. Because the full-state reader has many more input dimensions than the contrast reader, we treat the readout-coverage gap as a performance gap; claims about readout-external information rest on rank-matched comparisons. We then connect diagnosis to behavior. A label-free logit correction tests the decision-rule gap during generation, while held-out decoding and minimal-pair subspace interventions distinguish the availability of readout-external information from its causal use. Acoustic controls characterize how much of that information is explained by measured surface cues. These analyses yield four contributions. First, we introduce a generation-aligned framework that exactly decomposes the distance between emitted behavior and full-state decodability into endpoint validity, a decision-rule gap, and a readout-coverage gap. Second, we show that the decision-rule diagnosis is behaviorally actionable. A standard label-free logit correction [12, 13] improves the emitted answer across all ten conditions with negligible format cost (Section V-B). Third, in rank-matched comparisons, we identify emotion information that remains linearly accessible outside the native readout and generalizes to held-out speakers. Fourth, matched minimal-pair interventions show that the selected readout-external directions have limited influence on the emitted answer, separating information availability from causal use. I Related Work Prosody-sensitive evaluation of speech language models Speech language models route speech through an encoder and projector into a text LLM [14, 2, 15, 16, 1]. Benchmarks such as Dynamic-SUPERB, AIR-Bench, and SD-Eval include emotion and paralinguistic tasks [6, 7, 8], and controlled studies show that models often rely more on lexical than acoustic cues [17]. These works establish behavioral gaps but do not localize whether a cue is lost, attenuated, or retained but underused. We study speech without engineered text–audio conflict and localize these downstream failures within the audio-to-answer computation. Label-free correction of class-dependent readout offsets Prompted multiple-choice answers exhibit systematic position and token biases [18]. Prior label-free methods estimate such preferences from content-free inputs [12], answer-side scoring statistics [19], or the mean predicted distribution over an unlabeled batch [13]. We adopt the latter estimator but use it as a behavioral intervention rather than offline rescoring, applying the offset at the first answer step, resuming full-vocabulary generation, and scoring the emitted string. This measures both accuracy gain and format stability, making the correction a behavioral test of the decision-rule gap. Representation probing in speech models Beyond the option scores, linear probing shows that emotion and prosody are recoverable from frozen speech encoders [20, 21, 22, 23, 24, 25, 26]. Layer-wise probing inside speech language models further shows that such attributes remain recoverable deep in the language stack [27]. However, decodability does not imply use [28]; we therefore treat probing as an availability diagnostic rather than evidence that the complete system recruits the cue. Subspace interventions and the selection trap The logit lens and hidden-state interventions provide tools for localizing computation [29, 30, 31, 32, 33]. Supervised subspace selection, however, can confound decodability with mechanism [34]. We address this problem by fixing the native readout space from the output head and treating held-out decoding and matched replacement as separate measurements of availability and use. In text-only models, a related knowledge–prediction gap has been reported on multiple-choice questions [35]; our framework additionally separates readout coverage from the decision rule over option scores and anchors both to the generated answer. Concurrent work retrieves sparse audio concepts [36] and studies text–audio conflict [37]; our focus is the availability and causal use of readout-external information in ordinary prompted speech. I Diagnosing the State-to-Answer Interface Figure 1 gives an overview of this section. It develops a diagnostic ladder that aligns the emitted token, the option scores, and the underlying hidden state at the same first-token event, and a decomposition of that state around the answer readout. Both concern the last mile; they do not by themselves localize losses earlier in the audio pathway. Figure 1: Overview of the generation-aligned diagnostic ladder. Left: audio and prompt tokens are processed by the encoder, projector, and language model, and the output head W maps the answer-position state xtx_t to full-vocabulary logits. Four readouts score the same answer event: greedy generation over the full vocabulary (AgenA_gen), the post-hoc argmax over the four option logits (AoptA_opt), a learned affine reader on the option-logit contrasts (AaffA_aff), and a learned affine reader on the full state (AstateA_state); dashed boxes mark the learned readers, and successive differences among the four accuracies give the gaps in Eq. (3). Right: xtx_t decomposes into its component in the answer-readout span (PVxtP_Vx_t) and the readout-external complement V⟂V , within which a supervised-selected subspace SdecodingS_decoding is compared against same-rank random subspaces (Sections I-D and I-E). I-A Three Views for One Answer For each audio–prompt pair, let xt∈ℝdx_t ^d be the model-native, post-normalization hidden state used to predict the first answer token, V the vocabulary, and W∈ℝ||×dW ^|V|× d the language-model output head. For option-token identifiers v1,…,v4v_1,…,v_4, define zt z_t =Wxt, =Wx_t, (1) zt,opt z_t,opt =(zt,v1,…,zt,v4), = (z_t,v_1,…,z_t,v_4 ), yt y_t =argmaxv∈zt,v, = *arg\,max_v z_t,v, at the same inference step. Thus, yty_t, zt,optz_t,opt, and xtx_t provide three views of one answer event: emitted behavior, native option preference, and the state available to the readout. I-B A Four-Level Diagnostic Ladder We operationalize these views as four levels of emotion-classification performance (Fig. 1, left), all evaluated on the same rows and with the same scoring rule; learned readers are assessed on the same held-out splits: • AgenA_gen: accuracy of the model’s actual generated answer, with any response outside the required format scored as incorrect; • AoptA_opt: accuracy of the post-hoc argmax restricted to the four option logits; • AaffA_aff: accuracy of a learned affine reader applied to three reference-relative option-logit contrasts; • AstateA_state: accuracy of a regularized affine reader applied to the full answer-position state. An off-option emission or a mismatch in native tie-breaking can make AgenA_gen differ from AoptA_opt. For the two learned readers, let ct=(zt,v1−zt,v4,zt,v2−zt,v4,zt,v3−zt,v4)⊤c_t=(z_t,v_1-z_t,v_4,\,z_t,v_2-z_t,v_4,\,z_t,v_3-z_t,v_4) , and set haff=cth_aff=c_t and hstate=xth_state=x_t. Both predictions take the form y^r=argmaxk(Urhr+br)k,r∈aff,state. y_r= *arg\,max_k(U_rh_r+b_r)_k, r∈\aff,state\. (2) AaffA_aff and AstateA_state are the corresponding held-out accuracies. Both readers are affine and include a bias; U distinguishes their learned weights from the fixed output head W, and the subscript identifies only whether the input is ctc_t or xtx_t. Sharing a function class makes the comparison interpretable. The contrast reader can relearn combinations and offsets of the existing option contrasts but cannot access information outside them, whereas the state reader applies the same rule type to the complete state. Their successive differences telescope: Astate−Agen=Aopt−Agen⏟Δendpoint+Aaff−Aopt⏟Δdecision+Astate−Aaff⏟Δcoverage.A_state-A_gen= A_opt-A_gen_ _endpoint+ A_aff-A_opt_ _decision+ A_state-A_aff_ _coverage. (3) Δendpoint _endpoint is the accuracy difference between emitted generation and the option-only decision. It primarily captures emission and formatting behavior, including off-option responses and native tie-breaking mismatches, so we report it without a separate mechanistic analysis. A near-zero Δendpoint _endpoint certifies that the option-restricted view is a faithful anchor for the two explanatory gaps. Δdecision _decision, the decision-rule gap, measures the gain available from a better rule over the existing contrasts. Δcoverage _coverage, the readout-coverage gap, measures the additional performance supported by the full state. Here, AstateA_state is a decodability reference rather than attainable model performance. The identity is exact by telescoping, and finite-sample gap estimates are reported with uncertainty. We analyze the two explanatory gaps in ladder order, beginning with the decision-rule gap. I-C Logit Correction of the Decision-Rule Gap A positive Δdecision _decision means that an affine rule improves on the native option-only choice using the same contrasts. One possible source is stable option bias [18]. We test it with the unlabeled-batch estimator, part of a broader family of label-free corrections [13, 12, 19], and score its effect on subsequent generation. For each utterance x under a fixed prompt variant, let qk(x)q_k(x) be the model’s probability for option k, normalized over the four prompted options. Averaging this quantity over a condition- and prompt-matched unlabeled set C estimates how strongly the model favors each option overall. We define p^k=1||∑xi∈qk(xi),bk=−logp^k+14∑j=14logp^j, p_k= 1|C| _x_i q_k(x_i), b_k=- p_k+ 14 _j=1^4 p_j, (4) Here, p^k p_k is the estimated average preference and bkb_k reverses and centers it. A frequently favored option receives a smaller offset, whereas an underpreferred option receives a larger one. If all four options are favored equally, every offset is zero, so neither their relative ordering nor their competition with off-option tokens changes. Because a marginal option preference can also reflect the target class distribution, interpreting it as bias requires a target-prior assumption; here the intended target prior is uniform, and the retained four-class subsets are nearly balanced. We add bkb_k to the corresponding option-token logit only at the first answer step and then continue ordinary full-vocabulary generation. Let AoffsetA_offset be the resulting accuracy under the same parser used for AgenA_gen. The gain Aoffset−AgenA_offset-A_gen therefore measures whether this simple correction improves the answers the model actually emits. Because the gain is scored at the generation endpoint, where the offsets also shift the options’ competition with off-option tokens, it tests the bias account of Δdecision _decision rather than estimating that gap directly. Ground-truth labels are used only afterward to score the gain; they do not enter the correction. In our experiments, C contains the target-batch inputs themselves, making the procedure label-free but transductive. I-D Localizing the Readout-Coverage Gap A positive Δcoverage _coverage shows that the full hidden state supports better linear emotion decoding than the three option-logit contrasts. It does not show where the additional decodable information lies, partly because the full-state reader receives many more input dimensions. This subsection therefore asks whether emotion information is concentrated in the model’s native answer readout or also remains available outside it. We use rank-matched comparisons here and report a matched-budget refit in Supplementary Section S4. We first identify the hidden-state directions that can directly change the relative option logits. Using the fourth option as reference, define C=[Wv1,:−Wv4,:Wv2,:−Wv4,:Wv3,:−Wv4,:]∈ℝ3×d,V3=row(C).C= bmatrixW_v_1,:-W_v_4,:\\ W_v_2,:-W_v_4,:\\ W_v_3,:-W_v_4,: bmatrix ^3× d, V_3=row(C). (5) We call V3V_3 the prompt-specific answer-readout space (Fig. 1, right). It has three dimensions because four option scores have three independent relative contrasts; the rows of C are linearly independent in the analyzed systems. The output head W is fixed; the prompt enters only by determining which four option-token rows of W define C. The contrast reader observes ct=Cxtc_t=Cx_t, so it can access only the component of the state in V3V_3. Let PV3P_V_3 and PV3⟂P_V_3 denote the orthogonal projections onto V3V_3 and its complement. Then xt=PV3xt+PV3⟂xt,CPV3⟂xt=0.x_t=P_V_3x_t+P_V_3 x_t, CP_V_3 x_t=0. (6) Thus, the component in V3V_3 can directly change the relative logits of the four answer options. Information in V3⟂V_3 may still be present in the hidden state, but it cannot directly change these relative logits at the final answer position. Throughout, readout-external is used in this geometric sense, meaning outside the span of the option-token rows of the output head, not unrelated to the task. This decomposition is exact at the final answer position. To study how the same information is organized before the final readout, we move to an intermediate layer L∗L^*. We select L∗L^* on the training split as the layer with the highest logit-lens accuracy, obtained by applying the model’s final normalization and output head to the answer-position state (Section IV-B). At L∗L^*, V3V_3 is therefore a reference aligned with the final readout, not an exact decomposition of the final logits. For the external subspaces, we remove the full option-row span V4=row(Wv1:v4)V_4=row(W_v_1:v_4). Because V3⊂V4V_3⊂ V_4, their complements satisfy V4⟂⊂V3⟂V_4 ⊂ V_3 . A direction selected in V4⟂V_4 is therefore also external to the relative option readout represented by V3V_3. The decoding comparisons below use rank-three spaces matched to V3V_3; the interventions in Section I-E instead use rank-four spaces matched to V4V_4, which also carries the absolute option logits that matter during unconstrained generation. We conduct two rank-three comparisons at L∗L^*. First, we compare held-out emotion decoding from V3V_3 with decoding from random three-dimensional subspaces rand3⊂V4⟂⊂V3⟂rand_3⊂ V_4 ⊂ V_3 . This tests whether the native answer-readout directions carry more emotion information than a random readout-external slice of the same dimension. Neither space is selected using emotion labels, making this the appropriate comparison for evaluating the relative informativeness of the native readout. Second, we ask whether a generalizable emotion signal can be found outside the native readout. For each prompt, we project the training states onto V4⟂⊂V3⟂V_4 ⊂ V_3 , fit a supervised multinomial logistic model, and define SdecodingS_decoding from the three leading right-singular directions of its coefficient matrix. We then fit a decoder in SdecodingS_decoding and evaluate it on held-out speakers. An advantage over rand3rand_3 shows that selected readout-external directions contain emotion information that generalizes beyond the training speakers. Because SdecodingS_decoding is selected using emotion labels whereas V3V_3 is not, their accuracies do not provide a direct ranking of the native and external spaces. Construction details are given in Section IV-B and the supplementary material. I-E Causal Interventions on Readout-External Information Held-out decoding shows what information is available outside the readout, but not whether that information affects the model’s answer. We therefore replace selected components of the answer-position state at L∗L^* and continue generation. The decoding analysis uses V3V_3, which represents the three relative contrasts among four options. The intervention instead uses the full option-row span V4V_4 defined above, because unconstrained generation also depends on the absolute option logits and their competition with off-option tokens. We compare three rank-four spaces. V4V_4 is the model’s native option-readout space. Sintervention⊂V4⟂S_intervention⊂ V_4 contains the four leading supervised readout-external directions. It extends the rank-three decoding space, so Sdecoding⊂SinterventionS_decoding⊂ S_intervention. Finally, rand4⊂V4⟂rand_4⊂ V_4 is a same-rank random control. Replacing V4V_4 tests whether the answer responds to information directly aligned with the native readout. Replacing SinterventionS_intervention tests whether selected information outside that readout can influence the answer, while rand4rand_4 controls for a generic state perturbation. For each held-out minimal pair, the receiver and donor share the same speaker and transcript but express different emotions. Let hrh_r and hdh_d be their answer-position states at L∗L^* under the same prompt. For S∈V4,Sintervention,rand4S∈\V_4,S_intervention,rand_4\, we construct h~r(S)=hr+PS(hd−hr). h_r^(S)=h_r+P_S(h_d-h_r). (7) This operation replaces only the receiver’s component in S with the donor’s component. We then continue generation from the edited state and measure two outcomes. The answer-change rate Rchg(S)R_chg(S) records any change from the receiver’s original answer. The donor-following rate Rdon(S)R_don(S) counts only the cases in which the answer changes to the donor’s emotion category, and therefore measures content-specific transfer. Each reported effect is the paired difference from the rand4rand_4 arm. A full-state replacement checks that downstream generation can respond to a state change at L∗L^*. For the depth analysis, we repeat the same intervention at several layers. Implementation and statistical details are given in Section IV-B. I-F Controls for Surface Acoustic Confounds The rank-three analysis tests whether SdecodingS_decoding supports held-out emotion decoding outside the native readout. One possible explanation is that this performance is driven mainly by simple surface acoustic cues that covary with the emotion labels. Such cue–label relationships can occur in acted-emotion corpora; for example, overall recording level can itself support decoding [9, 10]. We test this explanation with three controls. First, we decode emotion from clip-level acoustic-prosodic descriptors alone; the resulting accuracy AdescA_desc measures their predictive strength. Second, we regress the SdecodingS_decoding coordinates on those descriptors and decode from the residuals; the accuracy drop measures how much decoding depends on the measured cues. Third, we equalize the loudness of every clip, re-extract the states, and repeat the decoding to test dependence on absolute level. An extended descriptor panel and a nonlinear removal variant provide stronger versions of the same control; panel composition and protocols are given in Section IV-C. IV Experimental Setup IV-A Models, Corpora, and Evaluation We evaluate Qwen2.5-Omni-7B, Qwen2-Audio-7B, Audio-Flamingo-3, Kimi-Audio-7B, and Phi-4-M [1, 2, 3, 4, 5]. The task is four-way classification of happy, sad, angry, and neutral. CREMA-D [11] contributes 4,900 clips from 91 speakers, and VESUS [38] contributes 10,073 clips from 10 speakers, giving 10 model–corpus conditions. Four Latin-square prompt variants rotate the emotions through the four option positions. Under each model’s tokenizer, every selected option verbalizer is a single native vocabulary token. Generation is greedy over the full vocabulary without masking non-option tokens. A strict prefix parser maps valid answer surfaces to emotion labels; refusals, ambiguous answers, and off-format responses are incorrect. All four ladder levels use the same option rows, with the generated answer, option logits, and hidden state recorded at the same first-answer-token event. AgenA_gen and AoptA_opt require no fitting; AaffA_aff and AstateA_state use five speaker-disjoint outer folds. Standardization, penalty selection, and reader fitting are confined to the training speakers in each fold. The observational unit is the clip. Speaker-clustered resampling keeps all clips and prompt variants from one speaker together. We report paired, per-condition 95% intervals without family-wise adjustment. IV-B Subspace Decoding and Causal Replacement We split speakers into fixed training and held-out sets. Training speakers select L∗L^* by logit-lens accuracy, construct SdecodingS_decoding, and fit the decoders; held-out speakers are reserved for evaluation. The random-space results average rand3rand_3 over 20 independent draws and report the spread across draws. The matched replacements use the speaker split and subspaces defined above. Within each corpus, every system and intervention arm uses the same receiver–donor pairs, and the random arm uses one fixed rand4rand_4. Effects are paired differences from the random arm, with uncertainty clustered by speaker. We also report the one-sided 95% upper bound for each readout-external effect as a share of the corresponding readout-aligned effect. A full-state replacement provides a perturbability control at L∗L^*. IV-C Controls for Surface Acoustic Confounds Surface acoustic cues can covary with emotion labels, so we test whether they explain the held-out decodability of SdecodingS_decoding. We use a ten-descriptor base panel and a twenty-descriptor extended panel. AdescA_desc fits the same decoder family to the descriptors alone using the same speaker split. We then regress the SdecodingS_decoding coordinates on each descriptor panel and decode from the residuals, with all statistics estimated on training speakers only. The nonlinear variant replaces linear regression with gradient-boosted trees. The input-side control RMS-equalizes each clip, re-extracts the answer-position states under the same prompts and L∗L^*, and repeats the subspace analysis. The Supplementary Material provides the remaining experimental configuration details, including model checkpoints and prompt templates, tokenizer and endpoint audits, reader fitting and speaker splits, subspace construction and random-space sampling, receiver–donor pairing, and acoustic descriptor definitions and control protocols. V Results We organize the results around the generation-aligned performance ladder. We first report its endpoint, decision-rule, and readout-coverage gaps, then test label-free logit correction, the availability of readout-external information, its causal use, and finally controls for measured surface acoustic cues. V-A Both Gaps Are Systematic, but Their Relative Importance Varies Figure 2: Which gap dominates differs by condition. Each point is one model–corpus condition, with color denoting the system and shape the corpus. The dashed line marks equal gaps and the shading separates the two regimes: points in the blue region above the line lose more at the readout (coverage-dominant), and points in the tan region below it lose more at the decision rule (decision-dominant). Exact values and intervals are in Table I. Table I reports all three performance-ladder terms across the ten conditions, whose mean AgenA_gen is 0.4740.474. Δendpoint _endpoint is effectively zero throughout. The emitted answer achieves the same accuracy as the option favored by the model’s logits, so no meaningful performance is lost at this interface. Both explanatory gaps are positive. Δdecision _decision ranges from +0.0264+0.0264 to +0.2067+0.2067, showing that the default decision over the option logits falls short of a fitted rule on those same logits. Δcoverage _coverage ranges from +0.0062+0.0062 to +0.3679+0.3679, showing that the fitted logit rule in turn falls short of a reader of the full answer-position state. The confidence intervals for both gaps exclude zero in every condition. Figure 2 compares the relative sizes of the two explanatory gaps. Qwen2-Audio × CREMA-D is decision-dominant, with Δdecision _decision at +0.197+0.197 compared with a +0.060+0.060 readout-coverage gap. Phi-4-M × CREMA-D is coverage-dominant; its +0.368+0.368 readout-coverage gap is the largest in the study, whereas Δdecision _decision is +0.075+0.075. Qwen2.5-Omni × CREMA-D has substantial losses at both transitions, including the largest Δdecision _decision (+0.207+0.207) and a +0.138+0.138 readout-coverage gap. The remaining conditions lie between these patterns, with Kimi-Audio showing the widest overall separation between generated behavior and state decodability. TABLE I: The distance between generated behavior and linear decodability decomposes into an endpoint-validity term and two explanatory gaps. Rows are ordered by AgenA_gen. Δdecision=Aaff−Aopt _decision=A_aff-A_opt and Δcoverage=Astate−Aaff _coverage=A_state-A_aff. System Corpus AgenA_gen AoptA_opt AaffA_aff AstateA_state Δdecision _decision [95% CI] Δcoverage _coverage [95% CI] Audio-Flamingo-3 CREMA-D 0.9060 0.9060 0.9426 0.9487 +0.0366+0.0366 [+0.0289,+0.0446+0.0289,+0.0446] +0.0062+0.0062 [+0.0022,+0.0104+0.0022,+0.0104] Qwen2-Audio CREMA-D 0.7025 0.7025 0.8995 0.9594 +0.1970+0.1970 [+0.1909,+0.2030+0.1909,+0.2030] +0.0599+0.0599 [+0.0540,+0.0658+0.0540,+0.0658] Audio-Flamingo-3 VESUS 0.5667 0.5667 0.6335 0.6955 +0.0668+0.0668 [+0.0244,+0.1065+0.0244,+0.1065] +0.0620+0.0620 [+0.0122,+0.1037+0.0122,+0.1037] Qwen2.5-Omni CREMA-D 0.5265 0.5265 0.7332 0.8714 +0.2067+0.2067 [+0.1933,+0.2190+0.1933,+0.2190] +0.1382+0.1382 [+0.1275,+0.1492+0.1275,+0.1492] Kimi-Audio CREMA-D 0.3994 0.3994 0.5852 0.8572 +0.1858+0.1858 [+0.1741,+0.1977+0.1741,+0.1977] +0.2720+0.2720 [+0.2577,+0.2857+0.2577,+0.2857] Kimi-Audio VESUS 0.3704 0.3704 0.4104 0.6684 +0.0400+0.0400 [+0.0263,+0.0530+0.0263,+0.0530] +0.2581+0.2581 [+0.2277,+0.2921+0.2277,+0.2921] Qwen2-Audio VESUS 0.3677 0.3677 0.4358 0.6029 +0.0681+0.0681 [+0.0501,+0.0888+0.0501,+0.0888] +0.1671+0.1671 [+0.1451,+0.1909+0.1451,+0.1909] Qwen2.5-Omni VESUS 0.3590 0.3590 0.4302 0.6290 +0.0712+0.0712 [+0.0508,+0.0917+0.0508,+0.0917] +0.1988+0.1988 [+0.1635,+0.2345+0.1635,+0.2345] Phi-4-M CREMA-D 0.2793 0.2793 0.3544 0.7223 +0.0751+0.0751 [+0.0685,+0.0820+0.0685,+0.0820] +0.3679+0.3679 [+0.3508,+0.3838+0.3508,+0.3838] Phi-4-M VESUS 0.2594 0.2595 0.2859 0.5660 +0.0264+0.0264 [+0.0130,+0.0401+0.0130,+0.0401] +0.2801+0.2801 [+0.2419,+0.3250+0.2419,+0.3250] Robustness analyses reproduce the gaps with nonlinear logit-side decoding, matched regularization budgets, and delete-one-speaker resampling; complete results are in Supplementary Section S4. The ladder reveals no single universal failure profile. Some conditions have a larger decision-rule gap, others have a larger readout-coverage gap, and several show substantial gaps at both transitions. V-B Label-Free Logit Correction Recovers Part of the Decision-Rule Gap Figure 3: A label-free logit correction, measured on the emitted answer. Gains are grouped by system, with a solid CREMA-D bar and a hatched VESUS bar. Bars show only the generated-accuracy gain over the uncorrected baseline; baseline and corrected accuracies, together with transmission and parseability diagnostics, are in Supplementary Table S7. Figure 3 shows that label-free logit correction improves the emitted answer in all ten conditions, with accuracy gains from +0.0098+0.0098 to +0.1392+0.1392 (per-condition values in Supplementary Table S7). These generation-time gains recover part of the decision-rule gap without ground-truth labels. The largest gain is +0.1392+0.1392 on Qwen2.5-Omni × CREMA-D. The correction also helps across performance regimes. Phi-4-M gains +0.0399+0.0399 and +0.0178+0.0178 with its coverage-dominant profile, while Audio-Flamingo-3 gains +0.0098+0.0098 on CREMA-D from a 0.9060 baseline. Across conditions, the offset realizes 0.27 to 0.87 of the supervised decision-rule gap. The label-free correction is therefore effective across all tested model–corpus conditions, although the size of the gain varies. Because the offsets are estimated from the unlabeled evaluation batch itself, the correction is transductive; applying it to a single isolated example would require other target-domain data (Section VII). Because the offsets modify logits during unconstrained generation, they could also make the model produce answers outside the required format. We therefore check whether the corrected option is actually emitted and whether the answer remains parseable. The corrected option is emitted on 95.53% to 100% of rows, with parseability unchanged except for a 0.0019 loss on Qwen2.5-Omni × VESUS. The remaining difference from the supervised gap indicates that systematic option-prior bias is one contributor rather than its complete explanation. V-C Readout-External Emotion Information Generalizes Figure 4: Availability does not imply effective use. (a) Held-out decoding accuracy at L∗L^* for the native readout space V3V_3, a label-free random subspace rand3⊂V4⟂⊂V3⟂rand_3⊂ V_4 ⊂ V_3 averaged over 20 draws, and the supervised space SdecodingS_decoding selected in the same complement; the dashed line marks chance. Panels (b) and (c) score the rank-four minimal-pair replacements at L∗L^*, shown as paired differences from the rand4rand_4 arm for V4V_4 (circles, dark blue) and SinterventionS_intervention (squares, light blue). Panel (b) counts any answer change, whereas panel (c) counts only changes to the donor emotion. Bars are speaker-clustered 95% intervals, and the horizontal axis is symmetric-logarithmic. All panels share their row order. Exact values for (a), (b), and complete donor-following values for (c) are in Supplementary Tables S8, S11, and S13. Having tested the decision-rule gap at the output, we turn to the readout-coverage gap and ask whether generalizable emotion information remains decodable outside the native option readout. Fig. 4(a) shows the held-out decoding results for the native, random, and selected readout-external subspaces, with exact values in Supplementary Table S8. At the training-selected L∗L^*, held-out decoding from Sdecoding⊂V4⟂⊂V3⟂S_decoding⊂ V_4 ⊂ V_3 reaches 0.481 to 0.949, and its improvement over rand3rand_3 has a speaker-clustered interval excluding zero in every condition. The answer-position state therefore retains linearly accessible emotion information outside the native option contrasts. Phi-4-M illustrates the distinction. SdecodingS_decoding reaches 0.668 on CREMA-D and 0.481 on VESUS, whereas V3V_3 reaches only 0.342 and 0.259. The selection-matched V3−rand3V_3-rand_3 contrast measures how informative the native readout is relative to a same-rank random space. It is positive with an interval excluding zero in nine of the ten conditions, ranging from +0.037+0.037 to +0.272+0.272. The exception is Phi-4-M × VESUS at −0.018-0.018 [−0.033,+0.001-0.033,+0.001], and its CREMA-D condition is only +0.022+0.022 [+0.004,+0.040+0.004,+0.040], so Phi-4-M’s native option contrasts carry little more emotion information than a random subspace of the same rank. The next smallest contrast is Kimi-Audio × VESUS at +0.037+0.037 [+0.026,+0.047+0.026,+0.047], which shows that a weakly informative native readout is not confined to one language-model family. Raw accuracy contrasts compress near ceiling. Audio-Flamingo-3 × CREMA-D’s +0.068+0.068 difference, for example, sits on a 0.869 random-space baseline. Individual rand3rand_3 draws vary by up to 0.05; per-draw results and the Sdecoding−rand3S_decoding-rand_3 intervals are in Supplementary Section S6. V-D Minimal-Pair Interventions Reveal Limited Causal Use Held-out decodability establishes availability; the matched interventions now measure use. Fig. 4(b,c) shows both outcomes. Replacing the readout-aligned V4V_4 component increases answer change over the random arm in all ten conditions, by +0.0181+0.0181 to +0.3631+0.3631. It also significantly increases donor following in eight conditions; the two Phi-4-M conditions are nonsignificant. These results show that the model’s answer is causally sensitive to changes in the native readout space. By contrast, replacing the readout-external SinterventionS_intervention component has much less influence on the answer. It produces a significant answer-change effect in five conditions, but no effect exceeds +0.0206+0.0206. Only one condition shows a significant increase in donor following. These results indicate that the selected readout-external information has only limited influence on the emitted answer. We perform two additional tests to rule out the possibility that the weak SinterventionS_intervention effects arise only from small edits or an unresponsive downstream pathway. Exact values, per-condition L∗L^*, and remaining diagnostics are in Supplementary Table S11 and Section S7. Figure 5: Depth profile of causal access in Qwen2-Audio × CREMA-D. Donor-following effects of matched rank-four minimal-pair replacement at eight depths, with speaker-clustered 95% intervals; hollow markers mark intervals containing zero, and the vertical axis is symmetric-logarithmic. Exact values are in Supplementary Section S7-E. Qwen2-Audio × CREMA-D is the only condition with a significant readout-external donor-following effect at L∗L^* (+0.0106+0.0106 [+0.0058,+0.0154+0.0058,+0.0154]). We therefore repeat the intervention at eight layers to ask where this causal influence is strongest; Fig. 5 shows the resulting donor-following effects. When the replacement is applied at layer 16, the external effect is near zero. It peaks at layer 20, where it matches the readout-aligned effect at the same layer, and becomes weaker when the replacement is applied closer to the output. By contrast, the readout-aligned effect grows toward the output and reaches +0.4983+0.4983. In this condition, readout-external information has its strongest causal influence in the middle of the model, whereas readout-aligned information becomes increasingly influential near the final readout. We further track the particular component injected at layer 20 to determine whether it reaches the final answer. Relative to the random control, this intervention shifts the option logits toward the donor emotion by +0.190+0.190 [+0.176,+0.204+0.176,+0.204]. However, the shift is usually too small to change which option has the highest score. The mid-stack external pathway therefore reaches the final logits but rarely changes the emitted answer. This depth pattern is established only for Qwen2-Audio × CREMA-D (protocols and exact values in Supplementary Sections S7-E and S7-F). These interventions show that readout-external emotion information has limited causal access at the answer. V-E Readout-External Decodability Persists under Controls for Measured Surface Cues Figure 6: Readout-external decodability under progressively stronger acoustic controls. Each line is one model–corpus condition, with color denoting the system, solid circles CREMA-D, and dashed squares VESUS. Shown is held-out SdecodingS_decoding accuracy for the raw states, after input loudness equalization, and after residualizing ten or twenty acoustic-prosodic descriptors linearly or with gradient-boosted trees. Dotted lines mark the descriptor-only decoder for each corpus and the dashed gray line marks chance. Exact values and intervals are in Supplementary Tables S17, S19, and S20. The causal interventions above test whether readout-external information affects the answer. Here we return to the rank-three SdecodingS_decoding used in Fig. 4(a) and ask whether its held-out decodability can be explained by measured surface cues. Recording level alone separates several class pairs in these corpora (Supplementary Table S16), making shallow clip statistics a plausible source of readout-external decodability. Figure 6 applies the controls defined in Section IV-C. Linear residualization of the measured cues reduces held-out SdecodingS_decoding accuracy by 0.039 to 0.255. After nonlinear residualization of the extended descriptor panel, accuracy remains at 0.347 to 0.752, above the 0.25 chance level in every condition. Input-side loudness equalization changes accuracy by at most 0.022. Exact values are in Supplementary Tables S17, S19, and S20. Thus, the measured surface acoustic cues and absolute recording level do not fully explain the held-out decodability of SdecodingS_decoding. VI Discussion The central result is not simply that a hidden-state decoder outperforms the generated answer. The diagnostic ladder identifies two downstream gaps between information available at the answer position and the answer the model emits. One arises when the model converts its option logits into a choice; the other arises because the option contrasts expose only part of the information available in the hidden state. Both gaps are positive in every evaluated condition, so a system can suffer from both problems at once. They are aggregate properties of a model–corpus condition rather than mutually exclusive explanations of individual errors. More broadly, low generated accuracy need not mean that the relevant emotion evidence never reached the language model. VI-A Why the Decision Rule Loses Available Evidence Option logits combine evidence from the audio with option-token priors, positional preferences, and prompt-conditioned response habits. A stable preference for one option can therefore shift the default argmax even when the relative scores still contain useful emotion evidence [18]. This interpretation is consistent with the label-free correction. Estimating and removing marginal option preferences improves generation in all ten conditions and recovers 0.27 to 0.87 of the supervised decision-rule gap. The three largest decision-rule gaps also produce the three largest correction gains. Thus, the gap is not only diagnostic; it indicates when a logit-side repair is likely to help. The correction recovers only part of the gap because a fixed offset captures only the stable component of the mismatch. The remaining difference from the supervised affine reader may reflect class-dependent scaling or example-dependent boundaries that marginal option frequencies cannot estimate. This account predicts that reducing stable option preferences, through prompts or training procedures, should reduce both the decision-rule gap and the benefit of the offset correction. VI-B Why Decodable Information Has Limited Influence The readout-coverage results reveal a different mismatch. Emotion remains decodable from selected directions outside the native option readout and generalizes to held-out speakers, yet replacing those directions usually has little effect on the answer (no answer-change effect exceeds +0.0206+0.0206; Section V-D). A plausible explanation is objective mismatch. Audio front ends and intermediate language-model states may preserve rich prosodic information, while next-token and audio-instruction training reward only the information needed to produce the target text. They do not directly require all emotion-discriminative directions to align with the few output directions that separate the prompted option tokens. A supervised decoder is explicitly trained to find such directions; the native readout is not. On this account, readout-external information can persist as a usable representation without being part of the model’s normal answer pathway. This explains why decodability and behavioral influence diverge, a distinction that probing studies must preserve [28, 39]. It is especially important here because SdecodingS_decoding and SinterventionS_intervention are selected with emotion labels. Successful decoding shows that the information is available to a supervised linear reader, not that the model naturally uses the same directions [34]. The acoustic controls further show that the measured surface cues do not fully explain this decodability, although unmeasured acoustic properties may still contribute. The depth profile offers a more specific hypothesis about routing. In Qwen2-Audio × CREMA-D, readout-external replacement has its strongest donor-directed effect in the middle of the stack and becomes weaker near the output, while the readout-aligned effect grows. This pattern is consistent with progressive consolidation, in which intermediate layers can use emotion information in several directions while later layers increasingly concentrate behaviorally relevant information into token-aligned coordinates. Information that is not transferred into those coordinates may be overwritten or lose access to the answer. Because this pattern is established in one condition, it is best treated as a mechanism to test across models rather than a universal depth profile. The weak low-rank replacement effects do not imply that every direction outside the readout is functionless. They show that the selected linear component has limited causal access under the tested intervention. Emotion information could also be distributed across more directions or participate through nonlinear interactions that a rank-four replacement does not capture. The strong response to readout-aligned and full-state replacements nevertheless shows that the downstream pathway can respond to state changes at the intervention layer; the main limitation lies in how the selected external information is routed. VI-C Implications for Repair and Evaluation The two gaps suggest different repairs. A decision-rule gap can be addressed by calibrating the existing option logits, without changing the hidden representation. A readout-coverage gap instead requires changing how hidden-state information reaches the answer, for example through a learned readout adapter, targeted fine-tuning of the output mapping, or auxiliary supervision that aligns prosodic evidence with the option contrasts. If such training reduces Δcoverage _coverage and increases the causal effect of readout-external directions, it would support the routing explanation above. These results also change how generative paralinguistic systems should be evaluated. Generated answers, option logits, and hidden states should be measured at the same answer event; diagnostic readers should be evaluated on identity-disjoint splits; and claims about information use should include dimension-matched causal controls. Reporting these views alongside accuracy separates a failure to represent emotion from a failure to expose or use information that is already present at the answer position. VII Limitations Empirical and statistical coverage We evaluate only five models. Although they use several audio front ends, they cover only two language-model families, and four use Qwen-family backbones. Our experiments focus on four-way English emotion classification on CREMA-D and VESUS, using single-token multiple-choice answers. Results may differ for other architectures, spontaneous speech, other languages, other paralinguistic tasks, or free-form answers. VESUS contains only 10 speakers, which limits the precision of speaker-clustered estimates. Diagnostic scope AaffA_aff reads only three option-logit contrasts, whereas AstateA_state reads the full d-dimensional hidden state. Although both use linear classifiers, the state reader has access to many more input dimensions. A larger Δcoverage _coverage can therefore arise for two reasons: the hidden state may contain information that the native readout does not expose, or the full-state reader may benefit from its larger input space. We therefore interpret Δcoverage _coverage as a performance gap rather than a pure measure of readout geometry. Acoustic interpretation Our acoustic controls account for recording level and twenty measured acoustic descriptors, but they do not cover every property of the speech signal. The remaining decodable information may therefore still include acoustic cues that we did not measure. It should not, by itself, be interpreted as an abstract representation of emotion. Correction and intervention scope The logit correction does not use emotion labels, but it estimates its offsets from the full unlabeled evaluation batch. It therefore requires a batch of examples from the target domain and must be recalibrated for a new domain. It cannot be applied to one isolated example without other target-domain data. The matched replacements are diagnostic tests rather than a trained repair. They ask whether replacing a selected part of the hidden state can change the model’s answer; they do not teach the model a new readout or routing mechanism. The one significant readout-external donor-following effect therefore shows only a small amount of causal influence. It does not close the readout-coverage gap. VIII Conclusion Behavioral errors alone do not reveal where task information stops influencing a generated answer. By aligning behavior, option logits, and hidden states at the same first-token event, our diagnostic ladder separates endpoint validity, the decision-rule gap, and the readout-coverage gap. Both explanatory gaps are positive in all ten evaluated conditions. A label-free logit correction recovers part of the decision-rule gap during generation. Meanwhile, emotion information remains decodable outside the native readout under controls for measured acoustic cues, but replacing the selected external component seldom moves the answer toward the donor emotion; this bounds the causal influence of the directions we selected, not of all readout-external information. These results distinguish information availability from behavioral use and motivate different responses: correcting the rule over existing option logits or adapting how hidden-state information is read out and routed. The same three views can be captured at any prompted answer token; in speech, they show where paralinguistic information available at the answer position stops contributing to the emitted response. Reported alongside accuracy, they turn a benchmark score into a diagnosis of the state-to-answer interface. References [1] J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y. Fan, K. Dang, B. Zhang, X. Wang, Y. Chu, and J. Lin, “Qwen2.5-omni technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2503.20215 [2] Y. Chu, J. Xu, Q. Yang et al., “Qwen2-audio technical report,” arXiv preprint arXiv:2407.10759, 2024. [3] A. Goel, S. Ghosh, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. H. Yang, R. Duraiswami, D. Manocha, R. Valle, and B. Catanzaro, “Audio flamingo 3: Advancing audio intelligence with fully open large audio language models,” arXiv preprint arXiv:2507.08128, 2025. [4] Kimi Team, “Kimi-audio technical report,” 2025. [5] M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann et al., “Phi-4 technical report,” arXiv preprint arXiv:2412.08905, 2024. [6] C.-y. Huang, K.-H. Lu, S.-H. Wang, C.-Y. Hsiao, C.-Y. Kuan, H. Wu, S. Arora, K.-W. Chang, J. Shi, Y. Peng et al., “Dynamic-superb: Towards a dynamic, collaborative, and comprehensive instruction-tuning benchmark for speech,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, p. 12 136–12 140. [7] Q. Yang, J. Xu, W. Liu, Y. Chu, Z. Jiang, X. Zhou, Y. Leng, Y. Lv, Z. Zhao, C. Zhou et al., “Air-bench: Benchmarking large audio-language models via generative comprehension,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, p. 1979–1998. [8] J. Ao, Y. Wang, X. Tian, D. Chen, J. Zhang, L. Lu, Y. Wang, H. Li, and Z. Wu, “Sd-eval: A benchmark dataset for spoken dialogue understanding beyond words,” Advances in Neural Information Processing Systems, vol. 37, p. 56 898–56 918, 2024. [9] K. R. Scherer, “Vocal communication of emotion: A review of research paradigms,” Speech Communication, vol. 40, no. 1–2, p. 227–256, 2003. [10] P. N. Juslin and P. Laukka, “Communication of emotions in vocal expression and music performance: Different channels, same code?” Psychological bulletin, vol. 129, no. 5, p. 770, 2003. [11] H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, “Crema-d: Crowd-sourced emotional multimodal actors dataset,” IEEE transactions on affective computing, vol. 5, no. 4, p. 377–390, 2014. [12] T. Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, “Calibrate before use: Improving few-shot performance of language models,” in Proceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 139. PMLR, 2021, p. 12 697–12 706. [13] H. Zhou, X. Wan, L. Proleev, D. Mincu, J. Chen, K. Heller, and S. Roy, “Batch calibration: Rethinking calibration for in-context learning and prompt engineering,” in International Conference on Learning Representations, 2024, arXiv:2309.17249. [14] D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y. Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,” in Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, p. 15 757–15 773. [15] C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,” in International Conference on Learning Representations, vol. 2024, 2024, p. 16 607–16 629. [16] Z. Kong, A. Goel, R. Badlani, W. Ping, R. Valle, and B. Catanzaro, “Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities,” arXiv preprint arXiv:2402.01831, 2024. [17] J. Chen, Z. Guo, J. Chun, P. Wang, A. Perrault, and M. Elsner, “Do audio LLMs really LISTEN, or just transcribe? measuring lexical vs. acoustic emotion cues reliance,” in Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, 2026, p. 5848–5877. [18] C. Zheng, H. Zhou, F. Meng, J. Zhou, and M. Huang, “Large language models are not robust multiple choice selectors,” in International Conference on Learning Representations, 2024, spotlight; arXiv:2309.03882. [19] S. Kumar, “Answer-level calibration for free-form multiple choice question answering,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022, p. 665–679. [20] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing, vol. 29, p. 3451–3460, 2021. [21] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, p. 1505–1518, 2022. [22] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning. PMLR, 2023, p. 28 492–28 518. [23] A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self-supervised speech representation model,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, p. 914–921. [24] M. De Seyssel, M. Lavechin, Y. Adi, E. Dupoux, and G. Wisniewski, “Probing phoneme, language and speaker information in unsupervised speech representations,” in Interspeech 2022, 2022, p. 1402–1406. [25] J. Wagner, A. Triantafyllopoulos, H. Wierstorf, M. Schmitt, F. Burkhardt, F. Eyben, and B. W. Schuller, “Dawn of the transformer era in speech emotion recognition: Closing the valence gap,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. [26] Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emotion2vec: Self-supervised pre-training for speech emotion representation,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, p. 15 747–15 760. [27] C.-K. Yang, N. Ho, Y.-J. Lee, and H.-y. Lee, “AudioLens: A closer look at auditory attribute perception of large audio-language models,” arXiv preprint arXiv:2506.05140, 2025. [28] Y. Belinkov, “Probing classifiers: Promises, shortcomings, and advances,” Computational Linguistics, vol. 48, no. 1, p. 207–219, 2022. [29] nostalgebraist, “Interpreting GPT: The logit lens,” 2020, lessWrong post; https://w.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens. [30] N. Belrose, I. Ostrovsky, L. McKinney, Z. Furman, L. Smith, D. Halawi, S. Biderman, and J. Steinhardt, “Eliciting latent predictions from transformers with the tuned lens,” arXiv preprint arXiv:2303.08112, 2023. [31] J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, Y. Singer, and S. Shieber, “Investigating gender bias in language models using causal mediation analysis,” Advances in neural information processing systems, vol. 33, p. 12 388–12 401, 2020. [32] A. Geiger, H. Lu, T. Icard, and C. Potts, “Causal abstractions of neural networks,” in Advances in Neural Information Processing Systems, vol. 34, 2021, p. 9574–9586. [33] K. Meng, D. Bau, A. Andonian, and Y. Belinkov, “Locating and editing factual associations in gpt,” Advances in neural information processing systems, vol. 35, p. 17 359–17 372, 2022. [34] A. Makelov, G. Lange, A. Geiger, and N. Nanda, “Is this the subspace you are looking for? an interpretability illusion for subspace activation patching,” arXiv preprint arXiv:2311.17030, 2023. [35] Y. Park, H. Pyun, and Y. Jo, “Bridging the knowledge-prediction gap in LLMs on multiple-choice questions,” in Proc. International Conference on Machine Learning, 2026, arXiv:2509.23782. [36] T. F. Chowdhury, D. H. Ta, S. Pan, J. Stoddard, and Z. Liao, “AR&D: A framework for retrieving and describing concepts for interpreting AudioLLMs,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2026, arXiv:2602.22253. [37] H. Cho, S. Yoo, J. Jang, C. Kim, and J. S. Chung, “Who wins the conflict? mechanistic interpretability of text bias in audio LLMs,” 2026. [38] J. Sager, R. Shankar, J. Reinhold, and A. Venkataraman, “VESUS: A crowd-annotated database to study emotion production and perception in spoken English,” in Interspeech, 2019. [39] Y. Xu, S. Zhao, J. Song, R. Stewart, and S. Ermon, “A theory of usable information under computational constraints,” in International Conference on Learning Representations (ICLR), 2020. Supplementary Material S1 Experimental Setup Details S1-A Systems Table S1 lists the five systems evaluated in the paper, with the checkpoint, architecture, and parameter count of each. For compactness, subsequent supplementary tables abbreviate them as Omni, Q2A, AF3, Kimi, and Phi-4. We selectively evaluate Qwen2.5-Omni [1], Phi-4-M [5], Audio-Flamingo-3 [3], Qwen2-Audio [2], and Kimi-Audio [4]. These systems are frequently represented in related work and span several widely used speech-language-model architectures. All five expose hidden states along a speech-understanding, text-response pathway and provide the discrete answer endpoint required by our analysis. TABLE S1: The five evaluated systems. Parameter counts are taken from the public model cards and are total architecture sizes, including text/decoder branches where applicable. System HuggingFace checkpoint Architecture # Params Qwen2.5-Omni-7B Qwen/Qwen2.5-Omni-7B Whisper-style audio tower + 28-layer LLM 7 B Audio-Flamingo-3 nvidia/audio-flamingo-3-hf Whisper-large-v3 + 28-layer Qwen2.5-7B LLM 8.3 B Phi-4-M microsoft/Phi-4-multimodal-instruct Conformer audio tower + 32-layer LLM 5.6 B Qwen2-Audio-7B Qwen/Qwen2-Audio-7B-Instruct Whisper-large-v3 audio tower + 32-layer Qwen2-7B LLM 8.4 B Kimi-Audio-7B moonshotai/Kimi-Audio-7B-Instruct Whisper-large-v3 encoder + 28-layer Qwen2.5-7B LLM 9.8 B S1-B Corpus Composition The clip counts in Table S2 are filtered subsets of the published corpora, not the full releases. • CREMA-D. We use the four-emotion subset \angry, happy, sad, neutral\ of the published six-emotion corpus, dropping disgust and fear, and keep every clip of the retained classes. The source corpus contains fewer neutral clips than non-neutral clips, so the resulting 4,900-clip subset is not class-balanced: its majority class accounts for 25.94% of clips, against the 25% four-way chance rate. • VESUS. We use the four-emotion subset \happy, sad, angry, neutral\ of the five-emotion corpus, dropping fearful. VESUS reads a phonetically balanced, semantically neutral script of more than 250 short phrases, each spoken by 10 actors in every emotion, giving 10,073 clips whose majority class accounts for 25.01%. Because neither subset is exactly balanced, we report the empirical majority-class rates rather than treating 25% as an exact baseline. TABLE S2: Per-class clip counts of the two evaluation corpora. Corpus happy sad angry neutral total CREMA-D 1,271 1,271 1,271 1,087 4,900 VESUS 2,518 2,517 2,519 2,519 10,073 S1-C Prompt Protocol System prompts. All five systems are queried with their official chat templates, unmodified. Qwen2.5-Omni receives the canonical system message distributed with the model: “You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.” Phi-4-M’s speech-understanding template (<|user|><|audio_1|>…<|end|><|assistant|>) does not include a system turn by design. Audio-Flamingo-3, Qwen2-Audio, and Kimi-Audio are each queried with a single user turn carrying the audio and the text prompt; for these we do not add a system message, following each model’s recommended speech-understanding format. Prompt bank. Every clip is presented under the four prompt variants of Table S3. They share one instruction template and differ only in the Latin-square assignment of emotions to option letters, which places each emotion at each letter position exactly once across the four variants and so controls for positional bias. The four variants of a clip are repeated measurements of the same audio and are kept together in the same split and the same bootstrap cluster, as detailed in Section S1-D. TABLE S3: The four-variant emotion prompt bank. The Latin-square design guarantees that each emotion appears at each letter position exactly once across B1–B4. ID Prompt text Instruction template: “Listen to the audio and identify the speaker’s emotion. (A) A (B) B (C) C (D) D. Answer with just the letter A, B, C, or D:” B1 A = happy, B = sad, C = angry, D = neutral B2 A = sad, B = angry, C = neutral, D = happy B3 A = angry, B = neutral, C = happy, D = sad B4 A = neutral, B = happy, C = sad, D = angry Output parsing. Generation is greedy over the full vocabulary, and the processor never masks non-option tokens. The parser locates a standalone option marker in leading position, such as A, (A), or A., and the corresponding patterns for B, C, and D. Markers are bounded by non-letter characters so that letters inside words such as “Answer” are not matched, and the selected option is mapped to the emotion label assigned under the active Latin-square variant. Any generation without a valid leading option marker, including refusals and free-form prose, is counted as incorrect in every main-text analysis; it is never re-parsed into a class by scanning the rest of the response. Layer-selection readout. The criterion that fixes L∗L^* (Section IV-B) uses a separate readout from the option letters that the analyses score. At each layer, the model’s own final normalization and unembedding are applied to the answer-position state, and the score of a class is the log-sum-exp over that class’s emotion-word tokens: \ happy, Happy, joyful\ for happy, \ sad, Sad, upset\ for sad, \ angry, Angry\ for angry, and the corresponding set for neutral. Each listed verbalizer is a single native vocabulary token under the corresponding model tokenizer. The highest-scoring class is the layerwise prediction, and L∗L^* is the layer with the highest accuracy on the training split. These emotion words are never scored as answers; they enter only the layer-selection criterion. S1-D Readers, Splits, and Uncertainty All ladder levels are evaluated on identical rows. The affine and state readers use five speaker-disjoint outer folds, with feature standardization and every supervised fitting decision confined to the training side. The affine reader selects its penalty by nested cross-validation, whereas the main state reader uses a fixed penalty. Thus the tuned reader is the one subtracted in Δcoverage _coverage; Section S4 reports the refit in which both readers receive the same nested selection budget. The audio clip is the observational unit. A resampled speaker carries all of that speaker’s clips and all four prompt variants, preserving their dependence. Intervals are paired and reported per condition without family-wise adjustment. This construction yields 91 speaker clusters on CREMA-D and 10 on VESUS; the latter necessarily produces less precise condition-level intervals. S2 Endpoint Audit Table S4 reports three implementation checks for the answer endpoint. Answer surfaces. For each system we enumerate candidate surfaces with its own tokenizer and select the one whose tokens carry the full-vocabulary top-1 mass. Every selected option verbalizer is a single native vocabulary token, so its logit is obtained from one output-head row rather than an aggregation across subtokens. The choice of surface is not cosmetic. Audio-Flamingo-3 puts all of its mass on parenthesis-merged tokens and none on bare letters, so an analysis that assumed bare letters would have scored a token that system never emits. Phi-4 places 0.9998 and 0.9992 of its top-1 mass on bare letters, with the small remainder on the parenthesis variant. Spaced surfaces receive no mass in any condition. Off-format generations. The strict parser accepts only a leading option marker. Under the uncorrected generation protocol, all generated answers satisfy this requirement. State-to-logit reconstruction and conditioning. Passing each saved post-normalization state back through the native language-model head reproduces the stored option-logit contrasts with a mean absolute error one to two orders of magnitude below the tolerance derived from each condition’s logit scale. The option-contrast matrix is well conditioned everywhere, with a condition number between 2.51 and 2.94 for four systems and 8.35 for Phi-4. TABLE S4: Endpoint audit. Surface is the selected answer surface and its full-vocabulary top-1 in-option rate; the alternative surface is given where the sweep was stored. Off-fmt is the fraction of generations the strict parser rejected. MAE/tol is the state-to-logit reconstruction error against its tolerance, and κ is the condition number of the option-contrast matrix. System Corpus Surface In-option Off-fmt κ MAE/tol AF3 CREMA-D paren 1.0000 0.0000 2.94 0.026/0.425 AF3 VESUS paren 1.0000 0.0000 2.94 0.026/0.420 Kimi CREMA-D bare 1.0000 0.0000 2.51 0.026/0.363 Kimi VESUS bare 1.0000 0.0000 2.51 0.026/0.368 Phi-4 CREMA-D bare 0.9998 0.0000 8.35 0.053/0.875 Phi-4 VESUS bare 0.9992 0.0000 8.35 0.054/0.825 Omni CREMA-D bare — 0.0000 2.51 0.002/0.280 Omni VESUS bare — 0.0000 2.51 0.002/0.284 Q2A CREMA-D bare — 0.0000 2.60 0.002/0.265 Q2A VESUS bare — 0.0000 2.60 0.002/0.266 For Audio-Flamingo-3, the parenthesized form is the valid answer surface; the bare and spaced forms are not selected. The surface sweep was not stored for the four Qwen conditions, which were run on an earlier pass of the pipeline, but all of their generated answers pass the strict parser and use one of the requested option forms. S3 The Performance Ladder in Full Table S5 gives every rung and both explanatory gaps with speaker-clustered intervals for all 10 conditions. AMLPA_MLP is the nonlinear control of Section S4: a multilayer perceptron on the same three option contrasts. TABLE S5: The full ladder, ordered by AgenA_gen. Intervals are paired speaker-clustered bootstraps and are not family-wise adjusted. System Corpus AgenA_gen AoptA_opt AaffA_aff AMLPA_MLP AstateA_state Δendpoint _endpoint Δdecision _decision [95% CI] Δcoverage _coverage [95% CI] AF3 CREMA-D 0.9060 0.9060 0.9426 0.9434 0.9487 0.0000 +0.0366+0.0366 [+0.0289,+0.0446+0.0289,+0.0446] +0.0062+0.0062 [+0.0022,+0.0104+0.0022,+0.0104] Q2A CREMA-D 0.7025 0.7025 0.8995 0.9033 0.9594 0.0000 +0.1970+0.1970 [+0.1909,+0.2030+0.1909,+0.2030] +0.0599+0.0599 [+0.0540,+0.0658+0.0540,+0.0658] AF3 VESUS 0.5667 0.5667 0.6335 0.6257 0.6955 0.0000 +0.0668+0.0668 [+0.0244,+0.1065+0.0244,+0.1065] +0.0620+0.0620 [+0.0122,+0.1037+0.0122,+0.1037] Omni CREMA-D 0.5265 0.5265 0.7332 0.7448 0.8714 0.0000 +0.2067+0.2067 [+0.1933,+0.2190+0.1933,+0.2190] +0.1382+0.1382 [+0.1275,+0.1492+0.1275,+0.1492] Kimi CREMA-D 0.3994 0.3994 0.5852 0.6126 0.8572 0.0000 +0.1858+0.1858 [+0.1741,+0.1977+0.1741,+0.1977] +0.2720+0.2720 [+0.2577,+0.2857+0.2577,+0.2857] Kimi VESUS 0.3704 0.3704 0.4104 0.4230 0.6684 0.0000 +0.0400+0.0400 [+0.0263,+0.0530+0.0263,+0.0530] +0.2581+0.2581 [+0.2277,+0.2921+0.2277,+0.2921] Q2A VESUS 0.3677 0.3677 0.4358 0.4425 0.6029 0.0000 +0.0681+0.0681 [+0.0501,+0.0888+0.0501,+0.0888] +0.1671+0.1671 [+0.1451,+0.1909+0.1451,+0.1909] Omni VESUS 0.3590 0.3590 0.4302 0.4385 0.6290 0.0000 +0.0712+0.0712 [+0.0508,+0.0917+0.0508,+0.0917] +0.1988+0.1988 [+0.1635,+0.2345+0.1635,+0.2345] Phi-4 CREMA-D 0.2793 0.2793 0.3544 0.3759 0.7223 0.0000 +0.0751+0.0751 [+0.0685,+0.0820+0.0685,+0.0820] +0.3679+0.3679 [+0.3508,+0.3838+0.3508,+0.3838] Phi-4 VESUS 0.2594 0.2595 0.2859 0.2949 0.5660 0.0000 +0.0264+0.0264 [+0.0130,+0.0401+0.0130,+0.0401] +0.2801+0.2801 [+0.2419,+0.3250+0.2419,+0.3250] S4 Robustness of the Readout-Coverage Gap Three controls target three different alternatives to the coverage gap. Table S6 reports the second and third; the first is the AMLPA_MLP column of Table S5. The affine function class is not the bottleneck. Replacing the affine reader on the three option contrasts with a multilayer perceptron on the same contrasts does not absorb the state advantage. Astate−AMLPA_state-A_MLP ranges from +0.0053+0.0053 to +0.3464+0.3464, and its interval excludes zero in all 10 conditions. The nonlinear reader beats the affine one on the same contrasts in nine conditions, so the option scores do carry some nonlinearly accessible structure; it is simply far smaller than what the full state supports. The selection budget is not the explanation. AaffA_aff selects its penalty by nested cross-validation while AstateA_state uses a fixed penalty, so the subtracted term is the tuned one. Refitting both ends under a matched nested budget changes the coverage gap by at most 0.01640.0164 in absolute accuracy (Qwen2.5-Omni × VESUS) and leaves five conditions slightly lower than reported. Every coverage interval still excludes zero. The main text reports the fixed-penalty specification; the matched refit confirms the result without systematically favoring either reader. No single speaker drives either gap. Recomputing both terms with each speaker’s rows removed in turn, without refitting, leaves both terms positive in every replicate of every condition. This check does not use the bootstrap’s resampling assumptions at all, which matters most for the 10-speaker corpus. TABLE S6: Capacity-matched refit and delete-one-speaker jackknife. Δ is the change in the coverage gap under the matched budget relative to Table S5. Capacity-matched refit Jackknife range System Corpus AstateA_state Δcoverage _coverage [95% CI] Δ Δdecision _decision Δcoverage _coverage AF3 CREMA-D 0.9487 +0.0062+0.0062 [+0.0022,+0.0104+0.0022,+0.0104] +0.0000+0.0000 [+0.0348,+0.0372+0.0348,+0.0372] [+0.0057,+0.0068+0.0057,+0.0068] Q2A CREMA-D 0.9592 +0.0597+0.0597 [+0.0538,+0.0657+0.0538,+0.0657] −0.0002-0.0002 [+0.1961,+0.1980+0.1961,+0.1980] [+0.0589,+0.0606+0.0589,+0.0606] AF3 VESUS 0.6911 +0.0576+0.0576 [+0.0045,+0.1017+0.0045,+0.1017] −0.0044-0.0044 [+0.0546,+0.0789+0.0546,+0.0789] [+0.0527,+0.0800+0.0527,+0.0800] Omni CREMA-D 0.8715 +0.1383+0.1383 [+0.1273,+0.1495+0.1273,+0.1495] +0.0001+0.0001 [+0.2051,+0.2086+0.2051,+0.2086] [+0.1369,+0.1402+0.1369,+0.1402] Kimi CREMA-D 0.8562 +0.2710+0.2710 [+0.2569,+0.2846+0.2569,+0.2846] −0.0010-0.0010 [+0.1845,+0.1876+0.1845,+0.1876] [+0.2703,+0.2751+0.2703,+0.2751] Kimi VESUS 0.6760 +0.2657+0.2657 [+0.2338,+0.3047+0.2338,+0.3047] +0.0076+0.0076 [+0.0365,+0.0449+0.0365,+0.0449] [+0.2458,+0.2662+0.2458,+0.2662] Q2A VESUS 0.6034 +0.1676+0.1676 [+0.1447,+0.1923+0.1447,+0.1923] +0.0005+0.0005 [+0.0602,+0.0720+0.0602,+0.0720] [+0.1586,+0.1737+0.1586,+0.1737] Omni VESUS 0.6455 +0.2153+0.2153 [+0.1787,+0.2506+0.1787,+0.2506] +0.0164+0.0164 [+0.0659,+0.0769+0.0659,+0.0769] [+0.1876,+0.2100+0.1876,+0.2100] Phi-4 CREMA-D 0.7221 +0.3678+0.3678 [+0.3507,+0.3837+0.3507,+0.3837] −0.0002-0.0002 [+0.0739,+0.0760+0.0739,+0.0760] [+0.3656,+0.3702+0.3656,+0.3702] Phi-4 VESUS 0.5659 +0.2800+0.2800 [+0.2420,+0.3248+0.2420,+0.3248] −0.0001-0.0001 [+0.0230,+0.0296+0.0230,+0.0296] [+0.2628,+0.2907+0.2628,+0.2907] S5 Logit Correction Details Offsets are estimated per condition and per prompt variant from the marginal option probabilities of the evaluation rows, using no emotion labels, and are centered before use. For every condition, the offsets and the corrected accuracy they imply were written to disk before the corrected-generation pass ran, so each row of Table S7 compares a generated result against a prediction fixed in advance. The four Qwen conditions were re-estimated on the full corpora for this table; their baselines reproduce AgenA_gen to four decimals. Prediction and generation agree exactly in five conditions. Qwen2.5-Omni × VESUS is the only condition in which correction reduces parseability: 0.0019 of rows leave the option set. The remaining disagreements change one valid option into another and therefore do not create a format cost. TABLE S7: Label-free logit correction. Predicted is the corrected accuracy registered before the generation pass; generated is the observed one. Flip gives the predicted and observed fraction of rows whose answer changes. Recovery is the gain divided by Δdecision _decision. System Corpus Baseline Predicted Generated Gain Agreement Flip pred./obs. Parseability Recovery AF3 CREMA-D 0.9060 0.9158 0.9158 +0.0098+0.0098 1.0000 0.0131 / 0.0131 1.000 → 1.000 0.268 Q2A CREMA-D 0.7025 0.8177 0.8177 +0.1152+0.1152 1.0000 0.1401 / 0.1401 1.000 → 1.000 0.585 AF3 VESUS 0.5667 0.6137 0.6137 +0.0470+0.0470 1.0000 0.1155 / 0.1155 1.000 → 1.000 0.703 Omni CREMA-D 0.5265 0.6657 0.6657 +0.1392+0.1392 1.0000 0.2137 / 0.2137 1.000 → 1.000 0.674 Kimi CREMA-D 0.3994 0.5343 0.5289 +0.1295+0.1295 00.9685 0.3833 / 0.3607 1.000 → 1.000 0.697 Kimi VESUS 0.3704 0.3920 0.3898 +0.0194+0.0194 00.9553 0.3025 / 0.2947 1.000 → 1.000 0.485 Q2A VESUS 0.3677 0.4268 0.4268 +0.0591+0.0591 1.0000 0.3616 / 0.3616 1.000 → 1.000 0.868 Omni VESUS 0.3590 0.4043 0.4036 +0.0446+0.0446 00.9981 0.2008 / 0.2007 1.000 → 00.998 0.627 Phi-4 CREMA-D 0.2793 0.3193 0.3192 +0.0399+0.0399 00.9993 0.3506 / 0.3501 1.000 → 1.000 0.532 Phi-4 VESUS 0.2594 0.2772 0.2772 +0.0178+0.0178 00.9995 0.2791 / 0.2787 1.000 → 1.000 0.674 The recovery ratio is reported per condition rather than summarized, because it is unstable when its denominator is small and because the supervised affine reader is a diagnostic reference rather than a target the label-free offset is expected to reach. The main text quotes the 0.27 to 0.87 range across all 10 conditions. S6 Readout Subspaces and Held-Out Decoding TABLE S8: Held-out rank-three subspace decodability at L∗L^*, plotted in main-text Fig. 4(a). rand3rand_3 is averaged over 20 independent draws. Intervals for V3−rand3V_3-rand_3 are speaker-clustered over the held-out speakers and exclude zero except on Phi-4-M × VESUS. System Corpus V3V_3 rand3rand_3 V3−rand3V_3-rand_3 [95% CI] SdecodingS_decoding Qwen2-Audio CREMA-D 0.890 0.618 +0.272+0.272 [+0.260,+0.284+0.260,+0.284] 0.949 Qwen2.5-Omni CREMA-D 0.732 0.528 +0.204+0.204 [+0.190,+0.218+0.190,+0.218] 0.858 Audio-Flamingo-3 CREMA-D 0.937 0.869 +0.068+0.068 [+0.060,+0.077+0.060,+0.077] 0.940 Kimi-Audio CREMA-D 0.602 0.467 +0.136+0.136 [+0.117,+0.155+0.117,+0.155] 0.823 Phi-4-M CREMA-D 0.342 0.320 +0.022+0.022 [+0.004,+0.040+0.004,+0.040] 0.668 Qwen2-Audio VESUS 0.463 0.359 +0.104+0.104 [+0.075,+0.132+0.075,+0.132] 0.606 Qwen2.5-Omni VESUS 0.464 0.351 +0.112+0.112 [+0.062,+0.151+0.062,+0.151] 0.561 Audio-Flamingo-3 VESUS 0.670 0.561 +0.109+0.109 [+0.082,+0.135+0.082,+0.135] 0.690 Kimi-Audio VESUS 0.408 0.372 +0.037+0.037 [+0.026,+0.047+0.026,+0.047] 0.563 Phi-4-M VESUS 0.259 0.277 −0.018-0.018 [−0.033,+0.001-0.033,+0.001] 0.481 With seed 0, half of the speakers are assigned to the training split and the remainder to the held-out split; construction is prompt-specific. Let WoptW_opt stack the four option-token output rows, V4=row(Wopt)V_4=row(W_opt), and PV4P_V_4 project onto that space. Let H contain training states at L∗L^*, and let fN_f be the final normalization. We standardize H⟂=f(H)(I−PV4)H_ =N_f(H)(I-P_V_4), fit an L2 multinomial logistic regression (C=0.01C=0.01, 3,000 iterations), and define B⟂ B_ =BstdD−1(I−PV4)=UΣR⊤, =B_stdD^-1(I-P_V_4)=U R , (S1) Sdecoding S_decoding =spanRi,:i=13, =span\R_i,:\_i=1^3, where BstdB_std is the standardized coefficient matrix and D contains the fitted feature scales. Reprojection and re-orthonormalization reduce leakage into V4V_4 below 10−610^-6. Because the relative-contrast space V3V_3 is contained in V4V_4, Sdecoding⊂V4⟂⊂V3⟂S_decoding⊂ V_4 ⊂ V_3 . The rank-four space SinterventionS_intervention used in the causal analysis is built from the leading four discriminant directions. Because both spaces come from the same singular basis, SdecodingS_decoding is contained in SinterventionS_intervention by construction. The rank-four intervention therefore contains the decoded directions, although causal effects need not be monotone under subspace expansion. The causal analysis uses the matched rank-four geometry. Held-out decodability uses a separate standardized L2 multinomial probe (C=0.5C=0.5, 2,000 iterations), fitted on the training split and scored on the held-out split. Correctness is averaged over four prompt variants, and 95% intervals use 2,000 speaker-bootstrap resamples. For example, Phi-4 × CREMA-D uses 986 clips from 45 training speakers and 1,014 clips from 46 held-out speakers. Each rand3rand_3 draw is a standard-normal sample projected into V4⟂V_4 and re-orthonormalized, with the raw seed reused across the four prompts so that only the per-prompt projector differs. S6-A Averaging the random reference over draws A single random subspace is a noisy reference. Repeating the entire probe over 20 independent draws (Table S9) shows that same-rank draws vary by 0.0170.017 to 0.0530.053 in standard deviation, enough to move a small contrast across zero. The decoding results in the main text therefore use the draw-averaged reference: per-clip correctness is averaged over the 20 draws before scoring and bootstrapping, exactly as it is averaged over the four prompts. The native contrast is positive on all 20 draws in seven conditions. It is positive on 16 draws for Kimi-Audio × VESUS, 17 for Phi-4-M × CREMA-D, and only 2 for Phi-4-M × VESUS, confirming that small native contrasts can depend on the random draw. By contrast, Sdecoding−rand3S_decoding-rand_3 is positive on all 20 draws in all ten conditions. TABLE S9: The random reference across 20 independent rank-three draws. sd and range are over draws, not over speakers. The last two columns count the draws on which each contrast is positive. System Corpus mean sd range V3>rand3V_3>rand_3 Sdecoding>rand3S_decoding>rand_3 Qwen2-Audio CREMA-D 0.618 0.053 [0.506,0.719] 20/20 20/20 Qwen2.5-Omni CREMA-D 0.528 0.030 [0.479,0.583] 20/20 20/20 Audio-Flamingo-3 CREMA-D 0.869 0.024 [0.831,0.908] 20/20 20/20 Kimi-Audio CREMA-D 0.467 0.030 [0.399,0.521] 20/20 20/20 Phi-4-M CREMA-D 0.320 0.019 [0.296,0.361] 17/20 20/20 Qwen2-Audio VESUS 0.359 0.022 [0.315,0.394] 20/20 20/20 Qwen2.5-Omni VESUS 0.351 0.021 [0.317,0.392] 20/20 20/20 Audio-Flamingo-3 VESUS 0.561 0.027 [0.511,0.607] 20/20 20/20 Kimi-Audio VESUS 0.372 0.034 [0.305,0.429] 16/20 20/20 Phi-4-M VESUS 0.277 0.017 [0.249,0.311] 02/20 20/20 Table S10 adds the intervals omitted from the main text. Sdecoding−rand3S_decoding-rand_3 is positive with an interval excluding zero in all ten conditions. Adding SdecodingS_decoding to V3V_3 also improves decoding in eight conditions; the two nonsignificant Audio-Flamingo-3 increments reflect that its native readout is already highly informative. TABLE S10: Held-out subspace decodability with intervals. V3−rand3V_3-rand_3 compares two spaces that are both fixed without emotion labels; SdecodingS_decoding is selected on the training split, so its margin over rand3rand_3 includes a supervised search advantage. n.s. marks an interval containing zero. System Corpus SdecodingS_decoding V3−rand3V_3-rand_3 Sdecoding−rand3S_decoding-rand_3 [V3,Sdecoding]−V3[V_3,S_decoding]-V_3 Qwen2-Audio CREMA-D 0.949 +0.272+0.272 [+0.260,+0.284+0.260,+0.284] +0.332+0.332 [+0.321,+0.342+0.321,+0.342] +0.059+0.059 [+0.048,+0.070+0.048,+0.070] Qwen2.5-Omni CREMA-D 0.858 +0.204+0.204 [+0.190,+0.218+0.190,+0.218] +0.330+0.330 [+0.309,+0.350+0.309,+0.350] +0.125+0.125 [+0.103,+0.147+0.103,+0.147] Audio-Flamingo-3 CREMA-D 0.940 +0.068+0.068 [+0.060,+0.077+0.060,+0.077] +0.071+0.071 [+0.062,+0.080+0.062,+0.080] +0.003+0.003 [−0.003,+0.011-0.003,+0.011] n.s. Kimi-Audio CREMA-D 0.823 +0.136+0.136 [+0.117,+0.155+0.117,+0.155] +0.357+0.357 [+0.325,+0.386+0.325,+0.386] +0.221+0.221 [+0.185,+0.255+0.185,+0.255] Phi-4-M CREMA-D 0.668 +0.022+0.022 [+0.004,+0.040+0.004,+0.040] +0.348+0.348 [+0.320,+0.375+0.320,+0.375] +0.328+0.328 [+0.294,+0.361+0.294,+0.361] Qwen2-Audio VESUS 0.606 +0.104+0.104 [+0.075,+0.132+0.075,+0.132] +0.246+0.246 [+0.179,+0.322+0.179,+0.322] +0.144+0.144 [+0.103,+0.199+0.103,+0.199] Qwen2.5-Omni VESUS 0.561 +0.112+0.112 [+0.062,+0.151+0.062,+0.151] +0.209+0.209 [+0.155,+0.257+0.155,+0.257] +0.099+0.099 [+0.077,+0.128+0.077,+0.128] Audio-Flamingo-3 VESUS 0.690 +0.109+0.109 [+0.082,+0.135+0.082,+0.135] +0.129+0.129 [+0.105,+0.160+0.105,+0.160] +0.017+0.017 [−0.032,+0.070-0.032,+0.070] n.s. Kimi-Audio VESUS 0.563 +0.037+0.037 [+0.026,+0.047+0.026,+0.047] +0.192+0.192 [+0.140,+0.241+0.140,+0.241] +0.156+0.156 [+0.110,+0.201+0.110,+0.201] Phi-4-M VESUS 0.481 −0.018-0.018 [−0.033,+0.001-0.033,+0.001] n.s. +0.204+0.204 [+0.144,+0.287+0.144,+0.287] +0.226+0.226 [+0.175,+0.294+0.175,+0.294] S7 Minimal-Pair Activation Replacement S7-A Pairing, per-arm rates, and statistical units The primary causal pass uses strict minimal pairs from the held-out split: receiver and donor share speaker and transcript but differ in emotion. The donor representation is the state of one real clip under the same prompt, not a training-set or class-average state, and the same clip pair is used across all arms and prompts. The training split still fixes L∗L^*, SinterventionS_intervention, and the random subspace before any held-out intervention. Within the 2,000-clip stratified sample, 866 of 1,014 held-out CREMA-D clips and 474 of 1,000 held-out VESUS clips have an eligible minimal-pair donor. We sample 400 eligible receivers per condition and evaluate four prompts, giving 1,600 rows in each of the ten conditions. Because receiver and donor share a speaker, intervals cluster the paired outcomes by speaker: 46 clusters on CREMA-D and 5 on VESUS. Table S11 reports the primary per-condition effects that main-text Fig. 4(b) plots, together with each condition’s L∗L^*. TABLE S11: Matched rank-four activation replacement at L∗L^* using held-out minimal pairs. The in-span arm replaces V4V_4 and the readout-external arm replaces SinterventionS_intervention; each effect is the paired difference in answer-change rate from the same rand4rand_4 arm, with speaker-clustered 95% intervals, and n.s. marks an interval containing zero. The last column places the readout-external effect on a common scale as its one-sided 95% upper bound divided by the in-span effect of the same condition. System Corpus L∗L^* In-span effect [95% CI] Readout-external effect [95% CI] Relative 95% UB Qwen2-Audio CREMA-D 28 +0.3344+0.3344 [+0.3115,+0.3572+0.3115,+0.3572] +0.0206+0.0206 [+0.0136,+0.0276+0.0136,+0.0276] 7.9% Qwen2.5-Omni CREMA-D 27 +0.3631+0.3631 [+0.3218,+0.4044+0.3218,+0.4044] +0.0037+0.0037 [+0.0005,+0.0070+0.0005,+0.0070] 1.8% Audio-Flamingo-3 CREMA-D 25 +0.1156+0.1156 [+0.0915,+0.1397+0.0915,+0.1397] +0.0025+0.0025 [+0.0002,+0.0048+0.0002,+0.0048] 3.8% Kimi-Audio CREMA-D 27 +0.1950+0.1950 [+0.1582,+0.2318+0.1582,+0.2318] +0.0050+0.0050 [+0.0019,+0.0081+0.0019,+0.0081] 3.9% Phi-4-M CREMA-D 29 +0.0606+0.0606 [+0.0465,+0.0747+0.0465,+0.0747] −0.0063-0.0063 [−0.0127,+0.0002-0.0127,+0.0002] n.s. −1.4%-1.4\% Qwen2-Audio VESUS 31 +0.2437+0.2437 [+0.1617,+0.3258+0.1617,+0.3258] +0.0000+0.0000 [−0.0017,+0.0017-0.0017,+0.0017] n.s. 0.6% Qwen2.5-Omni VESUS 27 +0.1994+0.1994 [+0.1110,+0.2877+0.1110,+0.2877] +0.0000+0.0000 [−0.0017,+0.0017-0.0017,+0.0017] n.s. 0.7% Audio-Flamingo-3 VESUS 25 +0.1319+0.1319 [+0.0839,+0.1798+0.0839,+0.1798] −0.0013-0.0013 [−0.0054,+0.0029-0.0054,+0.0029] n.s. 1.7% Kimi-Audio VESUS 24 +0.0181+0.0181 [+0.0119,+0.0243+0.0119,+0.0243] +0.0088+0.0088 [+0.0053,+0.0122+0.0053,+0.0122] 64.1% Phi-4-M VESUS 28 +0.0269+0.0269 [+0.0185,+0.0352+0.0185,+0.0352] −0.0006-0.0006 [−0.0058,+0.0046-0.0058,+0.0046] n.s. 13.9% TABLE S12: Minimal-pair answer-change rates by arm at L∗L^*. V, SinterventionS_intervention, and Random are rank matched; Full replaces the entire state. System Corpus V SinterventionS_intervention Random Full Q2A CREMA-D 0.3350 0.0213 0.0006 0.6275 Omni CREMA-D 0.3631 0.0037 0.0000 0.5081 AF3 CREMA-D 0.1175 0.0044 0.0019 0.8831 Kimi CREMA-D 0.1963 0.0063 0.0013 0.3550 Phi-4 CREMA-D 0.0744 0.0075 0.0138 0.2744 Q2A VESUS 0.2450 0.0013 0.0013 0.2881 Omni VESUS 0.2000 0.0006 0.0006 0.2762 AF3 VESUS 0.1350 0.0019 0.0031 0.5713 Kimi VESUS 0.0300 0.0206 0.0119 0.3144 Phi-4 VESUS 0.0413 0.0138 0.0144 0.1363 Subtracting the random rate gives positive in-span effects in all ten conditions, from +0.0181+0.0181 to +0.3631+0.3631. The readout-external effect reaches +0.0206+0.0206 at most, and its relative upper bound is at most 7.9% in eight conditions. The two larger ratios occur where the in-span denominator is small; Kimi-Audio × VESUS is the only one of them with a detected readout-external answer-change effect. S7-B Donor-content outcome Answer change asks whether an edit moves the answer; donor following asks whether it moves specifically toward the donor emotion. Table S13 reports both matched effects on this stricter outcome. TABLE S13: Minimal-pair donor-following effects relative to the same random arm, with speaker-clustered 95% intervals. System Corpus In-span effect [95% CI] Readout-external effect [95% CI] Q2A CREMA-D +0.2281+0.2281 [+0.2056,+0.2506+0.2056,+0.2506] +0.0106+0.0106 [+0.0058,+0.0154+0.0058,+0.0154] Omni CREMA-D +0.2400+0.2400 [+0.2025,+0.2775+0.2025,+0.2775] +0.0006+0.0006 [−0.0006,+0.0018-0.0006,+0.0018] n.s. AF3 CREMA-D +0.1088+0.1088 [+0.0857,+0.1318+0.0857,+0.1318] +0.0013+0.0013 [−0.0005,+0.0030-0.0005,+0.0030] n.s. Kimi CREMA-D +0.1181+0.1181 [+0.0894,+0.1468+0.0894,+0.1468] +0.0006+0.0006 [−0.0015,+0.0027-0.0015,+0.0027] n.s. Phi-4 CREMA-D +0.0031+0.0031 [−0.0072,+0.0134-0.0072,+0.0134] n.s. −0.0031-0.0031 [−0.0077,+0.0014-0.0077,+0.0014] n.s. Q2A VESUS +0.1494+0.1494 [+0.0969,+0.2019+0.0969,+0.2019] −0.0006-0.0006 [−0.0017,+0.0004-0.0017,+0.0004] n.s. Omni VESUS +0.1156+0.1156 [+0.0624,+0.1689+0.0624,+0.1689] +0.0000+0.0000 [+0.0000,+0.0000+0.0000,+0.0000] n.s. AF3 VESUS +0.0950+0.0950 [+0.0445,+0.1455+0.0445,+0.1455] −0.0013-0.0013 [−0.0050,+0.0025-0.0050,+0.0025] n.s. Kimi VESUS +0.0106+0.0106 [+0.0055,+0.0157+0.0055,+0.0157] +0.0000+0.0000 [−0.0048,+0.0048-0.0048,+0.0048] n.s. Phi-4 VESUS −0.0050-0.0050 [−0.0113,+0.0013-0.0113,+0.0013] n.s. −0.0031-0.0031 [−0.0064,+0.0001-0.0064,+0.0001] n.s. Only Qwen2-Audio × CREMA-D shows readout-external donor following above random. Its +0.0106+0.0106 effect shows limited content-specific causal influence under a compatible replacement. The Phi-4-M in-span arm does not transfer donor content in either corpus, so its already small answer-change effects provide a weak scale reference. S7-C Perturbation-magnitude diagnostic Minimal pairing makes donor and receiver states more similar, so a null effect can coincide with a smaller edit. Table S14 reports the applied relative state change in all ten conditions and, where an earlier cross-speaker pass is available, the minimal-to-cross-speaker ratio. TABLE S14: Mean relative intervention magnitude ‖h~r−hr‖/‖hr‖\| h_r-h_r\|/\|h_r\|. Ratios compare minimal-pair with cross-speaker replacement where available; the last column compares the two arms within the minimal-pair pass. System Corpus Minimal V Minimal SinterventionS_intervention V ratio SinterventionS_intervention ratio Sintervention/VS_intervention/V Q2A CREMA-D 0.0786 0.1750 0.98 0.95 2.23 Omni CREMA-D 0.0585 0.0233 0.91 0.93 0.40 AF3 CREMA-D 0.1327 0.0987 – – 0.74 Kimi CREMA-D 0.0453 0.0203 – – 0.45 Phi-4 CREMA-D 0.0197 0.0226 0.71 0.88 1.15 Q2A VESUS 0.0248 0.0279 0.65 0.76 1.12 Omni VESUS 0.0296 0.0066 0.56 0.73 0.22 AF3 VESUS 0.0818 0.0307 – – 0.38 Kimi VESUS 0.0128 0.0146 – – 1.14 Phi-4 VESUS 0.0126 0.0137 0.40 0.68 1.09 For Qwen2-Audio × CREMA-D, minimal pairing retains 95% of the cross-speaker SinterventionS_intervention edit magnitude. Within the minimal-pair pass, that edit is 2.23 times the V4V_4 edit, yet its answer-change effect is +0.0206+0.0206 rather than +0.3344+0.3344, ruling out a weaker external edit as the explanation in this condition. Across all ten conditions, SinterventionS_intervention is at least as large as V4V_4 in five. Kimi-Audio × VESUS belongs to this set, but both edits are the smallest in the study, matching its status as a weak-intervention boundary case. S7-D Routing capacity above L∗L^* The selected L∗L^* lies one to four blocks below the top of each stack (Table S11). A weak SinterventionS_intervention effect could therefore have a simple explanation: the remaining blocks might be unable to route any V4⟂V_4 component into the option logits. The full-state arm tests this possibility. Relative to the in-span replacement, it adds the donor’s entire V4⟂V_4 component. If the remaining blocks were unresponsive to that component, the two arms would have the same effect. Instead, the full arm changes the answer more often than the in-span arm in every condition, from 0.288 versus 0.245 on Qwen2-Audio × VESUS to 0.883 versus 0.118 on Audio-Flamingo-3 × CREMA-D (Table S12). The additional changes generally move toward the donor emotion. Relative to the unpatched baseline, the full arm raises donor following in all ten conditions: by +0.161+0.161 to +0.839+0.839 in the eight non-Phi-4-M conditions, and by +0.046+0.046 and +0.009+0.009 in the two Phi-4-M conditions. The weak SinterventionS_intervention effects therefore cannot be explained solely by a downstream pathway that is unresponsive to readout-external content. S7-E Minimal-pair depth scan: protocol and consistency TABLE S15: Minimal-pair replacement at eight depths of Qwen2-Audio × CREMA-D, on all 866 eligible receivers, with the readout-external subspace refit at each depth. Each cell is the paired difference from the same-rank random arm, with speaker-clustered 95% intervals; n.s. marks an interval containing zero, and L=28L=28 is the L∗L^* used in Table S11. Donor following Answer change L In-span V4V_4 Readout-external SinterventionS_intervention In-span V4V_4 Readout-external SinterventionS_intervention 16 +0.0003+0.0003 [−0.0003,+0.0008-0.0003,+0.0008] n.s. +0.0012+0.0012 [+0.0001,+0.0023+0.0001,+0.0023] +0.0000+0.0000 [−0.0008,+0.0008-0.0008,+0.0008] n.s. +0.0014+0.0014 [−0.0002,+0.0031-0.0002,+0.0031] n.s. 18 +0.0020+0.0020 [+0.0004,+0.0036+0.0004,+0.0036] +0.0072+0.0072 [+0.0039,+0.0105+0.0039,+0.0105] +0.0032+0.0032 [+0.0012,+0.0051+0.0012,+0.0051] +0.0162+0.0162 [+0.0123,+0.0200+0.0123,+0.0200] 20 +0.0136+0.0136 [+0.0101,+0.0171+0.0101,+0.0171] +0.0144+0.0144 [+0.0099,+0.0190+0.0099,+0.0190] +0.0185+0.0185 [+0.0144,+0.0226+0.0144,+0.0226] +0.0372+0.0372 [+0.0308,+0.0437+0.0308,+0.0437] 22 +0.0681+0.0681 [+0.0595,+0.0768+0.0595,+0.0768] +0.0089+0.0089 [+0.0058,+0.0121+0.0058,+0.0121] +0.0941+0.0941 [+0.0861,+0.1021+0.0861,+0.1021] +0.0196+0.0196 [+0.0157,+0.0236+0.0157,+0.0236] 24 +0.0710+0.0710 [+0.0624,+0.0796+0.0624,+0.0796] +0.0110+0.0110 [+0.0073,+0.0147+0.0073,+0.0147] +0.1028+0.1028 [+0.0948,+0.1107+0.0948,+0.1107] +0.0280+0.0280 [+0.0233,+0.0327+0.0233,+0.0327] 26 +0.1723+0.1723 [+0.1598,+0.1849+0.1598,+0.1849] +0.0115+0.0115 [+0.0083,+0.0148+0.0083,+0.0148] +0.2610+0.2610 [+0.2492,+0.2727+0.2492,+0.2727] +0.0188+0.0188 [+0.0149,+0.0226+0.0149,+0.0226] 28 +0.2307+0.2307 [+0.2126,+0.2487+0.2126,+0.2487] +0.0095+0.0095 [+0.0062,+0.0128+0.0062,+0.0128] +0.3346+0.3346 [+0.3176,+0.3516+0.3176,+0.3516] +0.0147+0.0147 [+0.0111,+0.0183+0.0111,+0.0183] 31 +0.4983+0.4983 [+0.4684,+0.5281+0.4684,+0.5281] +0.0064+0.0064 [+0.0036,+0.0091+0.0036,+0.0091] +0.6544+0.6544 [+0.6261,+0.6828+0.6261,+0.6828] +0.0084+0.0084 [+0.0053,+0.0115+0.0053,+0.0115] Table S15 repeats the minimal-pair intervention at layers 16, 18, 20, 22, 24, 26, 28, and 31 of Qwen2-Audio × CREMA-D. Main-text Fig. 5 plots the donor-following columns. The pairing, receiver split, V4V_4, and rand4rand_4 are the same as in the primary pass. At each depth, SinterventionS_intervention is refit on the training speakers using the construction in Section S6. All 866 held-out receivers with an eligible same-speaker, same-transcript donor are included under four prompt variants, giving 3,464 rows per cell. Intervals cluster on receiver speaker. At layer 28, the selected L∗L^*, the readout-external answer-change effect is +0.0147+0.0147 [+0.0111,+0.0183+0.0111,+0.0183], close to the primary-pass estimate of +0.0206+0.0206 [+0.0136,+0.0276+0.0136,+0.0276] obtained from a 400-receiver sample. Beyond the peak at layer 20, the decline is not layer-by-layer monotone; the effects at layers 22 through 26 sit within one another’s intervals, and the ordering claim the table supports is that the external effect is largest at layer 20 and smallest at layers 16 and 31. S7-F Propagation of the injected readout-external component We repeat the layer-20 and layer-24 SinterventionS_intervention replacements on 400 receivers and record the induced answer-position difference δ(L′)=hpatched(L′)−hunpatched(L′)δ(L )=h_patched(L )-h_unpatched(L ) at every later layer, together with the endpoint option logits. After a layer-20 replacement, the component of δ in the injected subspace retains 0.88 of its original norm at layer 31. The perturbation therefore persists. Its overlap with V4V_4 grows from zero at injection to about 0.07 of ‖δ‖\|δ\| by layer 26 and then remains near that level, showing that part of the perturbation reaches the native readout span. The endpoint logits also move toward the donor emotion. Relative to the random arm, the donor-option logit minus the mean of the other three option logits shifts by +0.190+0.190 [+0.176,+0.204+0.176,+0.204] after the layer-20 replacement and by +0.312+0.312 [+0.289,+0.335+0.289,+0.335] after the layer-24 replacement. Thus the injected component reaches the final option scores with the donor’s sign, but usually not strongly enough to change their ordering. Because the last saved state may include the model’s final normalization, this endpoint statement uses the recorded option logits rather than the state-norm decomposition. S8 Controls for Measured Surface Acoustic Cues These analyses test whether the held-out decodability of SdecodingS_decoding can be explained by measured surface acoustic cues. We define the descriptor panels, quantify how predictive the cues are, remove them from the subspace coordinates, and separately remove absolute level from the input audio. S8-A Descriptor Panel Ten clip-level acoustic-prosodic descriptors are computed once per recording and are model-independent: duration; voiced-frame fraction; five fundamental-frequency statistics (mean, standard deviation, range, terminal value, and slope of the F0 track); and three RMS-energy statistics (mean, standard deviation, and max-minus-min spread). RMS descriptors enter all emotion-corpus analyses in log units. In the residualization and decoding analyses, missing descriptor values, standardization statistics, ordinary-least-squares coefficients, and class means are all estimated on training-speaker rows only and then applied to held-out rows. The descriptor-only decoder AdescA_desc uses the same probe family, regularization-selection protocol, and speaker split as the subspace decoders. An extended panel used for robustness adds ten further descriptors: spectral tilt (the regression slope of the long-term average spectrum in dB over log-frequency), spectral centroid mean and standard deviation, local jitter, local shimmer, harmonics-to-noise ratio, and the first four DCT coefficients of the time-interpolated log-F0 contour. Jitter, shimmer, and harmonics-to-noise ratio are computed with Praat via parselmouth; coverage is complete on both corpora. The nonlinear removal variant replaces the ordinary-least-squares residualization with per-dimension gradient-boosted trees, again fit on training-speaker rows only. S8-B Stimulus-Level Loudness Statistics Table S16 reports, for every pair of emotion classes, the class-mean RMS-level difference in dB and how well dB level alone separates the pair. Folded AUC is max(AUC,1−AUC) (AUC,1-AUC) and is therefore orientation-free. Uncertainty is a speaker-clustered bootstrap with 2,000 resamples. On CREMA-D, level alone is a strong class separator for several pairs; on VESUS the gaps are smaller. These statistics motivate treating surface loudness as an explicit alternative explanation rather than an afterthought. TABLE S16: Stimulus-level RMS-loudness differences between emotion classes. Δ is the class-mean dB RMS gap (first class minus second); folded AUC measures how well dB RMS alone separates the pair. Speaker-clustered bootstrap 95% intervals. Corpus Pair Δ [95% CI] AUC [95% CI] CREMA-D happy vs sad +8.01[+7.48,+8.56]+8.01\,[+7.48,+8.56] 0.92[0.90,0.93]0.92\,[0.90,0.93] happy vs angry −5.70[−6.34,−5.10]-5.70\,[-6.34,-5.10] 0.77[0.75,0.80]0.77\,[0.75,0.80] happy vs neutral +4.77[+4.21,+5.33]+4.77\,[+4.21,+5.33] 0.80[0.76,0.83]0.80\,[0.76,0.83] sad vs angry −13.71[−14.47,−12.96]-13.71\,[-14.47,-12.96] 0.98[0.97,0.99]0.98\,[0.97,0.99] sad vs neutral −3.24[−3.63,−2.85]-3.24\,[-3.63,-2.85] 0.79[0.76,0.82]0.79\,[0.76,0.82] angry vs neutral +10.47[+9.81,+11.15]+10.47\,[+9.81,+11.15] 0.95[0.93,0.96]0.95\,[0.93,0.96] VESUS happy vs sad +4.26[+2.41,+6.03]+4.26\,[+2.41,+6.03] 0.70[0.62,0.81]0.70\,[0.62,0.81] happy vs angry −1.23[−3.44,+0.74]-1.23\,[-3.44,+0.74] 0.58[0.50,0.71]0.58\,[0.50,0.71] happy vs neutral +2.56[+1.30,+3.98]+2.56\,[+1.30,+3.98] 0.68[0.58,0.81]0.68\,[0.58,0.81] sad vs angry −5.48[−7.67,−3.49]-5.48\,[-7.67,-3.49] 0.74[0.65,0.87]0.74\,[0.65,0.87] sad vs neutral −1.70[−4.04,+0.63]-1.70\,[-4.04,+0.63] 0.57[0.51,0.71]0.57\,[0.51,0.71] angry vs neutral +3.79[+1.52,+5.91]+3.79\,[+1.52,+5.91] 0.72[0.58,0.86]0.72\,[0.58,0.86] S8-C Base Descriptor Removal TABLE S17: Linear controls for the ten measured acoustic-prosodic descriptors at L∗L^*. AdescA_desc is held-out four-class accuracy from the descriptors alone, with no hidden state. The resid columns re-decode each rank-three projection after removing the descriptors, with the regression fit on training speakers only. The last column is the paired SdecodingS_decoding drop with speaker-clustered 95% intervals; n.s. marks an interval containing zero. The V3V_3 and SdecodingS_decoding columns match Table S8. System Corpus AdescA_desc V3V_3 V3V_3 resid SdecodingS_decoding SdecodingS_decoding resid SdecodingS_decoding drop [95% CI] Qwen2-Audio CREMA-D 0.640 0.890 0.759 0.949 0.856 +0.094+0.094 [+0.075,+0.115+0.075,+0.115] Qwen2.5-Omni CREMA-D 0.640 0.732 0.613 0.858 0.655 +0.203+0.203 [+0.176,+0.230+0.176,+0.230] Audio-Flamingo-3 CREMA-D 0.640 0.937 0.852 0.940 0.847 +0.093+0.093 [+0.067,+0.121+0.067,+0.121] Kimi-Audio CREMA-D 0.640 0.602 0.464 0.823 0.610 +0.214+0.214 [+0.189,+0.237+0.189,+0.237] Phi-4-M CREMA-D 0.640 0.342 0.314 0.668 0.413 +0.255+0.255 [+0.230,+0.280+0.230,+0.280] Qwen2-Audio VESUS 0.343 0.463 0.426 0.606 0.537 +0.069+0.069 [+0.048,+0.089+0.048,+0.089] Qwen2.5-Omni VESUS 0.343 0.464 0.422 0.561 0.493 +0.068+0.068 [+0.042,+0.095+0.042,+0.095] Audio-Flamingo-3 VESUS 0.343 0.670 0.644 0.690 0.651 +0.039+0.039 [−0.014,+0.077-0.014,+0.077] n.s. Kimi-Audio VESUS 0.343 0.408 0.361 0.563 0.488 +0.076+0.076 [+0.040,+0.111+0.040,+0.111] Phi-4-M VESUS 0.343 0.259 0.262 0.481 0.423 +0.059+0.059 [+0.022,+0.090+0.022,+0.090] Table S17 reports the base control summarized in main-text Fig. 6. It gives the descriptor-only reference AdescA_desc, decoding after linear removal of the ten descriptors, and the paired drop in SdecodingS_decoding accuracy. S8-D Subspace–Descriptor Associations Table S18 reports, for each condition and subspace, the single descriptor best predicted from the rank-three projection, as out-of-sample R2R^2 under an ordinary-least-squares fit on training speakers. The pooled column predicts the raw descriptor; the within-class column first centers both the descriptor and the projection by their training-speaker class means, so it measures covariation that is not explained by class membership. The dominant descriptors are energy statistics in nearly every condition, and the within-class values are substantially smaller than the pooled ones, indicating that much of the pooled association reflects class structure. These associations are reported descriptively, without a multiplicity correction across descriptors and conditions. TABLE S18: Strongest acoustic-descriptor association per subspace: the top-R2R^2 descriptor, with held-out OLS R2R^2 in parentheses. Pooled: descriptor predicted directly from the rank-three projection. Within-class: both centered by training-speaker class means first. Condition V3V_3 top (pooled) V3V_3 top (within) SdecodingS_decoding top (pooled) SdecodingS_decoding top (within) Qwen2-Audio × CREMA-D rms_std (0.69) rms_mean (0.18) rms_std (0.63) rms_mean (0.06) Qwen2.5-Omni × CREMA-D rms_std (0.62) rms_mean (0.16) rms_std (0.62) rms_mean (0.09) Audio-Flamingo-3 × CREMA-D rms_std (0.73) rms_std (0.31) rms_std (0.65) rms_std (0.10) Kimi-Audio × CREMA-D rms_mean (0.64) rms_mean (0.32) rms_std (0.75) rms_std (0.37) Phi-4-M × CREMA-D rms_mean (0.17) duration (0.12) rms_std (0.62) rms_mean (0.17) Qwen2-Audio × VESUS rms_std (0.18) rms_mean (0.07) rms_mean (0.22) rms_mean (0.17) Qwen2.5-Omni × VESUS rms_std (0.10) rms_std (0.10) rms_mean (0.15) rms_mean (0.13) Audio-Flamingo-3 × VESUS rms_std (0.35) rms_mean (0.22) rms_mean (0.36) rms_mean (0.32) Kimi-Audio × VESUS rms_mean (0.12) rms_mean (0.04) rms_mean (0.12) rms_mean (0.06) Phi-4-M × VESUS rms_mean (0.02) rms_mean (0.01) rms_mean (0.13) rms_mean (0.09) S8-E Extended Panel and Nonlinear Removal Table S19 repeats the residualized decoding of the main text under descriptor removal of increasing strength: the ten-descriptor panel removed linearly, the extended twenty-descriptor panel removed linearly, and the extended panel removed with gradient-boosted trees. Held-out SdecodingS_decoding decodability remains above the 0.25 chance level in every condition under every variant. The descriptor-only reference also strengthens slightly with the extended panel, from 0.640 to 0.664 on CREMA-D and from 0.343 to 0.378 on VESUS, confirming that the added features carry usable information that SdecodingS_decoding nevertheless exceeds. TABLE S19: Held-out SdecodingS_decoding decodability after descriptor removal of increasing strength. resid-10: ordinary least squares on the ten-descriptor panel (main text). resid-20: the same on the extended twenty-descriptor panel. GBRT-20: gradient-boosted-tree removal of the extended panel. Chance is 0.25. System Corpus SdecodingS_decoding resid-10 resid-20 GBRT-20 Qwen2-Audio CREMA-D 0.949 0.856 0.832 0.752 Qwen2.5-Omni CREMA-D 0.858 0.655 0.630 0.562 Audio-Flamingo-3 CREMA-D 0.940 0.847 0.835 0.752 Kimi-Audio CREMA-D 0.823 0.610 0.574 0.494 Phi-4-M CREMA-D 0.668 0.413 0.385 0.347 Qwen2-Audio VESUS 0.606 0.537 0.514 0.463 Qwen2.5-Omni VESUS 0.561 0.493 0.453 0.410 Audio-Flamingo-3 VESUS 0.690 0.651 0.610 0.568 Kimi-Audio VESUS 0.563 0.488 0.474 0.424 Phi-4-M VESUS 0.481 0.423 0.380 0.349 S8-F Input-Side Loudness Equalization The residualization analyses remove surface cues from the state side. The input-side check removes absolute level from the audio itself: every clip is RMS-equalized to a fixed target with peak limiting, answer-position states are re-extracted under the main protocol (the same clips, the four multiple-choice prompts, and the per-condition L∗L^*), and the subspace decoding is repeated. Two arms are scored. The replication arm rebuilds SdecodingS_decoding and refits the probe on equalized training-speaker rows with the split held fixed; V3V_3 is a function of the model weights and is reused unchanged. The transfer arm applies the probes fit on raw states, without refitting, to the equalized held-out rows. TABLE S20: Raw versus loudness-equalized held-out decodability under the main protocol. eq: SdecodingS_decoding and probe rebuilt on equalized audio with the split held fixed. transfer: raw-fit probe applied without refitting to equalized held-out clips. The last column is the paired raw-minus-equalized SdecodingS_decoding difference with speaker-clustered bootstrap 95% intervals. System Corpus V3V_3 raw V3V_3 eq SdecodingS_decoding raw SdecodingS_decoding eq SdecodingS_decoding transfer SdecodingS_decoding raw−-eq [95% CI] Qwen2-Audio CREMA-D 0.890 0.891 0.949 0.949 0.950 +0.000+0.000 [−0.006,+0.006-0.006,+0.006] Qwen2.5-Omni CREMA-D 0.732 0.710 0.858 0.857 0.843 +0.001+0.001 [−0.008,+0.009-0.008,+0.009] Audio-Flamingo-3 CREMA-D 0.937 0.933 0.940 0.939 0.939 +0.001+0.001 [−0.004,+0.007-0.004,+0.007] Kimi-Audio CREMA-D 0.602 0.589 0.823 0.802 0.793 +0.022+0.022 [+0.008,+0.036+0.008,+0.036] Phi-4-M CREMA-D 0.342 0.329 0.668 0.655 0.645 +0.013+0.013 [−0.002,+0.028-0.002,+0.028] Qwen2-Audio VESUS 0.463 0.460 0.606 0.584 0.599 +0.022+0.022 [−0.002,+0.047-0.002,+0.047] Qwen2.5-Omni VESUS 0.464 0.448 0.561 0.544 0.551 +0.016+0.016 [+0.001,+0.036+0.001,+0.036] Audio-Flamingo-3 VESUS 0.670 0.649 0.690 0.694 0.673 −0.004-0.004 [−0.013,+0.006-0.013,+0.006] Kimi-Audio VESUS 0.408 0.401 0.563 0.570 0.563 −0.007-0.007 [−0.014,+0.001-0.014,+0.001] Phi-4-M VESUS 0.259 0.258 0.481 0.475 0.472 +0.007+0.007 [−0.013,+0.028-0.013,+0.028] Table S20 shows both arms. Held-out SdecodingS_decoding accuracy changes by at most 0.022 under replication and 0.030 under transfer across all ten conditions, and the paired raw-minus-equalized interval includes zero in eight of ten. Absolute recording level therefore does not explain most of the readout-external decodability. This control removes only absolute level, not energy dynamics, fundamental frequency, or spectral cues; gain normalization in the audio front ends may also contribute to the observed robustness.