Paper deep dive

Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace

Jiajun Fan, Jingyuan Li, Prashanth Gurunath Shivakumar, Qi Luo, Jia-Hong Huang, M. Maruf, Roger Ren, Yile Gu, Rahul Pandey, Ge Liu, Ivan Bulyko

Year: 2026Venue: arXiv preprintArea: cs.SDType: PreprintEmbeddings: 54

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%

Last extracted: 8/27/2026, 4:14:18 AM

Summary

This paper investigates the internal representations of the Qwen3-Omni audio language model using a logit lens to read intermediate activations. The authors demonstrate that the model forms a verbalizable, multilingual 'middle-layer workspace' (layers 35-80%) where audio-driven concepts are legible before the model generates an answer. Key findings include the model's ability to reconstruct hidden multi-hop reasoning chains (e.g., inferring 'Nixon' from garbled audio via 'Watergate' and 'scandal'), language-agnostic concept representation (significant Chinese readouts on English inputs), and the extraction of paralinguistic information (emotion, source) not present in text transcripts. The study uses waveform-swap controls and activation patching to isolate audio-specific signals from text priors.

Entities (8)

Qwen3-Omni · model · 98%Logit Lens · method · 95%Middle-Layer Workspace · concept · 92%Nixon · entity · 90%Watergate · event · 90%Activation Patching · method · 88%Waveform-Swap Control · method · 85%MMAU · dataset · 80%

Relation Signals (7)

Qwen3-Omni → uses → Logit Lens

confidence 95% · Reading a base Qwen3-Omni with a logit lens at the audio-token positions

Qwen3-Omni → exhibits → Middle-Layer Workspace

confidence 92% · we find that the answer to a spoken question becomes legible - in words - in the model's middle layers

Qwen3-Omni → infers → Nixon

confidence 90% · on a clip whose verbatim transcription is empty garbling, it reconstructs Watergate and scandal... and resolves to Nixon

Qwen3-Omni → infers → Watergate

confidence 90% · reconstructs Watergate and scandal

Activation Patching → demonstrates → Causal Use

confidence 88% · activation patching shows it is causally used and committed before the last fifth of the layers

Waveform-Swap Control → isolates → Audio-Driven Signal

confidence 85% · isolates the audio-driven signal from a prior over the printed options

Qwen3-Omni → evaluatedon → MMAU

confidence 80% · Core controls run on a curated 140-clip MMAU [21] set

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:An audio language model is a black box in a specific way: we see what it says, never what it works out on the way there, and chain-of-thought monitoring helps only if the model writes its reasoning down. Reading a base Qwen3-Omni with a logit lens at the audio-token positions, we find that the answer to a spoken question becomes legible - in words - in the model's middle layers, before it emits any token. Five findings follow. (1) The readout carries concepts in neither the question, the options, nor the model's own transcription: on a clip whose verbatim transcription is empty garbling, it reconstructs Watergate and scandal, passes through the role president, and resolves to Nixon - a hidden multi-hop chain, read with no chain-of-thought. (2) The content is language-agnostic: one audio-inferred concept surfaces in several scripts at once, and 38% of top-1 readouts are Chinese on English inputs. (3) It is paralinguistic: given the same clip as audio and as the model's own emotion-free caption, the audio mind forms the sound source, speaker role, or affect that the caption discards, and answers correctly more often. (4) The audio-driven signal is absent at the input, turns on about a tenth of the way into the network, separates most cleanly from the text prior in the middle band (35-80% of depth), and activation patching shows it is causally used and committed before the last fifth of the layers. (5) Deleting single layers maps the pipeline: reading the sound in is localized to the entry layers and answer delivery to the output layer, while retrieval is distributed across the interior. Throughout, a waveform-swap control - identical text, only the sound changed - isolates the audio-driven signal from a prior over the printed options. This is a qualitative account of what an audio model works out before it speaks: the quantities are controls, not benchmark scores.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

50,294 characters extracted from source content.

Expand or collapse full text

Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace Jiajun Fan 1,2 , Jingyuan Li 1 , Prashanth Gurunath Shivakumar 1 , Qi Luo 1 , Jia-Hong Huang 1 , M. Maruf 1 , Roger Ren 1 , Yile Gu 1 , Rahul Pandey 1 , Ge Liu 2 , Ivan Bulyko 1 1 Amazon AGI Foundations 2 University of Illinois Urbana-Champaign Abstract An audio language model is a black box in a specific way: we see what it says, never what it works out on the way there, and chain-of-thought monitoring helps only if the model writes its reasoning down. Reading a base Qwen3-Omni with a logit lens at the audio-token positions, we find that the answer to a spoken question becomes legible—in words—in the model’s middle layers, before it emits any token. Five findings follow. (1) The readout carries concepts in neither the question, the options, nor the model’s own transcription: on a clip whose verbatim transcription is empty garbling, it reconstructs Watergate and scandal, passes through the role president, and resolves to Nixon—a hidden multi-hop chain, read with no chain-of-thought. (2) The content is language-agnostic: one audio-inferred concept surfaces in several scripts at once, and 38% of top-1 readouts are Chinese on English inputs. (3) It is paralinguistic: given the same clip as audio and as the model’s own emotion-free caption, the audio mind forms the sound source, speaker role, or affect that the caption discards, and answers correctly more often. (4) The audio-driven signal is absent at the input, turns on about a tenth of the way into the network, separates most cleanly from the text prior in the middle band (35–80% of depth), and activation patching shows it is causally used and committed before the last fifth of the layers. (5) Deleting single layers maps the pipeline: reading the sound in is localized to the entry layers and answer delivery to the output layer, while retrieval is distributed across the interior. Throughout, a waveform-swap control—identical text, only the sound changed—isolates the audio-driven signal from a prior over the printed options. This is a qualitative account of what an audio model works out before it speaks: the quantities are controls, not benchmark scores. Date: August 27, 2026 1 Introduction An audio language model has been a black box: we see what it says, never what it thinks. Chain-of-thought helps only if the model writes its reasoning out—and a model that answers a spoken question directly writes nothing down. In this work we read a base audio LLM (Qwen3-Omni) during the forward pass, as it listens to a spoken question and before it answers, and watch intermediate concepts—the source of a sound, a speaker’s role, an inferred event—take shape in its middle layers and become legible as words, though they appear in neither the question nor the answer. What makes this possible is that a model’s intermediate computation is, to a meaningful degree, legible. By projecting intermediate activations into vocabulary space, the logit lens [20] and its calibrated successors [3] 1 arXiv:2608.24958v1 [cs.SD] 25 Aug 2026 surface partially formed predictions well before those tokens are produced. Building on this, Gurnee et al.[12] argue that these readable representations are not incidental: language models maintain a small, privileged set of verbalizable representations concentrated in the middle layers that is used for internal reasoning, broadcast onward, and selectively recruited—a functional global workspace [1,5]. Their analysis exposes three functional bands: sensory early layers, a workspace band in the middle (∼35–80% of depth) where verbalizable concepts are represented, and motor late layers that align to the output. Those results were established for text, code, and image inputs; whether the structure extends to other modalities was explicitly left open. Audio was not tested. This matters beyond curiosity. Speech agents consume raw audio, may reason over paralinguistic content a transcript discards, and can emit speech or tool calls directly; if their decision-relevant concepts were legible before they act, that would be a lever for interpretability-based oversight—precisely the regime a chain-of-thought monitor [2,14] cannot see, since stated reasoning need not be produced and can be unfaithful [15,25]. But the workspace picture was established only for token inputs. When an audio LLM listens to a spoken question, does it form a middle-layer, verbalizable structure in which the answer—derived from the sound—can be read out, or is any readable structure merely a text prior over the accompanying prompt? We study base Qwen3-Omni-30B-A3B-Instruct [23], whose audio encoder injects features at audio-token positions into a 48-layer text Thinker. We apply a logit lens at those positions and read, for each answer option, the best rank achieved by its first token. Our central control fixes the question and the options byte-identical and swaps only the waveform (the real clip, a mismatched clip, or silence), which separates a genuinely sound-driven readout from a prior over the printed text. We organize the evidence as five findings. 1.We can read the model’s mind, with no chain-of-thought (Section 4.1): on a clip whose verbatim transcription is empty garbling, the readout assembles water+gate, then scandal, then the role president, and resolves to Nixon—a hidden multi-hop chain. 2. The thinking space is multilingual (Section 4.2): the same audio-inferred concept surfaces in several languages at once, and 38.5% of top-1 readouts are Chinese on entirely English inputs. 3.When speech carries extra information, the audio mind registers it and the text mind does not (Section 4.3): given the same clip as audio and as the model’s own emotion-free caption, only the audio mind forms the true sound source, speaker role, or affect. 4. The thought lives in the middle and forms in depth order (Section 4.4): the audio-driven gap over the text prior is null at the input, significant from∼12% depth, most cleanly separable from the prior in the workspace band, and causally used before the motor layers. 5. The answer is distributed, and hallucinations are born early (Section 4.5): deleting layers localizes listening to the entry layers and delivery to the output layer, while no interior layer is essential; a text-mode hallucination usually takes over early and is never revised. This is a qualitative account, and we are deliberately conservative about what it shows: a logit lens rather than the more faithful Jacobian lens, one model family, a small sound-dominated slice, and a readout that is sparse across audio positions. The quantities we report are controls—they exist to rule out a text prior, not to score the model on a benchmark. 2 Related Work Reading transformer internals. The logit lens [20] and tuned lens [3] decode intermediate states into vocabulary space; circuit- and feature-level work [6,19,24] studies how such content is computed. Gurnee et al.[12]characterize a verbalizable middle-layer global workspace—verbalizable, reused, broadcast, and gated representations—and introduce the Jacobian lens, a faithful alternative to the logit lens, to map it. Reading such internal state is of interest for oversight because chain-of-thought monitors [2,14] depend on a model’s stated reasoning, which need not be produced and can be unfaithful [15, 25]. Latent language and audio lenses. Text models represent meaning in a language-agnostic “semantic hub” [26,27]. Closest to us, two lines already apply a vocabulary projection to audio inputs: the Semantic Hub Hypothesis [27] shows audio content is lens-readable in the model’s dominant language, and AudioLens 2 [30] tracks how surface auditory attributes (e.g. gender, emotion) evolve across layers. Our question is different in two ways. First, we ask not whether any audio content is lens-readable, but whether the particular global-workspace organization of Gurnee et al.[12]—a verbalizable middle band carrying task-relevant, inferred content—appears for audio. Second, we add a waveform-swap control that isolates the audio-driven signal from a text prior over the printed options; without such a control a lens readout cannot be attributed to the sound rather than to the accompanying prompt. To our knowledge this control, and the resulting localization of an audio-driven, task-relevant signal to the workspace band, are new. What speech models are trained to do, and why we read a base one. The readout we study sits under a fast- moving post-training stack. Rewards defined over the reasoning process improve the consistency and scalability of audio-LLM reasoning [9]; procedure-aware reinforcement learning does the same for tool-augmented models [31]; and executable pipelines push multi-hop reasoning out of the activations and into code the system runs [22]. All three make reasoning more externally visible. We take the opposite starting point—a base model, answering directly, with nothing written down—and ask what is already legible inside it before any of that machinery is added. 3 Method Model and lens. We read the Thinker of Qwen3-Omni-30B-A3B-Instruct [23]:L=48 layers, widthd=2048, vocabulary|V|=152,064, untied output embeddingW U . The logit lens [20] decodes an intermediate residual state h (ℓ) p by applying the terminal read-out prematurely, lens h (ℓ) p = softmax W U norm(h (ℓ) p ) .(1) For a target tokentwe take its rank under this distribution; rank 1 meanstis the top token. Ranks are comparable across layers, where raw probabilities and logit scale drift with depth. We read at audio-token positions and report the best (minimum) rank over those positions and over the workspace-band layers (Thinker layers 17–38,∼35–80% depth, taken a priori from the text-model account of Gurnee et al.[12], not tuned on our data); the four-way prediction is the option of minimum rank. The logit lens is a cheap proxy rather than the Jacobian lens of Gurnee et al.[12]; relative to a faithful lens our readouts should, if anything, under-detect, and early-layer readouts are noisy by construction, so our claims concern the middle band. Audio-swap control. Audio is injected at audio-pad positions, which precede the question in the causal stream, so an audio-position readout cannot attend to the trailing options. We hold the written question and every option token byte-identical and replace only the waveform with (a) the real clip, (b) a mismatched clip of the same category, or (c) silence. This isolates an audio-driven readout from a text prior over the printed options. Because option positions are imbalanced (an always-majority baseline scores∼46%, not 25%), our primary metrics are balanced accuracy and the paired McNemar test. Two-mind comparison. To ask what the sound adds over its words, we feed the same clip twice—as the real waveform, and as the model’s own emotion-free caption of it supplied as text with no waveform—with the identical question and options in both runs. Causal tests. Decodability need not mean use. We test use with activation patching: on clips the model answers correctly with audio and incorrectly once the audio’s mel-features are zeroed, we copy the clean run’s residual stream into the corrupted run at one band of layers and at the audio-token positions, and let the model finish. Separately, we delete one layer at a time (replacing its output with its input, an identity skip) and measure the resulting accuracy drop; single-clip answers are noisy under bf16 mixture-of-experts routing, so we average over clips. Data. Core controls run on a curated 140-clip MMAU [21] set (non-counting answers whose first token is uniquely discriminative, an 83:57 sound:speech split); corpus-level statistics draw on larger MMAU pools as noted (338 music-track clips, 948 brain-map grids—one grid is one clip’s layer×position readout, 863,770 workspace cells—the 1000-clip MMAU-mini, 55 emotion clips), and on a 500-question spoken TriviaQA [13]. 3 0 50 100 1. water + gate (= "Watergate") 0 50 100 2. scandal 0 50 100 3. 美国总统 / president audio (time →) question + options 0 50 100 4. Nixon (answer) absentrank 1 pinned concept (darker = better rank) layer depth (input → output, 0–100%) ▉ = the model is reading a real word here (darker inside the workspace band) Reading one clip’s mind, concept by concept — the hidden chain, top to bottom, until the answer Q: “Which president was forced to resign due to the event?” answer: Richard Nixon Figure 1 Reading one clip’s mind, concept by concept (Nixon clip). One row per concept word; each is that word’s logit-lens rank over the full input (x: audio positions, then question+options) against layer depth (y). Darker = better rank; the shaded band is the workspace. The event concepts (water/gate, scandal) are rank 1 from the early layers and the actor concepts (president, Nixon) strengthen deeper—an event-then-actor progression rather than a strict token-by-token order; the answer is reached over the audio, not only at the printed option. 4 Findings 4.1 Finding 1: We Can Read the Mind—No Chain-of-Thought Needed Reading the forward pass at the audio positions, we can watch the model’s train of thought—not just what it answers but how it gets there. The most striking cases are multi-hop: the workspace surfaces a concept in neither the question nor any option, inferred from the sound, and only then resolves to the answer. The Nixon clip (flagship). Our cleanest case asks which president was forced to resign (answer: Nixon). What makes it clean is that the model’s own two routes for turning this audio into text both fail to name the chain. Its verbatim transcription is empty garbling (“The ote — [End of Audio]”); its free-form caption recovers only the generic political scene (“. . . president of. . . political officer. . . politician. . . senator. . . ”), never Watergate, scandal, or Nixon. Yet the middle-layer readout assembles exactly that chain (Figure 1). At adjacent audio positions it reaches rank 1 on water and gate—reconstructing “Watergate”, which never appears as a single token—while scandal is rank 1 from the earliest layers. The event so characterized, the person and the role surface together as the readout deepens: Nixon and president both reach rank 1 across the sensory–workspace transition, with the role persisting in the workspace band as Chinese zˇongtˇong. We therefore read the chain as event-then-actor rather than a strict token-by-token ordering. Nixon is a printed option, but it is decoded over the audio positions, not merely read off the choice—this is the one clip where the audio-side readout reaches the answer name. That two independent audio-to-text routes miss these concepts makes plain re-transcription unlikely; a failed ASR does not prove the words were never spoken, so we bound that residual possibility as a limitation. The Kennedy clip (a partial case, stated honestly). A clip narrating the Kennedy assassination (answer: Dallas) is weaker, and we include it to mark the boundary (Figure 2). Here the model’s ASR partly succeeds (it transcribes “the assassination of President John F.”), so assassination and president are largely transcribed, not inferred, and we do not count them. What goes beyond the transcript is the late naming of the victim Kennedy (absent from prompt and answer); the answer Dallas is read from the printed option. This partly- transcription-driven chain, in contrast to Nixon’s fully-inferred one, is how we scope the “read the mind” claim honestly. 4 020406080100 layer depth % (input → output) death / 身亡 assassin assassination perpetrated WORKSPACEsensorymotor (a) At the audio: the model infers the EVENT 020406080100 layer depth % (input → output) Dallas (answer) option "A" Texas WORKSPACEsensorymotor (b) At output: the ANSWER commits JFK-assassination clip, Q="which U.S. city?" (answer: Dallas) — "assassination" is in neither prompt nor output Marker = concept in top-10 logit-lens readout (larger = better rank). (a) key audio position: death → assassination emerge in the workspace band. (b) generation position: Dallas commits in the late/motor layers. Figure 2 Reading an audio LLM’s workspace before it speaks (Kennedy clip; Q: “which U.S. city?”; answer: Dallas; larger marker = better rank). (a) At an audio position the event concept assassination stabilizes at rank 1 across the workspace band alongside assassin and perpetrated. It appears in neither the printed question nor the answer, but the model’s own ASR partly transcribes it, so we treat this clip as illustrative rather than pure inference; the beyond-transcript readout here is the late victim name Kennedy (Section 4.1). (b) At the generation position the answer Dallas commits only in the motor layers: concept in the middle band, answer at the output. Answer Top readout Layer Depth Rank # ws. hits Dallas assassinationL3062%156 Kidney kidneyL1735%144 LionlionsL1940%137 Birdni ˇ aolèi (birds) L3369%112 Music músicaL1838%111 TrainrailL1940%127 Whipbi ̄ an (whip)L3267%17 Table 1 Seven curated thinking-path cases. For each we report the strongest (lowest-rank) workspace-band readout of a task-relevant concept over the audio span, the layer (of 48) where it is most stably rank 1, its depth, its rank among the 152,064-token vocabulary, and the number of workspace cells with the concept in the top-8. For Dallas we list assassination as the workspace-band scene concept, but it is partly transcribed by the model’s own ASR (Section 4.1). Kidney is speech-content recall; the rest are sound-source identifications. The pattern recurs across regimes. The flagship is one clip, but the readout structure is not unique to it: Table 1 collects seven clips, curated to illustrate the range, for which the answer concept is read at rank 1 in the workspace band over the audio positions before any output token. (The population-level claim is the waveform-swap and band statistics of Section 4.4, not these hand-picked cases.) They span three regimes. In multi-hop inference an intermediate concept absent from the prompt is surfaced over the audio and only then resolved: besides Dallas, an archaeology clip surfaces excavation before Howard Carter and a treaty clip surfaces negotiation before Egypt and Israel. In speech-content recovery the readout recovers a spoken detail: on a Kidney clip the specific organ, named only in the audio, becomes rank 1 in the band, while the question-echoed transplant does not count. In sound-source identification (Lion, Bird, Music, Train, Whip) the readout names the source directly from the acoustics. The recurrence of non-English scripts at rank 1—niˇaolèi (birds), música (music)—is the multilingual signature of Section 4.2. It is the answer we are reading, not frequent words. A lens can surface generically frequent tokens, so we run the same readout looking for ten unrelated placebo words (banana, guitar, planet, umbrella, tuesday, . . . ). Across the ten clips the true answer concept fills 1–34% of workspace cells, while the placebos fill 0.12%—essentially never. The single visible exception is trumpet on the whip clip (1.3%), where a whip-crack genuinely resembles a brass transient. The signal is sparse across time. Unlike text, where readable content spreads over many tokens, only 4–30% of audio positions carry the concept in the workspace band (e.g. 5 of 34 positions on the Kennedy clip, 10 of 37 on the archaeology clip). A randomly chosen audio position often reads as noise. This is why we aggregate 5 Archaeology → Howard Carter burial 棺 coffin遗址 ruinsexcavationtreasures Egypt Watergate → Nixon scandal -gatecontroversy Telephone patent → 1876 telecommunicationswired Skateboard → Skateboard 滑雪 skiing box size ∝ in-clip count · each word corpus-rare (in ≤8 of 948 grids) and spoken nowhere in the clip (a) The neighbourhood that lights up — words spoken nowhere in the clip, in no question, no option 0%50%100% layer depth % (input → output) workspace 35–80%late 88–98% Skateboard → skiing skate 滑雪 skiing Voyage → continents discovery continents Train → subway rail 地铁 subway Lion → wolf, tiger lion wolf† 嗥 howl 老虎 tiger (b) The timetable — the associate never comes first evoking concept associate The workspace holds a concept's whole neighbourhood, summoned from world knowledge and then set aside. († wolf is a perceptual competitor — early because it rides its own acoustic evidence.) Figure 3 Watching the model free-associate. (top) Spontaneous neighbour words that light up at the audio positions—spoken nowhere in the clip, in no question or option; corpus-rare. (bottom) Timetable: for each family, the first readable layer of the evoking concept (blue) vs. each associate (orange); the associate always follows its evoker. over positions by taking the best rank rather than averaging, which would wash the signal out. The workspace also holds the neighbourhood. Beyond the task-relevant concept, semantic neighbours of what the model hears light up spontaneously—spoken nowhere in the clip, in neither question nor option—and are then set aside. On a skateboard clip (a rolling sound; answer Skateboard), a single audio position read down the layers shows the schedule: the raw sound is verbalized (gˇundòng, “rolling”), the category forms (skate/skateboard), and only then the neighbour ignites (huáxuě, skiing, a sibling board-sport)—held for several layers but never rank 1, then dropped (Figure 3). Across the families we could verify, the associate always follows its evoker, and the farthest, evidence-free one (lˇaohˇu, tiger) lands latest,∼92% depth. The richest cases are whole neighbourhoods: the archaeology clip reconstructs a dig scene (burial, coffin, ruins, excavation, treasures). A neighbour counts as genuine only if absent from the clip’s transcript on inspection and corpus-rare (≤8 of 948 grids); literally spoken words are excluded. This is largely not an unembedding-geometry artifact: because the lens projects ontoW U , a strengthening evoker would lift its embedding neighbours for free, but five of six associates fall outside their evoker’s top-100 cosine neighbours, and the one near pair (skiing, the 7th neighbour of skate) still ignites strictly after its evoker, stays below rank 1, and is dropped—a schedule static similarity does not encode. This is the audio analogue of spreading activation [4], read layer by layer with depth playing the role of time. 4.2 Finding 2: The Thinking Space Is Multilingual Is the model re-transcribing the sound, or holding a concept? Though clip and prompt are entirely in English, one concept appears in several languages at once. On a music clip the idea of music appears in the workspace band as English music, Chinese y ̄ınyuè, Spanish música, German Musik and Italian musica at once (Figure 4); president surfaces as English, Chinese zˇongtˇong and Korean; water, on the Nixon clip, as English, Spanish agua and French ’eau. Tallying the top-1 readout across all audio-region workspace cells (863,770 cells over 948 clips, restricted to real-word readouts), the inner vocabulary is 52.6% English, 38.5% Chinese and≤4.3% others—so more than a third of leading readouts are Chinese, on English inputs. This is not the lens’s known bias toward frequent CJK tokens. A frequency control shows the Chinese form is clip-specific: y ̄ınyuè is rank 1 on 30% of music clips but 6% of the 610 non-music clips (a 5×lift), and Chinese bird forms show 9–20× lift on birdsong, where a globally frequent token would lift near 1. 6 0 35 80 100 layer depth % workspace audio span (time →) questionoptions (“Lion” here) With audio — “lion” forms across the workspace over the audio span → answers “Lion” (correct) 0 35 80 100 layer depth % workspace written description (time →) questionoptions (“Lion” here) Pure text — the model's own caption of the same clip → answers “Wolf” (wrong) Q: “identify the source of the roars.” options: Lion ✓ / Dog / Wolf / Bear · pinned word “lion” (darker = higher rank) Figure 4 The workspace is multilingual. The same concept surfaces across languages at the audio positions (most at rank 1). The model holds a language-agnostic concept rendered into whichever token sits nearest, not a memorized English string. 0 35 80 100 layer depth % workspace audio span (time →) questionoptions (“Lion” here) With audio — “lion” forms across the workspace over the audio span → answers “Lion” (correct) 0 35 80 100 layer depth % workspace written description (time →) questionoptions (“Lion” here) Pure text — the model's own caption of the same clip → answers “Wolf” (wrong) Q: “identify the source of the roars.” options: Lion ✓ / Dog / Wolf / Bear · pinned word “lion” (darker = higher rank) Figure 5 Two minds, one clip. The roaring clip read two ways, with the concept lion pinned (darker = better rank). Top (audio): lion lights up across the workspace over the audio span; the model answers Lion. Bottom (the model’s own emotion-free caption): lion appears almost nowhere but the printed option, and the model answers Wolf. 4.3 Finding 3: When Speech Carries Extra Information, the Audio Mind Registers It The sound also carries how something was said—an emotion, a speaker’s role, an acoustic source—which no transcript preserves. Reading the same clip twice, as audio and as the model’s own emotion-free caption of it, shows where that content lives. On clips where the two minds disagree, the audio mind forms the true, speech-specific concept and answers correctly while the caption mind does not: a roar reads lion (the caption run answers wolf ; Figure 5); a clip reads priest where the caption hears only “father”; a sarcastic voice reads the inferred affect frustration, named nowhere in the prompt and nowhere in the caption. Quantifying the disagreement clips, the audio mind forms the correct speaker role on 88.9% of them versus 70.4% for the caption mind (n=27), and 33 of the 39 affects it reads are absent from the caption mind’s readout. The behavioral shadow of this needs no lens—over the 1000-clip MMAU-mini, answering from the audio beats answering from the caption on every track, by 7.6 points overall and 11.1 on the sound track—but the point is not the score: it is that the concept the answer depends on is formed in one mind and not in the other. This also distinguishes the effect from prior attribute-tracking [30]: the affect is a finer, inferred state named nowhere in the transcript, contrasted against a caption mind that does not form it. The opposite regime, for calibration. Hearing does not always help, and we state the boundary rather than hide it. On spoken TriviaQA the answer is a stored fact and the spoken clip carries words identical to the written question, so the sound adds nothing the text does not already supply, and the model does at least as well reading as listening. Reading both minds on the items it gets right from text and wrong from speech (n=87), the answer concept appears in the text mind’s workspace on 83% of them versus 62% for the speech mind—a partial retrieval loss, mixed with items where the concept does reach rank 1 in the speech workspace and the spoken output still garbles it. We therefore treat this task as the boundary of the effect, not as a clean mechanism. 7 BandReal Sil. Mism. R−S R−M Sensory41.0 38.0 31.8+3.0 +9.2 Workspace 40.0 21.8 32.2 +18.2 +7.8 Motor48.9 34.7 26.3 +14.2 +22.6 Table 2 Balanced four-way readout accuracy (%) by functional band. R−S (real−silence) tests separation from the text prior—null in the sensory band (p=0.78), largest and highly significant in the workspace band (p=4.7×10 −5 ) where silence alone collapses to chance, intermediate in motor (p=1.5×10 −3 ). R−M (real−mismatch, position-matched) is significant in the workspace (p=0.015) but grows toward motor (p<10 −3 ), so raw magnitude keeps sharpening toward the output while separability from the prior peaks mid-stack. p: paired two-sided McNemar. 4.4 Finding 4: The Thought Lives in the Middle, and Forms in Depth Order The findings above read the workspace band by assumption. We now show that the audio-driven signal is most cleanly decodable in the middle band, turns on early, and is causally used—committed before the output—not merely present. Is the readout driven by the sound? Under the audio-swap control (question and options byte-identical; only the waveform real / mismatched / silence), the correct answer is read far more strongly under real audio (40.0% balanced accuracy) than under silence (21.8%, at chance, where a majority guess scores 25%; paired McNemarp=4.7×10 −5 ). Because silence has fewer audio positions and our best-rank metric takes a minimum over positions, we also rest the claim on the position-matched real-vs-mismatch contrast, where the two conditions have nearly equal audio-position counts (47.9 vs. 43.6 mean positions): real (40.0%) still beats mismatch (32.2%) in the workspace band (p=0.015). Threshold-free, the median rank of the correct answer among the 152,064 vocabulary tokens degrades with signal quality in the same order: #2,297, #5,040, #26,241. Where: the audio signal separates from the text prior in the middle band. Sweeping the readout band by band (Table 2), the real−silence gap—the audio content the text prior alone cannot supply—is negligible in the early sensory band (real≈silence, both explained by the text prior; gap +3.0,p=0.78), largest in the workspace band (silence collapses to chance there, 38.0→21.8, while real holds; gap +18.2,p=4.7×10 −5 ), and intermediate in the motor band (+14.2,p=1.5×10 −3 ). We are precise about what this shows and report both contrasts: the workspace is where the audio signal is most cleanly separable from the text prior—the silence baseline falls to chance there and nowhere else—not where its raw magnitude is greatest, since under the position-matched real−mismatch contrast magnitude keeps growing toward the output. When: the signal turns on early and does not fade. Reading the real−silence gap layer by layer (Figure 6), the two conditions are statistically indistinguishable through the first several layers—the encoder has deposited acoustic features, but nothing the sound adds is yet answer-relevant—and then, a little more than a tenth of the way up, the gap opens and stays open: 37 of the 49 readout depths (the embedding output plus the 48 layers) reachp <0.05 (paired McNemar, one-sided, as the audio-driven direction is predicted a priori), 35 surviving Benjamini–Hochberg control, concentrated in the workspace band (18 of 22, versus 9 of the 17 sensory layers). This transition is not the model beginning to process the audio—the encoder writes acoustic features into the residual stream from the first layer—but the depth at which the sound’s content becomes verbalizable and decision-relevant. Depth order on a single clip. The staging is visible on individual clips, not only on average. On the Kennedy clip (Figure 2) the audio-inferred concepts appear in order: the event (death, assassination) is readable from ∼8% depth and holds through the workspace, while the victim’s name (Kennedy) surfaces only in the last layers and the answer commits at the output. The Nixon clip repeats the pattern (Figure 1): scandal and the reconstructed water+gate early, the role president in the workspace, the answer by the middle layers. 8 020406080100 layer depth % (input → output) 0 10 20 30 real − silence gap (pts) gap significant from ∼12% depth workspace band significant (paired McNemar) not significant Figure 6 The audio-driven gap over the text prior, layer by layer. Balanced four-way readout accuracy with real audio minus the same quantity with silence, with a 95% bootstrap band; filled markers are depths where the paired test is significant. The gap is null at the input, becomes significant at∼12% depth, and stays open, densest inside the workspace band. Causally used, and committed before the late layers. Decodability need not mean use. On clips the model gets right with audio and wrong once the audio is zeroed, patching the real-audio activations back at one band restores the correct answer: 10/10 clips flip back with a sensory-band patch (95.7% of the clean−corrupt logit gap recovered) and 9/10 with a workspace-band patch (89.1%), while patching only the motor band restores nothing (0/10, 4.8%). So the audio content is causally used and committed before the motor band. Because an early patch propagates downstream, this localizes coarsely: it shows the content is used and committed before the last fifth of the network, not that the workspace band alone is responsible—a workspace-band patch at the text positions, which have already attended to the audio, also restores the answer (9/10). Not one checkpoint’s quirk. Repeating the band-localized audio-swap on an architecturally different model— Qwen2.5-Omni-7B [28], a dense 28-layer Thinker versus our sparse mixture-of-experts—reproduces the signature: the real−silence gap is small in the sensory band and large and highly significant in the workspace band (p < 10 −3 ), the same sensory-null-to-workspace-live ordering as Table 2. 4.5 Finding 5: The Answer Is Distributed, and Hallucinations Are Born Early Reading is correlational; deleting is causal. Bypassing one layer at a time (an identity skip) and measuring accuracy over 40 clips gives a clean functional map (Figure 7). Only L0—the encoder-to-text entry, where the audio is read in—is clearly critical: deleting it costs 65 points of accuracy. Every interior layer costs at most 10 points, and the band means excluding L0 are all within one point of zero (sensory−0.2, workspace +0.7, motor +1.0). The answer is therefore not carried by any one interior layer; the network routes around whichever one is removed. Breaking accuracy is one thing, and which ability breaks is another. On the Kennedy clip we track three separable functions per deleted layer: perception (does the sound-inferred event assassination still form at the audio positions?), retrieval (does the recalled answer concept still surface?), and delivery (does the model still emit a valid answer?). Only deleting L0–L1 stops the event forming from the sound; only deleting L47 breaks the output; and no single layer, deleted, stops the answer concept from surfacing. The pipeline is thus: read the sound in (L0–L1), hold and recall the answer across the interior, deliver it out (L47)—and only the two ends are irreplaceable. Where a hallucination is born. The same readout localizes errors. Take the clips the model answers correctly from the real audio but wrongly from its own caption: the caption dropped the decisive acoustic detail, so the text-only mind falls back on a plausible neighbour. Across 73 such clips we trace the depth at which the wrong concept first takes over the text mind (Figure 8). It is usually an early commitment—median onset 9 010203040 deleted layer (0 = encoder →text entry, 47 = output) 0 10 20 30 40 50 60 70 accuracy drop (pts, 40 clips) L0: −65 pts workspace band every interior layer ≤10 pts (a) Deleting one layer: how much accuracy is lost 010203040 deleted layer Perception (event @ audio) Retrieval (answer concept) Delivery (valid answer) no single layer breaks retrieval (b) Which ability breaks (red = lost) Figure 7 Deleting one layer at a time. (a) Accuracy drop from bypassing each layer, over 40 clips. Only the entry layer L0 is critical (−65 points); every interior layer costs at most 10. (b) Which ability is lost. Perception fails only for L0–L1, delivery only for L47, and retrieval for no single layer—listening and delivery are localized to the two ends, retrieval is distributed across the interior. sensory ( <35%) workspace (35--80%) motor ( >80%) depth where the wrong concept takes over 0 10 20 30 40 50 clips (of 73) 46 10 17 median onset 12% depth Figure 8 Where a text-mode hallucination is born. Depth at which the wrong concept first takes over the caption-fed run, on 73 clips the model answers correctly from the audio. Most wrong answers are early commitments, formed inside the sensory band and carried forward unchanged. 12% depth, 58.9% formed by 20% depth, 46 of 73 inside the sensory band—and 84.9% of the wrong answers form before the output layer rather than slipping at the end. A text-mode hallucination is thus typically not a late slip: the model locks onto a plausible-but-wrong concept at roughly the same depth where correct answers first emerge (Section 4.4) and never revises it. Reading the two minds side by side both detects the hallucination—the audio mind holds the right concept where the text mind holds the wrong one—and shows where it entered. 5 Conclusion We asked whether the verbalizable middle-layer global workspace found in text, code, and image models also exists in an audio LLM. Reading the Thinker of a base Qwen3-Omni model with a logit lens at the audio positions, we find evidence that it does: the answer-relevant concept becomes readable before any token is emitted, it is driven by the sound (waveform-swap control) and causally used before the output (activation patching), and the readable content is conceptual rather than a re-transcription—multilingual, paralinguistic, and free-associating. Deleting layers one at a time adds the functional map: listening and delivery are localized to the two ends of the stack, while retrieval is distributed across the interior. This is precisely the regime a chain-of-thought monitor cannot see, since stated reasoning need not be produced and can be unfaithful. 10 The account is deliberately scoped. The logit lens is a cheap proxy—relative to a faithful lens such as the Jacobian lens of Gurnee et al.[12]it should, if anything, under-detect, and early-layer readouts are noisy by construction, which is why our claims concern the middle band; the patching test likewise localizes coarsely, showing the audio content is committed before the motor band rather than pinning it to the workspace alone. We use “workspace” and “access” in the functional, information-posted-for-report sense of Baars[1]and Dehaene and Naccache[5], and make no claim about subjective experience. The natural next steps are a corpus-scale replication under a faithful lens, larger forced-alignment sets, and a test of whether safety-relevant decisions—tool calls, refusals, fabrication—are readable this way before a speech agent acts. Two uses would follow from that test. As a training signal: reinforcement learning is now the standard way to shape generative models [7,8,10,11,29], and a workspace readout offers a process-level target where a scalar reward sees only the final answer. As a monitor: deployed multimodal systems are latency- and memory-bound, which is why so much work goes into adaptive inference and pruning [16–18]; a logit-lens probe is by comparison almost free, one matrix product per position read. References [1] Bernard J. Baars. A Cognitive Theory of Consciousness. Cambridge University Press, New York, 1988. [2]Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. CoRR, abs/2503.11926, 2025. doi: 10.48550/ARXIV.2503.11926. URLhttps://doi.org/10.48550/ arXiv.2503.11926. [3]Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. CoRR, abs/2303.08112, 2023. doi: 10.48550/ARXIV.2303.08112. URL https://doi.org/10.48550/arXiv.2303.08112. [4] Allan Collins and Elizabeth Loftus. A spreading activation theory of semantic processing. Psychological Review, 82:407–428, 11 1975. doi: 10.1037//0033-295X.82.6.407. [5] Stanislas Dehaene and Lionel Naccache. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition, 79 1-2:1–37, 2001. URLhttps://api.semanticscholar.org/CorpusID: 1762431. [6] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. https://transformer-circuits.pub/2021/framework/index.html. [7]Jiajun Fan and Changnan Xiao. Generalized data distribution iteration. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 6103–6184. PMLR, 2022. URL https://proceedings.mlr.press/v162/fan22c.html. [8]Jiajun Fan, Yuzheng Zhuang, Yuecheng Liu, Jianye Hao, Bin Wang, Jiangcheng Zhu, Hao Wang, and Shu-Tao Xia. Learnable behavior control: Breaking atari human world records via sample-efficient behavior selection. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/forum?id=FeWvD0L_a4. [9] Jiajun Fan, Roger Ren, Jingyuan Li, Rahul Pandey, Prashanth Gurunath Shivakumar, Ivan Bulyko, Ankur Gandhe, Ge Liu, and Yile Gu. Incentivizing consistent, effective and scalable reasoning capability in audio llms via reasoning process rewards. CoRR, abs/2510.20867, 2025. doi: 10.48550/ARXIV.2510.20867. URL https://doi.org/10.48550/arXiv.2510.20867. [10]Jiajun Fan, Shuaike Shen, Chaoran Cheng, Yuxin Chen, Chumeng Liang, and Ge Liu. Online reward-weighted fine-tuning of flow matching with wasserstein regularization. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=2IoFFexvuw. 11 [11]Jiajun Fan, Tong Wei, Chaoran Cheng, Yuxin Chen, and Ge Liu. Adaptive divergence regularized policy optimization for fine-tuning generative models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=aXO0xg0ttW. [12]Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul C. Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, and Jack Lindsey. Verbalizable representations form a global workspace in language models. CoRR, abs/2607.15495, 2026. doi: 10.48550/ARXIV.2607.15495. URLhttps://doi.org/10.48550/ arXiv.2607.15495. [13]Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.https://doi.org/10.21227/de50-f985, April 2025. URLhttps: //doi.org/10.21227/de50-f985. Accessed on Y-M-D. [14]Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca D. Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo, Victoria Krakovna, Shane Legg, David Lindner, David Luan, Aleksander Madry, Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong, Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojciech Zaremba, Bowen Baker, Rohin Shah, and Vladimir Mikulik. Chain of thought monitorability: A new and fragile opportunity for AI safety. CoRR, abs/2507.11473, 2025. doi: 10.48550/ARXIV.2507.11473. URL https://doi.org/10.48550/arXiv.2507.11473. [15]Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamile Lukosiute, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. Measuring faithfulness in chain-of-thought reasoning. CoRR, abs/2307.13702, 2023. doi: 10.48550/ARXIV.2307.13702. URLhttps://doi.org/10.48550/ arXiv.2307.13702. [16]Ye Li, Huanan Liu, Kangye Ji, Yuan Meng, Jiajun Fan, Yuansong Wang, Shiyu Qin, Chenglei Wu, Shu-Tao Xia, and Zhi Wang. Elegantvla: Learning when to think for efficient vision-language-action models. CoRR, abs/2605.29438, 2026. doi: 10.48550/ARXIV.2605.29438. URLhttps://doi.org/10.48550/arXiv.2605.29438. [17]Ye Li, Yuan Meng, Zewen Sun, Kangye Ji, Chen Tang, Jiajun Fan, Xinzhu Ma, Shu-Tao Xia, Zhi Wang, and Wenwu Zhu. SP-VLA: A joint model scheduling and token pruning approach for VLA model acceleration. In The Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum? id=RwdGIIjPlC. [18]Ye Li, Chen Tang, Yuan Meng, Jiajun Fan, Zenghao Chai, Xinzhu Ma, Zhi Wang, and Wenwu Zhu. PRANCE: joint token-optimization and structural channel-pruning for adaptive vit inference. IEEE Trans. Pattern Anal. Mach. Intell., 48(1):283–298, 2026. doi: 10.1109/TPAMI.2025.3605239. URLhttps://doi.org/10.1109/TPAMI. 2025.3605239. [19] Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the biology of a large language model. Transformer Circuits Thread, 2025. URLhttps: //transformer-circuits.pub/2025/attribution-graphs/biology.html. [20] nostalgebraist. interpreting GPT: the logit lens. LessWrong, 2020. URLhttps://w.lesswrong.com/posts/ AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens. Blog post. [21] S. Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. MMAU: A massive multi-task audio understanding and reasoning benchmark. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https://openreview.net/forum?id=TeVAZXr3yv. [22] Jiashuo Sun, Jimeng Shi, Yixuan Xie, Saizhuo Wang, Jash Rajesh Parekh, Pengcheng Jiang, Zhiyi Shi, Jiajun Fan, Qinglong Zheng, Peiran Li, Shaowen Wang, Ge Liu, and Jiawei Han. Retrieval is cheap, show me the 12 code: Executable multi-hop reasoning for retrieval-augmented generation. CoRR, abs/2605.12975, 2026. doi: 10.48550/ARXIV.2605.12975. URL https://doi.org/10.48550/arXiv.2605.12975. [23]Qwen Team. Qwen3-omni technical report. CoRR, abs/2509.17765, 2025. doi: 10.48550/ARXIV.2509.17765. URL https://doi.org/10.48550/arXiv.2509.17765. [24] Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam S. Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. CoRR, abs/2605.29358, 2026. doi: 10.48550/ARXIV.2605.29358. URL https://doi.org/10.48550/arXiv.2605.29358. [25] Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URLhttp://papers.nips.c/paper_files/paper/2023/ hash/ed3fea9033a80fea1376299fa7863f4a-Abstract-Conference.html. [26]Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do llamas work in english? on the latent language of multilingual transformers. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 15366–15394. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.ACL-LONG.820. URL https://doi.org/10.18653/v1/2024.acl-long.820. [27]Zhaofeng Wu, Xinyan Velocity Yu, Dani Yogatama, Jiasen Lu, and Yoon Kim. The semantic hub hypothesis: Language models share semantic representations across languages and modalities. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https://openreview.net/forum?id=FrFQpAgnGE. [28]Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report. CoRR, abs/2503.20215, 2025. doi: 10.48550/ARXIV.2503.20215. URLhttps://doi.org/10.48550/arXiv.2503.20215. [29]Bangji Yang, Hongbo Ma, Jiajun Fan, and Ge Liu. Batched contextual reinforcement. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=8Oc3Mx754M. [30]Chih-Kai Yang, Neo Ho, Yi-Jyun Lee, and Hung-Yi Lee. Audiolens: A closer look at auditory attribute perception of large audio-language models. In IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2025, Honolulu, HI, USA, December 6-10, 2025, pages 1–8. IEEE, 2025. doi: 10.1109/ASRU65441.2025.11434723. URL https://doi.org/10.1109/ASRU65441.2025.11434723. [31]Qinglong Zheng, Jiajun Fan, Chaoran Cheng, and Ge Liu. Procedure-aware reinforcement learning for tool- augmented large language models. In Third Conference on Language Modeling, 2026. URLhttps://openreview. net/forum?id=d4wrBuJ4xo. 13

Instant research discovery