Paper deep dive
How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoning
Hongxuan Wu, Yukun Zhang, Xueqing Zhou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/21/2026, 3:15:57 AM
Summary
This paper introduces 'PID Flow', a framework for layer-wise Partial Information Decomposition (PID) to analyze how multimodal Transformers process visual and linguistic information. Applied to LLaVA-1.5 and LLaVA-1.6, the study reveals a 'modal transduction' pattern where visual-unique information peaks early and decays, while language-unique information dominates late layers (approx. 82%). The authors use causal interventions (attention knockouts) to demonstrate that disrupting visual-to-language pathways increases trapped visual information and synergy, confirming that current MLLMs largely translate visual evidence into language representations rather than performing deep cross-modal fusion.
Entities (9)
Relation Signals (8)
PID Flow → uses → Partial Information Decomposition
confidence 98% · We address this question with a layer-wise framework based on Partial Information Decomposition (PID)... we introduce PID Flow
LLaVA-1.6-7B → exhibits → modal transduction
confidence 96% · Applying this framework to... LLaVA-1.6-7B... we uncover a consistent modal transduction pattern
LLaVA-1.5-7B → exhibits → modal transduction
confidence 96% · Applying this framework to LLaVA-1.5-7B... we uncover a consistent modal transduction pattern
PID Flow → appliedto → LLaVA-1.5-7B
confidence 95% · Applying this framework to LLaVA-1.5-7B and LLaVA-1.6-7B
PID Flow → appliedto → LLaVA-1.6-7B
confidence 95% · Applying this framework to LLaVA-1.5-7B and LLaVA-1.6-7B
Image→Question attention → disruptedby → attention knockouts
confidence 94% · we perform targeted Image→Question attention knockouts
modal transduction → characterizedby → Synergistic Information
confidence 85% · cross-modal synergy remains below 2%
modal transduction → characterizedby → Redundant Information
confidence 85% · semantic redundancy governing the detailed information fingerprint
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:When a multimodal Transformer answers a visual question, is the prediction driven by visual evidence, linguistic reasoning, or genuinely fused cross-modal computation -- and how does this structure evolve across layers? We address this question with a layer-wise framework based on Partial Information Decomposition (PID) that decomposes the predictive information at each Transformer layer into redundant, vision-unique, language-unique, and synergistic components. To make PID tractable for high-dimensional neural representations, we introduce \emph{PID Flow}, a pipeline combining dimensionality reduction, normalizing-flow Gaussianization, and closed-form Gaussian PID estimation. Applying this framework to LLaVA-1.5-7B and LLaVA-1.6-7B across six GQA reasoning tasks, we uncover a consistent \emph{modal transduction} pattern: visual-unique information peaks early and decays with depth, language-unique information surges in late layers to account for roughly 82\% of the final prediction, and cross-modal synergy remains below 2\%. This trajectory is highly stable across model variants (layer-wise correlations $>$0.96) yet strongly task-dependent, with semantic redundancy governing the detailed information fingerprint. To establish causality, we perform targeted Image$\rightarrow$Question attention knockouts and show that disrupting the primary transduction pathway induces predictable increases in trapped visual-unique information, compensatory synergy, and total information cost -- effects that are strongest in vision-dependent tasks and weakest in high-redundancy tasks. Together, these results provide an information-theoretic, causal account of how vision becomes language in multimodal Transformers, and offer quantitative guidance for identifying architectural bottlenecks where modality-specific information is lost.
Tags
Links
- Source: https://arxiv.org/abs/2602.15580v1
- Canonical: https://arxiv.org/abs/2602.15580v1
Trouble viewing inline? Open PDF directly →
Full Text
81,498 characters extracted from source content.
Expand or collapse full text
How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoning Hongxuan Wu Yukun Zhang Xueqing Zhou Abstract When a multimodal Transformer answers a visual question, is the prediction driven by visual evidence, linguistic reasoning, or genuinely fused cross-modal computation—and how does this structure evolve across layers? We address this question with a layer-wise framework based on Partial Information Decomposition (PID) that decomposes the predictive information at each Transformer layer into redundant, vision-unique, language-unique, and synergistic components. To make PID tractable for high-dimensional neural representations, we introduce PID Flow, a pipeline combining dimensionality reduction, normalizing-flow Gaussianization, and closed-form Gaussian PID estimation. Applying this framework to LLaVA-1.5-7B and LLaVA-1.6-7B across six GQA reasoning tasks, we uncover a consistent modal transduction pattern: visual-unique information peaks early and decays with depth, language-unique information surges in late layers to account for roughly 82% of the final prediction, and cross-modal synergy remains below 2%. This trajectory is highly stable across model variants (layer-wise correlations >>0.96) yet strongly task-dependent, with semantic redundancy governing the detailed information fingerprint. To establish causality, we perform targeted Image→ attention knockouts and show that disrupting the primary transduction pathway induces predictable increases in trapped visual-unique information, compensatory synergy, and total information cost—effects that are strongest in vision-dependent tasks and weakest in high-redundancy tasks. Together, these results provide an information-theoretic, causal account of how vision becomes language in multimodal Transformers, and offer quantitative guidance for identifying architectural bottlenecks where modality-specific information is lost. 1 Introduction When a multimodal large language model (MLLM) answers a visual question, what drives the prediction—visual evidence, linguistic reasoning, or a genuinely fused cross-modal computation? The answer has direct design consequences: if visual evidence is absorbed into language representations early in the network, then improving the visual encoder alone will yield diminishing returns, and the integration mechanism itself becomes the bottleneck. Despite rapid progress in MLLM capabilities, this question remains open. Existing interpretability tools offer partial views. Attention-based analyses reveal where information is routed but not what type of predictive content is carried. Probing classifiers diagnose what is linearly decodable at each layer but depend on auxiliary task design and do not decompose how multiple sources combine. Gradient-based attributions measure input sensitivity rather than the internal information structure of representations. What is missing is a representation-level account that, at each layer, quantifies how much predictive information is shared by both modalities, unique to one, or available only through their combination. We provide such an account using partial information decomposition (PID). Given visual and linguistic representations XVX_V and XLX_L at a given layer and a target variable Y, PID decomposes the joint predictive information into four non-negative terms: I(XV,XL;Y)=R⏟redundant+UV⏟vision-unique+UL⏟language-unique+S⏟synergistic. I(X_V,X_L;\,Y)\;=\; R U_V_redundant\;+\; U_V_vision-unique\;+\; U_L_language-unique\;+\; S U_V_synergistic\,. (1) Tracking the quadruple (R,UV,UL,S)(R,U_V,U_L,S) across all layers of a Transformer yields an information trajectory—a depth-resolved fingerprint of how multimodal evidence is injected, transformed, and consolidated through the network. Applying this framework to LLaVA-1.5-7B and LLaVA-1.6-7B across six GQA reasoning tasks, we uncover a consistent modal transduction pattern: visual-unique information peaks in early layers and decays monotonically with depth, while language-unique information surges in late layers and dominates the final prediction. Synergy—the information available only when both modalities are considered jointly—remains surprisingly small throughout. At the decision layer, language-unique information accounts for roughly 82% of total predictive information, vision-unique contributes approximately 6%, and synergy stays below 2%. This suggests that current MLLMs do not perform “emergent cross-modal fusion” so much as translate visual evidence into language-space representations for downstream reasoning. Contributions. This paper makes four contributions. 1. A mechanism taxonomy grounded in information trajectories. We define three mechanistic regimes—persistent synergy, modal transduction, and redundancy-dominant convergence—as formally separable trajectory signatures (Definitions 4.1–4.3). These provide a reusable vocabulary for characterizing cross-modal computation in any architecture that maintains distinguishable modality streams. 2. PID Flow: tractable PID estimation for high-dimensional neural representations. Direct PID computation on Transformer activations (typically d≥4096d≥ 4096) is infeasible. We introduce PID Flow, a three-stage pipeline—dimensionality reduction, normalizing-flow Gaussianization, and closed-form Gaussian PID—that exploits the bijective invariance of mutual information to yield stable estimates. The estimator is layer-independent, architecture-agnostic, and requires no task-specific probing design. 3. Empirical identification of modal transduction as the dominant regime in LLaVA. Across six semantically diverse tasks, both LLaVA-1.5 and LLaVA-1.6 exhibit strikingly consistent transduction trajectories (cross-model layer-wise correlations >0.96>0.96), with shared turning points around layers 18–20. Task semantics modulate the magnitude of redundancy and the strength of language dominance but do not alter the overarching regime—modal transduction is architecture-robust and task-general within the LLaVA family. 4. Causal validation via attention knockout. We block the Image→ attention pathway and show that the resulting PID shifts match three a priori predictions of the transduction hypothesis: vision-unique information increases (visual evidence becomes “trapped”), synergy rises (the model resorts to less efficient joint processing), and the total information budget grows (compensation is costly). Task-dependent effect sizes further reveal that semantic redundancy governs robustness to pathway disruption, providing the first information-theoretic, causal account of attention-knockout effects in multimodal Transformers. Beyond interpretation, PID trajectories offer actionable architectural guidance. If visual information is compressed in early layers, encoder improvements alone cannot change the downstream computation; preserving more UVU_V likely requires objective- or architecture-level interventions such as auxiliary losses that penalise premature transduction or cross-modal attention mechanisms that maintain a separate visual stream. More broadly, PID trajectories can identify bottleneck layers where modality-unique information is lost, guiding targeted modifications rather than wholesale redesign. The remainder of the paper is organized as follows. Section 2 reviews related work. Section 3 introduces partial information decomposition. Section 4 presents the layer-wise PID framework and the PID Flow estimator. Section 5 reports experiments and causal interventions. Section 6 concludes. Figure 1: Core research framework. (A) Multimodal input: image and question tokens are processed by a 32-layer LLaVA Transformer. (B) PID Flow estimation: layer-wise representations are compressed via mean pooling and PCA, Gaussianized by a normalizing flow, and decomposed into four PID components (R,UV,UL,SR,U_V,U_L,S). (C) Information trajectories reveal a three-stage modal transduction pattern: visual injection (Stage I), consolidation (Stage I), and language-dominant decision formation (Stage I), with UL≈82%U_L≈ 82\% and S<2%S<2\% at the final layer. Cross-model trajectory correlations exceed r>0.96r>0.96. 2 Related Work Modern vision–language models (VLMs) combine a vision encoder with a large language model (LLM) backbone through trainable adapters. CLIP (Radford et al., 2021) established strong cross-modal representations via contrastive pre-training, and instruction-tuned systems such as LLaVA (Liu et al., 2023) further improved performance on downstream vision–language tasks. Despite these advances, the mechanisms of multimodal reasoning remain unclear: when an MLLM answers a visual question, which parts of the computation depend on vision, which depend on language, and where (if anywhere) does genuinely joint cross-modal evidence emerge? We review three lines of work most relevant to our goal—mechanistic interpretability via attention, representation-level probing and information-theoretic diagnostics, and partial information decomposition (PID), together with causal interventions through attention knockout. Attention-based interpretability. A large body of work analyzes self- and cross-attention patterns in Transformers as a proxy for information routing. Tooling such as VLM-Interpret visualizes attention maps and relevance scores in LLaVA-style models, enabling interactive inspection of how image patches and text tokens are attended during reasoning (Ben Melech Stan et al., 2024). Beyond raw attention rollout, GMAR proposes a gradient-weighted multi-head attention rollout that re-weights heads by class-specific gradient importance, yielding sharper token-level saliency in Vision Transformers (Jo et al., 2025). Earlier analyses in language-only Transformers also show that some attention heads correlate with syntactic or semantic relations, while many heads appear redundant or allocate mass to special tokens (Clark et al., 2019). However, attention weights can be weakly correlated with feature importance measured by gradients or erasure, and different attention patterns may lead to similar predictions, raising concerns about using attention alone as a faithful explanation (Jain & Wallace, 2019). In the visual domain, relevance-propagation schemes that combine attention with gradient information (e.g., Deep Taylor-style decompositions) produce class-specific attribution maps that better align with human rationales for images and text (Chefer et al., 2021). In multimodal settings, fine-grained attention analyses together with causal interventions on visual features further highlight the gap between attention heatmaps and true decision mechanisms (Bi et al., 2025). Overall, attention-based methods primarily answer where the model routes information, but they do not directly quantify what type of predictive information is carried (redundant, unique, or synergistic), which motivates our information-structural approach. Probing and information-theoretic diagnostics. Probing methods diagnose representation content by training auxiliary models on intermediate activations. Linear probes provide simple layer-wise diagnostics of linear separability and representation accessibility (Alain & Bengio, 2016). Structural and edge probing demonstrate that contextual encoders implicitly encode rich linguistic structure, recovering parse trees and diverse syntactic phenomena from frozen representations (Hewitt & Manning, 2019; Tenney et al., 2019). Recent work formalizes probing in information-theoretic terms, treating probe accuracy as evidence about how much information a representation contains about a target and emphasizing controls and baselines to avoid confounds (e.g., lexical shortcuts) (Pimentel et al., 2020; Pimentel & Cotterell, 2021). Complementary to supervised probes, sparse autoencoder approaches learn interpretable feature dictionaries directly from activations; in the multimodal setting, SAE-based analyses can disentangle monosemantic features and study feature-level alignment and restructuring in VLMs (Lou et al., 2025). These methods are valuable for what is encoded, but they typically do not decompose how multiple sources combine to form predictions, especially in the presence of redundancy and synergy. Partial information decomposition. PID decomposes mutual information from multiple sources into redundant, unique, and synergistic components, providing a principled language for multi-source predictive structure (Williams & Beer, 2010; Bertschinger et al., 2014; Lyu et al., 2024; Murphy & Bassett, 2024). Exact PID is generally intractable for continuous high-dimensional variables, so recent work has focused on tractable estimators and relaxations, including Gaussian PID in closed form (Venkatesh et al., 2023), flow-based mappings to latent Gaussian spaces (Zhao et al., 2025), and dynamic extensions based on information rates for multivariate processes (Faes et al., 2025), as well as geometric viewpoints clarifying relationships among PID solutions and dependency structure (Kunert-Graf et al., 2020). PID has been used to quantify multimodal interactions (Liang et al., 2023), interpret diffusion-based text-to-image systems (Dewan et al., 2024), and design interaction-aware measures and objectives for multimodal learning and mixture-of-experts settings (Yang et al., 2025; Dissanayake et al., 2025; Xin et al., 2025). In contrast to most prior applications that focus on distribution-level summaries or specialized architectures, our work uses PID layer-wise to characterize the depth-wise evolution of information structure and to identify mechanistic regimes in large pretrained vision–language Transformers. Causal interventions via attention knockout. To move from correlational patterns to causal mechanism testing, we adopt attention knockout as an intervention on information routing. In language-only LLMs, Geva et al. (Geva et al., 2023) ablate specific attention connections to localize where factual knowledge is read out, measuring the impact on predicted answers. We extend this idea to multimodal models by selectively blocking attention from image tokens to question tokens (Image→ ), and analyzing how both prediction behavior and information structure change. Crucially, we pair knockout with PID trajectories: knockout provides a causal perturbation, while PID quantifies how the model reallocates predictive information across redundancy, modality-unique evidence, and synergy under intervention. From prior work to our approach. Taken together, existing lines of research provide complementary but incomplete views of multimodal reasoning. Attention-based analyses reveal where information may flow but do not characterize the type of predictive information being transferred; probing and representation diagnostics identify what is encoded at individual layers but largely ignore how multiple modalities combine; and prior applications of PID quantify multimodal interaction at an aggregate level, without resolving depth-wise dynamics or mechanistic regimes. Moreover, causal interventions such as attention knockout have been used to localize important pathways, but lack a principled account of why disrupting a pathway affects performance. In the next section, we introduce a layer-wise PID framework that bridges these gaps by jointly tracking the evolution of redundant, unique, and synergistic information across depth, and by pairing this analysis with targeted attention knockouts to obtain causal, information-theoretic explanations of multimodal computation. 3 Preliminaries: Partial Information Decomposition Our analysis builds on Partial Information Decomposition (PID), a framework that characterizes how multiple information sources jointly contribute to a prediction. Unlike mutual information, which quantifies only the total dependence between representations and outputs, PID explicitly distinguishes redundant, unique, and synergistic contributions. This distinction is essential for understanding multimodal reasoning, where visual and linguistic representations may overlap in meaning, contribute independently, or interact in a genuinely joint manner. 3.1 PID formulation Beyond mutual information. Mutual information I(X;Y)I(X;Y) measures how informative a variable X is about a target Y, but it is insufficient when multiple sources are involved. Given two sources XVX_V (vision) and XLX_L (language), I(XV,XL;Y)I(X_V,X_L;Y) does not reveal whether predictive information is shared by both modalities, specific to one modality, or only available through their combination. This limitation is particularly salient in vision–language models, where answers may rely on aligned concepts, modality-specific cues, or cross-modal binding. Decomposition. PID decomposes the joint mutual information into four non-negative components: I(XV,XL;Y)=R+UV+UL+S,I(X_V,X_L;Y)=R+U_V+U_L+S, (2) where R denotes redundant information shared by both sources, UVU_V and ULU_L denote vision-unique and language-unique information, respectively, and S denotes synergistic information that is available only when both sources are considered jointly. These components satisfy consistency relations with marginal and conditional mutual information (e.g., I(XV;Y)=R+UVI(X_V;Y)=R+U_V and I(XV;Y∣XL)=UV+SI(X_V;Y X_L)=U_V+S), ensuring that the decomposition is additive and interpretable. 3.2 Interpretation in multimodal models Semantic roles. In multimodal architectures, PID components admit a natural interpretation. Redundancy R captures aligned or overlapping concepts represented in both vision and language (e.g., common object categories). Vision-unique information UVU_V corresponds to visual attributes such as color, position, or fine-grained appearance that are not recoverable from text alone. Language-unique information ULU_L reflects linguistic structure, compositional logic, or task instructions. Synergy S captures genuinely joint evidence, such as binding an attribute to a specific object or resolving relations that require simultaneous access to both modalities. Table 1 summarizes these roles. Table 1: Semantic interpretation of PID components in multimodal models. Component Information-theoretic meaning Multimodal interpretation R shared predictive information aligned vision–language concepts UVU_V information unique to vision visual attributes (color, position, texture) ULU_L information unique to language linguistic or logical structure S information available only jointly cross-modal binding (e.g., object–attribute relations) Expected depth-wise behavior. Applied layer-wise to a multimodal Transformer, PID provides a language for describing how information structure evolves with depth. A priori, one may expect early layers to contain substantial vision-unique information, intermediate layers to increase redundancy as modalities align, and late layers to become dominated by language-unique information if visual evidence has been successfully transduced into the language representation. These expectations motivate our empirical analysis of PID trajectories across layers. 3.3 Why estimation is non-trivial High-dimensional challenge. Directly estimating PID from high-dimensional neural representations is statistically challenging. Transformer activations typically have thousands of dimensions, while available sample sizes are limited, making naive density estimation or discretization infeasible. Moreover, many existing estimators target mutual information rather than the full PID structure. Approach overview. To make layer-wise PID analysis tractable in modern vision–language models, we adopt a strategy that combines dimensionality reduction, invertible transformations, and closed-form Gaussian information measures. The goal is not to recover an exact ground-truth decomposition, but to obtain a stable and interpretable information-structural proxy that can be tracked across depth and compared under causal interventions. Full methodological and implementation details are provided in Section 4 and the Appendix. Summary. PID provides a principled framework for decomposing predictive information into redundant, modality-unique, and synergistic components. In the following sections, we apply PID layer-wise to large vision–language models and use it to identify consistent mechanistic regimes and to interpret the effects of causal interventions such as attention knockout. 4 Method: Layer-wise PID Analysis We propose a layer-wise analysis framework that quantifies how visual and linguistic evidence is organized across the depth of a multimodal large language model (MLLM). The method proceeds in four steps. We first define a layer-wise information state (Section 4.1), then operationalize distinct cross-modal mechanisms via trajectory-level signatures (Section 4.2), introduce PID Flow, a scalable estimator for high-dimensional representations (Section 4.3), and finally discuss estimation reliability and causal diagnostics (Section 4.4). 4.1 Layer-wise Information State Setup. Consider a multimodal Transformer with L blocks. We index layers such that ℓ=0 =0 corresponds to the embedding layer (before any Transformer block), and ℓ=1,…,L =1,…,L correspond to the outputs after each block. For a given input, token positions are partitioned into two index sets: the visual region V (image tokens) and the language region ℒL (question tokens). Let (ℓ)∈ℝN×dH^( ) ^N× d denote the hidden-state matrix after layer ℓ , where N is the sequence length and d is the hidden dimension. Modality-level summaries. We summarize each modality by mean pooling over its token region: V(ℓ) _V^( ) =1||∑i∈i(ℓ),L(ℓ)=1|ℒ|∑j∈ℒj(ℓ), = 1|V| _i h_i^( ), _L^( )= 1|L| _j h_j^( ), (3) where i(ℓ)h_i^( ) denotes the i-th token representation at layer ℓ . Mean pooling is a simple permutation-invariant summary; alternative pooling schemes yield qualitatively similar trajectories (Appendix G.1). Information state. Let Y denote the (discrete) target variable used for evaluation. We define the information state at layer ℓ as the PID quadruple ℐ(ℓ)=(R(ℓ),UV(ℓ),UL(ℓ),S(ℓ)), ^( )\;=\; (R^( ),\,U_V^( ),\,U_L^( ),\,S^( ) ), (4) induced by the joint predictive information I(V(ℓ),L(ℓ);Y)=R(ℓ)+UV(ℓ)+UL(ℓ)+S(ℓ). I\! (x_V^( ),x_L^( );Y )=R^( )+U_V^( )+U_L^( )+S^( ). (5) Trajectory. The layer-wise PID trajectory is the sequence =ℐ(ℓ)ℓ=0L. \;=\; \I^( ) \_ =0^L. (6) This trajectory serves as a quantitative information trace describing how multimodal evidence is injected, transformed, and consolidated throughout the network. Table 2 summarizes the PID components and their interpretation in multimodal large language models. Table 2: PID components at layer ℓ and their interpretation in MLLMs. Component Symbol Interpretation (MLLMs) Redundancy R(ℓ)R^( ) predictive information jointly encoded by both modalities Vision-unique UV(ℓ)U_V^( ) evidence available only from the visual representation Language-unique UL(ℓ)U_L^( ) evidence available only from the language representation Synergy S(ℓ)S^( ) evidence available only from the joint pair (PID sense) 4.2 Mechanism Taxonomy via Trajectory Signatures We characterize cross-modal computation through trajectory-level signatures defined on measurable PID quantities, without assuming linearity or input-independent attention. Total predictive information. We define the total predictive information at layer ℓ as Itot(ℓ)=R(ℓ)+UV(ℓ)+UL(ℓ)+S(ℓ). I_tot^( )\;=\;R^( )+U_V^( )+U_L^( )+S^( ). (7) Persistent synergy. Definition 4.1 (Persistent synergy). An MLLM exhibits persistent synergy if there exist thresholds τS>0 _S>0 and γ∈(0,1)γ∈(0,1) such that S(L)>τSandS(L)Itot(L)>γ. S^(L)> _S S^(L)I_tot^(L)>γ. (8) This condition indicates that the final decision relies on predictive information not recoverable from either modality alone. Modal transduction. Definition 4.2 (Modal transduction). An MLLM exhibits modal transduction if there exists a turning layer ℓ⋆∈1,…,L−1 ∈\1,…,L-1\ and a threshold η∈(0,1)η∈(0,1) such that UV(ℓ) is unimodal with a peak near ℓ⋆,UL(L)Itot(L)>η. U_V^( ) is unimodal with a peak near , U_L^(L)I_tot^(L)>η. (9) Operationally, this corresponds to early injection of vision-unique evidence followed by late-layer dominance of language-unique evidence. Redundancy-dominant convergence. Definition 4.3 (Redundancy-dominant convergence). An MLLM exhibits redundancy-dominant convergence if there exist ℓ0∈0,…,L−1 _0∈\0,…,L-1\ and ρ>1ρ>1 such that R(L)>R(ℓ0)andR(L)>ρ⋅maxUV(L),UL(L),S(L). R^(L)>R^( _0) R^(L)>ρ· \! \U_V^(L),\,U_L^(L),\,S^(L) \. (10) Thresholds are treated as descriptive hyperparameters rather than decision boundaries; all qualitative conclusions are robust across a wide range of values (Appendix G.6). Proposition 4.4 (Separability of mechanisms). Under non-degenerate conditions (Itot(L)>0I_tot^(L)>0 and at least one PID component strictly dominates), the three definitions above describe mutually exclusive regimes in the space of final-layer allocations. A proof sketch is provided in Appendix G.2. 4.3 PID Flow: Scalable Estimation for High-Dimensional States Overview. Direct PID estimation on Transformer representations is infeasible due to high dimensionality. We therefore introduce PID Flow, a three-stage pipeline combining dimension reduction, invertible Gaussianization, and closed-form computation under a Gaussian approximation. 4.3.1 Step 1: Dimensionality reduction We apply PCA to V(ℓ)x_V^( ) and L(ℓ)x_L^( ) to obtain ~V(ℓ),~L(ℓ)∈ℝd′ x_V^( ), x_L^( ) ^d , retaining 95%95\% of the variance in all experiments. PCA is not invertible; we treat this as a controlled approximation and assess sensitivity to d′d in Appendix G.3. 4.3.2 Step 2: Invertible Gaussianization On the reduced space, we learn a bijection fθ:ℝd′→ℝd′f_θ:R^d ^d using a normalizing flow. We define V(ℓ)=fθ(~V(ℓ)),L(ℓ)=fθ(~L(ℓ)). _V^( )=f_θ( x_V^( )), _L^( )=f_θ( x_L^( )). (11) Theorem 4.5 (Bijection invariance of mutual information). Let f be a bijection. Then I(f(X);Y)=I(X;Y)I(f(X);Y)=I(X;Y) and I(f(X1),f(X2);Y)=I(X1,X2;Y)I(f(X_1),f(X_2);Y)=I(X_1,X_2;Y). Remark. The theorem applies to the flow step. Any PID computed from (V(ℓ),L(ℓ))(z_V^( ),z_L^( )) therefore reflects the information structure of the reduced variables, up to the approximation introduced by PCA. 4.3.3 Step 3: Gaussian plug-in PID We model (V(ℓ),L(ℓ))(z_V^( ),z_L^( )) as approximately Gaussian, estimate parameters (^,^)( μ, ) by maximum likelihood, and compute PID components in closed form using the IminI_ redundancy rule (Appendix G.4). 4.4 Estimation Reliability and Diagnostics Consistency. Proposition 4.6 (Consistency (informal)). If the transformed variables are exactly Gaussian and the redundancy rule is well-defined, Gaussian plug-in estimates of mutual information and the induced PID components are consistent as the sample size n→∞n→∞. Attention knockout integration. Let αi→j _i→ j denote the pre-softmax attention logit from token i to token j. To probe pathway-level causality, we block a source→ attention pathway by applying αi→j←−∞,i∈,j∈,αi→j,otherwise, _i→ j← cases-∞,&i ,\,j ,\\ _i→ j,&otherwise, cases (12) and recompute the PID trajectory to analyze shifts Δℐ(ℓ) ^( ). 4.5 Summary We introduced a layer-wise PID analysis framework based on (i) an information-state representation, (i) trajectory-defined mechanism signatures, and (i) PID Flow, a scalable estimator combining PCA, invertible Gaussianization, and Gaussian plug-in PID. The next section applies this framework to LLaVA models and evaluates robustness across architectures and causal interventions. 5 Experiments Goal and design. Our experiments use Partial Information Decomposition (PID) to turn “information structure” from a descriptive artifact into a set of testable mechanistic claims about multimodal reasoning. We organize the evaluation around four questions: (Q1) Mechanism identification—does an MLLM operate closer to persistent synergy, modal transduction, or redundancy-dominant convergence? (Q2) Generality—is the mechanism consistent across tasks and stable across model variants? (Q3) Causal validation—do PID components shift in predictable ways when a hypothesized pathway is disrupted? (Q4) Task dependence—how do tasks differ in their demand for vision-unique information, redundancy, and synergy? Experimental suites. We conduct three suites of experiments. First, we compute full 32-layer PID trajectories on six representative GQA-style reasoning tasks for LLaVA-1.5-7B, and identify the dominant mechanism via trajectory signatures (Section 5.1). Second, we replicate the pipeline on LLaVA-1.6-7B and quantify cross-model alignment in both endpoint composition and depth-wise dynamics (Section 5.2). Third, we perform attention knockout that blocks the Image→ pathway, and analyze knockout-induced trajectory shifts to causally validate the transduction hypothesis and expose task-level heterogeneity (Section 5.3). Operationalization and reliability controls. At each layer ℓ , we aggregate image-token and question-token hidden states into modality-level representations and define the task label as the target variable Y. We estimate PID components ℐ(ℓ)=(R(ℓ),UV(ℓ),UL(ℓ),S(ℓ))I^( )=(R^( ),U_V^( ),U_L^( ),S^( )) and assemble trajectories across depth. We use (i) cross-model trajectory correlation, (i) stability of key turning layers, and (i) knockout-induced relative changes as primary metrics. All details on data filtering, representation extraction, estimator settings, hyperparameters, and statistical tests are deferred to the appendix. 5.1 Results I: Layer-wise Evidence for Modal Transduction (a) Choose Attribute (b) Choose Category (c) Choose Relation (d) Compare Attribute (e) Logical Object (f) Query Attribute Figure 2: Layer-wise PID trajectories for LLaVA-1.5-7B across six tasks. Each panel plots redundancy (RℓR ), vision-unique (UVℓU_V ), language-unique (ULℓU_L ), and synergy (SℓS ) as functions of Transformer depth. All tasks share a consistent pattern: UVU_V peaks early and decays, ULU_L surges in late layers, and S remains bounded throughout—the signature of modal transduction. Figure 2 presents full 32-layer PID trajectories for LLaVA-1.5-7B on six semantically diverse tasks: attribute recognition (ChooseAttr), category selection (ChooseCat), spatial relation reasoning (ChooseRel), attribute comparison (CompareAttr), logical selection (LogicalObj), and localization queries (QueryAttr). Despite substantial differences in task semantics, the trajectories share a strikingly consistent depth-wise structure, pointing to a shared computational mechanism. Three recurrent signatures identify modal transduction. Across all six tasks, the PID trajectories exhibit three depth-wise regularities that collectively satisfy the modal-transduction criteria defined in Section 4.2. (i) Visual-unique information decays with depth. UVU_V decreases from moderate early-layer values (2–8 bits) to low final-layer values (<<3 bits), indicating that visual evidence is progressively consumed and re-encoded rather than preserved as an independent stream. (i) Language-unique information dominates late layers. ULU_L follows a characteristic U-shaped trajectory—a mild early rise, a mid-layer plateau or trough (layers 10–15), and a sharp late-layer surge (layers 20–32) that ultimately controls the decision. (i) Synergy remains bounded. S stays below 3.5 bits and under 15% of total predictive information at every layer, inconsistent with a persistent-synergy regime. These signatures align with the theoretical predictions of Section 4.2: early visual injection, mid-layer consolidation, and late language-space decision formation. Final-layer composition confirms language dominance. Table 3 quantifies the endpoint. Averaged across tasks, ULU_L accounts for 82.4% of total predictive information at Layer 31, while UVU_V contributes only 6.4% (UL/UV≈18:1U_L/U_V≈ 18:1). The dominance is consistent but task-modulated. For the high-redundancy task ChooseCat, ULU_L accounts for 48.1% of the total, with substantial redundancy (R=7.74R=7.74 bits) absorbing the remainder. In contrast, ChooseRel, CompareAttr, and QueryAttr each exceed 93% ULU_L share, with UVU_V approaching zero. Synergy remains small even in intuitively cross-modal tasks (S=0.65S=0.65 bits for ChooseRel), providing quantitative evidence against persistent fusion at decision time. Table 3: Final-layer (Layer 31) PID decomposition for LLaVA-1.5-7B. Task R ULU_L UVU_V S UL/TotalU_L/Total ChooseAttr 1.13 14.18 2.01 1.71 74.5% ChooseCat 7.74 15.40 5.31 3.54 48.1% ChooseRel 0.13 18.83 0.50 0.65 93.6% CompareAttr 0.22 24.54 0.01 0.83 95.9% LogicalObj 0.07 12.62 0.58 0.35 92.7% QueryAttr 0.03 22.70 0.00 0.59 97.3% Avg. 1.55 18.05 1.40 1.28 82.4% A three-stage computation emerges from the trajectories. The depth-wise dynamics reveal a stable three-stage structure shared across tasks (Figure 4; Table 4). Stage I (Layers 0–10): visual injection. UVU_V drops rapidly while ULU_L increases modestly, consistent with early cross-attention injecting visual evidence into question tokens and initiating re-encoding in language space. Stage I (Layers 10–20): consolidation. UVU_V continues to decline more slowly; ULU_L temporarily stabilises or dips; redundancy R typically increases, reflecting growing shared semantics. Stage I (Layers 20–32): language-dominant decision formation. ULU_L rises sharply and UVU_V reaches its minimum, indicating that late-layer reasoning operates primarily in the language representation using already-transduced visual evidence. The turning points that demarcate these stages cluster tightly across tasks (Table 4): the UVU_V peak occurs at layers 0–1, the ULU_L trough near layer 13, and the ULU_L surge onset near layer 19. This clustering suggests that the stage structure is anchored by architectural depth rather than task-specific idiosyncrasies. Table 4: Cross-task consistency of modal-transduction turning points. Task Peak of UVU_V Trough of ULU_L Onset of ULU_L surge ChooseAttr 0 13 20 ChooseCat 1 14 19 ChooseRel 0 11 19 CompareAttr 1 – 17 LogicalObj 0 13 19 QueryAttr 1 13 20 Median 0.5 13 19 Task semantics modulate information fingerprints without changing the mechanism. While modal transduction is universal across the six tasks, task semantics shape the detailed information composition—what we term the task’s information fingerprint. ChooseCat exhibits the highest redundancy (R=7.74R=7.74 bits, 24.2% share), consistent with category concepts being strongly encoded in both modalities. Relational, comparative, and localization tasks (ChooseRel, CompareAttr, QueryAttr) instead show near-zero redundancy (R<0.3R<0.3 bits) and very high ULU_L shares (>>93%), indicating a pattern of language reasoning under visual conditioning: vision supplies entity- and attribute-level evidence early, while the decisive operations unfold in language space. The extreme case, QueryAttr, has UV≈0U_V≈ 0 at the final layer, consistent with multi-step reasoning proceeding almost entirely within language representations after early extraction. An information-theoretic reinterpretation of attention-knockout findings. The PID trajectories also offer a new lens on prior attention-knockout results. A pathway can have a small late-stage knockout effect not because vision is irrelevant, but because decision-relevant visual evidence has already been transduced into question representations at earlier layers. In this view, early-layer Image→ connections are causal bottlenecks for the UV→ULU_V\!→\!U_L transfer, whereas late-stage Question→ connections primarily convey the amplified ULU_L signal that dominates readout. We validate this interpretation with targeted interventions in Section 5.3. Figure 3: Layer-wise PID trajectories for LLaVA-1.5-7B across six tasks. All tasks exhibit decay of UVU_V, a late-stage surge of ULU_L, and bounded synergy S, supporting modal transduction as the dominant mechanism. Figure 4: Turning layers and final-layer composition. Transition points (UVU_V peak, ULU_L trough, ULU_L surge onset) cluster across tasks, and final-layer predictive information is dominated by ULU_L. 5.2 Results I: Cross-Model Consistency Section 5.1 establishes modal transduction in LLaVA-1.5-7B. To test whether these patterns generalise beyond a single implementation, we replicate the full pipeline on LLaVA-1.6-7B. The comparison is non-trivial: LLaVA-1.6 introduces dynamic high-resolution visual processing with variable image-token counts (up to 4×4× those of LLaVA-1.5) and a different Vicuna-family language backbone and training mix, while sharing the same CLIP ViT-L vision encoder and 32-layer Transformer depth. If modal transduction reflects a deeper organising principle—pretrained LLM priors, task semantics, or depth-induced computation—then three properties should be stable across models: (i) final-layer PID composition, (i) full layer-wise trajectories, and (i) key turning points. We test each in turn. (a) Choose Attribute (b) Choose Category (c) Choose Relation (d) Compare Attribute (e) Logical Object (f) Query Attribute Figure 5: Layer-wise PID trajectories for LLaVA-1.6-7B across six tasks. The depth-wise pattern mirrors LLaVA-1.5-7B (Figure 2): UVU_V decays, ULU_L surges late, and synergy remains bounded, confirming modal transduction in an architecturally distinct model. Final-layer invariance: language dominance persists. Table 5 compares the Layer 31 PID decomposition between the two models across all six tasks (Figure 6 provides a visual summary). Two conclusions emerge. First, language dominance is nearly unchanged. Averaged across tasks, the ULU_L share is 82.0% for LLaVA-1.5-7B and 80.7% for LLaVA-1.6-7B—a difference of only 1.3 percentage points. Task-level variation is similarly small (e.g., ChooseAttr: 74.5% vs. 71.8%; ChooseRel: 93.6% vs. 92.6%), and synergy stays bounded (S<4S<4 bits, <15%<15\% of total) in all cases. Second, task-level information fingerprints are preserved. ChooseCat remains the canonical high-redundancy case in both models (R≈7.7R≈ 7.7–7.87.8 bits; lowest ULU_L share ≈47≈ 47–48%48\%), while relational, comparative, and localization tasks retain strong language dominance (UL>90%U_L>90\%) and near-zero redundancy (R<0.3R<0.3 bits). Where models do differ, the change is in magnitude rather than organisation: LLaVA-1.6 shows higher total information for some tasks (e.g., ChooseRel: +15%+15\%; QueryAttr: +24%+24\%), consistent with improved visual resolution increasing the amount of information available for transduction without altering the transduction mechanism itself. Table 5: LLaVA-1.5-7B vs. LLaVA-1.6-7B: final-layer (Layer 31) PID comparison. Task Model R ULU_L UVU_V S Total ULU_L Share UL/UVU_L/U_V ChooseAttr 1.5-7b 1.13 14.18 2.01 1.71 19.03 74.5% 7.05×7.05× 1.6-7b 1.27 13.73 2.33 1.80 19.13 71.8% 5.89×5.89× Diff. +12% -3.2% +16% +5.3% +0.5% -2.7% -16% ChooseCat 1.5-7b 7.74 15.40 5.31 3.54 31.99 48.1% 2.90×2.90× 1.6-7b 7.69 14.17 5.20 3.30 30.36 46.7% 2.73×2.73× Diff. -0.6% -8.0% -2.1% -6.8% -5.1% -1.4% -5.9% ChooseRel 1.5-7b 0.13 18.83 0.50 0.65 20.11 93.6% 37.66×37.66× 1.6-7b 0.15 21.48 0.72 0.85 23.20 92.6% 29.83×29.83× Diff. +15% +14% +44% +31% +15% -1.0% -21% CompareAttr 1.5-7b 0.22 24.54 0.01 0.83 25.60 95.9% 2454×2454× 1.6-7b 0.26 23.23 0.08 0.83 24.40 95.2% 290×290× Diff. +18% -5.3% +700% 0% -4.7% -0.7% -88% LogicalObj 1.5-7b 0.07 12.62 0.58 0.35 13.62 92.7% 21.76×21.76× 1.6-7b 0.07 14.09 0.84 0.45 15.45 91.2% 16.77×16.77× Diff. 0% +12% +45% +29% +13% -1.5% -23% QueryAttr 1.5-7b 0.03 22.70 0.00 0.59 23.32 97.3% ∞ 1.6-7b 0.04 27.98 0.00 0.96 28.98 96.6% ∞ Diff. +33% +23% – +63% +24% -0.7% – Avg. 1.5-7b 1.55 18.05 1.40 1.28 22.28 82.0% ∼18× 18× 1.6-7b 1.58 19.11 1.53 1.37 23.59 80.7% ∼11× 11× Avg. Diff. +2% +5.9% +9.3% +7.0% +5.9% -1.3% – Figure 6: Final-layer PID composition across models. ULU_L dominates at Layer 31 in both LLaVA-1.5-7B and LLaVA-1.6-7B, while synergy remains bounded. Trajectory-level invariance: depth-wise dynamics are synchronised. Final-layer agreement does not guarantee mechanistic agreement across depth. Figure 7 overlays representative PID trajectories from both models, showing closely aligned depth-wise evolution: UVU_V decays monotonically, ULU_L follows the same U-shaped profile with a late surge, and S remains low throughout. Table 6 quantifies this alignment via cross-model Pearson correlations computed across the 32 layers for each PID component. All components are highly correlated (mean r: UL=0.982U_L=0.982, R=0.976R=0.976, UV=0.961U_V=0.961, S=0.927S=0.927), confirming that the two models match not only in endpoint composition but also in their depth-wise information dynamics. Synergy exhibits slightly lower correlation, consistent with higher estimation variance for synergistic terms in high-dimensional settings, but its qualitative behaviour—bounded and non-dominant—is invariant across models. Figure 7: Cross-model trajectory alignment. Solid: LLaVA-1.5-7B; dashed: LLaVA-1.6-7B. Depth-wise dynamics of all four PID components are closely synchronised. Table 6: Cross-model PID trajectory correlation (Pearson r across 32 layers). Task r(R)r(R) r(UL)r(U_L) r(UV)r(U_V) r(S)r(S) Mean ChooseAttr 0.968 0.993 0.978 0.941 0.970 ChooseCat 0.989 0.981 0.953 0.968 0.973 ChooseRel 0.972 0.996 0.945 0.892 0.951 CompareAttr 0.963 0.987 0.968 0.913 0.958 LogicalObj 0.981 0.974 0.972 0.925 0.963 QueryAttr 0.984 0.959 0.948 0.923 0.954 Avg. 0.976 0.982 0.961 0.927 0.962 Turning-point stability: stage transitions are depth-anchored. Table 7 compares three stage-transition landmarks across models: (i) the peak layer of UVU_V, (i) the trough of ULU_L, and (i) the onset of the late ULU_L surge. The median surge onset differs by only one layer (Layer 19 vs. 18), with cross-task dispersion of ±1.1± 1.1 layers. This stability indicates that stage transitions are anchored by architectural depth rather than visual-tokenisation details. The sole notable exception is QueryAttr, where the UVU_V peak is delayed by four layers in LLaVA-1.6; crucially, the downstream trough and surge onset remain aligned, preserving the overall three-stage organisation. Table 7: Cross-model comparison of key turning points. Task Turning point 1.5-7B 1.6-7B Diff. ChooseAttr UVU_V peak 1 0 −1-1 ULU_L trough 14 15 +1+1 ULU_L surge onset 20 19 −1-1 ChooseCat UVU_V peak 0 1 +1+1 ULU_L trough 13 15 +2+2 ULU_L surge onset 19 18 −1-1 ChooseRel UVU_V peak 0 0 0 ULU_L trough 8 – – ULU_L surge onset 19 18 −1-1 CompareAttr UVU_V peak 0 0 0 ULU_L trough – – – ULU_L surge onset 18 17 −1-1 LogicalObj UVU_V peak 0 0 0 ULU_L trough 14 15 +1+1 ULU_L surge onset 19 19 0 QueryAttr UVU_V peak 0 4 +4+4 ULU_L trough 14 15 +1+1 ULU_L surge onset 20 18 −2-2 Median UVU_V peak 0 0 0 ULU_L trough 14 15 +1+1 ULU_L surge onset 19 18 −1-1 Std. dev. – ±0.8± 0.8 ±0.9± 0.9 ±1.1± 1.1 In summary, modal transduction is stable across an architecturally distinct model at all three levels of analysis: final-layer composition (Table 5), full depth-wise dynamics (Table 6), and stage-transition landmarks (Table 7). These results indicate that the transduction pattern is not a model-specific artifact but a strategy shaped jointly by pretrained LLM priors, task semantics, and depth-induced computational stratification. 5.3 Results I: Causal Validation via Attention Knockout Experiments I–I establish modal transduction observationally and demonstrate cross-model invariance, but do not yet provide direct causal evidence. We now perform an attention knockout intervention—blocking a specific candidate pathway—and test whether the resulting PID shifts match mechanism-level predictions. Intervention design. For LLaVA-1.6-7B, we set the pre-softmax attention logits on all Image→ edges to −∞-∞ across all 32 layers, severing the dominant route through which visual evidence is injected into question tokens (implementation details in Appendix E.1). If modal transduction operates as a progressive transfer of UVU_V into ULU_L via this pathway, the knockout should produce three predictable shifts at the decision layer: (P1) ΔUV>0 U_V>0: visual information becomes “trapped” in the image stream instead of being consumed. (P2) ΔS>0 S>0: the model must resort to less efficient joint processing, increasing synergy. (P3) ΔTotal>0 \,Total>0: compensating for blocked transduction requires a larger total information budget. We further expect the magnitude of these effects to vary with task semantics: low-redundancy, vision-dependent tasks should show strong sensitivity, while high-redundancy tasks should be robust due to alternative informational routes. 5.3.1 Decision-layer effects Table 8 reports the Layer 31 PID composition under normal and knockout conditions for all six tasks; Figure 8 provides a visual summary. The results reveal three distinct response classes. Table 8: PID effects of Image→ attention knockout at Layer 31 on LLaVA-1.6-7B. Values are in bits; Δ:=(knockout−normal)/normal×100% :=(knockout-normal)/normal× 100\%. Bold entries mark changes exceeding 5%5\% or mechanism-critical shifts. Task Condition R ULU_L UVU_V S Total UL/UVU_L/U_V ΔTotal \,Total ChooseRel Normal 0.15 21.48 0.72 0.85 23.20 29.8×29.8× – (High vision dep.) Knockout 0.15 23.14 0.85 1.07 25.20 27.2×27.2× +8.6% Δ 0% +7.7% +18.1% +25.9% – -8.7% – LogicalObj Normal 0.07 14.09 0.84 0.45 15.45 16.8×16.8× – (High vision dep.) Knockout 0.06 14.63 0.90 0.49 16.09 16.3×16.3× +4.1% Δ -14% +3.8% +7.7% +9.8% – -3.0% – QueryAttr Normal 0.04 27.98 0.00 1.00 29.02 ∞ – (Pure language) Knockout 0.04 30.63 0.00 1.25 31.93 ∞ +10.0% Δ 0% +9.5% 0% +25.2% – – – CompareAttr Normal 0.26 23.23 0.08 0.83 24.40 290×290× – (Extract–compare) Knockout 0.26 23.14 0.01 0.82 24.24 2314×2314× -0.7% Δ 0% -0.4% -86.3% -1.2% – +8×+8× – ChooseAttr Normal 1.27 13.73 2.33 1.85 19.18 5.89×5.89× – (High redundancy) Knockout 1.27 13.27 2.47 1.76 18.77 5.37×5.37× -2.1% Δ 0% -3.3% +6.0% -5.0% – -8.8% – ChooseCat Normal 7.69 14.17 5.20 3.50 30.56 2.73×2.73× – (High redundancy) Knockout 7.70 13.79 5.43 3.37 30.29 2.54×2.54× -0.9% Δ +0.1% -2.7% +4.5% -3.9% – -7.0% – Figure 8: Final-layer effects of Image→ attention knockout. Vision-dependent tasks (ChooseRel, LogicalObj) show the strongest PID shifts; high-redundancy tasks (ChooseAttr, ChooseCat) are largely robust. Type A—strong response in vision-dependent tasks. ChooseRel and LogicalObj validate all three predictions: UVU_V and S increase, and the total information budget grows (ChooseRel: +8.6%+8.6\%; LogicalObj: +4.1%+4.1\%). These tasks require sustained visual grounding, and the Image→ pathway is a causal bottleneck for their transduction. ULU_L can still increase under knockout, but only at a higher total-information cost—the hallmark of a less efficient compensatory mechanism. Type B—compensation in language-dominated tasks. QueryAttr shows the largest ΔTotal \,Total (+10.0%+10.0\%) while UVU_V remains effectively zero: the model compensates by expanding language-unique capacity (ΔUL=+9.5% U_L=+9.5\%) and synergy (ΔS=+25.2% S=+25.2\%), confirming that even nominally language-internal reasoning is affected when the upstream transduction channel is severed. CompareAttr is atypical: UVU_V drops sharply (−86.3%-86.3\%) with negligible change in total budget, consistent with an “extract–compare” computation where visual extraction completes early and later layers prefer fully transduced representations. Type C—robustness in high-redundancy tasks. ChooseAttr and ChooseCat exhibit minimal impact (ΔTotal=−2.1% \,Total=-2.1\% and −0.9%-0.9\%). Although UVU_V increases slightly, both ULU_L and S decrease, suggesting that high redundancy provides alternative informational routes—and may even reduce cross-modal interaction overhead when the direct transduction channel is removed. Robustness here is governed by multi-path redundancy rather than pathway strength alone. Task-dependence score. To summarise task sensitivity in a single metric, we define Dep:=(ΔUV+ΔS+ΔTotal)/ 3Dep:=( U_V+ S+ \,Total)\,/\,3 (percentage units). This score ranks ChooseRel as most dependent, followed by QueryAttr and LogicalObj, while ChooseAttr and ChooseCat are near zero or negative—consistent with redundancy-driven robustness. 5.3.2 Layer-wise knockout dynamics Figure 9 compares the full 32-layer PID trajectories under normal and knockout conditions for ChooseRel, the task with the strongest causal response. The knockout induces a characteristic cascade that mirrors the three-stage observational pattern from Experiments I–I, now with causal force: Stage I (Layers 0–10). Early layers fail to establish a strong visually conditioned language state: UVU_V remains elevated relative to the baseline, and the initial ULU_L rise is attenuated. Stage I (Layers 10–20). The gap between normal and knockout trajectories widens as mid-layer consolidation proceeds without the full benefit of early transduction. Stage I (Layers 20–32). Late layers compensate by sharply increasing both ULU_L and S, but at the cost of a higher total information budget—confirming that the compensatory regime, while partially effective, is less efficient than the intact transduction pathway. This depth-resolved profile transforms the observational three-stage structure into a causal account: blocking the primary transduction route at its origin propagates through the entire network, with each stage exhibiting predictable deviations from the unperturbed trajectory. Figure 9: Layer-wise knockout effects for ChooseRel. Solid: normal; dashed: knockout. Blocking Image→ attention traps visual evidence (elevated UVU_V), delays ULU_L growth, and forces late-layer compensation at higher total-information cost. In summary, attention knockout provides direct causal evidence for modal transduction. Blocking Image→ attention systematically increases UVU_V and S in vision-dependent tasks and raises the total information budget required for decision formation. Task-level heterogeneity reveals that semantic redundancy governs knockout robustness, and the PID lens upgrades prior knockout findings into a quantitative account of which information components are carried by which pathways. 6 Conclusion We introduced a layer-wise information-theoretic framework that combines Partial Information Decomposition with scalable normalizing-flow estimation to characterize how visual and linguistic evidence is organised across the depth of multimodal Transformers. Across six reasoning tasks and two LLaVA model variants, the framework reveals a consistent modal transduction mechanism: visual-unique information is injected early and progressively absorbed into language representations, language-unique information dominates late-layer decision formation (approximately 82% of total predictive information at the final layer), and synergy remains bounded throughout (<<2%). Causal validation via Image→ attention knockout confirms that blocking this pathway traps visual evidence, forces compensatory synergy, and inflates the total information budget—with effect sizes governed by task-level semantic redundancy. These results reframe multimodal reasoning in current MLLMs as language-space decision making under visually conditioned representations, rather than emergent cross-modal fusion. This characterisation carries concrete design implications: if visual evidence is compressed in early layers, improving the encoder alone will have limited downstream impact; preserving modality-unique information likely requires architectural or objective-level interventions—such as auxiliary losses that penalise premature transduction, or cross-attention mechanisms that maintain a separate visual stream—targeted at the bottleneck layers identified by PID trajectories. Limitations and future work. Our analysis is subject to several limitations that suggest directions for future research. First, the PID estimation pipeline introduces cumulative approximations (mean pooling, PCA, flow-based Gaussianization, IminI_ redundancy); while ordinal patterns are robust, absolute bit values should be interpreted with caution, and future work should quantify estimation error more precisely. Second, our experiments are restricted to the LLaVA family, which shares a common encode-project-decode architecture; models with structurally different fusion mechanisms (e.g., cross-attention–based architectures such as Flamingo) may exhibit qualitatively different PID trajectories, and testing this hypothesis is an important next step. Third, the attention knockout is applied uniformly across all 32 layers; layer-specific interventions would provide finer-grained causal evidence for the three-stage transduction structure. Finally, GQA tasks are relatively simple; more demanding benchmarks requiring multi-step visual reasoning may reveal stronger synergistic contributions than observed here. We view layer-wise PID trajectories as a general-purpose diagnostic that can be extended to these settings, and we will release all code and data to support such extensions. 7 Impact Statement This work aims to advance the understanding of multimodal reasoning in large vision–language models through interpretability and causal analysis. The contributions are methodological and analytical, with no direct application, deployment, or use of sensitive data, human subjects, or automated decision-making systems. The potential societal impacts are consistent with those commonly associated with research on model analysis and interpretability, and we do not identify specific ethical concerns requiring further discussion. Generative AI tools were used solely for language editing and stylistic refinement; all scientific content and conclusions are the authors’ own. References Alain & Bengio (2016) Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes. In International Conference on Learning Representations, 2016. URL https://openreview.net/forum?id=HJ4-rAVtl. Also available as arXiv:1610.01644. Ben Melech Stan et al. (2024) Ben Melech Stan, G., Aflalo, E., Rohekar, R. Y., Bhiwandiwalla, A., Tseng, S.-Y., Olson, M. L., Gurwicz, Y., Wu, C., Duan, N., and Lal, V. Lvlm-interpret: An interpretability tool for large vision-language models. arXiv preprint, 2024. Preprint. Bertschinger et al. (2014) Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., and Ay, N. Quantifying unique information. Entropy, 16(4):2161–2183, 2014. doi: 10.3390/e16042161. Bi et al. (2025) Bi, J., Guo, J., Tang, Y., Wen, L. B., Liu, Z., and Xu, C. Unveiling visual perception in language models: An attention head analysis approach. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4135–4144, 2025. Chefer et al. (2021) Chefer, H., Gur, S., and Wolf, L. Transformer interpretability beyond attention visualization. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 782–791, 2021. Clark et al. (2019) Clark, K., Khandelwal, U., Levy, O., and Manning, C. D. What does bert look at? an analysis of bert’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, p. 276–286, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10.18653/v1/w19-4828. Dewan et al. (2024) Dewan, S., Zawar, R., Saxena, P., Chang, Y., Luo, A., and Bisk, Y. Interpreting diffusion via partial information decomposition. In Advances in Neural Information Processing Systems, volume 37, p. 50485–50510, 2024. Dissanayake et al. (2025) Dissanayake, P., Hamman, F., Halder, B., Sucholutsky, I., Zhang, Q., and Dutta, S. Quantifying knowledge distillation using partial information decomposition. In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics. PMLR, 2025. Faes et al. (2025) Faes, L., Sparacino, L., Mijatovic, G., Antonacci, Y., Ricci, L., Marinazzo, D., and Stramaglia, S. Partial information rate decomposition. arXiv preprint, 2025. Preprint. Geva et al. (2023) Geva, M., Bastings, J., Filippova, K., and Globerson, A. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 12216–12235, Singapore, 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.751. URL https://aclanthology.org/2023.emnlp-main.751/. Hewitt & Manning (2019) Hewitt, J. and Manning, C. D. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 4129–4138, 2019. Hudson & Manning (2019) Hudson, D. A. and Manning, C. D. GQA: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6700–6709, 2019. Jain & Wallace (2019) Jain, S. and Wallace, B. C. Attention is not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 11–20, 2019. doi: 10.18653/v1/D19-1002. Jo et al. (2025) Jo, S., Jang, G., and Park, H. Gmar: Gradient-driven multi-head attention rollout for vision transformer interpretability. In Proceedings of the IEEE International Conference on Image Processing (ICIP), 2025. Krishna et al. (2016) Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision, 123(1):32–73, 2016. doi: 10.1007/s11263-016-0981-9. Online first: 2016; print issue: 2017. Kunert-Graf et al. (2020) Kunert-Graf, J., Sakhanenko, N., and Galas, D. Partial information decomposition and the information delta: A geometric unification. Entropy, 22(12):1333, 2020. Liang et al. (2023) Liang, P. P., Cheng, Y., Fan, X., Ling, C. K., Nie, S., Chen, R. J., Deng, Z., Allen, N., Auerbach, R., Mahmood, F., Salakhutdinov, R., and Morency, L.-P. Quantifying & modeling multimodal interactions: An information decomposition framework. In Advances in Neural Information Processing Systems, volume 36, p. 66654–66689, 2023. Liu et al. (2023) Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. In Advances in Neural Information Processing Systems, volume 36, p. 73669–73684, 2023. Oral Presentation. Lou et al. (2025) Lou, H., Li, C., Ji, J., and Yang, Y. Sae-v: Interpreting multimodal models for enhanced alignment. In Proceedings of the 42nd International Conference on Machine Learning, volume 267, p. 28142–28161. PMLR, 2025. Lyu et al. (2024) Lyu, A., Clark, A., and Raviv, N. Explicit formula for partial information decomposition. arXiv preprint, 2024. Preprint. Murphy & Bassett (2024) Murphy, K. A. and Bassett, D. S. Information decomposition in complex systems via machine learning. Proceedings of the National Academy of Sciences, 121(13):e2312988121, 2024. doi: 10.1073/pnas.2312988121. Pimentel & Cotterell (2021) Pimentel, T. and Cotterell, R. A Bayesian framework for information-theoretic probing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 2869–2887, Online and Punta Cana, Dominican Republic, 2021. doi: 10.18653/v1/2021.emnlp-main.229. Pimentel et al. (2020) Pimentel, T., Valvoda, J., Hall Maudslay, R., Zmigrod, R., Williams, A., and Cotterell, R. Information-theoretic probing for linguistic structure. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p. 4609–4622, 2020. doi: 10.18653/v1/2020.acl-main.420. Radford et al. (2021) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, volume 139, p. 8748–8763. PMLR, 2021. Tenney et al. (2019) Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Van Durme, B., Bowman, S. R., Das, D., and Pavlick, E. What do you learn from context? Probing for sentence structure in contextualized word representations. In International Conference on Learning Representations, p. 3179–3197, 2019. Venkatesh et al. (2023) Venkatesh, P., Bennett, C., Gale, S., Ramirez, T. K., Heller, G., Durand, S., Olsen, S., and Mihalas, S. Gaussian partial information decomposition: Bias correction and application to high-dimensional data. In Advances in Neural Information Processing Systems, volume 36, p. 78253–78272, 2023. Williams & Beer (2010) Williams, P. L. and Beer, R. D. Nonnegative decomposition of multivariate information. arXiv preprint, 2010. Also published in PLOS Computational Biology. Xin et al. (2025) Xin, J., Yun, S., Peng, J., Choi, I., Ballard, J., Chen, T., and Long, Q. I2moe: Interpretable multimodal interaction-aware mixture-of-experts. In International Conference on Machine Learning, 2025. Yang et al. (2025) Yang, Z., Wang, H., and Hu, D. Efficient quantification of multimodal interaction at sample level. In International Conference on Machine Learning, 2025. Zhao et al. (2025) Zhao, W., Balachandran, A., Tian, C., and Liang, P. P. Partial information decomposition via normalizing flows in latent gaussian distributions. In Advances in Neural Information Processing Systems, 2025. Symbol Table Table 9: Symbol table for key notation. Symbol Meaning Description V Vision modality Visual input stream (image tokens) L Language modality Linguistic input stream (question tokens) Y Target variable Prediction target (logit or probability of correct answer) ℓ Layer index Transformer layer, ℓ∈0,…,L ∈\0,…,L\ L Model depth Total number of Transformer layers (32 in LLaVA) V(ℓ)x_V^( ) Vision representation Aggregated visual representation at layer ℓ L(ℓ)x_L^( ) Language representation Aggregated language representation at layer ℓ I(⋅;⋅)I(·;·) Mutual information Shannon mutual information Itot(ℓ)I_tot^( ) Total predictive information I(V(ℓ),L(ℓ);Y)I(x_V^( ),x_L^( );Y) R(ℓ)R^( ) Redundant information Predictive information shared by vision and language UV(ℓ)U_V^( ) Vision-unique information Predictive information available only from vision UL(ℓ)U_L^( ) Language-unique information Predictive information available only from language S(ℓ)S^( ) Synergistic information Predictive information available only jointly ℐ(ℓ)I^( ) Information state (R(ℓ),UV(ℓ),UL(ℓ),S(ℓ))(R^( ),U_V^( ),U_L^( ),S^( )) T PID trajectory Layer-wise sequence ℐ(ℓ)ℓ=0L\I^( )\_ =0^L ΔC(ℓ) C^( ) Knockout-induced change Relative change of component C under attention knockout Appendix A Notation Summary This appendix summarizes the notation used throughout the paper to ensure clarity and consistency, especially for information-theoretic quantities and layer-wise analysis in multimodal large language models (MLLMs). Modalities and representations. We denote the vision modality by V and the language modality by L. At Transformer layer ℓ , the aggregated modality-level representations are V(ℓ)x_V^( ) for vision and L(ℓ)x_L^( ) for language. Aggregation is performed over modality-specific token sets as described in Section 4. Target variable. The target variable Y denotes the model’s prediction target and is implemented as the logit (or logit-transformed probability) of the correct answer token. All information quantities are computed with respect to Y. Partial Information Decomposition (PID). At each layer ℓ , the joint predictive information is decomposed as I(V(ℓ),L(ℓ);Y)=R(ℓ)+UV(ℓ)+UL(ℓ)+S(ℓ),I(x_V^( ),x_L^( );Y)=R^( )+U_V^( )+U_L^( )+S^( ), (13) where R(ℓ)R^( ) is redundant information shared by both modalities, UV(ℓ)U_V^( ) and UL(ℓ)U_L^( ) are vision-unique and language-unique information, respectively, and S(ℓ)S^( ) is synergistic information that arises only from the joint pair. Information state and trajectory. We define the layer-wise information state as ℐ(ℓ)=(R(ℓ),UV(ℓ),UL(ℓ),S(ℓ))I^( )=(R^( ),U_V^( ),U_L^( ),S^( )), and the PID trajectory as the sequence =ℐ(ℓ)ℓ=0LT=\I^( )\_ =0^L. Total information. The total predictive information at layer ℓ is denoted by Itot(ℓ):=R(ℓ)+UV(ℓ)+UL(ℓ)+S(ℓ).I_tot^( ):=R^( )+U_V^( )+U_L^( )+S^( ). Attention knockout notation. For attention knockout experiments, we denote the relative change of a PID component C∈R,UV,UL,S,ItotC∈\R,U_V,U_L,S,I_tot\ at layer ℓ by ΔC(ℓ)(%)=CKO(ℓ)−CBase(ℓ)CBase(ℓ)×100%, C^( )(\%)= C_KO^( )-C_Base^( )C_Base^( )× 100\%, where BaseBase and KOKO refer to the normal and knockout conditions, respectively. Notation consistency. Throughout the paper, we use (V,L)(V,L) consistently to denote vision and language modalities. Any alternative labels (e.g., question/image tokens) refer to the same underlying distinction and are unified under this notation. Appendix B Experimental Settings and Implementation Details B.1 Models and Comparative Design We study the LLaVA family, which instantiates the dominant multimodal design of a visual encoder followed by a projection into an LLM backbone. Importantly, LLaVA maintains an explicit separation between visual and textual token streams, which makes the modality-level random variables required by our PID analysis well-defined. Our main comparison contrasts LLaVA-1.5-7B and LLaVA-1.6-7B (Vicuna-based). Both models use a 32-layer Transformer language backbone, enabling layer-wise alignment at the level of representational hierarchy, while differing meaningfully in visual processing. LLaVA-1.5-7B uses CLIP ViT-L/14 with fixed resolution (336×336336×336), yielding a fixed number of visual tokens (576576). These tokens are mapped by a two-layer MLP into the Vicuna embedding space (dimension 40964096; Vicuna-v1.5-7B). In contrast, LLaVA-1.6-7B supports dynamic high-resolution and multi-scale processing (up to 672×672672×672), producing a variable number of visual tokens (typically 576576–23042304), and is trained on an expanded dataset with an updated Vicuna version (v1.6). This controlled perturbation allows us to test whether the observed information-flow patterns reflect architecture-specific details (e.g., resolution and token count) or an architecture-robust mechanism driven by LLM priors and task semantics. B.2 Tasks, Semantic Coverage, and Sample Scale We conduct our experiments on the GQA dataset (Hudson & Manning, 2019), a large-scale benchmark for real-world visual reasoning and compositional question answering built on images from Visual Genome (Krishna et al., 2016). GQA provides scene-graph annotations with objects, attributes, and relations, and uses a functional program engine to generate diverse yet controlled questions with reduced language bias and balanced answer distributions. Its questions are organized along structural and semantic dimensions, covering existence, attribute, category, relation, and logical queries, which makes it particularly suitable for probing how multimodal large language models integrate visual and linguistic information during visual question answering. Our GQA-based VQA subset. Starting from the validation split of GQA, we construct a focused VQA subset tailored to our analysis. We keep question groups where state-of-the-art MLLMs such as LLaVA variants achieve reasonably high accuracy, and discard overly easy yes/no verification questions. From six remaining groups that span attribute and category selection, spatial reasoning, and object-level comparison or logical inference, we sample image–question pairs that are correctly answered by the models under study. This results in a compact benchmark with reliable model predictions and object-level bounding boxes, which we exploit to separate object-related patches from the rest of the image when studying cross-modal information flow. We construct six representative visual reasoning subtasks from the GQA validation set: ChooseAttr, ChooseCat, ChooseRel, CompareAttr, LogicalObj, and QueryAttr. Collectively, the suite covers attribute recognition, category identification, spatial relations, attribute comparison, logical composition, and spatial localization. The task suite is designed to (i) provide semantic diversity without duplicating the same reasoning template, and (i) induce meaningful variation in information structure. In particular, tasks differ in visual dependence (e.g., spatial relations and logical composition typically require stronger grounding) and cross-modal redundancy (e.g., category concepts often have strong shared representations in vision and language). For each task and model, we retain at least 900900 valid samples, which is sufficient for stable layer-wise estimation and reduces finite-sample bias in mutual-information proxies. Appendix C Layer-wise PID Flow Estimation and Gaussian Decomposition C.1 Why Layer-wise Independent Estimation Our object of interest is the predictive information structure at each layer. Because the representation distribution changes substantially with depth, we train PID-flow estimators independently for each layer. Sharing a single estimator across layers would introduce coupling artifacts and bias trajectory comparisons. For each task, model, and layer, we train two bivariate flows: one for (XLℓ,Y)(X_L ,Y) and one for (XVℓ,Y)(X_V ,Y). C.2 PID Flow Objective and Operational Semantics For a fixed layer ℓ , we learn invertible mappings (Z^Q,Z^Y)=fQℓ(XLℓ,Y),(Z^I,Z^Y′)=fIℓ(XVℓ,Y),( Z_Q, Z_Y)=f_Q (X_L ,Y), ( Z_I, Z_Y )=f_I (X_V ,Y), (14) such that each transformed pair is approximately standard Gaussian. This enables closed-form Gaussian mutual information and a numerically stable PID proxy satisfying non-negativity and additivity. We emphasize that these PID terms are operational proxies used to characterize trajectories and intervention responses, rather than claims of exact ground-truth PID in the underlying high-dimensional system. C.3 Flow Architecture and Optimization We use RealNVP with 8 coupling blocks per bivariate flow, hidden dimension 256, ReLU activations, and batch normalization after each block. We optimize with Adam (learning rate 10−410^-4, batch size 128, 10,000 steps), weight decay 10−510^-5, and gradient clipping at 1.01.0. All runs use fixed random seeds and CUDA deterministic mode. C.4 Gaussian MI and PID Construction For a Gaussian pair (A,B)(A,B) with correlation ρAB _AB, mutual information is I(A;B)=−12log(1−ρAB2).I(A;B)=- 12 (1- _AB^2). (15) Using estimated correlations, we compute I(ZQ;ZY)I(Z_Q;Z_Y), I(ZI;ZY)I(Z_I;Z_Y), and I(ZQ,ZI;ZY)I(Z_Q,Z_I;Z_Y). We define redundancy by the minimum-information rule: Rℓ=minI(ZQ;ZY),I(ZI;ZY),R = \I(Z_Q;Z_Y),\,I(Z_I;Z_Y)\, (16) and the remaining components by additivity: ULℓ=I(ZQ;ZY)−Rℓ,UVℓ=I(ZI;ZY)−Rℓ,Sℓ=I(ZQ,ZI;ZY)−I(ZQ;ZY)−I(ZI;ZY)+Rℓ.U_L =I(Z_Q;Z_Y)-R , U_V =I(Z_I;Z_Y)-R , S =I(Z_Q,Z_I;Z_Y)-I(Z_Q;Z_Y)-I(Z_I;Z_Y)+R . (17) By construction, Rℓ+ULℓ+UVℓ+Sℓ=I(ZQ,ZI;ZY)R +U_L +U_V +S =I(Z_Q,Z_I;Z_Y) up to numerical tolerance. C.5 Numerical Stability and Quality Checks For each task–model–layer configuration, we verify (i) non-negativity of all PID components, (i) additivity error <1%<1\%, and (i) seed stability via repeated training. We focus on robust properties—relative dominance, turning points, and trajectory trends—and report variance ranges to ensure that the observed patterns reflect mechanisms rather than estimator noise. Appendix D Cross-Model Consistency Metrics To quantify architecture robustness, we align PID trajectories between LLaVA-1.5 and LLaVA-1.6 for each task and component. We compute Pearson correlation across layers and normalized mean absolute error (nMAE). We additionally identify key layers (e.g., dominance switches or curvature peaks) and report their absolute offsets, localizing any discrepancies to specific components and depth ranges. Appendix E Attention Knockout: Implementation and Paired Evaluation E.1 Knockout Design We implement a unidirectional knockout that blocks Image→ attention by setting the corresponding attention logits to −∞-∞ before softmax, while preserving Question→ and all self-attention pathways. The intervention is applied simultaneously to all 32 layers, targeting the hypothesized modal-transduction route. E.2 Mechanistic Intervention and Paired Computation The knockout modifies only the forward computation graph and leaves parameters unchanged, constituting a clean mechanistic intervention. We evaluate knockout and baseline on identical samples (as in Section B.2), enabling paired comparisons. E.3 PID Under Knockout and Task-level Dependence Under knockout, we recompute PID trajectories even when predictions become incorrect, enabling analysis of how information structure reorganizes under disruption. We report relative changes ΔCℓ(%)=CKOℓ−CBaseℓCBaseℓ×100%, C (\%)= C _KO-C _BaseC _Base× 100\%, (18) and define a task-level dependence score as the average of ΔUV U_V, ΔS S, and ΔITotal I_Total. Appendix F Evaluation, Statistics, and Reproducibility We report absolute PID values (bits) as well as normalized proportions. Key summary metrics include the final-layer ratio ULL/UVLU_L^L/U_V^L, the synergy share SL/ITotalLS^L/I_Total^L, and cross-layer turning points. Statistical stability is assessed using bootstrap confidence intervals (1,000 resamples), ANOVA across tasks, and paired t-tests for knockout effects, reporting Cohen’s d as effect size. All experiments use fixed random seeds (42), deterministic computation, fixed data splits, and disjoint sets for flow training and PID evaluation. We will release code, preprocessing scripts, and visualization pipelines upon publication to support full reproducibility. Appendix G Supplementary Material for Layer-wise PID Analysis This appendix provides supplementary material for the method introduced in Section 4. It consolidates robustness checks, proof sketches, and implementation details that support the theoretical and empirical validity of the proposed layer-wise PID analysis framework. G.1 Alternative Pooling Rules and Robustness In the main text, modality-level representations are summarized using mean pooling over visual and language token regions (Section 4.1). Here we assess the robustness of PID trajectories to alternative pooling strategies. We consider (i) attention-weighted pooling over language tokens using the self-attention distribution, and (i) max pooling over token dimensions. Across all tasks and models, the qualitative behavior of PID trajectories is preserved, including the relative dominance of UV(ℓ)U_V^( ), UL(ℓ)U_L^( ), and S(ℓ)S^( ), as well as the location of turning points. While absolute magnitudes vary mildly, all mechanism classifications defined in Section 4.2 remain unchanged. G.2 Separability of Mechanism Definitions This section provides a proof sketch for Proposition 4.4. Let ℐ(L)=(R(L),UV(L),UL(L),S(L))I^(L)=(R^(L),U_V^(L),U_L^(L),S^(L)) denote the final-layer information allocation, and assume Itot(L)>0I_tot^(L)>0. Each mechanism definition in Section 4.2 imposes a set of strict inequality constraints on ℐ(L)I^(L). Persistent synergy requires proportional dominance of S(L)S^(L); modal transduction requires dominance of UL(L)U_L^(L) together with an intermediate-layer peak in UV(ℓ)U_V^( ); and redundancy-dominant convergence requires R(L)R^(L) to exceed all other components by a fixed margin. Under non-degeneracy (at least one component strictly dominates), these regions do not overlap in the simplex defined by R(L)+UV(L)+UL(L)+S(L)=Itot(L)R^(L)+U_V^(L)+U_L^(L)+S^(L)=I_tot^(L). G.3 Sensitivity to PCA Dimensionality We evaluate the sensitivity of PID trajectories to the retained PCA dimension d′d . Across d′∈16,32,64d ∈\16,32,64\, corresponding to approximately 90%90\%–98%98\% variance retained, the qualitative structure of PID trajectories is stable. In particular, the ordering of modality dominance at the final layer and the depth of modality-transduction turning points are preserved. This suggests that PCA primarily removes redundant noise rather than decision-critical information. G.4 Gaussian Plug-in PID: Closed-form Details After the flow transformation described in Section 4.3, we model (V(ℓ),L(ℓ),Y)(z_V^( ),z_L^( ),Y) as jointly Gaussian. For Gaussian variables, mutual information admits the closed form I(A;B)=−12log(1−ρAB2),I(A;B)=- 12 \! (1- _AB^2 ), (19) where ρAB _AB denotes the Pearson correlation coefficient. All marginal and joint mutual information terms are computed from empirical covariance estimates. PID components are then obtained via the defining relations in Section 3, adopting the IminI_ redundancy rule. Numerical non-negativity is enforced up to a tolerance of 10−610^-6. G.5 Consistency of Gaussian Plug-in Estimation Under correct model specification, i.e., when the transformed variables are exactly Gaussian, maximum-likelihood covariance estimation is consistent. By the continuous mapping theorem, the induced mutual information estimates and the resulting PID components are consistent as the sample size tends to infinity. When Gaussianity is violated, bias may arise; in practice, we mitigate this via flow-capacity sweeps, normality diagnostics, and cross-model consistency checks. G.6 Threshold Sensitivity All thresholds introduced in Section 4.2 (e.g., τS _S, γ, η, and ρ) are varied within reasonable ranges. The qualitative classification of mechanisms remains stable across these ranges, indicating that our conclusions are not driven by finely tuned hyperparameters.