Paper deep dive
Geometric Similarity in VLM Low-Level Vision Representations
Shao-Jun Xia, Huixin Zhang, Zhen Lei, Anlan Sun, Yuner Zhang, Xiaoyang Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/2/2026, 9:42:51 AM
Summary
This paper introduces GeoSim, a unified four-level framework for analyzing the geometric similarity of low-level vision representations in Vision-Language Models (VLMs). The study investigates 24 low-level tasks across autoregressive (AR) models and diffusion transformers (DiTs) to understand how hidden-layer representations are organized. GeoSim evaluates global similarity, local geometry, sparse feature decomposition, and topological verification to reveal organizing principles and limits of cross-task and cross-model transferability.
Entities (10)
Relation Signals (5)
GeoSim → analyzes → Vision-Language Models
confidence 95% · GeoSim analyzes task-conditioned representations from global similarity, local geometry, sparse feature decomposition, and topological verification perspectives.
GeoSim → evaluates → Low-Level Vision Tasks
confidence 93% · In this paper, we systematically investigate representational similarity across 24 low-level tasks
GeoSim → appliesto → Autoregressive Models
confidence 92% · Our formulation applies to the analysis of hidden states in AR models and feature maps in DiTs
GeoSim → appliesto → Diffusion Transformers
confidence 92% · Our formulation applies to the analysis of hidden states in AR models and feature maps in DiTs
Sparse Autoencoder → usedin → Sparse Feature Decomposition
confidence 90% · Level 3 decomposes representations into shared and task-specific sparse features... SAE representation decomposition
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-language models (VLMs) have emerged as powerful candidates for universal vision backbones, with representative architectures including autoregressive (AR) models and diffusion transformers (DiTs). Yet, adapting them efficiently for all-in-one low-level image restoration remains a challenge. Crucially, the field lacks an understanding of how VLMs organize hidden-layer representations and whether these structurally distinct paradigms share a common geometric organization for pixel-level perception. Such shared organization is a prerequisite for building highly transferable, unified restoration VLMs and adapters. In this paper, we systematically investigate representational similarity across 24 low-level tasks spanning 5 categories. We propose GeoSim, a unified four-level framework that analyzes task-conditioned representations from global similarity, local geometry, sparse feature decomposition, and topological verification perspectives. Our formulation applies to the analysis of hidden states in AR models and feature maps in DiTs across same- and cross-task/model settings. Our results reveal the organizing principles of low-level visual representations while exposing their limits in cross-task and cross-model agreement. Ultimately, GeoSim provides an interpretability lens for probing latent transferability in low-level vision and diagnosing model limitations in task- or model-specific scenarios.
Tags
Links
- Source: https://arxiv.org/abs/2610.00848v1
- Canonical: https://arxiv.org/abs/2610.00848v1
Trouble viewing inline? Open PDF directly →
Full Text
111,000 characters extracted from source content.
Expand or collapse full text
GEOMETRIC SIMILARITY IN VLM LOW-LEVEL VISION REPRESENTATIONS Shao-Jun Xia * Duke University shaojun.xia@duke.edu Huixin Zhang * Texas A&M University zhanghui21@tamu.edu Zhen Lei * Florida State University zl24a@fsu.edu Anlan Sun † UC San Diego ans108@ucsd.edu Yuner Zhang University of Pennsylvania yunerzh@seas.upenn.edu Xiaoyang Chen † UNC Chapel Hill xiaoyang chen@med.unc.edu * Equal contribution. † Corresponding authors. ABSTRACT Vision-language models (VLMs) have emerged as powerful candidates for univer- sal vision backbones, with representative architectures including autoregressive (AR) models and diffusion transformers (DiTs). Yet, adapting them efficiently for all-in-one low-level image restoration remains a challenge. Crucially, the field lacks an understanding of how VLMs organize hidden-layer representations and whether these structurally distinct paradigms share a common geometric organi- zation for pixel-level perception. Such shared organization is a prerequisite for building highly transferable, unified restoration VLMs and adapters. In this paper, we systematically investigate representational similarity across 24 low-level tasks spanning 5 categories. We propose GeoSim, a unified four-level framework that analyzes task-conditioned representations from global similarity, local geometry, sparse feature decomposition, and topological verification perspectives. Our for- mulation applies to the analysis of hidden states in AR models and feature maps in DiTs across same- and cross-task/model settings. Our results reveal the organizing principles of low-level visual representations while exposing their limits in cross- task and cross-model agreement. Ultimately, GeoSim provides an interpretability lens for probing latent transferability in low-level vision and diagnosing model limitations in task- or model-specific scenarios. Our code is available on GitHub. 1INTRODUCTION Vision-language models (VLMs) have demonstrated remarkable capabilities across a broad spec- trum of visual understanding tasks (Zhang et al., 2024). In high-level vision, they have achieved success in image classification (Radford et al., 2021), object detection (Li et al., 2022c), semantic segmentation (Li et al., 2022a), instance segmentation (Huynh et al., 2022), and open-vocabulary recognition (Zhong et al., 2022). More recently, VLMs have also begun to extend into low-level vision, where a single model is expected to solve diverse perceptual tasks such as restoration, re- moval, enhancement, reconstruction, and photometric correction (Ai et al., 2024; Jiang et al., 2025a). Unlike high-level vision—which primarily maps inputs to invariant semantic categories—low-level tasks must carefully balance fine-grained pixel statistics, frequency components, and local image structures with high-level perceptual context. Consequently, modern all-in-one restoration systems increasingly rely on shared internal representations to transfer knowledge across diverse perceptual tasks. Understanding how these representations are organized is therefore becoming a fundamental problem, with important implications for model interpretability, efficient knowledge transfer, con- tinual learning, and the principled design of scalable all-in-one architectures. Despite rapid progress in low-level VLMs, existing studies have focused primarily on improving task performance through prompting (Potlapalli et al., 2023; Conde et al., 2024), agentic instruc- tion (Jiang et al., 2024; Chen et al., 2024a), and multimodal representations (Luo et al., 2024). Comparatively little is known about the internal organization of the representations that enable these 1 arXiv:2610.00848v1 [cs.CV] 1 Oct 2026 models to solve diverse perceptual tasks. In particular, it remains unclear whether different low- level tasks rely on shared computational mechanisms, whether they naturally organize into coherent perceptual families, or whether each task occupies an independent region of representation space. Without understanding this organization, it remains difficult to explain why knowledge transfers across some tasks but not others, how parameter sharing should be designed, or whether a single representation space can effectively support increasingly diverse low-level capabilities. The geometry of hidden representations provides a powerful lens for studying the internal organiza- tion of deep models. Classical representation analysis methods, including RSA (Kriegeskorte et al., 2008), SVCCA (Raghu et al., 2017), and CKA (Kornblith et al., 2019), characterize neural represen- tations through pairwise similarity, shared linear subspaces, or relational structure. In VLMs, con- trastive models such as CLIP and ALIGN learn a shared embedding space by encouraging matched image–text pairs to exhibit higher similarity than mismatched pairs (Radford et al., 2021; Jia et al., 2021). Subsequent work has shown that large multimodal models develop increasingly shared rep- resentational structures across architectures and, in some cases, across modalities (Huh et al., 2024; Jha et al., 2025). More recent layer-wise analyses further reveal that visual representations become progressively aligned with textual feature spaces (Venhoff et al., 2025), object-specific information is refined through subsets of visual tokens (Neo et al., 2025), and visual information is increasingly transferred into text-token representations in deeper layers (Lin et al., 2025). Complementary advances in interpretability suggest that concepts, decisions, and intermediate computations are often organized as linear directions or low-dimensional manifolds in hidden space (Marks & Tegmark, 2024; Joshi et al., 2025; Gurnee et al., 2026; Ma et al., 2026). Sparse au- toencoders (SAEs) further decompose hidden activations into interpretable sparse features (Huben et al., 2024; Gao et al., 2025; Pach et al., 2025; Shu et al., 2025), while recent evidence indicates that semantically equivalent visual and textual inputs can induce shared modality-invariant task rep- resentations (Luo et al., 2025a). Collectively, these studies suggest that neural representations often possess rich geometric structure. However, nearly all existing analyses focus on semantic concepts, language alignment, or simplified multimodal architectures. Whether similar geometric principles govern perceptual representations in low-level vision, and how they extend from object and modal- ity comparisons to task-conditioned low-level transformations, remains largely unexplored. Under- standing whether such tasks exhibit a coherent geometric organization is therefore important not only for interpretability but also as a foundation for predicting task transfer, designing modular ar- chitectures, identifying shared computational primitives, and scaling unified restoration models to increasingly diverse perceptual capabilities. Motivated by these observations, this paper investigates the geometric organization of low-level visual task representations in modern VLMs. Specifically, we ask: RQ1. Do low-level visual tasks exhibit meaningful geometric clustering in VLM hidden repre- sentations, and how does this organization evolve across network depth? RQ2. Do low-level task representations exhibit shared and interpretable latent structures within and across task families? RQ3. To what extent is the geometric organization of low-level tasks universal across different VLM architectures? To answer these questions, we introduce GeoSim (Figure 1), a unified four-level framework for char- acterizing the geometry of low-level visual task representations. GeoSim analyzes task-conditioned representations from four complementary perspectives: global similarity structure, local geometric structure, sparse feature decomposition, and topological verification. This unified framework en- ables systematic comparisons across tasks, layers, architectures, and model families, providing a comprehensive view of how perceptual representations are organized within modern VLMs. Our main contributions are summarized as follows: • We present a systematic geometric study of 24 low-level vision tasks across multiple VLM fam- ilies and establish a reusable benchmark with unified prompt templates and evaluation protocols. • We reveal how task geometry evolves across depth and quantify global, local, and cross-model agreement with Inter-task/Intra-task Cosine Similarity, Distance Correlation (dCor), Nearest Neighbor Graph Similarity (NNGS), Local Intrinsic Dimensionality (LID), Procrustes Align- ment Residual (PAR), and topological analyses. 2 Low-Level Vision Tasks Super-ResolutionHDR Reconstruction Reconstruction Lens Flare RemovalReflection Removal Raindrop RemovalShadow Removal Removal ColorizationHarmonization InpaintingLight Enhancement Style TransferEdge Detection Generation/Enhancement Relighting White Balance Correction Contrast Enhancement Photometric Correction Artifact Removal Dehazing Deraining Defocus Deblurring Demoiréing Desnowing Motion Deblurring Denoising Underwater Restoration Restoration GeoSim Framework Level 1: Global Similarity Structure Level 2: Local Geometric Structure Level 3: Sparse Feature Decomposition Level 4: Topological Verification Analysis Representation Image Gallery Pure Visual Representation Specific Task: Derain, Dehaze, Denoise, ...... Input CNN Local Feature Space √ Output Input Specific Task: Derain, Dehaze, Denoise, ...... Output Transformer Context-aware Feature Space √ Mixed Representation Specific Task: Derain, Dehaze, Denoise, ...... TextPrompt: “This is a [task name] task” Input Output VLM Joint Feature Space ? Figure 1: Overview of VLM representations for low-level tasks and the GeoSim framework. • We establish a comprehensive pipeline including Sparse Autoencoder (SAE)-based feature de- composition, Mutual Information (MI), and cross-model Maximum Mean Discrepancy (MMD) to distinguish task-specific and cross-task shared sparse representations. 2RELATED WORK All-in-one image restoration. Image restoration has gradually evolved from task-specific net- works toward unified models capable of handling diverse degradations within a single architec- ture (Jiang et al., 2025a). Recent work leverages VLM representations (Luo et al., 2024), visual instructions with diffusion priors (Luo et al., 2025b), and autoregressive visual token generation (Ra- jagopalan et al., 2026) for restoration. These advances demonstrate that increasingly diverse restora- tion tasks can be handled within a shared representation space, yet the internal organization of the learned representations remains largely unexplored. Beyond restoration, Li et al. (2025) used component-wise tuning to show that distortion recognition in an MLLM depends most strongly on adapting its visual encoder, with further gains from the alignment projector. This component-level evidence motivates our study of how low-level tasks are organized across layers and models. Task transferability and task relationships. Understanding relationships among visual tasks has long been a central topic in computer vision because tasks with similar representations are expected to transfer knowledge more effectively. Taskonomy (Zamir et al., 2018) systematically quantified transfer relationships among a large collection of visual tasks, Task2Vec (Achille et al., 2019) pro- posed compact task embeddings for directly measuring task similarity, and later studies leveraged representation similarity to construct task taxonomies and guide multi-task learning (Dwivedi & Roig, 2019; Standley et al., 2020). Sachdeva et al. (2026) provided a comprehensive study of pos- itive and negative transfer across visual perception tasks in VLMs. Beyond explicit transfer, recent work demonstrated that VLMs can spontaneously develop cross-modal task representations from in-context examples (Luo et al., 2025a) and perform cross-task visual in-context learning through implicit text-driven prompts (Xia et al., 2025). While these studies characterize whether tasks trans- fer knowledge, they provide limited insight into why such transfer occurs. In particular, how trans- ferability is reflected in the geometric organization of hidden representations within modern VLMs remains largely unknown. Geometric representation analysis. Geometric representation analysis seeks to understand neu- ral networks by characterizing the structure of their hidden representations. These techniques have recently been applied to VLMs to study shared image–text embedding spaces (Radford et al., 2021), cross-model representational convergence (Huh et al., 2024), language-space interpretation of visual features (Jiang et al., 2025b), modality-invariant task representations (Luo et al., 2025a), and pro- gressive visual–language alignment (Venhoff et al., 2025). Nevertheless, most existing studies focus on semantic concepts, language alignment, or individual geometric properties. A unified frame- 3 Table 1: Notation used throughout Section 3. SymbolMeaning xInput image n, bSamples per task in each sampling repeat (100); number of sampling repeats (10) i, j, sImage indices (i, j); sampling-repeat index (s) p, q, t∈TTask indices (p, q for pairs, t for single task) ℓ, d, d PCA Layer index (ℓ ∗ : stable layer); feature dimension (d); retained principal-component dimension (d PCA ) h ℓ p (x)∈ R d Representation of image x for task p at layer ℓ H ℓ p ∈ R n×d Task representation matrix [h ℓ p (x 1 ); . . . ; h ℓ p (x n )] M 1 , M 2 Evaluated VLMs fSAE feature index O(d PCA )Group of d PCA × d PCA orthogonal matrices ∥·∥ F Frobenius norm work that systematically integrates global, local, distributional, sparse, and topological analyses for understanding low-level perceptual task representations remains lacking. SAEs for model interpretability. SAEs have emerged as an effective tool for interpreting hidden representations in large neural networks (Huben et al., 2024; Gao et al., 2025; Templeton et al., 2024). Recently, SAE-based analysis has been applied to vision and vision-language models. Uni- versal SAEs learn a single sparse concept space jointly across several vision encoders (Thasarathan et al., 2025), whereas Matryoshka SAEs optimize nested sparsity levels within CLIP represen- tations (Zaigrajew et al., 2025). Pach et al. (2025) further demonstrated that SAEs can recover monosemantic and steerable features from VLM representations. These studies establish SAEs as an effective tool for interpreting individual concepts within VLMs. While in our work, we employ SAEs to characterize shared and task-specific sparse representations across multiple low-level vi- sion tasks and multiple VLM architectures, providing a feature-level perspective on the geometric organization of perceptual task representations. Extended related work is provided in Appendix A. 3PROPOSED FRAMEWORK 3.1FRAMEWORK OVERVIEW We introduce GeoSim, a unified framework for characterizing the geometric organization of low- level visual task representations in VLMs. Rather than relying on a single similarity measure, GeoSim analyzes representations from four complementary perspectives: Level 1 characterizes the global similarity structure among tasks and its evolution across depth, Level 2 examines local geo- metric structure, Level 3 decomposes representations into shared and task-specific sparse features, and Level 4 examines cross-model universality. Notation is summarized in Table 1, and the motiva- tion for each level is detailed in Appendix B.2. 3.2LEVEL 1: GLOBAL SIMILARITY STRUCTURE Inter-Task and Intra-Task similarity. For an input image x, let h ℓ p (x) ∈ R d denote the hidden representation extracted for task p at layer ℓ. Given n sampled images, we stack their representa- tions into H ℓ p = [h ℓ p (x 1 );... ;h ℓ p (x n )] ∈ R n×d . H ℓ p is formed in each sampling repeat s, with s suppressed when unambiguous. We compute the average inter-task cosine similarity between every pair of tasks S ℓ (p,q) = 1 b b X s=1 1 n 2 n X i=1 n X j=1 cos h ℓ p (x s i ),h ℓ q (x s j ) ,(1) where b denotes the number of sampling repeats. Given a single task t, we sample n distinct images. The average intra-task cosine similarity is defined as S ℓ (t) = 1 b b X s=1 1 n 2 n X i=1 n X j=1 cos h ℓ t (x s i ),h ℓ t (x s j ) .(2) 4 Collecting all pairwise similarities produces layer-wise task similarity matrix S ℓ = [S ℓ (p,q)] p,q∈T , which summarizes the global organization of low-level task representations at layer ℓ. Cross-model similarity. Cosine similarity characterizes relationships between tasks within a sin- gle model but does not directly compare the geometric structure of representations across archi- tectures. We therefore additionally employ dCor (Sz ́ ekely et al., 2007), which measures statistical dependence between two representation matrices based on pairwise distance structure. For the same task t, let H ℓ,M 1 t and H ℓ,M 2 t denote the representation matrices extracted from models M 1 and M 2 , respectively. In cross-model expressions, ℓ refers to the selected layer ℓ ∗ M of each model, which need not coincide across models. The overall cross-model statistic is defined as the average dCor over the b sampling repeats: dCor ℓ (M 1 ,M 2 ) = 1 b b X s=1 dCov s p dVar s 1 dVar s 2 .(3) Here, dCov s and dVar s 1 , dVar s 2 denote the distance covariance and distance variances in repeat s, computed from double-centered pairwise Euclidean distances as detailed in Appendix B.3.1. Layer-wise evolution. Representation geometry evolves substantially across transformer layers. To identify where the global organization of low-level tasks stabilizes, we compute the Frobenius distance D ℓ full = ∥S ℓ − S ℓ+1 ∥ F between consecutive task similarity matrices. A large value of D ℓ full indicates substantial reorganization of task relationships between adjacent layers, whereas a consistently small value indicates a stable task similarity structure. We designate the final layer within this stable region as ℓ ∗ and use it for the subsequent analyses (Appendix C.2). Visualization. Node positions in the task relation graph (Figure 5) are obtained by classical multi- dimensional scaling (MDS) (Torgerson, 1952) of the cross-model consensus dissimilarities derived from S ℓ (p,q). 3.3LEVEL 2: LOCAL GEOMETRIC STRUCTURE Local intrinsic dimensionality (LID). We first characterize the local complexity of each task representation space using LID, estimated by maximum likelihood (Levina & Bickel, 2004) from the k nearest neighbors of each sample within the same task and averaged over samples and sampling repeats (Appendix B.3.2). k-nearest-neighbor graph similarity (NNGS). To directly evaluate local neighborhood preserva- tion across models, we construct k-nearest-neighbor graphs over corresponding samples and mea- sure the fraction of shared neighbors. For representation matrices U and V (H ℓ,M 1 t ,H ℓ,M 2 t ) ex- tracted from two models, the neighborhood similarity (Huh et al., 2024) is NNGS(U,V ) = 1 b b X s=1 1 n n X i=1 |N k (x s i ;U )∩ N k (x s i ;V )| k ,(4) where N k (x s i ;U ) denotes the k nearest neighbors of sample i in representation space U . Procrustes alignment residual (PAR). Neighborhood similarity evaluates local topology, but does not capture whether two representation spaces can be globally aligned after removing coor- dinate differences. Therefore, we further measure geometric alignment using orthogonal Procrustes analysis (Sch ̈ onemann, 1966). For task t and models M 1 ,M 2 , we first center each representation matrix and apply principal compo- nent analysis (PCA) separately, retaining d PCA ≤ min(d M 1 ,d M 2 ,n− 1) dimensions. The resulting matrices are denoted as H ℓ,M 1 t ,H ℓ,M 2 t ∈ R n×d PCA . The alignment residual is defined as PAR(t,M 1 ,M 2 ) =min Q∈O(d PCA ) ∥H ℓ,M 1 t − H ℓ,M 2 t Q∥ F .(5) A smaller PAR indicates stronger geometric agreement up to an orthogonal transformation. 3.4LEVEL 3: SPARSE FEATURE DECOMPOSITION SAE representation decomposition. At the stable layer ℓ ∗ , we write z t (x) = h ℓ ∗ t (x) ∈ R d . For each VLM, we train a single ReLU SAE with an ℓ 1 sparsity penalty on z t (x), shared across all 5 low-level vision tasks, so that feature usage can be compared within a common learned dictionary; a(x) ∈ R D SAE denotes its sparse code and D SAE the dictionary size (Appendix B.3.3; training details in Appendix C.3). The SAE is trained on task-balanced samples, and all feature statistics are computed on held-out samples. Feature activation statistics. For SAE feature f , we define its activation rate for task t as ρ f (t) = 1 N t X x∈D t 1[a f (x) > τ ],(6) where D t is the held-out sample set for task t, N t = |D t |, a f (x) denotes the activation value of feature f , and τ is an activation threshold. The activation rate ρ f (t) measures how frequently a feature is used by a given task. Task selectivity index (TSI). We quantify whether a feature is concentrated on a small number of tasks using TSI: TSI(f ) = max t∈T ρ f (t) P t∈T ρ f (t) .(7) A high TSI(f ) indicates that feature f is preferentially activated by a particular task, whereas a lower value indicates broader usage across tasks. Here, ρ f (t) is the activation probability defined in Eq. (6). Features are categorized as task-specific or shared according to predefined thresholds θ high and θ low . Feature–task mutual information (MI). To quantify whether individual SAE features encode task identity, we measure the MI between feature activation and task labels. For each SAE feature f , we define its binary activation state as Z f = 1[a f (x) > τ ], where τ is the activation threshold, Z f ∈0, 1. The MI between feature activation and task identity is computed as I(Z f ;T ) = X z∈0,1 X t∈T P (z,t) log P (z,t) P (z)P (t) ,(8) where P (z,t) is estimated from held-out samples across all tasks. Higher values indicate that a feature activation pattern carries more information about task identity, suggesting stronger task se- lectivity. Cross-model sparse representation comparison. To evaluate whether different VLMs develop similar sparse representations, we compare feature activation-rate distributions across models. For a given task t, let A M 1 t and A M 2 t denote the empirical distributions of the activation rates ρ f (t) obtained from the SAEs of models M 1 and M 2 , respectively. Each feature active on at least one held- out task contributes one scalar rate, while features inactive across all held-out tasks are excluded. We measure their distributional similarity using the MMD (Gretton et al., 2012) with a radial basis function (RBF) kernel κ: MMD 2 (A M 1 t ,A M 2 t ) = E A M 1 t φ− E A M 2 t φ 2 H κ ,(9) where φ denotes the feature mapping induced by kernel κ, E A M 1 t φ, E A M 2 t φ are shorthand for E ρ∼A M 1 t [φ(ρ)], E ρ∼A M 2 t [φ(ρ)], with ρ ∈ [0, 1] denoting a feature activation rate. We report the squared MMD between these empirical distributions. Lower values indicate more similar task- conditioned activation-rate distributions, without establishing correspondence between individual SAE features across models. 3.5LEVEL 4: TOPOLOGICAL VERIFICATION For each task t, we project the stable-layer representations H ℓ ∗ t to a low-dimensional point cloud e H t ⊂ R d PH with a fixed UMAP (McInnes et al., 2018) configuration (d PH ≪ d), which mitigates distance concentration in the ambient space. Using persistent homology (PH), we summarize the birth and death of connected components and one-dimensional loops (homology degreesH 0 andH 1 ) in persistence diagrams, and compare models on the same task with the 2-Wasserstein and bottleneck distances between their diagrams. Level 4 serves as a supplementary cross-check; construction details and results are given in Appendices B.3.4 and D.2. 6 Table 2: Low-level vision task suite organized by task family. FamilyTasks RestorationArtifact Removal, Defocus Deblurring, Motion Deblurring, Dehazing, Demoir ́ eing, Denoising, Deraining, Desnowing, Underwater Restoration RemovalLens Flare Removal, Raindrop Removal, Reflection Removal, Shadow Removal Generation/EnhancementColorization, Harmonization, Inpainting, Light Enhancement, Style Transfer, Edge Detection ReconstructionSuper-Resolution, HDR Reconstruction Photometric CorrectionRelighting, White Balance Correction, Contrast Enhancement 4EXPERIMENTS 4.1EXPERIMENTAL SETUP Datasets. We construct a benchmark suite containing 24 low-level vision tasks grouped into five perceptual families: restoration, removal, generation/enhancement, reconstruction, and photometric correction (see Table 2). For tasks such as artifact removal, colorization, and inpainting, we con- struct datasets by synthesizing degraded inputs from clean images. Dataset sources and construction details are provided in Appendix C. For each task, we sample n = 100 distinct input images from representative low-level vision datasets in each of b = 10 fixed sampling repeats. To minimize the influence of instruction variation, we use a fixed prompt template for each task and keep the prompt format consistent across models. Implementation Details. We analyze unified VLMs spanning visual understanding and image generation across multiple architectural families and parameter scales (see Table 3). All experiments were conducted using eight NVIDIA RTX 6000 GPUs, each with 48 GB of memory, and eight NVIDIA B200 GPUs, each with 180 GB of memory. 4.2RESULTS Our main results are organized according to the four task/model conditions, with the metrics in Appendix Table 4. 4.2.1CONDITION I: SAME TASK, SAME MODEL Table 3: VLMs used in our analyses. ScaleModels Base (∼7–8B) Emu3-Chat (8B) (Wang et al., 2024; 2026), Anole-7B (Chern et al., 2024), Janus-Pro-7B (Chen et al., 2025), InternVL3.5-8B (Wang et al., 2025), Qwen3-VL-8B (Bai et al., 2025) Large (>13B) Qwen-Image-Edit (∼20B) (Wu et al., 2025), Emu3.5-Image (∼34B) (Cui et al., 2025) Condition I fixes the task and model while varying the input image. As shown in Figure 2, all seven models exhibit aligned instruction-induced displacements, with mean cosine similarities of 0.80–0.96 at the selected analysis layer. Despite architectural differences, these transformations are stable across images at that layer, while their consistency can vary across depth. Appendix Figure 9 shows that local geometric complexity is task-dependent. The 24 tasks follow a consistent ordering across models (ρ = 0.80), with Denoising having the lowest LID in every model. Each task exhibits a stable global direction and a characteristic local manifold complexity. Key Takeaway I Low-level visual tasks exhibit stable within-task geometry, with aligned instruction-induced transformations across images and consistently organized local task manifolds across models. 4.2.2CONDITION I: SAME TASK, DIFFERENT MODELS Condition I compares the same task across different models to assess cross-model geometric consistency. As shown in Figure 4, the 21 model pairs exhibit strong global agreement (mean dCor = 0.77), but weaker local consistency (mean chance-adjusted NNGS = 0.27). The held-out, normalized PAR exceeds the mean-prediction baseline of 1.0 for 16 of 21 pairs. Overall, models share global relational geometry but not local neighborhoods or a common coordinate frame. Level-3 SAE evidence in Figure 3 further narrows the scope of this cross-model agreement by demonstrating that same-task cross-model activation-rate MMD is lower than the matched within- 7 Emu3 0.00.51.0 0.5 1.0 Anole 0.00.51.0 Janus-Pro 0.00.51.0 InternVL3.5 0.00.51.0 0.5 1.0 Qwen3-VL 0.00.51.0 Qwen-Image-Edit 0.00.51.0 Emu3.5 0.00.51.0 Average Intra-task Cosine Similarity Relative Depth RestorationRemovalGeneration/EnhancementReconstructionPhotometric Correction Figure 2: Condition I: Consistency of the instruction-induced displacement across network depth. Each panel is one model; each of the 24 curves is one task, colored by perceptual family. Anole vs. Emu3 Anole vs. Emu3.5 Anole vs. InternVL3.5 Anole vs. Janus-Pro Anole vs. Qwen-Image-Edit Anole vs. Qwen3-VL Emu3 vs. Emu3.5 Emu3 vs. InternVL3.5 Emu3 vs. Janus-Pro Emu3 vs. Qwen-Image-Edit Emu3 vs. Qwen3-VL Emu3.5 vs. InternVL3.5 Emu3.5 vs. Janus-Pro Emu3.5 vs. Qwen-Image-Edit Emu3.5 vs. Qwen3-VL InternVL3.5 vs. Janus-Pro InternVL3.5 vs. Qwen-Image-Edit InternVL3.5 vs. Qwen3-VL Janus-Pro vs. Qwen-Image-Edit Janus-Pro vs. Qwen3-VL Qwen-Image-Edit vs. Qwen3-VL 0.0050.010.02 0.88× 0.68× 0.76× 0.80× 0.71× 0.90× Task-conditioned MMD RBF-MMD² Anole Emu3 Emu3.5 InternVL3.5 Janus-Pro Qwen-Image-Edit Qwen3-VL 0.000.250.500.751.00 TSI 0.474 TSI 0.263 TSI 0.493 TSI 0.386 TSI 0.347 TSI 0.439 TSI 0.546 Feature selectivity Fraction of live SAE features 0.00500.00750.0100 0.0057 0.0088 0.0039 0.0072 0.0082 0.0068 0.0052 Task information Mean I(Z f ; T) Same taskReferenceAll 3 seeds Aggregate sharedintermediatetask-specific Figure 3: Condition I: Task-conditioned cross-model and within-model reference MMD (left), per-model TSI composition (center), and feature–task MI (right). model, different-task reference for only 6 of 21 model pairs when aggregated over 24 tasks and three SAE seeds, with only Anole–Qwen3-VL, Emu3–Janus-Pro, and Emu3.5–Qwen3-VL satisfying this relation across all three seeds. Moreover, model pairs with high global agreement do not generally show lower same-task MMD than the different-task reference. Across models, task-specific features account for only 0.9–15.1% of live features, while the shared-feature fraction varies from 13.4% to 72.4%, indicating that sparse features are broadly reused across tasks within each model. Key Takeaway I Cross-model universality is primarily global in that models preserve within-task relational ge- ometry despite limited local-neighborhood agreement and the absence of a shared coordinate frame. Level-3 SAE evidence shows the same boundary at the activation-rate distribution level, where 6 of 21 model pairs exhibit lower same-task MMD than the different-task reference. 8 0.00 0.25 0.50 0.75 dCor 0.0 0.2 0.4 NNGS (k = 10) 0.0 0.5 1.0 PAR 0.00 0.05 0.10 0.15 r - W a s s e r s t e i n ( H 1 ) Emu3-Janus Emu3-Emu3.5 Emu3-Q3-VL Anole-Janus Anole-Emu3 Janus-Q3-VL Emu3.5-Janus Emu3.5-QIE Emu3-QIE Emu3-IVL3.5 Anole-Emu3.5 Anole-Q3-VL IVL3.5-Janus Emu3.5-Q3-VL IVL3.5-Q3-VL Janus-QIE Anole-IVL3.5 QIE-Q3-VL Emu3.5-IVL3.5 Anole-QIE IVL3.5-QIE Figure 4: Condition I: All 21 model pairs, averaged over 24 tasks and ordered by dCor. Restoration Removal Generation/Enhancement Reconstruction Photometric Correction Artifact Removal Defocus Deblurring Motion Deblurring Dehazing Denoising Deraining Desnowing Underwater Restoration Raindrop Removal Reflection Removal Shadow Removal Colorization Harmonization Inpainting Light Enhancement Style Transfer Edge Detection HDR Reconstruction Relighting White Balance Correction Contrast Enhancement Demoiréing Lens Flare Removal Super-Resolution Figure 5: Condition I: Consensus relation graph over the 24 tasks. Node positions are visualized with MDS; edge width increases with consensus similarity. 4.2.3CONDITION I: DIFFERENT TASKS, SAME MODEL Condition I examines how low-level visual tasks are organized within a shared model represen- tation. The consensus graph in Figure 5 identifies 36 reliable task relations, including 20 cross- family links. Removal and Photometric Correction tasks form the most densely connected region, with Contrast Enhancement the most connected task, while Reconstruction lies nearby and the nine Restoration tasks spread more thinly across the graph. In contrast, most Generation/Enhancement tasks (e.g., Colorization, Harmonization, and Style Transfer) are more isolated from the main cluster. The heatmaps in Figure 12 (Appendix) show reproducible task structure across models, but the within- versus across-family gap is only 0.135 standard deviations (SD), below the typical cross- model disagreement of 0.71-SD. Thus, task geometry is better explained by shared visual mecha- nisms than by broad perceptual families. Key Takeaway I Cross-task geometry is meaningful but mechanism-centric, with tasks sharing degradation or restoration operations forming the strongest neighborhoods, whereas the predefined perceptual families only weakly capture the representation structure. 4.2.4CONDITION IV: DIFFERENT TASKS, DIFFERENT MODELS Condition IV explores whether feature activation-rate distributions are similar across tasks and mod- els through an exhaustive Level-3 MMD sweep. In Figure 6, the left panel summarizes reciprocal percentile-rank agreement across 21 model pairs, while the right panel presents the 15 most con- sistently close task pairs using mean within-model-pair percentile and top-decile prevalence. Since raw MMD values are not comparable across SAE model pairs, all results are rank-normalized within each model pair before aggregation. Same-family task pairs also rank 3.1 percentile points worse than cross-family pairs on average, with the same trend in 16 of 21 model pairs. Key Takeaway IV Despite the difficulty of cross-task, cross-model comparison, a small set of specific task/model combinations still exhibits consistently low activation-rate MMD under the joint screening cri- teria. These similarities are pair-specific, and same-family task pairs are not closer on average. 9 1 3 57 9 11 13 1517 192123 01 Artifact Removal 02 Defocus Deblurring 03 Motion Deblurring 04 Dehazing 06 Denoising 07 Deraining 08 Desnowing 09 Underwater Restoration 11 Raindrop Removal 12 Reflection Removal 13 Shadow Removal 14 Colorization 15 Edge Detection 16 Harmonization 17 Inpainting 18 Light Enhancement 19 Style Transfer 20 HDR Reconstruction 22 Contrast Enhancement 23 Relighting 24 White Balance Correction Task index Cross-model consensus 0 5 10 15 20 Dehazing vs. Motion Deblurring Denoising vs. Style Transfer Motion Deblurring vs. Style Transfer Reflection Removal vs. Relighting Motion Deblurring vs. Shadow Removal Denoising vs. Relighting Raindrop Removal vs. Style Transfer Denoising vs. Shadow Removal Denoising vs. Reflection Removal Denoising vs. Motion Deblurring Dehazing vs. Denoising Dehazing vs. Raindrop Removal Denoising vs. Raindrop Removal Motion Deblurring vs. Raindrop Removal Raindrop Removal vs. Shadow Removal Mean rank percentile Closest task pairs 020406080100 Top-10% prevalence (%) Cross-familySame-familyTop-10% prevalence Mean rank percentile (%) 0255075100 05 Demoiréing 10 Lens Flare Removal 21 Super-Resolution Figure 6: Condition IV: Cross-task, cross-model activation-rate distribution similarity from the Level-3 sweep. Results are aggregated using MMD across seven models and 24 tasks. 4.3DISCUSSION Our results reveal meaningful geometric organization of low-level visual tasks (RQ1). Within each model, task instructions induce consistent global displacement directions across images, while LID follows a similar task-dependent ordering across architectures. This organization is also inter- pretable and partly shared at the feature level (RQ2): sparse features are broadly reused across tasks within each model, and the inter-task geometry is better explained by shared degradation and trans- formation mechanisms than by the predefined perceptual families. Overall, global similarity and local geometry provide complementary views of a mechanism-centric task structure. Cross-model universality is substantial but limited (RQ3). Different VLMs preserve similar global distance re- lationships for the same task, but exhibit weaker agreement in local neighborhoods and coordinate alignment. This suggests that universality emerges mainly through shared relational organization rather than pointwise feature correspondence. Sparse feature analysis may reveal recurring visual primitives, but feature-level sharing should be established through explicit cross-model validation. Limitations. Our study has several limitations: (i) analyzing a single selected layer per model may overlook layer-specific geometric and sparse-feature dynamics; (i) standardized prompts together with a transformer-dominated model set may limit generalization to more diverse instructions and architectures; (i) task comparisons may also reflect dataset differences, since different tasks draw on different source datasets; and (iv) projection-based topology and cross-model SAE findings re- quire further validation across layers, methods, and model pairs. 5CONCLUSION In this study, we present a comprehensive geometric analysis of how VLMs represent low-level visual tasks and the relations among cross-task representations. We propose GeoSim, a four-level analysis framework comprising global similarity, local geometry, sparse feature decomposition, and topological verification. This work establishes a benchmark for future VLM research on represen- tation interpretability, transfer learning, and representation alignment across low-level tasks. More importantly, our analysis provides actionable insights into how low-level task representations are or- ganized in VLMs, offering guidance for the development of VLMs in all-in-one image restoration. REFERENCES Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. A high-quality denoising dataset for smartphone cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern 10 Recognition, p. 1692–1700, 2018. Abdullah Abuolaim and Michael S. Brown. Defocus deblurring using dual-pixel data. In Proceed- ings of the European Conference on Computer Vision, p. 111–126, 2020. Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Char- less C. Fowlkes, Stefano Soatto, and Pietro Perona. Task2Vec: Task embedding for meta-learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 6430–6439, 2019. Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown. When color constancy goes wrong: Correcting improperly white-balanced images. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, p. 1535–1544, 2019. Eirikur Agustsson and Radu Timofte. NTIRE 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, p. 1122–1131, 2017. Yuang Ai, Huaibo Huang, Xiaoqiang Zhou, Jiexiang Wang, and Ran He. Multimodal prompt perceiver: Empower adaptiveness generalizability and fidelity for all-in-one image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 25432–25444, 2024. Cosmin Ancuti, Codruta O. Ancuti, and Christophe De Vleeschouwer. D-HAZY: A dataset to evaluate quantitatively dehazing algorithms. In Proceedings of the IEEE International Conference on Image Processing, p. 2226–2230, 2016. Pablo Arbel ́ aez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hier- archical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):898–916, 2011. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631, 2025. Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P. Calmon, and Himabindu Lakkaraju. Inter- preting CLIP with sparse linear concept embeddings (SpLiCE). In Advances in Neural Informa- tion Processing Systems, volume 37, p. 84298–84328, 2024. Jianrui Cai, Shuhang Gu, and Lei Zhang. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing, 27(4):2049–2062, 2018. Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Sixiang Chen, Tian Ye, Renjing Pei, Kaiwen Zhou, Fenglong Song, and Lei Zhu. RestoreAgent: Autonomous image restoration agent via multimodal large language models. In Advances in Neural Information Processing Systems, volume 37, p. 110643–110666, 2024a. Tianwei Chen, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, and Hajime Nagahara. Learning more may not be better: Knowledge transferability in vision-and-language tasks. Jour- nal of Imaging, 10(12):300, 2024b. Xiangyu Chen, Zhengwen Zhang, Jimmy S. Ren, Lynhoo Tian, Yu Qiao, and Chao Dong. A new journey from SDRTV to HDRTV. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4500–4509, 2021. 11 Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-Pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811, 2025. Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. ANOLE: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06135, 2024. David Cohen-Steiner, Herbert Edelsbrunner, and John Harer. Stability of persistence diagrams. Discrete & Computational Geometry, 37(1):103–120, 2007. David Cohen-Steiner, Herbert Edelsbrunner, John Harer, and Yuriy Mileyko. Lipschitz functions have L p -stable persistence. Foundations of Computational Mathematics, 10(2):127–139, 2010. Marcos V. Conde, Gregor Geigle, and Radu Timofte. InstructIR: High-quality image restoration following human instructions. In Proceedings of the European Conference on Computer Vision, p. 1–21, 2024. Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang. DoveNet: Deep image harmonization via domain verification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8391–8400, 2020. Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, Xinghang Li, Jirong Liu, Yang Liu, Zhuoyan Luo, Jinsheng Wang, Wenxuan Wang, Yueze Wang, Chengyuan Wang, Fan Zhang, Yingli Zhao, Ting Pan, Xianduo Li, Zecheng Hao, Wenxuan Ma, Zhuo Chen, Yulong Ao, Tiejun Huang, Zhongyuan Wang, and Xinlong Wang. Emu3.5: Native multimodal models are world learners. arXiv preprint arXiv:2510.26583, 2025. Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yihang Luo, and Chen Change Loy. Flare7K++: Mixing synthetic and real datasets for nighttime flare removal and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(11):7041–7055, 2024. Kshitij Dwivedi and Gemma Roig. Representation similarity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 12387–12396, 2019. Majed El Helou, Ruofan Zhou, Johan Barthas, and Sabine S ̈ usstrunk. VIDIT: Virtual image dataset for illumination transfer. arXiv preprint arXiv:2005.05460, 2020a. Majed El Helou, Ruofan Zhou, Sabine S ̈ usstrunk, Radu Timofte, et al. AIM 2020: Scene relighting and illumination estimation challenge. In Proceedings of the European Conference on Computer Vision Workshops, p. 499–518, 2020b. Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Removing rain from single images via a deep detail network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1715–1723, 2017. Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In International Confer- ence on Learning Representations, volume 2025, p. 26721–26754, 2025. Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch ̈ olkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012. Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar, Julius Tarng, Adam Pearce, Chris Olah, and Joshua Batson. When models manipulate manifolds: The geometry of a counting task. arXiv preprint arXiv:2601.04480, 2026. Robert Huben, Hoagy Cunningham, Logan Smith, Aidan Ewart, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. In International Conference on Learning Representations, volume 2024, p. 7827–7845, 2024. 12 Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The Platonic represen- tation hypothesis. In Proceedings of the International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, p. 20617–20642. PMLR, 2024. Dat Huynh, Jason Kuen, Zhe Lin, Jiuxiang Gu, and Ehsan Elhamifar. Open-vocabulary instance seg- mentation via robust cross-modal pseudo-labeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7020–7031, 2022. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1125–1134, 2017. Rishi Jha, Collin Zhang, Vitaly Shmatikov, and John Morris. Harnessing the universal geometry of embeddings. In Advances in Neural Information Processing Systems, volume 38, p. 45963– 45987, 2025. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Proceedings of the International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, p. 4904–4916. PMLR, 2021. Junjun Jiang, Zengyuan Zuo, Gang Wu, Kui Jiang, and Xianming Liu. A survey on all-in-one image restoration: Taxonomy, evaluation and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(12):11892–11911, 2025a. Nick Jiang, Anish Kachinthaya, Suzanne Petryk, and Yossi Gandelsman. Interpreting and editing vision-language representations to mitigate hallucinations. In International Conference on Learn- ing Representations, volume 2025, p. 63582–63605, 2025b. Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, and Jinwei Gu. AutoDIR: Automatic all-in-one image restoration with latent diffusion. In Proceedings of the European Conference on Computer Vision, p. 340–359, 2024. Yeying Jin, Xin Li, Jiadong Wang, Yan Zhang, and Malu Zhang. Raindrop clarity: A dual-focused dataset for day and night raindrop removal. In Proceedings of the European Conference on Com- puter Vision, p. 1–17, 2024. Abhinav Joshi, Divyanshu Bhatt, and Ashutosh Modi. Geometry of decision making in language models. In Advances in Neural Information Processing Systems, volume 38, p. 144139–144196, 2025. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Proceedings of the International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, p. 3519–3529. PMLR, 2019. Alexey Kravets, Da Li, Chuan Li, Da Chen, and Vinay P. Namboodiri. Interpretability transfer from language to vision via sparse autoencoders. In Proceedings of the International Conference on Machine Learning, 2026. Nikolaus Kriegeskorte, Marieke Mur, and Peter A. Bandettini. Representational similarity analysis — connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2:4, 2008. Akshay Kulkarni, Tsui-Wei Weng, Vivek Narayanaswamy, Shusen Liu, Wesam A. Sakla, and Kow- shik Thopalli. Interpretable and steerable concept bottleneck sparse autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 2918–2927, 2026. Pierre-Yves Laffont, Zhile Ren, Xiaofeng Tao, Chao Qian, and James Hays. Transient attributes for high-level understanding and editing of outdoor scenes. ACM Transactions on Graphics, 33(4): 149:1–149:11, 2014. 13 Sergio Lanza, Jae Hee Lee, and Stefan Wermter. Extraction and analysis of multimodal concepts in vision language models through sparse autoencoders. In Proceedings of the International Conference on Artificial Neural Networks, p. 312–323, 2026. Elizaveta Levina and Peter J. Bickel. Maximum likelihood estimation of intrinsic dimension. In Advances in Neural Information Processing Systems, volume 17, 2004. Boyi Li, Kilian Q. Weinberger, Serge Belongie, Vladlen Koltun, and Ren ́ e Ranftl. Language-driven semantic segmentation. In International Conference on Learning Representations, 2022a. Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restora- tion for unknown corruption. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 17452–17462, 2022b. Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29:4376–4389, 2020. Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, p. 10965–10975, 2022c. Yuan Li, Zitang Sun, Yen-Ju Chen, and Shin’ya Nishida. Investigate the low-level visual perception in vision-language based image quality assessment. arXiv preprint arXiv:2512.09573, 2025. Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selec- tive remapping of visual concepts during adaptation. In International Conference on Learning Representations, volume 2025, p. 24444–24469, 2025. Baihan Lin and Nikolaus Kriegeskorte. The topology and geometry of neural representations. Pro- ceedings of the National Academy of Sciences, 121(42):e2317881121, 2024. Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, p. 5334–5342, 2025. Jiaying Liu, Dong Liu, Wenhan Yang, Sifeng Xia, Xiaoshuai Zhang, and Yuanying Dai. A compre- hensive benchmark for single image compression artifact reduction. IEEE Transactions on Image Processing, 29:7845–7860, 2020. Yun-Fu Liu, Da-Wei Jaw, Shih-Chia Huang, and Jenq-Neng Hwang. DesnowNet: Context-aware deep network for snow removal. IEEE Transactions on Image Processing, 27(6):3064–3073, 2018. Grace Luo, Trevor Darrell, and Amir Bar. Vision-language models create cross-modal task repre- sentations. In Proceedings of the International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, p. 41157–41176. PMLR, 2025a. Wenyang Luo, Haina Qin, Zewen Chen, Libin Wang, Dandan Zheng, Yuming Li, Yufan Liu, Bing Li, and Weiming Hu. Visual-instructed degradation diffusion for all-in-one image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 12764–12777, 2025b. Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sj ̈ olund, and Thomas Sch ̈ on. Controlling vision-language models for multi-task image restoration. In International Conference on Learn- ing Representations, volume 2024, p. 16226–16246, 2024. Yanbiao Ma, Fei Luo, Linfeng Zhang, Chuangxin Zhao, Mingxuan Wang, Yinan Wu, Zhe Qian, Yang Lu, Long Chen, Zhao Cao, Xiaoshuai Hao, Ji-Rong Wen, and Jungong Han. Reason- ing emerges from constrained inference manifolds in large language models. arXiv preprint arXiv:2605.08142, 2026. 14 Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In Conference on Language Modeling, 2024. Leland McInnes, John Healy, and James Melville. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018. Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 3883–3891, 2017. Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. Towards inter- preting visual information processing in vision-language models. In International Conference on Learning Representations, volume 2025, p. 57172–57189, 2025. Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. Sparse autoencoders learn monosemantic features in vision-language models. In Advances in Neural Information Processing Systems, volume 38, p. 95706–95742, 2025. Arghya Pal and Vineeth N. Balasubramanian. Zero-shot task transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 2189–2198, 2019. Vaishnav Potlapalli, Syed Waqas Zamir, Salman Khan, and Fahad Shahbaz Khan. PromptIR: Prompting for all-in-one image restoration. In Advances in Neural Information Processing Sys- tems, volume 36, p. 71275–71293, 2023. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the International Conference on Machine Learning, volume 139 of Proceedings of Ma- chine Learning Research, p. 8748–8763. PMLR, 2021. Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems, volume 30, 2017. Sudarshan Rajagopalan, Kartik Narayan, and Vishal M. Patel. RestoreVAR: Visual autoregressive generation for all-in-one image restoration. In International Conference on Learning Represen- tations, 2026. Yusung Ro, Jaehyun Choi, and Junmo Kim. Beyond semantics: Disentangling information scope in sparse autoencoders for CLIP. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 3252–3261, 2026. Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. Effects of degradations on deep neural network architectures. arXiv preprint arXiv:1807.10108, 2018. Bhuvan Sachdeva, Karan Uppal, Abhinav Java, and Vineeth N. Balasubramanian. Understanding task transfer in vision-language models. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, p. 28754–28763, 2026. Peter H. Sch ̈ onemann. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 31(1):1–10, 1966. Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models. In Findings of the Conference on Empirical Methods in Natural Language Processing, p. 1690– 1712, 2025. Huizhen Shu, Xuying Li, Hongxu Lin, Wenjie Sun, and Hui Li. Steering vision-language models with joint sparse autoencoders. arXiv preprint arXiv:2606.25657, 2026. Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from RGBD images. In Proceedings of the European Conference on Computer Vision, p. 746–760, 2012. 15 Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In Proceedings of the International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, p. 9120–9132. PMLR, 2020. G ́ abor J. Sz ́ ekely, Maria L. Rizzo, and Nail K. Bakirov. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 35(6):2769–2794, 2007. Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monose- manticity: Extracting interpretable features from Claude 3 Sonnet. Transformer Circuits Thread, 2024. Harrish Thasarathan, Julian Forsyth, Thomas Fel, Matthew Kowal, and Konstantinos G. Derpanis. Universal sparse autoencoders: Interpretable cross-model concept alignment. In Proceedings of the International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, p. 59304–59325. PMLR, 2025. Anthony Tiong, Junqi Zhao, Boyang Li, Junnan Li, Steven Hoi, and Caiming Xiong. What are we measuring when we evaluate large vision-language models? an analysis of latent factors and biases. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 3427–3454, 2024. Warren S. Torgerson. Multidimensional scaling: I. theory and method. Psychometrika, 17(4):401– 419, 1952. Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M. Patel. TransWeather: Transformer-based restoration of images degraded by adverse weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 2353–2363, 2022. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(86):2579–2605, 2008. Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, and Neel Nanda. How visual representations map to language feature space in multimodal LLMs. In Explainable AI for Com- puter Vision Workshop at the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. Renjie Wan, Boxin Shi, Ling-Yu Duan, Ah-Hwee Tan, and Alex C. Kot. Benchmarking single- image reflection removal algorithms. In Proceedings of the IEEE International Conference on Computer Vision, p. 3942–3950, 2017. Jifeng Wang, Xiang Li, and Jian Yang. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1788–1797, 2018. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. InternVL3.5: Advancing open-source multimodal models in versa- tility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 16 Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tiejun Huang, and Zhongyuan Wang. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. Xinlong Wang, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Zhen Li, Yuqi Wang, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Chunlei Men, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Zhongyuan Wang, and Tiejun Huang. Multimodal learning with next- token prediction for large multimodal models. Nature, 650(8101):327–333, 2026. Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep Retinex decomposition for low- light enhancement. In Proceedings of the British Machine Vision Conference, 2018. Alex H. Williams, Erin Kunz, Simon Kornblith, and Scott W. Linderman. Generalized shape metrics on neural representations. In Advances in Neural Information Processing Systems, volume 34, p. 4738–4750, 2021. Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-Image technical report. arXiv preprint arXiv:2508.02324, 2025. Shao-Jun Xia, Huixin Zhang, and Zhengzhong Tu. T2T-VICL: Cross-task visual in-context learning via implicit text-driven VLMs. arXiv preprint arXiv:2511.16107, 2025. Xin Yu, Peng Dai, Wenbo Li, Lan Ma, Jiajun Shen, Jia Li, and Xiaojuan Qi. Towards efficient and scale-robust ultra-high-definition image demoir ́ eing. In Proceedings of the European Conference on Computer Vision, p. 646–662, 2022. Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting CLIP with hierarchi- cal sparse autoencoders. In Proceedings of the International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, p. 73918–73956. PMLR, 2025. Amir R Zamir, Alexander Sax, William B Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, p. 3712–3722, 2018. Jinghao Zhang, Jie Huang, Mingde Yao, Zizheng Yang, Hu Yu, Man Zhou, and Feng Zhao. Ingredient-oriented multi-degradation learning for image restoration.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 5825–5835, 2023. Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5625–5644, 2024. Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, and Jianfeng Gao. RegionCLIP: Region-based language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 16793–16803, 2022. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ADE20K dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 5122–5130, 2017. Afra Zomorodian and Gunnar Carlsson. Computing persistent homology. Discrete & Computational Geometry, 33(2):249–274, 2005. 17 AEXTENDED RELATED WORK A.1IMAGE RESTORATION AND TASK CONDITIONING Early approaches primarily relied on shared CNN- or Transformer-based representations to model multiple degradation types, including AirNet (Li et al., 2022b) and TransWeather (Valanarasu et al., 2022). IDR (Zhang et al., 2023) introduced an ingredient-oriented formulation that explicitly dis- entangles degradation factors into reusable restoration components. Subsequent methods further improved task conditioning by incorporating prompts or natural-language instructions. Promp- tIR (Potlapalli et al., 2023) employs learnable prompts to adapt a unified restoration backbone to different degradations, while InstructIR (Conde et al., 2024) replaces task-specific embeddings with human-readable language instructions. A.2TASK TRANSFERABILITY Taskonomy was subsequently extended to zero-shot task transfer (Pal & Balasubramanian, 2019). Transferability has also been studied for VLMs: Chen et al. (2024b) investigated knowledge trans- ferability across multimodal tasks, while Tiong et al. (2024) analyzed latent factors underlying VLM benchmarks and showed that benchmark performance may reflect evaluation biases rather than gen- uine task competence. A.3GEOMETRIC REPRESENTATION ANALYSIS Geometric representation analysis seeks to understand neural networks by characterizing the struc- ture of their hidden representations. Beyond the analyses discussed in Section 2, existing geometric analysis methods can be broadly divided into four categories: (1) global similarity methods, in- cluding RSA (Kriegeskorte et al., 2008), CKA (Kornblith et al., 2019), and dCor (Sz ́ ekely et al., 2007), quantify representational similarity between activation spaces; (2) alignment methods, such as Procrustes analysis (Williams et al., 2021), explicitly compare geometric structures across mod- els; (3) manifold-based analyses estimate intrinsic geometric properties through measures such as LID (Levina & Bickel, 2004); and (4) distributional approaches, including MMD (Gretton et al., 2012), compare activation distributions. Topological RSA (Lin & Kriegeskorte, 2024) further pro- vides a controlled continuum between complete distance geometry and neighborhood structure. A.4SPARSE AUTOENCODERS ACROSS DOMAINS Early work showed that SAEs can recover approximately monosemantic features from language- model activations (Huben et al., 2024). Subsequent studies established scalable training and evalua- tion protocols (Gao et al., 2025) and demonstrated that learned sparse features can support interpre- tation and causal steering (Templeton et al., 2024). More recently, SAE-based analysis has been extended to vision and vision-language models. SpLiCE (Bhalla et al., 2024) decomposes CLIP embeddings into sparse language-grounded con- cepts, while PatchSAE (Lim et al., 2025) applies SAEs to patch-level visual activations to identify localized concepts and studies how prompt adaptation remaps existing visual concepts. Later work showed that interpretable SAE features are not necessarily steerable (Kulkarni et al., 2026), and that semantic labels alone do not reveal whether a visual feature uses local or global evidence (Ro et al., 2026). Other recent studies also extend sparse representations across modalities by transferring language-side interpretations to vision encoders (Kravets et al., 2026), extracting unified visual, linguistic, and multimodal concepts within a single VLM (Lanza et al., 2026), and jointly learning visual and textual sparse features for behavioral steering (Shu et al., 2026). BADDITIONAL FRAMEWORK DETAILS B.1OVERVIEW OF THE ANALYSIS LEVELS GeoSim analyzes representations from four complementary perspectives that progressively reveal different aspects of their organization: 1 1. Global Similarity Structure: How are different low-level visual tasks organized in repre- sentation space, and how does this organization evolve across network depth? 2. Local Geometric Structure: Do tasks with similar global representations also preserve comparable neighborhood structures and manifold properties? 3. Sparse Feature Decomposition: Which latent features are shared across tasks, and which features encode task-specific information? 4. Cross-model Universality: To what extent do different VLM architectures develop similar geometric structures for the same perceptual tasks? Together, these four levels provide a hierarchical characterization of low-level visual representations, progressing from task-level organization to local geometry, feature-level mechanisms, and cross- model consistency. The notation used throughout the framework is summarized in Table 1. B.2MOTIVATION AND COMPLEMENTARITY OF THE ANALYSIS LEVELS B.2.1GLOBAL AND LOCAL GEOMETRY Level 1 investigates the global organization of low-level task representations. We first quantify the similarity between different tasks within each VLM to determine whether perceptually related tasks form coherent representation clusters. We then evaluate whether the resulting global structures are consistent across different VLM architectures and how they evolve throughout the network. Level 1 characterizes the global relationships among low-level tasks but cannot determine whether these similarities are preserved locally. Level 2 therefore examines neighborhood structure and manifold geometry to distinguish genuine geometric correspondence from global averaging effects. While Level 1 captures global task organization through aggregate similarity, it cannot show whether these relationships hold at local scales. Level 2 addresses this gap by examining three complemen- tary properties of representation spaces, namely local manifold complexity, neighborhood consis- tency, and coordinate-invariant geometric alignment. Together, these measures reveal whether global task similarity reflects genuine structural correspondence or merely high-dimensional averaging. Level 2 complements the global similarity analysis in Level 1 by testing whether task relationships persist at finer geometric scales. High global similarity accompanied by consistent neighborhood structure and alignment provides stronger evidence of shared representation geometry, whereas dis- crepancies indicate that global correlations may obscure local differences. B.2.2SPARSE FEATURES AND TOPOLOGICAL VERIFICATION Levels 1–2 characterize representation geometry through continuous metrics, including similarity, neighborhood structure, and alignment. Level 3 moves beyond geometric correspondence to identify the sparse latent features associated with shared and task-specific representation structures. Specifi- cally, we decompose hidden representations using SAEs and analyze task-dependent feature activa- tion patterns as an interpretable complement. Level 3 provides a feature-level interpretation of the geometric patterns identified in Levels 1–2, but depends on SAE training choices such as dictionary size and sparsity regularization. Level 4 there- fore examines topological properties of representation spaces using geometric invariants, providing an additional analysis that is less dependent on a particular feature decomposition. While Levels 1–3 analyze representation space through continuous geometry and sparse SAE acti- vations, they remain sensitive to metric and dictionary choices. Level 4 introduces a complementary topological perspective using persistent homology (PH). Because persistence diagrams are invariant to isometric transformations of the point cloud, Level 4 does not depend on a shared coordinate frame; it still depends on projection and filtration choices, and thus provides a cross-check under different assumptions rather than an assumption-free refinement. Level 4 does not replace the geometric and feature-level analyses in Levels 1–3. Instead, it evaluates whether their conclusions remain consistent when representation structure is summarized through topological invariants. Agreement across levels provides stronger evidence that observed task or- ganization reflects intrinsic properties of VLM representations rather than artifacts of a particular metric, feature decomposition, or analysis method. 2 B.3ADDITIONAL ANALYSIS METRICS B.3.1LEVEL 1: GLOBAL SIMILARITY STRUCTURE Cross-model similarity. For the dCor statistic in Eq. (3), pairwise Euclidean distances are com- puted as a s ij =∥h ℓ,M 1 t (x s i )− h ℓ,M 1 t (x s j )∥ 2 , b s ij =∥h ℓ,M 2 t (x s i )− h ℓ,M 2 t (x s j )∥ 2 ,(10) and double-centered to obtain A s ij = a s ij − ̄a s i· − ̄a s ·j + ̄a s · and B s ij = b s ij − ̄ b s i· − ̄ b s ·j + ̄ b s · . The squared distance covariance and variances are then computed as dCov s 2 = 1 n 2 X i,j A s ij B s ij , dVar s 1 2 = 1 n 2 X i,j A s ij 2 , dVar s 2 2 = 1 n 2 X i,j B s ij 2 . (11) B.3.2LEVEL 2: LOCAL GEOMETRIC STRUCTURE Local intrinsic dimensionality (LID). We first characterize the local complexity of each task representation space using LID, estimated by maximum likelihood (Levina & Bickel, 2004). For task t at layer ℓ, the LID of sample x s i in sampling repeat s is d LID ℓ k (x s i ;t) = " 1 k− 1 k−1 X o=1 log r k (h ℓ t (x s i )) r o (h ℓ t (x s i )) # −1 ,(12) where r o (h ℓ t (x s i )) denotes the Euclidean distance from h ℓ t (x s i ) to its o-th nearest neighbor among samples from the same task representation space. The average LID of task t is obtained by averaging over samples and sampling repeats: LID ℓ (t) = 1 b b X s=1 1 n n X i=1 d LID ℓ k (x s i ;t).(13) B.3.3LEVEL 3: SPARSE FEATURE DECOMPOSITION SAE architecture. At the stable layer ℓ ∗ , we denote the extracted representation as z t (x) = h ℓ ∗ t (x) ∈ R d . For each VLM, we train a single SAE shared across all low-level vision tasks, al- lowing feature usage to be compared within a common learned dictionary. The SAE encoder maps the representation into a sparse latent code a(x) = ReLU(W e z t (x) + b e ),(14) where W e ∈ R D SAE ×d and b e ∈ R D SAE are the encoder parameters, a(x) ∈ R D SAE is the sparse feature activation vector, and D SAE denotes the SAE dictionary size. The original representation is reconstructed by ˆz t (x) = W d a(x) + b d ,(15) where W d ∈ R d×D SAE and b d ∈ R d are the decoder parameters. The SAE is optimized using a reconstruction loss with an ℓ 1 sparsity penalty: L SAE = E x 1 d ∥z t (x)− ˆz t (x)∥ 2 2 + λ D SAE ∥a(x)∥ 1 .(16) Task-pair feature sharing. While I(Z f ;T ) in Eq. (8) measures whether individual features en- code task identity, it does not quantify whether two tasks rely on shared sparse features. Therefore, for task pairs with shared underlying image identities, we measure the similarity between their bi- nary SAE activation patterns. Let F p and F q denote the task-wise binary SAE activation patterns for tasks p and q, respectively. Their dependency is quantified using MI: I(F p ;F q ) = X c∈0,1 X e∈0,1 P pq (c,e) log P pq (c,e) P p (c)P q (e) ,(17) where c and e denote the binary activation states associated with tasks p and q, respectively. Here, P pq (c,e) is their joint probability, while P p (c) and P q (e) are the corresponding marginal probabil- ities, estimated from paired samples. Higher values indicate stronger feature-level overlap between two tasks, providing an information-theoretic complement to the geometric similarity measured in Levels 1–2. 3 B.3.4LEVEL 4: TOPOLOGICAL VERIFICATION Persistent homology. The point cloud e H t is obtained using the UMAP preprocessing described in Section 3.5. Given e H t , we construct the Vietoris–Rips filtration (Zomorodian & Carlsson, 2005): VR ε ( e H t ) = n σ ⊆ e H t : diam(σ)≤ ε o , ε≥ 0,(18) which forms a nested sequence of simplicial complexes as ε increases. At each filtration scale, the m-th homology groupG m (VR ε ( e H t )) captures topological structures of dimension m, summarized by the Betti number β m (ε) = rankG m VR ε ( e H t ) .(19) We focus on β 0 , corresponding to connected components, and β 1 , corresponding to one-dimensional loops. Following common usage, we refer to degrees m = 0, 1 as H 0 and H 1 . Each topological feature appears at a birth scale α and disappears at a death scale δ > α. The resulting birth–death pairs form the persistence diagram: PD m ( e H t ) =(α 1 ,δ 1 ), (α 2 ,δ 2 ),....(20) Cross-model topological consistency. To quantify whether different VLMs produce similar topo- logical structures, we compare persistence diagrams from models M 1 and M 2 for the same task t. Specifically, we compute the r-Wasserstein distance: d W PD M 1 m , PD M 2 m = inf γ X ξ∈PD M 1 m ∥ξ− γ(ξ)∥ r ∞ 1/r ,(21) where γ denotes a matching between persistence diagrams, with diagonal points added to handle unequal numbers of features. As r → ∞, the Wasserstein distance reduces to the bottleneck dis- tance: d B PD M 1 m , PD M 2 m = inf γ sup ξ∈PD M 1 m ∥ξ− γ(ξ)∥ ∞ .(22) We report the Wasserstein distance with r = 2 as the primary metric because it captures the ag- gregate deformation of persistence diagrams, while the bottleneck distance is reported as a comple- mentary worst-case measure. We additionally assess sampling variability for both Conditions I and IV: in Figure 11, intervals show the spread across the b = 10 frozen sampling repeats, whereas in Figure 13, they are bootstrap percentile intervals from B PH = 50 resamples. Stability considerations. The stability of persistence diagrams under perturbations of the input point cloud has been established for the bottleneck distance (Cohen-Steiner et al., 2007) and, under additional Lipschitz and bounded-total-persistence assumptions, extended to finite r-Wasserstein distances (Cohen-Steiner et al., 2010). These results justify using persistence-based distances as ro- bust summaries of geometric structure, but do not eliminate sensitivity introduced by dimensionality reduction or filtration choices. CDATA DETAILS C.1DATASETS This section provides additional details about the datasets used to construct the 24 low-level vision tasks in our experiments. C.1.1RESTORATION LIU4K. LIU4K (Liu et al., 2020) is a high-resolution dataset for single-image compression artifact removal, containing 1,500 training images and 200 testing images at 4K resolution. Since it provides only clean images, we generate compressed inputs using the compression settings adopted in our experiments and retain the original images as references. 4 DPDD. DPDD (Abuolaim & Brown, 2020) is a real-world defocus deblurring dataset captured using a Canon EOS 5D Mark IV dual-pixel camera. It contains 500 scenes and 2,000 images. Each scene provides a defocused image, left and right dual-pixel sub-aperture views, and an all-in-focus reference image. The official split consists of 350 training scenes, 74 validation scenes, and 76 testing scenes. GoPro. GoPro (Nah et al., 2017) is a motion deblurring dataset in which blurry images are synthe- sized by averaging consecutive high-frame-rate frames, with the corresponding sharp frames used as references. It contains 3,214 blurry and sharp pairs at a resolution of 1280× 720, including 2,103 pairs from 22 sequences for training and 1,111 pairs from 11 sequences for testing. D-HAZY. D-HAZY (Ancuti et al., 2016) is a synthetic dehazing benchmark that generates hazy images from haze-free RGB images and their associated depth maps using the atmospheric scattering model, producing aligned hazy and haze-free pairs. We use its NYU Depth v2 subset, generated from the 1,449 images of NYU Depth v2 (Silberman et al., 2012), which cover 464 indoor scenes captured using a Microsoft Kinect camera. UHDM. UHDM (Yu et al., 2022) is a real-world 4K image demoir ́ eing dataset containing 5,000 paired moir ́ e and moir ́ e-free images, including 4,500 pairs for training and 500 pairs for testing. The images were collected using different display devices and cameras and cover moir ́ e patterns with diverse colors, orientations, frequencies, and spatial scales. SIDD. SIDD (Abdelhamed et al., 2018) is a real-world smartphone denoising dataset containing approximately 30,000 noisy images from 10 representative scenes captured using five smartphone cameras. The images cover different illumination conditions, ISO levels, exposure times, and scene contents. Corresponding nearly noise-free reference images are obtained from repeated captures of the same scenes. Rain14000. Rain14000 (Fu et al., 2017) is a synthetic deraining dataset constructed from 1,000 clean images. Fourteen rainy images are generated for each clean image using rain patterns with different orientations and intensities, resulting in 14,000 rainy and clean pairs. The standard split contains 12,600 pairs generated from 900 clean images for training and 1,400 pairs generated from the remaining 100 clean images for testing. Snow100K. Snow100K (Liu et al., 2018) is a synthetic desnowing dataset containing 100,000 snowy images produced by compositing snow masks onto snow-free photographs, split evenly into 50,000 training and 50,000 testing images. Each synthesized image is accompanied by its snow-free reference and the corresponding snow mask, and the test set is further partitioned into light, medium, and heavy subsets according to snow particle size, so that the benchmark covers snow particles with different sizes, densities, and transparencies. The dataset additionally provides a set of realistic snowy photographs without references. We use the paired synthetic images, taking the snowy image as the input and the snow-free image as the reference. UIEB. UIEB (Li et al., 2020) is a real-world underwater image enhancement benchmark contain- ing 950 images with diverse color casts, visibility conditions, illumination patterns, and underwater environments. Among them, 890 images have reference images selected through human evaluation, while the remaining 60 challenging images do not have reference targets. We use the paired subset for underwater restoration. C.1.2REMOVAL Flare7K++. Flare7K++ (Dai et al., 2024) is a large-scale lens flare removal dataset that extends Flare7K by combining 7,000 synthetic flare images, including 5,000 scattering flares and 2,000 reflective flares, with 962 real captured flare images and 23,949 flare-free background images. Flare- corrupted inputs in the released paired test set are generated by compositing flare layers with clean backgrounds, while the original backgrounds serve as references. Raindrop Clarity. Raindrop Clarity (Jin et al., 2024) is a real-world raindrop removal dataset cov- ering both daytime and nighttime conditions. It contains 15,186 image pairs or triplets, including 5,442 daytime samples and 9,744 nighttime samples. The dataset provides raindrop-corrupted im- ages, clear background images, and observations captured under different focal settings. It includes both background-focused cases, in which raindrops appear blurred, and raindrop-focused cases, in which the background may be out of focus. 5 SIR 2 . SIR 2 (Wan et al., 2017) is a real-world benchmark for single-image reflection removal. It contains 40 controlled indoor scenes, including 20 solid-object scenes and 20 postcard scenes, together with 100 wild scenes. The controlled scenes provide reflection-contaminated images and their corresponding transmission and reflection layers. Multiple observations are captured using different aperture settings and glass thicknesses, producing reflections with different strengths, blur levels, and spatial structures. ISTD. ISTD (Wang et al., 2018) is a shadow removal dataset containing 1,870 image triplets, each consisting of a shadow image, its corresponding shadow-free image, and a binary shadow mask. The official split contains 1,330 triplets for training and 540 triplets for testing. The dataset covers shadows with different shapes, strengths, and boundary characteristics. C.1.3GENERATION AND ENHANCEMENT ADE20K. ADE20K (Zhou et al., 2017) is a scene-understanding dataset covering diverse indoor and outdoor scenes, objects, materials, and spatial layouts. Its standard scene-parsing benchmark contains 20,210 training images, 2,000 validation images, and 3,000 testing images, with annotations for 150 semantic categories. For colorization, we use the RGB images rather than their semantic annotations. Each RGB image is converted to grayscale as input, while the original color image is retained as the reference. iHarmony4. iHarmony4 (Cong et al., 2020) is a large-scale image harmonization benchmark com- bining four sub-datasets: HCOCO, HAdobe5K, HFlickr, and Hday2night. It contains 73,146 syn- thesized composite images, each accompanied by a harmonized reference image and a foreground mask. The composite foregrounds exhibit inconsistencies with their backgrounds in color, illumina- tion, contrast, and overall appearance. Natural Images. The Natural Images dataset, proposed by Roy et al. (2018), contains 6,899 high- resolution natural images from eight object categories, including airplane, car, cat, dog, flower, fruit, motorbike, and person. It is split into 5,724 training images and 1,175 validation images. Since the dataset provides only intact images, we generate masked images and use the original images as ground-truth targets for image inpainting. LOL. LOL (Wei et al., 2018) is a real-world low-light image enhancement dataset containing 500 aligned low-light and normal-light image pairs captured from the same scenes under different exposure conditions. The standard split contains 485 pairs for training and 15 pairs for testing. The low-light images exhibit reduced visibility, color distortion, amplified noise, and loss of local details. Night2Day. The Night2Day dataset distributed with pix2pix (Isola et al., 2017) is derived from the Transient Attributes Database (Laffont et al., 2014). It contains approximately 20,000 natural- scene images captured by stationary webcams at different times of day. Nighttime and daytime observations of the same underlying scenes are used as source and reference images for style transfer. BSDS500. BSDS500 (Arbel ́ aez et al., 2011) is a standard benchmark for natural-image edge and boundary detection. It contains 500 images, divided into 200 training images, 100 validation images, and 200 testing images. Each image is annotated by multiple human subjects, resulting in several valid boundary maps representing perceptually meaningful object and region contours. C.1.4RECONSTRUCTION DIV2K. DIV2K (Agustsson & Timofte, 2017) is a high-quality image super-resolution dataset containing 1,000 images at approximately 2K resolution, including 800 training images, 100 valida- tion images, and 100 testing images. The corresponding low- and high-resolution images are used as inputs and references for the super-resolution task. HDRTV1K. HDRTV1K (Chen et al., 2021) is an ultra-high-definition dataset for SDRTV-to- HDRTV conversion. It contains 1,235 training pairs and 117 testing pairs at 4K resolution, derived from HDR10 video content. The paired SDRTV and HDRTV images differ in dynamic range, color gamut, contrast, and highlight appearance, with SDRTV images serving as source observations and HDRTV images as references. 6 Input Image Task Prompt “This is a deraining task.” Neutral Prompt “Keep the image unchanged.” Task-Similarity Matrices 퐒⁰퐒²⁰퐒²³퐒³¹ 20–23 Layer ℓ Smoothed 푫 퐟퐮퐥 퓵 풉 풑 퓵∗ 풙 ℓ* = 23 ℓ* Task Neutral Image TokenText TokenLast Token Emu3 Decoder TaskNeutralDelta ℓ = 0 − ℓ = 1 − ⋮ ℓ = 20 − ℓ = 21 − ℓ = 22 − ℓ* = 23 − ⋮ ℓ = 31 Layer − All layers 풉 풑 퓵 풙 Figure 7: Feature extraction and stable-layer selection. Illustrated using a typical AR model (Emu3). Input images with task and neutral prompts produce the cached states whose difference defines h ℓ p (x). C.1.5PHOTOMETRIC CORRECTION Rendered WB. The Rendered WB dataset (Afifi et al., 2019) is a large-scale benchmark for white- balance correction. Its main set contains 62,535 rendered images captured using seven DSLR cam- eras, with corresponding correctly white-balanced references. An additional cross-camera test set contains 2,881 rendered images captured using four mobile phones and one DSLR camera. Each scene is rendered under several camera-specific white-balance settings to produce realistic color casts. SICE. SICE (Cai et al., 2018) is a multi-exposure contrast enhancement dataset containing 589 indoor and outdoor scene sequences and 4,413 images in total. Each sequence depicts the same static scene under several exposure levels. Candidate enhanced images are generated using multiple multi-exposure fusion and HDR methods, after which subjective evaluation is used to select a high- quality reference image for each sequence. VIDIT. For relighting, we use Track 1 of the AIM 2020 Scene Relighting Challenge (El Helou et al., 2020b), based on VIDIT (El Helou et al., 2020a). VIDIT contains 390 synthetic scenes at a resolution of 1024× 1024, each rendered under 40 illumination settings formed by eight illumi- nation directions and five color temperatures. The scenes are divided into 300 training scenes, 45 validation scenes, and 45 testing scenes. Track 1 follows a one-to-one relighting setting, in which an image under a source illumination is transformed into the same scene under a predefined target color temperature and illumination direction. C.2FEATURE EXTRACTION PROTOCOL All images are EXIF-oriented, converted to RGB, and resized to 512× 512. For each image x and task p, the frozen model is run with the fixed task instruction “This is a/an [task name] task” and with the neutral instruction “Keep the image unchanged” as the baseline. For AR models, the attention mask identifies the final valid prompt position, which can attend to both the image and the complete instruction. We use the difference between its task and neutral states, h ℓ p (x) = u ℓ task,last (x)− u ℓ neutral,last (x).(23) For Qwen-Image-Edit, a DiT model, the corresponding representation is the task-minus-neutral dif- ference between spatial means of the condition-image feature maps at the first denoising step. Figure 7 illustrates how we extract hidden-layer representations, using a typical AR model as an example. At each layer of the AR model, we cache the last-token hidden states for the task-prompt and neutral-prompt input sequences, along with the sample IDs and extraction metadata. Eq. (23) is applied when the cache is loaded. Each frozen manifest, a saved ordered list of sample IDs reused across models, selects n rows in its recorded order. To choose the stable layer ℓ ∗ , we average the b task-similarity matrices per layer, compute D ℓ full = S ℓ − S ℓ+1 F .(24) 7 Model푴₁ Model푴₂ x 풛 풑 풙= ℎ " ℓ∗ 푥 Base ID 70% Train 30% Held-Out Encoder ReLU + ℓ₁ Sparse Code 풂풙 Decoder &풛 풑 풙 Held-OutActivation Patterns 푨 풑 푴 ퟏ 푨 풒 푴 ퟐ ConditionII Task t, M₁Task t, M₂ MMD² ConditionIV Task p, M₁ Task q, M₂ Task q, M₁ Task p, M₂ MMD² MMD² Mean MMD² Reciprocal SAE Figure 8: SAE training and its use in Conditions I and IV. Per-model SAEs are fit on task- balanced training data and reused to compare held-out activation-rate distributions. Neither condi- tion assumes aligned SAE feature coordinates. We smooth the change curve with a three-transition rolling median. Here, “full” denotes the com- plete task-similarity matrix. The task margin is the mean within-task similarity minus the mean cross-task similarity. Within 15–95% of network depth, the stable range begins at the first two- transition run below the 0.35-quantile change threshold whose task margin is at least the median candidate margin. The range ends when the change threshold is exceeded, and its final layer is used for all single-layer geometric analyses. If no candidate meets both criteria, the layer with the highest combined normalized margin and inverse-change score is selected and flagged as a fallback. SAE layer selection uses the separate resampling protocol in Appendix C.3. C.3SAE TRAINING DETAILS For the SAE analyses in Conditions I and IV, stable-layer selection uses 50 within-task bootstrap resamples of 100 rows, drawn with replacement from a 50% selection subset, and applies the crite- rion in Appendix C.2. These resamples are distinct from the b = 10 frozen sampling repeats and the B PH = 50 topological resamples. We train one SAE per model on the ℓ ∗ representations from all 24 tasks. Samples are first split by base-content identity into 70/30 training and evaluation sets, before any balancing or normalization. In the paired-control setting, identical samples are kept in the same split across tasks. Each task con- tributes n SAE = 500 training samples, with replacement used only when necessary. The training-set mean and scalar root-mean-square deviation are then used to apply the same normalization to both splits. The trained SAE with a ReLU activation function and an ℓ 1 penalty in Eqs. (14)–(16) uses D SAE = 8d. For each model, λ is selected on an inner 80/20 split from 0.1, 0.3, 1, 3, 10, 30 after 2,000 steps per candidate. Eligible candidates require explained variance ≥ 0.80 and active fraction≤ 0.05. Among these, we choose the one closest to the target active fraction 0.02, break- ing ties by explained variance and then smaller λ. Final fits use seeds 0, 1, 2 and AdamW for 5,000 steps (learning rate 10 −3 , batch size 1,024, zero weight decay, gradient clipping 1.0, and unit-norm decoder columns). All seeds must also satisfy the stated dead-feature, cross-seed, and train–evaluation checks, including explained variance≥ 0.80, active fraction≤ 0.05, dead-feature fraction≤ 0.95, cross-seed explained-variance and active-fraction ranges≤ 0.05 and≤ 0.02, and train–evaluation explained-variance gap≤ 0.15. Level-3 metrics use only held-out codes and define a feature as active when a f > τ = 10 −6 . Among features active on at least one held-out sample, we label a feature task-specific if TSI(f ) > θ high = 0.85, shared if TSI(f ) < θ low = 0.30, and intermediate otherwise. Crucially, metrics such as TSI, MI, and MMD are not used during λ selection. Conditions I and IV reuse these per-model SAEs and evaluation codes. Because their feature coor- dinates are unaligned, cross-model MMD compares the unordered activation-rate distributions rather than raw coordinates. The RBF kernel uses a bandwidth equal to the square root of the median pos- itive squared pairwise distance among the pooled activation rates in each comparison. Condition I uses different models with the same task. Condition IV uses different models and tasks, averages the two reciprocal task assignments, and ranks the result within each model pair. Figure 8 illustrates the SAE training process and its use in both conditions. 8 Table 4: Task/model conditions, and corresponding analysis levels and metrics. ConditionSetupLevel(s)Metrics Same Task, Same Model One task and model, with images varied L1 (primary) L2 (supplementary) Intra-task cosine (Eq. (2)), Frobenius distance (Eq. (24)), LID (Eqs. (12)–(13)) Same Task, Different Models One task with models varied; identically ordered images and independent SAEs L1–L3 (primary) L4 (supplementary) dCor (Eq. (3)), NNGS (Eq. (4)), PCA+PAR (Eq. (5)), TSI (Eq. (7)) and feature–task MI (Eq. (8)), RBF-MMD (Eq. (9)), r-Wasserstein distance (Eq. (21)), bottleneck distance (Eq. (22)) Different Tasks, Same Model One model with tasks variedL1 (primary)Inter-task cosine (Eq. (1)), MDS (visualization) Different Tasks, Different Models Tasks and models varied; no image-ID or SAE-coordinate alignment L3 (primary) L4 (supplementary) Task-conditioned activation-rate MMD extension of Eq. (9), averaged reciprocally and then ranked within each model pair; H 0 /H 1 r-Wasserstein and bottleneck distances (Eqs. (21) and (22)) DSUPPORTING MATERIALS D.1MAPPING BETWEEN CONDITIONS AND ANALYSIS LEVELS Table 4 maps the four evaluation conditions in Section 4.2 to the corresponding GeoSim analysis level(s) and the associated evaluation metrics. Overall, the table serves as a roadmap for the exper- imental analysis by linking each evaluation condition to its analysis objective and the appropriate metrics, providing an overview of the complete evaluation protocol. D.2VISUALIZATION RESULTS AND ILLUSTRATIONS This section collects the supplementary visual evidence referenced by the main results. The figures examine task-wise local dimensionality, its relationship to cross-model alignment, and within-model inter-task organization, and report the Level-3 SAE and Level-4 topological analyses for Conditions I and IV. Where applicable, captions state the unit of analysis, aggregation rule, and uncertainty summary needed to interpret the corresponding panel. D.2.1CONDITION I: SAME TASK, SAME MODEL Condition I holds the task and the model fixed and varies only the input image, asking whether the representation an instruction induces is a stable property of the task or is dominated by image con- tent. It evaluates Level 1 through the intra-task similarity in Eq. (2) and Level 2 through the LID in Eqs. (12)–(13). Both figures analyze the instruction-induced displacement, defined as the difference between the last-token hidden states obtained under the task prompt versus the neutral prompt (for Qwen-Image-Edit, the difference of spatially averaged condition-image features), computed over b = 10 frozen sampling repeats of n = 100 distinct images per task. Figure 2 evaluates Eq. (2) at every layer, plotting one curve per task against relative depth. The results indicate that within-task consistency varies across layers. Specifically, in five models the task curves begin at 0.95 or above at the first layer and separate only with increasing depth, whereas Qwen-Image-Edit starts lower for some tasks and fluctuates more across depth. Emu3.5 is the exception: it exhibits a sharp early rise, before showing greater separation among task curves at later layers. Consequently, the 0.80–0.96 interval quoted at the analysis layer does not describe the network as a whole. Figure 9 evaluates Eqs. (12)–(13) on these same displacements at the single layer ℓ ∗ chosen by Eq. (24), applying the Levina–Bickel estimator at k = 10. Furthermore, each sampling repeat contains no duplicate images, as the estimator degenerates when the nearest-neighbor distance van- ishes. Tasks are ordered by their cross-model mean, yielding an ordering that remains nearly iden- tical across models (mean pairwise Spearman ρ = 0.80, with Denoising lowest in all seven). This indicates that local complexity is governed by the task rather than the architecture, although the es- timator is downward-biased with n = 100 samples in a space of several thousand dimensions and only the ordering is interpreted. 9 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 LID (k = 10) Emu3 Anole Janus-Pro InternVL3.5 Qwen3-VL Qwen-Image-Edit Emu3.5 Mean Denoising Motion Deblurring Reflection Removal Style Transfer Raindrop Removal Relighting Underwater Restoration HDR Reconstruction Lens Flare Removal Light Enhancement White Balance Correction Shadow Removal Deraining Contrast Enhancement Demoiréing Defocus Deblurring Harmonization Colorization Edge Detection Super- Resolution Desnowing Dehazing Inpainting Artifact Removal Figure 9: Condition I: Same Task, Same Model. LID per task, estimated with the Levina–Bickel estimator at k = 10 at each model’s analysis layer ℓ ∗ . Tasks are ordered by their cross-model mean, shown as the black line. 4681012141618 LID (k = 10) 0.7 0.8 0.9 1.0 1.1 1.2 P A R Artifact Removal Defocus Deblurring Motion Deblurring Dehazing Denoising Deraining Desnowing Underwater Restoration Raindrop Removal Reflection Removal Shadow Removal Colorization Harmonization Inpainting Light Enhancement Style Transfer Edge Detection HDR Reconstruction Relighting White Balance Correction Contrast Enhancement 4681012141618 LID (k = 10) 0.1 0.2 0.3 0.4 0.5 0.6 N N G S ( k = 1 0 ) Artifact Removal Defocus Deblurring Motion Deblurring Dehazing Denoising Deraining Desnowing Underwater Restoration Raindrop Removal Reflection Removal Shadow Removal Colorization Harmonization Inpainting Light Enhancement Style Transfer Edge Detection HDR Reconstruction Relighting White Balance Correction Contrast Enhancement RestorationRemovalGeneration/EnhancementReconstructionPhotometric Correction Demoiréing Lens Flare Removal Super-Resolution Demoiréing Lens Flare Removal Super-Resolution Figure 10: Condition I: Same Task, Different Models. Local dimensionality against cross-model agreement, one point per task. The abscissa is the seven-model mean LID of Figure 9. The ordinates are the held-out PAR (left) and the chance-adjusted neighborhood overlap (right), each averaged over the 21 model pairs. The dotted line marks the PAR baseline of 1.0, the residual obtained by predicting the mean. D.2.2CONDITION I: SAME TASK, DIFFERENT MODELS Condition I holds the task fixed while varying the model, testing whether different architectures arrive at similar geometry for the same task. The sampling manifests ensure that the i-th row of every representation matrix across models corresponds to the same image, providing the pairing re- quired by dCor, NNGS, and PAR. Figure 4 reports four metrics over the 21 model pairs, averaged across the 24 tasks and ordered by dCor. PAR is computed after projecting each model’s represen- tations separately onto its top 64 principal components, and Level 4 uses the topological pipeline in Section 3.5. Agreement turns out to differ between global and local comparisons. The dCor in Eq. (3) averages 0.77, whereas the chance-adjusted NNGS averages only 0.27. We use k = 10 for both LID and NNGS and report (NNGS− c)/(1− c), where NNGS is the raw overlap defined in Eq. (4) and c = k/(n− 1) ≈ 0.101 for n = 100. The PAR in Eq. (5) is likewise reported in a held-out form: the orthogonal map is fitted on 80% of the rows and scored on the remaining 20%, with the test residual normalized by the Frobenius norm of the transformed held-out target matrix. Predicting the training-set target mean gives zero in the centered PCA space and therefore a normalized residual of 1.0. The residual averages 1.049, with 16 of the 21 pairs scoring above that 1.0 line. This 10 0.10.20.5 1 2 5 1020 H₀: connected components Persistence-diagram distance 0.00.10.20.30.40.50.6 H₁: loops Persistence-diagram distance Anole vs. Qwen3-VL Defocus Deblurring Anole vs. Qwen3-VL Dehazing Anole vs. Qwen3-VL Deraining Anole vs. Qwen3-VL Relighting Emu3 vs. Janus-Pro Defocus Deblurring Emu3 vs. Janus-Pro Dehazing Emu3 vs. Janus-Pro Light Enhancement Emu3.5 vs. Janus-Pro Dehazing Emu3.5 vs. Janus-Pro Light Enhancement Emu3.5 vs. Qwen3-VL Defocus Deblurring Emu3.5 vs. Qwen3-VL Deraining Emu3.5 vs. Qwen3-VL Light Enhancement InternVL3.5 vs. Janus-Pro Defocus Deblurring InternVL3.5 vs. Janus-Pro Denoising InternVL3.5 vs. Janus-Pro Light Enhancement InternVL3.5 vs. Janus-Pro Underwater Restoration Bottleneck mean Anole vs. Qwen3-VL Emu3 vs. Janus-Pro Emu3.5 vs. Janus-Pro Emu3.5 vs. Qwen3-VL InternVL3.5 vs. Janus-Pro W₂ mean + run interval Figure 11: Condition I: Same Task, Different Models. Level-4 2-Wasserstein and bottleneck distances for H 0 and H 1 across 16 model-pair and task combinations. indicates that an orthogonal alignment estimated on part of the data fails to generalize to the rest. The models therefore share a coarse ordering of distances without sharing either local neighborhoods or a coordinate frame. Figure 10 isolates the one systematic exception by plotting each task’s seven-model mean LID against its cross-model agreement. This yields a correlation of ρ = +0.853 against PAR and ρ = −0.764 against neighborhood overlap, with Denoising and Motion Deblurring the only tasks whose residuals fall markedly below 1.0 (at 0.74 and 0.83), whereas every task with an LID exceed- ing 15 scores above 1.1. This suggests that a task with fewer operative directions leaves the models less room to differ. Figure 3 shows the Level 3 results. For each of the 21 model pairs, we compute the same-task cross- model activation-rate MMD for all 24 tasks and average the results across tasks. We then compare this value with a matched within-model baseline computed from different-task pairs, averaging re- sults over three SAE random seeds. Under this comparison, 6 model pairs satisfy the criterion after seed aggregation, while 3 satisfy it consistently across all three seeds. The ratios in the left panel are the same-task cross-model MMD divided by the matched within-model, different-task refer- ence; only ratios below one are labeled. The TSI and feature–task MI panels instead characterize task selectivity within each individual model without requiring SAE features to be aligned across models. For Level 4, H 0 and H 1 denote the two homology degrees reported throughout this section. As the filtration radius increases, H 0 records connected components and their merging, summarizing mul- tiscale cluster and connectivity structure, while H 1 records the birth and death of one-dimensional loops. We report H 0 and H 1 separately rather than averaging them. A high H 0 distance with a low H 1 distance, for example, indicates different component structure but similar loop structure. Figure 11 presents the supplementary Level-4 analysis for 16 evaluated model-pair and task combi- nations. For both H 0 and H 1 , it reports mean 2-Wasserstein and bottleneck distances over b = 10 frozen sampling runs, each with n = 100 samples per task. Horizontal error bars show run-to-run variation in the 2-Wasserstein distance, and endpoints are the 2.5th and 97.5th percentiles of the ten 11 Emu3AnoleJanus-Pro InternVL3.5Qwen3-VLQwen-Image-EditEmu3.5 Restoration Removal Gen./Enh. Recon. Photom. Restoration Removal Gen./Enh. Recon. Photom. Restoration Removal Gen./Enh. Recon. Photom. Restoration Removal Gen./Enh. Recon. Photom. Restoration Removal Gen./Enh. Recon. Photom. Restoration Removal Gen./Enh. Recon. Photom. −2.0−1.5−1.0−0.50.00.51.01.52.0 Within-model z-score Figure 12: Condition I: Different Tasks, Same Model. Inter-task cosine similarity matrices, with one panel per model. Both axes carry the same 24 tasks in a fixed order grouped by perceptual family, with rules and labels marking the five categories. Each panel is standardized on its own off-diagonal entries, so color encodes how far a task pair departs from the typical pair of that model, red above and blue below, on the common scale of±2.2 SD given in the color bar. The diagonal is left blank because it carries intra-task rather than inter-task similarity. run-level distances. The 2-Wasserstein distance agrees in direction while supplying a complemen- tary topological check. D.2.3CONDITION I: DIFFERENT TASKS, SAME MODEL Condition I holds the model fixed while varying the task, asking how the 24 tasks are organized relative to one another inside a single network. It draws on Level 1 through the inter-task similarity of Eq. (1), whose collection over all task pairs forms the layer-wise task similarity matrix. Figure 12 shows that matrix at the analysis layer for every model. Because the raw ranges differ widely between models, such as 0.29–0.65 in Emu3 against 0.52–0.86 in Qwen3-VL, each panel is standardized on its own off-diagonal entries before plotting. What is compared across panels is therefore block structure rather than overall level. Under this standardization, the Removal block remains coherent in every model, whereas Generation/Enhancement shows no consistent block. Figure 5 aggregates the seven standardized matrices into a consensus, taking their mean across models as the signal and their spread as the noise. It draws an edge only where the consensus exceeds 0.714 SD, which represents the mean disagreement across models, ensuring the average signal is at least as large as the typical cross-model dispersion. We find 36 of the 276 task pairs qualify, and 20 of them cross a family boundary. The family gap is the mean consensus for within- family pairs minus that for cross-family pairs. The strongest are Defocus–Motion Deblurring at +1.66, Deraining–Desnowing at +1.63, and Reflection–Shadow Removal at +1.61, representing pairs that share a degradation mechanism. The cross-family edges show that relations in the graph are not confined to family boundaries. Seven tasks carry no edge for different reasons. Colorization and Harmonization sit below their model average similarity to every other task, with best consensus at−0.29 and−0.26, whereas Inpainting misses the threshold by 0.009 and is better read as undecided than as isolated. Unlike t-SNE (van der Maaten & Hinton, 2008) or UMAP (McInnes et al., 2018), classical MDS is deterministic and embeds the dissimilarities directly, which matters here because the relation graph contains only 24 nodes, a regime in which a t-SNE or UMAP layout is not stable. Node positions 12 derive from MDS on the consensus matrix, whose first two coordinates carry 31.5% of the structure with a rank correlation of 0.772 between plotted and true dissimilarity. Therefore, the layout should be read for grouping rather than for precise metric distance. Positions and edges encode different information, since coordinates summarize a task’s profile against all 23 others while an edge reflects a single pairwise value. D.2.4CONDITION IV: DIFFERENT TASKS, DIFFERENT MODELS Condition IV changes both task and model without assuming paired images or aligned SAE coor- dinates. Figure 13 follows the 10 reciprocal candidates retained by the exploratory Level-3 screen. The first panel gives their within-model-pair percentile ranks; its numerical annotations are raw re- ciprocal MMD values. The H 0 and H 1 panels report 2-Wasserstein means and bootstrap percentile intervals, the range across the two task assignments, and bottleneck distances. The final panel shows scale-normalized distances: the pairwise-distance matrix of each point cloud is divided by its own median nonzero distance before computing persistent homology. Topology uses n = 100, b = 10, and B PH = 50, remaining a supplementary and unpaired com- parison. For each assignment, interval bounds represent the 2.5th and 97.5th percentiles of the 50 bootstrap distances. The displayed interval spans from the lower of the two lower bounds to the higher of the two upper bounds. 02 4 Rank (%) 01020 H₀ PD distance 0.250.500.75 H₁ PD distance 0.51.0 Scale sensitivity Normalized W₂ Anole vs. Janus-Pro Denoising vs. Raindrop Removal Anole vs. Qwen-Image-Edit Relighting Emu3 vs. Emu3.5 Denoising vs. Style Transfer Emu3 vs. Emu3.5 Raindrop Removal vs. Style Transfer Emu3 vs. Janus-Pro Light Enhancement vs. Reflection Removal Emu3 vs. Qwen3-VL Raindrop Removal vs. Style Transfer Emu3.5 vs. Janus-Pro Denoising vs. Shadow Removal Emu3.5 vs. Qwen3-VL Artifact Removal vs. Underwater Restoration InternVL3.5 vs. Qwen-Image-Edit Artifact Removal vs. Underwater Restoration Janus-Pro vs. Qwen-Image-Edit Dehazing vs. Denoising 0.0078 0.0010 0.0177 0.0374 0.0100 0.0322 0.0173 0.0022 0.0014 0.0027 W₂ + intervalAssignment rangeBottleneckNormalized H₀ W₂Normalized H₁ W₂ Level-3 screen Lens Flare Removal vs. Figure 13: Condition IV: Different Tasks, Different Models. Level-3 screening rank and Level-4 H 0 /H 1 topological distances for the 10 reciprocal candidates, shown on raw and scale-normalized views. Dots show 2-Wasserstein means over the ten sampling runs and both task assignments, thin bars span the bootstrap percentile intervals of both assignments, and thick bars show the range between their means. 13