Paper deep dive

Sparse Concept Channels in Frozen 3D CT Vision Encoders

Farhad Nooralahzadeh, Lea Bogensperger, Christian Bluethgen, Michael Krauthammer

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 43

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 7/24/2026, 2:32:53 AM

Summary

The paper introduces the Concept Channel Probe (CCP), a training-free method to identify sparse, causal concept channels within frozen 3D medical vision-language models (VLMs). Using Pillar-0 (chest) and Merlin (abdominal) encoders, the authors demonstrate that specific radiological findings are encoded by approximately 10 sparse channels. This approach outperforms zero-shot prompting and matches supervised baselines in classification while significantly outperforming trained models like CT-CHAT in report generation efficiency and clinical efficacy metrics.

Entities (9)

Concept Channel Probe · method · 98%Merlin · model · 95%Pillar-0 · model · 95%CT-RATE · dataset · 92%RadChest-CT · dataset · 92%AUROC · metric · 90%BLEU · metric · 90%CT-CHAT · model · 90%F1 Score · metric · 90%

Relation Signals (7)

Pillar-0 → specializesin → Chest CT

confidence 95% · 3D chest vision-language model (Pillar-0)

Merlin → specializesin → Abdominal CT

confidence 95% · 3D abdominal VLM (Merlin)

Concept Channel Probe → uses → Merlin

confidence 95% · the same sparse probe replicates on an architecturally unrelated 3D abdominal VLM (Merlin)

Concept Channel Probe → uses → Pillar-0

confidence 95% · We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings.

Concept Channel Probe → evaluateson → CT-RATE

confidence 92% · We evaluate on three public cohorts: (1) CT-RATE

Concept Channel Probe → evaluateson → RadChest-CT

confidence 92% · (2) RadChest-CT... external chest CTs

Concept Channel Probe → outperforms → CT-CHAT

confidence 90% · Our training-free concept channel probe (CCP) method... outperforms published CT-CHAT on clinical efficacy

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large vision-language models are becoming increasingly dominant in 3D medical image interpretation, but we rarely know which internal units encode clinical findings or where that information lives in the representation. We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings. We show that (i) each radiological finding is encoded by a sparse set of ~10 vision-encoder channels that match full-feature classification performance and far exceed a zero-shot text prompting; (ii) turning off the channels tied to one finding, that finding's score collapses while unrelated labels stay stable; and (iii) the same sparse probe replicates on an architecturally unrelated 3D abdominal VLM (Merlin) suggesting a general property of frozen medical encoders. Our training-free concept channel probe (CCP) method, paired with a corpus-derived report template, outperforms published CT-CHAT on clinical efficacy and NLG metrics (F1 0.549 vs. 0.184; BLEU 0.483 vs. 0.373) at 22x lower latency. Our results provide a clear, reproducible characterization of how frozen medical encoders represent findings, demonstrating direct applicability across models.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

41,572 characters extracted from source content.

Expand or collapse full text

Sparse Concept Channels in Frozen 3D CT Vision Encoders Farhad Nooralahzadeh 1,2, Lea Bogensperger1, Christian Bluethgen3, Michael Krauthammer1 Abstract Large vision-language models are becoming increasingly dominant in 3D medical image interpretation, but we rarely know which internal units encode clinical findings or where that information lives in the representation. We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings. We show that (i) each radiological finding is encoded by a sparse set of ∼ 10 vision-encoder channels that match full-feature classification performance and far exceed a zero-shot text prompting; (i) turning off the channels tied to one finding, that finding’s score collapses while unrelated labels stay stable; and (i) the same sparse probe replicates on an architecturally unrelated 3D abdominal VLM (Merlin) suggesting a general property of frozen medical encoders. Our training-free concept channel probe (CCP) method, paired with a corpus-derived report template, outperforms published CT-CHAT on clinical efficacy and NLG metrics (F1 0.549 vs. 0.184; BLEU 0.483 vs. 0.373) at 22×lower latency. Our results provide a clear, reproducible characterization of how frozen medical encoders represent findings, demonstrating direct applicability across models. Introduction Foundation models for medical imaging are now strong enough to be considered as clinical decision support tools, yet their internal representations are poorly understood. We explore these representations by following the questions that matter in terms of explainability: What concepts does a vision encoder encode, and where, in which units and at what spatial resolution, does that information reside? Answering them without fine-tuning is attractive because it (a) avoids the cost and risk of retraining a large model, and (b) yields explanations about the deployed model rather than a modified copy. Figure 1: Concept Channel Probe (CCP) overview. We therefore take a deliberately training-free approach. As shown in Figure 1, we freeze a 3D vision-language model and cache its final image embedding xix_i. We then use multi-label volume annotations as statistical probes on xix_i - not to train a network, but to ask: which embedding dimensions linearly encode clinical findings? This leads to three contributions: (1) Sparsity Each radiological finding is encoded by ∼ 10 channels that achieve comparable classification performance to the full D-dimensional embedding, (2) Causality Zeroing a finding’s specific channels reduces its own probe AUROC score ∼ 20× more than other findings, and the specific channels cluster into clinically coherent groups; and (3) Generalization across backbones The sparse organization replicates on another 3D CT backbone (3D VLM with a different pretraining pipeline), indicating the phenomenon is a property of frozen VLMs rather than of one architecture. We then show the probe is useful and backbone-portable: the same CCP-10 detections and corpus-derived template verbalizer outperform trained CT VLMs on clinical and factual metrics at a fraction of the cost on underlying 3D VLMs. Related Work 3D medical vision-language models. Recent foundation models pair 3D CT encoders with text supervision, enabling zero-shot classification and report generation. Examples include CT-CHAT (Hamamci et al. 2026) and Pillar-0 (Agrawal et al. 2025) for chest CT, and Merlin (Blankemeier et al. 2024) for abdominal CT. These approaches optimize what the model predicts through pretraining and decoding. In contrast, we treat frozen encoders as interpretable systems and ask: where does finding information reside in the embeddings? We answer this through statistical probes rather than weight updates. Probing and interpretability of learned representations. Linear probes (Alain and Bengio 2016) test what information is linearly decodable from frozen features, while network dissection (Bau et al. 2017) aligns individual units with human concepts. Superposition analyses (Elhage et al. 2022) show that features can be distributed across many coordinates, and representation-engineering methods (Zou et al. 2023) read and steer concepts via mean-difference directions. Our concept channel probe (CCP) adapts these ideas to 3D medical encoders: we rank channels by calibration-set selectivity and fit a closed-form mean-difference detector on a sparse top-K subset, yielding a per-finding, training-free probe. Causal localization. Causal mediation analysis (Michel et al. 2019) and activation-level interventions for locating and editing knowledge in networks (Meng et al. 2022) establish necessity by ablating internal components. We apply causal intervention to frozen embeddings: zeroing each finding’s top-K channels selectively impacts its score, leaving others unchanged. Radiology report generation and evaluation. Report evaluation uses clinical metrics (RadGraph-F1 (Delbrouck et al. 2024), RadEval (Xu et al. 2025)). Rather than end-to-end generation, we decouple detection from verbalization: CCP produces reliable binary detections that a deterministic corpus template turns into FINDINGS text, which we evaluate under the same clinical and surface metrics. Method We avoid weight fine-tuning and opaque generative decoding by taking a training-free approach: each 3D image passes through a frozen vision encoder to produce embedding xix_i, on which all probes (CCP-K, sparsity, causal ablation, reports) operate without training. Frozen Backbone Inputs Let ViV_i denote a CT volume. Each backbone applies its own preprocessing; we cache the resulting frozen embedding per scan: xi=fV(preprocessV(Vi))∈ℝDx_i=f_V\! (preprocess_V(V_i) ) ^D (1) where fVf_V and D are fixed by the pretrained model (e.g., Pillar-0). We treat xix_i as a black-box input; backbone internals remain frozen. Our channel-ranking and ablation logic generalizes across architectures; only D and preprocessing differ. Concept Channel Probe (CCP-K) CCP-K (concept channel probe) is our primary linear probe on frozen channels. Given a multi-label task with C findings, for each finding c∈1,…,Cc∈\1,…,C\, we (i) rank channel coordinates j by selectivity score A~c,j A_c,j on calibration set (Eq. 2) and form top-K sets cKS^K_c; (i) fit a closed-form mean-difference direction wcw_c on each cKS^K_c (Eq. 3); (i) compute continuous scores scorec(i)score_c(i), calibrate thresholds τc _c on a held-out calibration split, and design binary detections did_i (Eq. 4); and (iv) further analyze finding-specific channels and causal ablation (Eqs. 5). The calibration split is used for channel ranking (wcw_c), score calibration, and τc _c. Here the finding index is c∈1,…,Cc∈\1,…,C\; channel index is j∈1,…,Dj∈\1,…,D\; circuit size K is a hyperparameter fixed before evaluation. ⁡(⋅) I\! (· ) is the indicator function (11 if the condition holds, 0 otherwise). Channel ranking and scoring. For pathology c, let Pc=i:finding c presentP_c=\i:finding c present\ and Nc=i:finding c absentN_c=\i:finding c absent\ on the calibration set. We define per-coordinate selectivity following network dissection (Bau et al. 2017): Ac,j=1|Pc||Nc|∑p∈Pc∑n∈Nc⁡(xp,j>xn,j),A~c,j=Ac,j−12A_c,j= 1|P_c||N_c|\!\! _p∈ P_c _n∈ N_c\! I\! (x_p,j>x_n,j ), A_c,j=A_c,j- 12 (2) Ac,jA_c,j is the pairwise win rate (equivalent to per-channel AUROC) and is invariant to finding prevalence—it measures separability, not class ratio. Centering to A~c,j A_c,j puts chance at zero. This makes ranking robust to the severe class imbalance in multi-label CT (prevalence ranges ∼ 7% to ∼ 45%). We rank channels by A~c,⋅ A_c,· and extract the top-K set cKS^K_c, then fit a mean-difference projection following (Zou et al. 2023) to score unseen volumes as a sparse alternative to fine-tuning (Hamamci et al. 2026): wc=μcKP−μcKN∥μcKP−μcKN∥⏟fit on calibration set,scorec(i)=⟨xi,cK,wc⟩=∑j∈cKxi,jwc,j⏟applied per volume w_c= μ^P_S^K_c-μ^N_S^K_c μ^P_S^K_c-μ^N_S^K_c _fit on calibration set, score_c(i)= x_i,S^K_c,\,w_c =\!\! _j ^K_c\!\!x_i,j\,w_c,j_applied per volume (3) Here μcKPμ^P_S^K_c and μcKNμ^N_S^K_c are class means over calibration positives and negatives, restricted to top-K channels. This mean-difference approach avoids bias from class imbalance, unlike pooled or least-squares fits. Critically, cKS^K_c and wcw_c are computed only on the calibration set. At inference, both are frozen; scoring requires extracting xi,cKx_i,S^K_c and computing its dot product with wcw_c. The test volume never enters μPμ^P or μNμ^N, ensuring no data leakage. Binary detections. We turn each continuous probe score into a present/absent prediction by comparing scorec(i)score_c(i) to a per-finding cutoff τc _c: di,c=⁡(scorec(i)≥τc),di=(di,1,…,di,C)∈0,1Cd_i,c= I\! (score_c(i)≥ _c ), d_i=(d_i,1,…,d_i,C)∈\0,1\^C (4) where di,c=1d_i,c=1 means CCP predicts finding c on volume i; did_i is also the detection vector fed to the report generation process afterwards. On the calibration split we select each τc _c by sweeping candidate cutoffs on the calibration scores. We use two operating points depending on the downstream task: for the report-generation detections did_i we choose the cutoff that maximizes per-finding F1, whereas for the classification benchmark (Table 1) we follow the CT-CLIP protocol and place τc _c at the dev ROC point closest to the top-left corner—a prevalence-independent operating point that does not depend on the positive/negative ratio. In both cases we then freeze cK,wc,τcc=1C\S^K_c,w_c, _c\_c=1^C and apply the same rule on the test split without refitting. Causal ablation. The CCP-K classification selects the global top-K coordinates for each finding. To investigate whether these channels are necessary (i.e., causally responsible for diagnostic scores), we perform controlled ablation following Michel et al. (2019) and Meng et al. (2022): for source finding c, define cablS^abl_c as the top-K coordinates with largest A~c,j A_c,j on the calibration split. We set xi,j′=0x _i,j=0 for j∈cablj ^abl_c before recomputing frozen probe scores on a held-out split. The drop matrix is: Δc→c′ _c→ c =AUROCc′(xi)−AUROCc′(xi′), =AUROC_c \! (\x_i\ )-AUROC_c \! (\x _i\ ), (5) sel =1C∑c=1C(Δc→c−1C−1∑c′≠cΔc→c′) = 1C _c=1^C ( _c→ c- 1C-1 _c ≠ c _c→ c ) where C is the number of findings. For each source c, the parenthesis in Eq. 5 is the gap between the on-target drop Δc→c _c→ c and the mean off-target drop over c′≠c ≠ c, and sel averages that gap over all c. Large sel means a finding’s specific channels are necessary for its own score but largely unnecessary for others. Training-free report generation We generate reports deterministically from did_i (Eq. 4) using a corpus-derived template. From training reports (excluding calibration examples), we extract: (i) T0T_0, the most frequent FINDINGS report (a common "normal chest" skeleton); and (i) uc\u_c\, the most common positive sentence per finding c with matching keywords. At test time, we edit T0T_0: (1) split sentences, tag keywords, mark negation; (2) remove sentences for absent findings, append ucu_c for di,c=1d_i,c=1, generate IMPRESSION. The verbalizer never introduces findings beyond did_i (no hallucinated positives). Dataset method AUROC F1 Accuracy PR CT-RATE (n=1564n=1564) Random 0.500 0.566 0.507 0.189 CT-CLIP zero-shot§ 0.731 0.707 0.668 0.323 CT-CLIP ClassFine§ 0.756 0.724 0.689 0.339 CT-CLIP VocabFine§ 0.756 0.738 0.705 0.353 Pillar-0 ClassFine† 0.833 0.800 0.778 0.431 Pillar-0 zero-shot∗ 0.682 0.693 0.652 0.291 CCP-Full— Pillar-0 0.793 0.782 0.760 0.400 CCP-10 — Pillar-0‡ 0.798 0.790 0.768 0.405 RadChest-CT (n=3630n=3630) Random 0.505 0.565 0.506 0.245 CT-CLIP zero-shot§ 0.629 0.637 0.592 0.335 CT-CLIP ClassFine§ 0.643 0.644 0.599 0.346 CT-CLIP VocabFine§ 0.650 0.677 0.636 0.346 Pillar-0 ClassFine† 0.713 0.609 0.570 0.335 Pillar-0 zero-shot ∗ 0.634 0.579 0.533 0.295 CCP-Full — Pillar-0 0.683 0.596 0.553 0.324 CCP-10 — Pillar-0‡ 0.686 0.622 0.581 0.333 Table 1: Multi-abnormality classification (classification metrics: mean AUROC; mean per-finding weighted F1, accuracy, and precision (PR) after ROC upper-left thresholds on calibration set: CT-RATE official test set (n=1564n=1564) and RadChest-CT (n=3630n=3630). §CT-CLIP rows extracted from Hamamci et al. (2026); †Pillar ClassFine: trained on CT-RATE train labels (∼ 39k volumes); encoder frozen. ∗zero-shot: Hamamci et al. (2026) present/absent prompt pairs ("c is present." / "c is not present.") embedded with each backbone’s text encoder, scored against cached xix_i via softmax over two cosines —the same recipe as cited CT-CLIP zero-shot, applied to Pillar-0 (1152-d) or Merlin (512-d, Clinical-Longformer); no label training. ‡CCP-10: training-free channel probe + calibration set thresholds. Bold = best supervised per column; bold underline = best training-free per column. Experimental setup Data. We evaluate on three public cohorts: (1) CT-RATE: 3D chest CTs with radiology reports and C=18C=18 multi-label finding annotations. We use the official validation split (15641564 CTs) for test evaluation and a 600600-volume calibration set from train for channel ranking, probe fitting, and threshold calibration. (2) RadChest-CT (Draelos et al. 2021): 36303630 external chest CTs with 8484 abnormality labels (5252 location-based). Following the protocol of Hamamci et al. (2026), detectors trained on CT-RATE dev are evaluated without refitting, using CT-RATE’s 18-finding ontology. (3) Merlin Abdominal CT (Blankemeier et al. 2024): paired abdominal CTs with reports and C=30C=30 multi-label finding annotations. Each finding is coded 11 (present), 0 (absent), or −1-1 (not assessed); we treat −1-1 as missing during channel ranking and probe fitting. Because prevalence is highly imbalanced across findings, we fit a detector for any finding with at least five positive calibration volumes and abstain otherwise; on CT-RATE all 1818 findings qualify, while a few of the rarest RadChest-CT and Merlin native labels fall below this floor. 3D VLM Backbones and K selection We run CCP on frozen precomputed vision-encoder embeddings from Pillar-0 (D=1152D=1152) and Merlin (D=512D=512). On Pillar-0 channel ranking and probe directions are fit on a 600600-volume calibration set; K is chosen on a disjoint 800800-volume train holdout by grid-searching K∈1,2,5,10,20,50,100,200,DK∈\1,2,5,10,20,50,100,200,D\ and taking the smallest K within 0.0050.005 mean AUROC of full-D (K∗=10K^*=10). We use the same K∗=10K^*=10 for Merlin without separate tuning. Classification metrics. F1, accuracy, and precision metrics follow CT-CLIP evaluation protocol (Hamamci et al. (2026)’s public evaluation code). Mean AUROC is threshold-free on raw scores. We report the numbers for CT-CLIP from the original paper since the public checkpoint is not available. Report evaluation metrics. For report generation, we measure (i) Clinical efficacy: micro/macro-F1 from a RadBERT-CT labeler on generated text vs. official CT-RATE volume labels (same GT as CCP), (i) Surface NLG: BLEU-1 and ROUGE-L vs. reference reports, (i) Factual structure: RadGraph-F1 (Delbrouck et al. 2024), and (iv) Reference-aligned: RadEval (Xu et al. 2025) F1RadBERT-CT (Wald et al. 2025) on generated vs. reference report pairs. Results Multi-label classification Table 1 summarizes results on CT-RATE and RadChest-CT. CCP uses frozen Pillar-0 embeddings. Baselines include: CT-CLIP zero-shot and ClassFine/VocabFine (cited from (Hamamci et al. 2026)); zero-shot prompts ("c is present/absent") scored via text-encoder similarity; and Pillar-0 ClassFine, a supervised linear head. CCP-10 is the best training-free method on both splits; supervised Pillar-0 ClassFine leads overall. On Pillar-0, training-free CCP-10 (0.7980.798) approaches supervised ClassFine (0.8330.833) without label training, while zero-shot stays lower (0.6280.628–0.6820.682 AUROC). On unseen RadChest-CT, CCP-10 remains best training-free (AUROC 0.6860.686, F1 0.6220.622), retaining ∼86% 86\% in-domain AUROC vs. zero-shot. Supervised CT-CLIP VocabFine leads F1 (0.6770.677); Pillar ClassFine reaches higher AUROC (0.7130.713) but lower F1 (0.6090.609). Training-free report generation We convert CCP-10 detections into reports on 15641564 test volumes, holding detections fixed and varying only the verbalizer. Table 2 reports clinical efficacy and NLG metrics. We compare: Structured template: PRESENT/ABSENT lists formatted as FINDINGS/IMPRESSION. Corpus-based template: edits normal-study template T0T_0 with positive sentences uc\u_c\. Frozen LLM: off-the-shelf LLM (Qwen3-VL-8B-Instruct) generates reports from lists. Retrieve-then-edit: k-N retrieves closest training report, then edits to match did_i. k-N constrained: k-N retrieval limited to normal reports, then edit to match did_i. CT-CHAT: trained chest CT VLM (Hamamci et al. 2026). Method Tr. μ 1 M-F1 B1 R-L Lt. Structured template no .441 .408 .015 .064 .24 Frozen LLM no .315 .317 .057 .081 1.15 Courpus-based Template. no .549 .492 .483 .379 .24 Retrieval-edit no .540 .484 .439 .314 .24 k-N retrieval no .544 .486 .414 .297 .24 CT-CHAT† yes .184 – .373 .326 5.5 Radiologist – .962 .944 – – – Table 2: Report generation on CT-RATE test (n=1564n=1564). Clinical Micor-F1 (μ 1), Macro-F1 (M-F1), evaluated by F1RadBERT-CT vs. official labels. NLG: BLEU-1 (B1) / ROUGE-L (R-L) vs. radiologist FINDINGS. Lateny (Lt.) as wall-clock time to produce one report end-to-end based on second per volume (s/v). CCP-10 corpus-derived template verbalizer is based on Pillar-0. †CT-CHAT: extracted from Hamamci et al. (2026). The CCP-10 corpus-based template is the strongest training-free verbalizer on all three axes: RadBERT-CT F1 0.5490.549 micro / 0.4920.492 macro (vs. cited CT-CHAT 0.1840.184), BLEU-1 0.4830.483 / ROUGE-L 0.3790.379 (vs. CT-CHAT 0.3730.373 / 0.3260.326), and latency 0.240.24 s/vol—∼ 23× faster than CT-CHAT (5.55.5 s/vol). Retrieve-then-edit matches clinical F1 (0.5400.540) but drops NLG (BLEU 0.4390.439, ROUGE-L 0.3140.314) due to non-corpus phrasing. Structured template and frozen LLM sacrifice prose style (BLEU ≤0.064≤ 0.064 and ∼ 0.01 respectively). Pairwise bootstrap confirms retrieve-then-edit and k-N lag corpus F1 (p<0.001p<0.001), with corpus F1 in tight 95% CI (Table 3). Statistical analysis. We assess uncertainty via paired volume bootstrap (n=1564n=1564 test volumes, B=2000B=2000 resamples; calibration/dev excluded). Point estimates report 95% bootstrap confidence intervals. Pairwise comparisons use two-sided bootstrap p-values with shared resampling indices. Classification uses mean macro-AUROC (threshold-free); report generation uses RadBERT-CT micro-F1 (Sec. Experimental setup). Table 3 summarizes primary comparisons. Comparison Δ 95% CI p Report generation (Clin-F1 vs. corpus template) Corpus template: 0.5490.549 [0.537, 0.560] (absolute CI). Structured template -0.078 [-0.086, -0.071] p<0.001p<0.001 Retrieve-then-edit -0.009 [-0.013, -0.004] p<0.001p<0.001 k-N retrieval -0.005 [-0.007, -0.002] p<0.001p<0.001 Frozen LLM (s2t) -0.233 [-0.245, -0.221] p<0.001p<0.001 Table 3: Paired bootstrap tests on CT-RATE test (n=1564n=1564, B=2000B=2000). Clin-F1: F1RadBERT-CT micro-F1; classification: mean macro-AUROC. Reference row for verbalizers: corpus template (CCP-10). Classification transfer. Sce. Calibration → Test AUROC μ-F1 cross-institution (Pillar-0 chest) A RadChest-CT valid → CT-RATE test 0.726 0.462 A CT-RATE dev → RadChest-CT all 0.686 0.622 cross-anatomy/backbone Merlin B abd. CT valid → abd. CT test (3030 findings) 0.839 0.832 — zero-shot† — 0.647 — supervised† — 0.641 anatomy mismatch, Merlin encoder/Chest labels C CT-RATE train → CT-RATE test 0.733 0.733 C CT-RATE train → RadChest-CT 0.605 – Report generation transfer. Sce. Calibration → Test F1 RG-F1 B-1 R-L BERT cross-institution (Pillar-0 chest) A RadChest-CT valid → CT-RATE test 0.448 0.284 0.471 0.352 – Merlin abdomen B abd. valid → abd. test – 0.262 0.214 0.226 0.677 — Merlin+RadLlama† – 0.293 0.102 – 0.588 Table 4: CCP-10 transfer summary. † extracted from Hamamci et al. (2026); Blankemeier et al. (2024) Transfer across datasets, backbones, and anatomy Since our concept channels probe (CCP) is defined entirely on frozen embeddings, the same recipe (K=10K=10) can be applied under three distinct transfer scenarios (Table 4): (A) Cross-institution: we use the same VLM backbone and chest anatomy, but different hospital/label ontology (Pillar-0 on CT-RATE vs. RadChest-CT; (B) Cross-anatomy/backbone: we apply the same CCP recipe to another 3D VLM backbone as Merlin on Abdominal CT dataset (Blankemeier et al. 2024); (C) Anatomy mismatch: we investigate whether the chest finding labels probed on an abdominal-pretrained Merlin encoder can be applied to CT-RATE and RadChest-CT. For each scenario we rank channels, fit probe directions, and calibrate thresholds entirely on the calibration split of the source dataset, followed by evaluation on the test target data split. Report generation metrics utilize a fixed, corpus-derived template verbalizer, varying only the underlying detection source. Scenario (A): Pillar-0 cross-institution. We ask whether CCP probes fit at one hospital transfer to the other while keeping the frozen 3D VLM backbone (Pillar-0) fixed. Probes are fit in each hospital’s native label space (1818 on CT-RATE dev; 8484 binary on RadChest-CT), but we evaluate on the target only through the mapped CT-RATE ontology similar to Hamamci et al. (2026). In classification, as shown in Table 4 (scenario A), the bidirectional transfer is asymmetric but strong RadChest-CT→ -RATE test reaches AUROC 0.7260.726 and F1 0.4620.462; CT-RATE→ -CT all volumes reaches 0.6860.686 / 0.6220.622 (also reported in Table 1). In report generation, Table 4 (scenario A), we keep the CT-RATE corpus template and test split fixed and swap only the detection source to probes calibrated on RadChest-CT, which yields F1 0.4480.448, BLEU 0.4710.471, and ROUGE-L 0.3520.352 vs. radiologist FINDINGS. It can be seen that without CT-RATE detector labels, cross-institution transfer retains ∼82% 82\% of F1 in-domain report generation (see 2) and still exceeds CT-CHAT performance. Scenario (B): Merlin cross-anatomy/backbone We apply CCP-10 to abdominal CT classification by ranking sparse channels in frozen Merlin contrastive embeddings, and calibrating per-finding probe directions and thresholds on the valid split with 3030 findings, and evaluating the frozen channels on the test set. As shown in Table 4 (scenario A), in classification task, CCP-10 reaches AUROC 0.8390.839 and F1 0.8320.832, exceeding Merlin zero-shot and supervised (Blankemeier et al. 2024) 0.6470.647 and 0.6410.641 F1, respectively. In report generation (Table 4, scenario B): with the same CCP-10 detections, we mine a corpus template from Merlin train FINDINGS, apply the same deterministic edit rule. Our proposed method yields RadGraph-F1 (RG-F1) 0.2620.262, BLEU-1 (B1) 0.2140.214, ROUGE-L (R-L) 0.2260.226, and BERTScore F1 (BERT) 0.6770.677. Compared to Merlin+RadLlama (Blankemeier et al. 2024), our training-free method improves BLEU-1 and BERTScore (0.2140.214 and 0.6770.677 vs. 0.1020.102 and 0.5880.588) but yields lower RadGraph-F1 (0.2620.262 vs. 0.2930.293). Regime (C): Abdominal encoder Merlin on chest Labels. anatomy mismatch We apply the same CCP-1010 recipe to CT-RATE chest findings but swap the frozen encoder to abdominal-pretrained Merlin—an intentional anatomy mismatch that complements Blankemeier et al. (2024)’s external benchmarks, where matched-anatomy zero-shot F1 drops from 0.7410.741 to 0.6470.647 under hospital distribution shift yet frozen Merlin still generalizes to external chest CT in their linear-probe evaluation (Blankemeier et al. 2024). Probes rank channels, fit directions, and calibrate thresholds on CT-RATE dev, then evaluate frozen readouts on official CT-RATE test (n=1564n=1564) and on all RadChest-CT volumes (n=3630n=3630; mapped ontology). In classification (Table 4, regime C), we can see on CT-RATE test, CCP-10 reaches AUROC 0.7330.733 and F1 0.7330.733; near original CT-CLIP zero-shot (0.7310.731 / 0.7070.707; but below chest-native Pillar-0 CCP-10 (0.7980.798 / 0.7900.790). Cross-institution evaluation on RadChest-CT all volumes yields AUROC 0.6050.605, well below regime (A) Pillar-0 transfer (0.6860.686 / 0.6220.622). Regime (C) therefore complements Merlin’s external chest benchmarks (Blankemeier et al. 2024), and we observe that the CCP procedure transfers across backbones, but encoder pretraining and probe labels must align, as shown in Scenarios A and B. Analysis and discussion CCP-10 succeeds on frozen chest and abdominal encoders. Here we analyze where that signal lives in the embedding. We ask: (i) Are findings encoded by sparse channels? (i) Are these channels causally necessary? (i) Does this hold across anatomies? Each finding lives in ∼ 10 channels. We rank channels by per-concept AUROC on the calibration set, apply CCP with varying top-K, and evaluate on 15641564 test volumes at K=10K=10. Fig. Analysis and discussion shows mean test AUROC across 18 CT-RATE findings vs. top-K channels on Pillar-0. Performance rises sharply from top-1 (0.7520.752) to top-10 (0.7980.798), saturates by top-50 (0.8040.804), and reaches 0.7930.793 with all D=1152D=1152 coordinates. Top-10 matches full-dimension AUROC (0.7980.798 vs. 0.7930.793) using only 1010 coordinates per finding (∼ 100× fewer), and beats zero-shot text prompting (0.6820.682). Top-10 can exceed full-embedding AUROC on individual findings (e.g., lung nodule 0.7340.734 vs. 0.6830.683, atelectasis 0.7460.746 vs. 0.6970.697) by dropping noisy dimensions. Table 5 lists each finding with its channel set c10S^10_c and test AUROC. Gaps are modest in both directions (|ΔAUROC|≤0.05| |≤ 0.05 for all 18 labels). Hiatal hernia shows the largest gap (top-10: 0.0270.027 below full). Fluid findings stay near ceiling (pleural effusion 0.928/0.9440.928/0.944, consolidation 0.905/0.8860.905/0.886). Causal ablation: Do sparse channels matter? We show that a few channels (K=10K=10) correlate strongly with each finding; however, correlation alone does not establish that the selected coordinates are necessary for discrimination. On CT-RATE test set (Pillar-0, n=1564n=1564), we zero cabl=cKS^abl_c=S^K_c (Eqs. 3, 5) for each source finding c and record Δc→c′ _c→ c for every scored finding c′c . Figure 3 displays the resulting 18×1818×18 matrix with source c on rows and scored c′c on columns (Eq. 5). We ablate the same dev-ranked top-K channels that CCP uses for each finding (cabl=cKS^abl_c=S^K_c). If those coordinates encoded only generic shared signal, zeroing finding c’s circuit would lower every detector by about the same amount; instead, the mean on-target AUROC drop is 0.00600.0060 versus 0.000030.00003 off-target (∼ 20×), so each finding’s channels are selectively necessary for its own score rather than broadly shared across all labels. Some off-diagonal entries remain elevated among clinically related findings (e.g., arterial and coronary calcification; emphysema with atelectasis and lung opacity), consistent with partial circuit overlap among related pathologies. Finding c10S^10_c top-1010 all D medical material 525, 923, 941, 904, 589, 1110, 848, 429, 870, 457 0.889 0.849 arterial wall calcification 792, 897, 1145, 564, 588, 607, 490, 981, 889, 97 0.864 0.863 cardiomegaly 897, 1103, 1145, 813, 423, 607, 792, 1045, 761, 429 0.878 0.863 pericardial effusion 552, 627, 761, 570, 1087, 923, 560, 872, 609, 446 0.813 0.767 coronary artery wall calcification 792, 1145, 588, 564, 490, 607, 897, 889, 407, 981 0.879 0.866 hiatal hernia 1078, 875, 607, 981, 1126, 457, 521, 588, 490, 407 0.652 0.679 lymphadenopathy 897, 1137, 1045, 1072, 813, 549, 985, 604, 634, 901 0.713 0.721 emphysema 771, 1100, 1009, 385, 792, 1107, 716, 429, 1069, 545 0.760 0.770 atelectasis 751, 584, 586, 623, 483, 613, 851, 968, 1039, 967 0.746 0.697 lung nodule 643, 508, 661, 663, 557, 386, 462, 528, 442, 543 0.734 0.683 lung opacity 404, 595, 721, 719, 517, 648, 743, 707, 594, 602 0.856 0.858 pulmonary fibrotic sequela 559, 586, 578, 613, 751, 454, 680, 1066, 727, 1064 0.682 0.692 pleural effusion 570, 609, 446, 395, 552, 761, 498, 659, 560, 444 0.928 0.944 mosaic attenuation pattern 897, 1103, 1007, 879, 858, 813, 784, 894, 982, 1045 0.792 0.786 peribronchial thickening 885, 564, 239, 1069, 429, 771, 457, 484, 904, 792 0.717 0.744 consolidation 689, 426, 648, 413, 551, 549, 600, 901, 730, 418 0.905 0.886 bronchiectasis 781, 716, 454, 1107, 1000, 961, 545, 397, 1100, 577 0.727 0.753 interlobular septal thickening 743, 1097, 1072, 428, 740, 482, 1013, 927, 719, 721 0.823 0.861 Table 5: Per-finding CCP-K∗K^* circuits on Pillar-0 (CT-RATE validation, n=1564n=1564, D=1152D=1152). For each finding, cK∗S^K^*_c lists dev-ranked channel indices (Eq. 3); top-K∗K^* and all-D columns are threshold-free test ROC-AUC. Figure 2: Sparsity recovery: test classification AUROC vs. number of top channels per finding. A handful of channels recover most of the full-feature performance on Pillar-0. Figure 3: Causal ablation on CT-RATE validation with frozen Pillar-0. Row c: AUROC changes after zeroing source finding c’s dev-ranked top-K=10K=10 circuit (cabl=cKS^abl_c=S^K_c); column c′c : scored finding. Cell (c,c′)(c,c ) shows Δc→c′ _c→ c from Eq. 5. A strong diagonal indicates selective necessity; off-diagonal blocks indicate shared circuits among related findings. Figure 4: Merlin abdominal CCP circuits by finding category (Figure 2c grouping). Each row is one of Merlin’s 1111 clinical categories (3030 findings total). Columns show embed coordinates with highest category-level selectivity (max AUROC−0.5-0.5 across findings in that category; valid split). Color intensity = selectivity. Merlin circuits by clinical category. We aggregate the top-1010 CCP circuits into 1111 finding categories, following Blankemeier et al. (2024). Fig. 4). Category-level selectivity (max over findings of A~c,j=AUROCc,j−0.5 A_c,j=AUROC_c,j-0.5 per coordinate) localizes sparse channels to clinically meaningful anatomy (e.g., lower-thorax calcification and cardiac findings share coordinates, whereas pleural effusion uses a separate circuit; peritoneal ascites and free air reach mean top-1010 AUROC 0.8950.895. Vasculature findings, by contrast, show heterogeneous top-1010 sets, consistent with diverse vascular appearances. Cross-category Jaccard overlap is near zero for unrelated groups (liver vs. lower thorax) but rises where pretraining anatomy overlaps (pancreas/spleen vs. lower thorax, 0.430.43), mirroring the partial circuit sharing among related chest findings in Fig. 3. Together, these results suggest contrastive 3D pretraining concentrates finding information in sparse coordinates rather than spreading it uniformly. CCP localizes, thresholds, and ablates these coordinates without training. Interpretability vs. supervised finetuning. Supervised finetuning (CT-CLIP ClassFine/VocabFine, Pillar ClassFine; Table 1) optimizes predictions by updating decision boundaries. CCP localizes where information lives by ranking channels and fitting a closed-form probe on frozen weights. On CT-RATE, CCP-10 exceeds reported CT-CLIP zero-shot and remains the best training-free probe, uniquely enabling sparsity and ablation analysis. On RadChest-CT, CCP-10 leads AUROC but lags VocabFine on weighted F1. Finetuning optimizes scores; CCP audits deployed encoders and provides channel-level explanations alongside predictions. Detection-grounded reports. We separate detection (CCP) from verbalization, avoiding end-to-end VLM failure modes. CT-CHAT achieves BLEU-1 0.3730.373 but F1 only 0.1840.184, drifting toward normal wording and missing abnormalities. CCP-10 + corpus template wins on both metrics (F1 0.5490.549, BLEU-1 0.4830.483) at ∼ 23× lower latency. Two checks support this design. Unrelated reports score BLEU-1 0.3850.385 against each other, so BLEU reflects boilerplate, not correctness. Corpus templates preserve institutional negations better than frozen LLMs. Retrieve-then-edit lags corpus template on F1, showing that with trustworthy detections, the bottleneck is faithful reconciliation to did_i, not generation. On abdominal CT, CCP-10 + Merlin template improve NLG scores over Merlin+RadLlama without fine-tuning. Trained RadLlama achieves higher RadGraph-F1 through richer entity encoding. Transfer across institutions, backbones, and anatomy. The three regimes in Table 4 form a single generalization story: Regime (A): Sparse chest circuits from one hospital retain signal at another (AUROC 0.6860.686 / 0.7260.726; report F1 ∼ 82% of in-domain). Institutional distribution shift does not erase channel structure. Regime (B): The CCP recipe is backbone-portable. On abdominal Merlin, CCP exceeds Merlin zero-shot and supervised F1 (0.8320.832 vs. 0.6470.647 / 0.6410.641). Regime (C): Chest probes on abdominal Merlin remain in-domain (0.7330.733) but lag chest-native Pillar-0 and degrade under hospital shift (0.6050.605 vs. 0.6860.686). Encoder and probe anatomy must align; the ranking procedure transfers across backbones, not across mismatched anatomies. This work presents one consistent picture: Frozen 3D CT encoders store findings in sparse causally selective channel subsets. CCP localizes and names these coordinates training-free, whereas finetuning improves what is predicted but not where it is encoded. CCP-10 achieves competitive performance while providing interpretability, grounding language in reliable detections rather than free-form generation. The probe-and-readout recipe transfers across hospitals and backbones when anatomy matches, and breaks down when it does not (regime C). For deployed foundation models, AUROC alone is an incomplete account of trust. Users need assurance that findings rest on stable internal features. CCP audits deployed models by probing, naming, ablating, and verbalizing. A limitation: rare findings have higher-variance channel estimates (few positive calibration volumes). We mitigate by reporting threshold-free AUROC, using prevalence-independent ROC operating points, and excluding low-count findings. Whether channel-level explanations remain stable under finer pathology labels, longitudinal follow-up, and multimodal inputs remains future work. Conclusion In this work we show how frozen medical VLMs encode radiological findings in sparse, causally selective channels. CCP localizes, ablates, and converts these into predictions and reports on chest (Pillar-0) and abdominal (Merlin) settings. The probe-and-template pipeline is competitive with finetuned VLMs on classification and report metrics while operating at a fraction of the latency. CCP can serve as a fast perception module in multi-agent workflows, supplying structured detections and channel evidence to routing agents, reducing latency in multi-step pipelines. We view CCP as a practical route to auditing what and where encoders represent findings, enabling lightweight, repeated inference without full VLM cost. Future work should test whether channel circuits remain stable under finer pathology labels, longitudinal follow-up, and multimodal inputs. We will also evaluate CCP inside multi-agent benchmarks with latency and interpretability as first-class metrics. References K. K. Agrawal, L. Liu, L. Lian, M. Nercessian, N. Harguindeguy, Y. Wu, P. Mikhael, G. Lin, L. V. Sequist, F. Fintelmann, et al. (2025) Pillar-0: a new frontier for radiology foundation models. arXiv preprint arXiv:2511.17803. Cited by: Related Work. G. Alain and Y. Bengio (2016) Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644. Cited by: Related Work. D. Bau, B. Zhou, A. Khosla, A. Oliva, and A. Torralba (2017) Network dissection: quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 6541–6549. Cited by: Related Work, Channel ranking and scoring.. L. Blankemeier, J. P. Cohen, A. Kumar, D. Van Veen, S. J. S. Gardezi, M. Paschali, Z. Chen, J. Delbrouck, E. Reis, C. Truyts, et al. (2024) Merlin: a vision language foundation model for 3d computed tomography. Research Square, p. rs–3. Cited by: Related Work, Experimental setup, Transfer across datasets, backbones, and anatomy, Transfer across datasets, backbones, and anatomy, Transfer across datasets, backbones, and anatomy, Table 4, Analysis and discussion. J. Delbrouck, P. Chambon, Z. Chen, M. Varma, A. Johnston, L. Blankemeier, D. Van Veen, T. Bui, S. Truong, and C. Langlotz (2024) RadGraph-XL: a large-scale expert-annotated dataset for entity and relation extraction from radiology reports. In Findings of the Association for Computational Linguistics ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand and virtual meeting, p. 12902–12915. External Links: Link Cited by: Related Work, Experimental setup. R. L. Draelos, D. Dov, M. A. Mazurowski, J. Y. Lo, R. Henao, G. D. Rubin, and L. Carin (2021) Machine-learning-based multiple abnormality prediction with large-scale chest computed tomography volumes. Medical image analysis 67, p. 101857. Cited by: Experimental setup. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah (2022) Toy models of superposition. Transformer Circuits Thread. Note: https://transformer-circuits.pub/2022/toy_model/index.html Cited by: Related Work. I. E. Hamamci, S. Er, C. Wang, F. Almas, A. G. Simsek, S. N. Esirgun, I. Dogan, O. F. Durugol, B. Hou, S. Shit, et al. (2026) Generalist foundation models from a multimodal dataset for 3d computed tomography. Nature Biomedical Engineering, p. 1–19. Cited by: Related Work, Channel ranking and scoring., Table 1, Experimental setup, Experimental setup, Multi-label classification, Training-free report generation, Transfer across datasets, backbones, and anatomy, Table 2, Table 4. K. Meng, D. Bau, A. J. Andonian, and Y. Belinkov (2022) Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), External Links: Link Cited by: Related Work, Causal ablation.. P. Michel, O. Levy, and G. Neubig (2019) Are sixteen heads really better than one?. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32, p. . External Links: Link Cited by: Related Work, Causal ablation.. T. Wald, I. E. Hamamci, Y. Gao, S. Bond-Taylor, H. Sharma, M. Ilse, C. Lo, O. Melnichenko, A. Schwaighofer, N. C. Codella, et al. (2025) Comprehensive language-image pre-training for 3d medical image understanding. arXiv preprint arXiv:2510.15042. Cited by: Experimental setup. J. Xu, X. Zhang, J. Abderezaei, J. Bauml, R. Boodoo, F. Haghighi, A. Ganjizadeh, E. Brattain, D. Van Veen, Z. Meng, D. W. Eyre, and J. Delbrouck (2025) RadEval: a framework for radiology text evaluation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, I. Habernal, P. Schulam, and J. Tiedemann (Eds.), Suzhou, China, p. 546–557. External Links: Link, Document, ISBN 979-8-89176-334-0 Cited by: Related Work, Experimental setup. A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A. Dombrowski, et al. (2023) Representation engineering: a top-down approach to ai transparency. arXiv preprint arXiv:2310.01405. Cited by: Related Work, Channel ranking and scoring..

Instant research discovery