Paper deep dive
ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology
Duncan Stothers, Ren-Chin Wu, William Lotter
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 8/25/2026, 8:06:18 AM
Summary
The paper introduces ADMIL (Attention-Distilled Multiple Instance Learning), a selective-compute framework for pathology slide analysis. It distills the attention distribution of a large ABMIL teacher model (using Virchow2 foundation model embeddings) into a lightweight tile-selection model called PriorNet (based on EfficientNet). At inference, PriorNet scores foreground tiles, selects the top-K most informative ones, and invokes the expensive foundation model only on that subset. This approach matches full-teacher performance on BRACS, PANDA, and CAMELYON16 datasets while avoiding over 98% of foundation model inference costs.
Entities (8)
Relation Signals (8)
ADMIL → evaluatedon → BRACS
confidence 98% · Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance
ADMIL → evaluatedon → CAMELYON16
confidence 98% · Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance
ADMIL → evaluatedon → PANDA
confidence 98% · Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance
PriorNet → trainedwith → KL-divergence
confidence 96% · PriorNet learns the teacher attention distribution from raw tile pixels with KL divergence
PriorNet → isbasedon → EfficientNet-B0
confidence 95% · PriorNet uses an EfficientNet-B0 (18) backbone
ADMIL → uses → PriorNet
confidence 95% · ADMIL ... distills an ABMIL teacher's attention into a lightweight tile-selection model, PriorNet.
ADMIL → uses → Virchow2
confidence 95% · We use Virchow2 as the foundation model tile encoder.
ADMIL → reduces → compute cost
confidence 94% · At deployment, ADMIL reduces compute cost by decreasing the number of foundation model calls.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Attention-based multiple instance learning (ABMIL) using pathology foundation model embeddings is effective for slide-level tasks, but exhaustive inference requires applying a large image encoder to every foreground tile despite the subsequent attention distribution often concentrating over a small subset of informative regions. We introduce ADMIL (Attention-Distilled Multiple Instance Learning), a selective-compute framework that distills an ABMIL teacher's attention into a lightweight tile-selection model, PriorNet. Using an EfficientNet architecture, PriorNet learns the teacher attention distribution from raw tile pixels with KL divergence; at inference, it scores the foreground pool, selects the top-K tiles, and invokes the expensive foundation model only on that subset before a selected-bag ABMIL student predicts the slide label. Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance at K=4, 8, and 128 tiles, respectively, avoiding >98% of foundation model (Virchow2) tile embeddings and model inference FLOPs. Random and teacher-attention oracle controls show that this result depends on task-relevant selection rather than tile-count reduction alone. Quantitative and qualitative analyses suggest that PriorNet recovers the teacher's tile ordering with high fidelity while focusing on task-relevant morphological regions. ADMIL shows that nearly all expensive tile encodings can be removed without sacrificing slide-level performance, providing a potential path for more efficient deployment in clinical settings where latency and compute costs are key considerations.
Tags
Links
- Source: https://arxiv.org/abs/2608.22066v1
- Canonical: https://arxiv.org/abs/2608.22066v1
Trouble viewing inline? Open PDF directly →
Full Text
37,006 characters extracted from source content.
Expand or collapse full text
ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology Duncan Stothers Ren-Chin Wu William Lotter Note: Correspondence: lotterb@ds.dfci.harvard.edu Abstract Attention-based multiple instance learning (ABMIL) using pathology foundation model embeddings is effective for slide-level tasks, but exhaustive inference requires applying a large image encoder to every foreground tile despite the subsequent attention distribution often concentrating over a small subset of informative regions. We introduce ADMIL (Attention-Distilled Multiple Instance Learning), a selective-compute framework that distills an ABMIL teacher’s attention into a lightweight tile-selection model, PriorNet. Using an EfficientNet architecture, PriorNet learns the teacher attention distribution from raw tile pixels with KL divergence; at inference, it scores the foreground pool, selects the top-K tiles, and invokes the expensive foundation model only on that subset before a selected-bag ABMIL student predicts the slide label. Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance at K=4K=4, 88, and 128128 tiles, respectively, avoiding >98% of foundation model (Virchow2) tile embeddings and model inference FLOPs. Random and teacher-attention oracle controls show that this result depends on task-relevant selection rather than tile-count reduction alone. Quantitative and qualitative analyses suggest that PriorNet recovers the teacher’s tile ordering with high fidelity while focusing on task-relevant morphological regions. ADMIL shows that nearly all expensive tile encodings can be removed without sacrificing slide-level performance, providing a potential path for more efficient deployment in clinical settings where latency and compute costs are key considerations. Figure 1: ADMIL overview. Stage 1: A lightweight image encoder (PriorNet) learns to estimate the attention distribution produced by a frozen foundation model-based ABMIL teacher. Stage 2: PriorNet selects the top-K tiles for generating foundation model embeddings, which are used to train a separate top-K ABMIL predictor. At deployment, ADMIL reduces compute cost by decreasing the number of foundation model calls. Introduction Pathology foundation models have become a standard source of tile-level representations for whole-slide image (WSI) analysis. A typical pipeline partitions each WSI into foreground tiles, embeds every tile with a foundation model encoder, and aggregates the resulting bag with attention-based multiple-instance learning (ABMIL) (10; 3; 14; 5; 13; 20; 22). This paradigm has shown strong performance for a range of tasks, including metastasis detection, tumor subtyping, and histologic grading (3; 12; 1; 2; 6). Its computational profile, however, is poorly matched to clinical deployment. A single WSI may contain thousands to tens of thousands of foreground tiles, so exhaustive foundation model inference can dominate latency, compute costs, and throughput. This cost is particularly striking because trained ABMIL models rarely use all tiles equally. Their slide prediction is often supported by a highly concentrated attention distribution over a small subset of morphologically informative regions for the particular task. The expensive encoder is therefore applied broadly even though the final classifier concentrates on a narrow evidence set. This mismatch suggests a selective-compute strategy: use a cheap model to identify likely high-value tiles before invoking the expensive pathology foundation model. We introduce Attention-Distilled Multiple Instance Learning (ADMIL), a framework to substantially reduce the inference cost of foundation model-based ABMIL while preserving performance. Given a trained ABMIL model, ADMIL first distills its attention distribution into a lightweight encoder, PriorNet, which is trained using KL divergence to predict the teacher-assigned attention weight for each tile. For a user-defined deployment budget K, a separate ABMIL is then trained using foundation model embeddings from only the K tiles receiving the highest PriorNet scores. At inference, PriorNet scores the full foreground pool, the foundation model embeds only the selected tiles, and the corresponding top-K ABMIL model produces the slide prediction. This design differs from conventional model compression. PriorNet does not replace the foundation model or the ABMIL architecture; it gates access to the expensive encoder. It also differs from heuristic tissue filtering and random subsampling because the selection rule is distilled from task-specific teacher behavior. The resulting pipeline preserves the high-capacity representation where it matters while avoiding most foundation model evaluations elsewhere. Our contributions are threefold: • We formulate attention distillation as a selective-compute policy for foundation model-based ABMIL and show that a lightweight PriorNet trained by KL divergence recovers useful teacher evidence. • Across BRACS, PANDA, and CAMELYON16, ADMIL matches the full-teacher headline performance at small tile budgets, avoiding 98.5%98.5\%–99.8%99.8\% of foundation model tile embeddings and reducing model inference FLOPs by 98.3%98.3\%–99.6%99.6\%. • We quantitatively and qualitatively evaluate tile selection fidelity, observing strong concordance between teacher- and student-selected tiles. Related Work Weakly supervised WSI classification. WSI classification is naturally formulated as multiple-instance learning, where a slide is a bag of tiles and supervision is available only at the bag level. Attention-based MIL introduced a permutation-invariant learned aggregation rule consisting of an interpretable, attention-weighted average of tile features (10). Large-scale studies and later variants such as CLAM, DSMIL, and TransMIL demonstrated the effectiveness of weakly supervised WSI learning (3; 14; 11; 17). When combined with foundation models, ABMIL remains a standard approach and often still outperforms other variants (16). ADMIL is nonetheless complementary to potential improvements in the aggregator: it targets the upstream cost of producing foundation model features for every candidate tile. Pathology foundation models. Self-supervised and multimodal pathology models have become strong general-purpose feature extractors. CTransPath and HIPT introduced transformer-based and hierarchical pretraining for histopathology (21; 4); UNI and CONCH demonstrated broad transfer across visual and multimodal tasks respectively (5; 13); and Virchow and Prov-GigaPath further scaled pathology pretraining and whole-slide modeling (20; 22). These models improve downstream performance but make exhaustive tile encoding increasingly expensive. ADMIL is designed specifically for this high-capacity, high-cost feature-extraction regime. Efficiency for gigapixel pathology. Several prior works have sought to reduce the computational burden of WSI analysis through new MIL strategies. ZoomMIL introduced a differentiable coarse-to-fine MIL network that progressively identifies informative regions and allocates higher-resolution processing only to selected areas (19). HDMIL similarly distinguishes between high- and low-resolution processing by distilling high-resolution attention into a low-resolution pre-screening network (7). Recent work has specifically targeted the cost of pathology foundation models. H0-mini focuses on distilling the features of large foundation models directly, resulting in a more efficient encoder (9). EAGLE uses a cheaper WSI-level foundation model to select a fixed set of 25 tiles for processing by a larger foundation model (15). Rather than pursuing efficiency through multi-resolution processing (e.g., ZoomMIL, HDMIL), compressed feature extraction (e.g., H0-mini), or task-agnostic tile selection using a separately pretrained model (e.g., EAGLE), ADMIL distills a task-specific foundation model ABMIL teacher directly into a lightweight attention predictor. The resulting PriorNet ranks tiles before foundation model inference, retaining the original high-capacity encoder for informative regions while avoiding its application to the vast majority of the slide. Methods: ADMIL Figure 2: Slide-level performance versus selected tile budget. Columns correspond to CAMELYON16, PANDA, and BRACS. The top row reports each dataset’s primary metric and the bottom row its secondary metric. The black dotted line denotes full-teacher performance, which is compared to ADMIL, a random tile selector, and a top-K teacher oracle. The red dashed vertical line indicates the K value where ADMIL reaches teacher performance. The x-axis is logarithmic and reports both raw K and the corresponding mean foreground fraction. Error bars correspond to 95% confidence intervals. Figure 3: Teacher-support retrieval by PriorNet. Values correspond to the fraction of the teacher top-K tiles found among the PriorNet top-T tiles, averaged across test slides within each dataset. Cells with T<KT<K are masked. All panels use the same color scale. Figure 4: Example attention maps. Each row contains a representative test slide from one dataset (left: CAMELYON16, right: PANDA). Panels show the WSI thumbnail, full-bag teacher attention, selected-bag student attention, and the top attended tiles. Figure 5: Sensitivity to the PriorNet supervision tile budget. Each panel compares PriorNet models trained with 5%, 10%, or 20% of each slide’s foreground candidates. The black dotted line denotes the full-teacher reference. Error bars correspond to 95% confidence intervals. The 10% setting is the default used for main results; the other curves assess robustness to the supervision fraction. Problem Setting and Full-Bag Teacher A slide is represented by a foreground tile bag X=xii=1NX=\x_i\_i=1^N, where N varies by slide. A frozen foundation model fθf_θ maps each tile to an embedding zi=fθ(xi)z_i=f_θ(x_i). The full-bag teacher computes attention logits ai=wa⊤tanh(Wazi+ba)a_i=w_a (W_az_i+b_a) and normalized attention αi=exp(ai)∑j=1Nexp(aj). _i= (a_i) _j=1^N (a_j). The teacher slide representation is h=∑iαizih= _i _iz_i, followed by a task head gψ(h)g_ψ(h). The teacher is trained on the full foreground bag and then frozen. Its exported attention distribution provides tile-level supervision for PriorNet. Attention Distillation with KL Divergence PriorNet is a lightweight image-space model, pϕp_φ, that produces one scalar score si=pϕ(xi)s_i=p_φ(x_i) per foreground tile. Scores are normalized across the slide: qi=exp(si)∑j=1Nexp(sj).q_i= (s_i) _j=1^N (s_j). PriorNet is trained with the KL distillation objective, ℒprior=DKL(∥)=∑i=1Nαilogαi+ϵqi+ϵ.L_prior=D_KL( α\,\|\,q)= _i=1^N _i _i+εq_i+ε. Rather than training PriorNet uniformly over all tiles, we use attention-guided sampling to increase the representation of highly attended tiles while reducing training cost. Specifically, for each dataset, we train on Z%Z\% of foreground tiles in the training set, distributed evenly across WSIs to target an equal number of tiles, M, per WSI. The retained tiles per WSI consist of the top M/2M/2 ranked by teacher attention and an additional M/2M/2 sampled from the remaining foreground pool. For the KL distillation loss, the teacher’s attention distribution is re-normalized over the sampled tiles. In the main experiments, we set Z=10%Z=10\% and additionally assess sensitivity using Z∈5,20%Z∈\5,20\\%. During evaluation, deployment, and top-K ABMIL training, PriorNet scores the complete foreground tile pool. Top-K Selection and Selected-Bag ABMIL Let πs _s denote the permutation that orders PriorNet scores from largest to smallest. ADMIL selects SK(X)=πs(1),…,πs(min(K,N)).S_K(X)=\ _s(1),…, _s( (K,N))\. Only selected tiles are embedded by the foundation model at inference: zi=fθ(xi),i∈SK(X).z_i=f_θ(x_i), i∈ S_K(X). A separate top-K ABMIL is trained under this selected-bag condition. For i∈SK(X)i∈ S_K(X), it computes ρi=wb⊤tanh(Wbzi+b),βi=exp(ρi)∑j∈SK(X)exp(ρj), _i=w_b (W_bz_i+b_b), _i= ( _i) _j∈ S_K(X) ( _j), then forms hK=∑i∈SK(X)βizi,y^K=gω(hK),h_K= _i∈ S_K(X) _iz_i, y_K=g_ω(h_K), with optimization using the original slide-level task loss. Methods: Evaluation Approach We compare ADMIL against several baselines, evaluating predictive performance, inference efficiency, and agreement between PriorNet predictions and teacher ABMIL attention values. Baselines and Controls Full foundation model ABMIL teacher. The original ABMIL teacher embeds every foreground tile using the foundation model and serves as the full-compute performance reference. Random top-K. For each slide, K foreground tiles are sampled uniformly without replacement and used to train and evaluate a separate selected-bag ABMIL architecture. This controls for whether any subset of the same size is sufficient. Teacher top-K oracle. The oracle uses the teacher’s own full-bag attention to select SKoracle(X)=TopK(αii=1N).S_K^oracle(X)=TopK(\ _i\_i=1^N). It is not deployable because obtaining α requires full-bag foundation model inference, but it estimates the performance attainable when teacher evidence is recovered perfectly. Attention Fidelity Analysis PriorNet’s ability to recapitulate the teacher’s attention is evaluated using correlation and retrieval metrics. For every slide, we compute the Pearson and Spearman correlation between PriorNet and teacher normalized attention values. We then average these values across test-set slides to obtain dataset-level agreement metrics. We also measure teacher-support retrieval: for teacher support size K and PriorNet retrieval depth T≥KT≥ K, we compute Rℓ(K,T)=|TopK(ℓ)∩TopT(ℓ)|min(K,Nℓ).R_ (K,T)= |TopK( α_ ) (q_ ) | (K,N_ ). We evaluate K,T∈1,2,4,8,16,32,64,128K,T∈\1,2,4,8,16,32,64,128\ and mask cells where T<KT<K. This analysis compares full-foreground teacher attention α with the full-foreground PriorNet distribution q; it is distinct from the selected-bag student attention β. Sensitivity Analysis We assess sensitivity to the training tile sampling parameter Z. The default Z=10%Z=10\% setting is used for all headline results; the 5%5\% and 20%20\% settings test whether the method’s behavior depends strongly on this supervision fraction. All other training and evaluation procedures are held fixed across settings. This analysis is intended to assess robustness to Z, rather than to select an alternative headline configuration. Inference Efficiency Assessment Inference efficiency is assessed based on percentage of foundation model embeddings avoided and FLOPs saved. These calculations are reported for a critical budget KcritK_crit, which corresponds to the number of tile embeddings needed to reach teacher performance for a particular dataset and task. Details of these calculations are provided in the Appendix. Methods: Experimental Setup Datasets and Tasks We evaluated ADMIL on three public WSI benchmarks. CAMELYON16 contains lymph node biopsies for breast cancer metastasis detection (12; 8), with 270 WSIs for model development and 129 for testing and an average of 12,391 foreground tiles per WSI. PANDA consists of prostate biopsies for six-class ISUP grade classification (2) with an average of 531 foreground tiles per WSI. As PANDA does not have a canonical test split, we split the data into 90/10% development/testing corresponding to 9,553 and 1,062 WSIs, respectively. BRACS contains breast biopsies and we assess the three-class benign, atypical, and malignant classification task (1), with 458 and 87 WSIs for development and testing, respectively, with an average of 2,513 foreground tiles per slide. We performed three global runs for each dataset using seeds 0, 1, and 2. For each global seed, we kept the test partition fixed and split the development set into a label-stratified 70/30 training/validation split using the global seed. PriorNet and all ABMIL training used the training split; model checkpoints were selected based on the validation split. PANDA was split at the patient level because it contains multiple slides per patient; CAMELYON16 and BRACS only contain one slide per patient. Statistical Analysis We report AUROC and AUPRC for CAMELYON16, quadratic weighted kappa (QWK) and within-1 accuracy for PANDA, and balanced accuracy and macro one-vs-rest AUROC for BRACS. Unless otherwise noted, presented metrics correspond to average test-set performance across the three independent runs. Uncertainty is estimated with 1,000 hierarchical bootstrap replicates. Models and Training Model Architectures. We use Virchow2 as the foundation model tile encoder (20). Full-bag teachers and selected-bag students use non-gated ABMIL with a 512-dimensional tanh attention layer, a scalar attention scorer, and a task head with a 512-dimensional hidden layer. PriorNet uses an EfficientNet-B0 (18) backbone initialized with ImageNet-1K V1 weights and a scalar output head. All models operated on foreground-only tiles of size 224×224224× 224. PriorNet Training. A separate PriorNet model was trained on each dataset for 20 epochs using AdamW with an initial learning rate of 10−410^-4, batch size corresponding to 1 WSI, weight decay of 10−510^-5, cosine learning-rate decay, mixed precision, dropout of 0.2, and gradient-norm clipping at 1.0. As described above, PriorNet was trained using a KL-divergence loss, with checkpoint selection based on the validation split. PriorNet training was independent of deployment budget K. ABMIL Training. Teacher and student ABMIL models were optimized with AdamW for 40 epochs with weight decay of 10−410^-4, batch size of 1, cosine learning rate decay, mixed-precision training, dropout of 0.1, and gradient-norm clipping at 1.0. Teachers used an initial learning rate of 10−410^-4; students used an initial rate of 10−310^-3, as we found that a higher learning rate facilitated convergence when training with fewer tiles. CAMELYON16 used class-balanced binary cross entropy as the loss function; PANDA and BRACS used class-balanced cross entropy. A separate ABMIL student was trained for every evaluated tile budget K. Checkpoints for all ABMIL models were selected based on the best validation headline metric during training. Results ADMIL Matches Full-Bag Performance at Small Tile Budgets Figure 2 shows the performance of ADMIL across tile budgets compared to the full-teacher ABMIL model. For each dataset, a low-budget regime is observed in which ABMIL matches teacher performance with a small percentage of foundation model embedded tiles. On the primary metrics, ADMIL first reaches the teacher threshold at K=4K=4 for BRACS balanced accuracy, K=8K=8 for PANDA QWK, and K=128K=128 for CAMELYON16 AUROC. These budgets correspond to 99.8%99.8\%, 98.5%98.5\%, and 99.0%99.0\% fewer foundation model tile embeddings, respectively (Table 1), which translates to similar levels of model inference FLOPs saved given PriorNet’s lightweight architecture. The secondary metrics show the same qualitative behavior. ADMIL reaches the mean teacher AUPRC on CAMELYON16 at K=128K=128 (0.977; 95% CI [0.959, 0.991]), the mean teacher within-1 accuracy on PANDA at K=16K=16 (0.918; 95% CI [0.909, 0.928]), and the mean teacher macro one-vs-rest AUROC on BRACS at K=32K=32 (0.860; 95% CI [0.819, 0.899]). ADMIL’s performance via PriorNet-based tile selection also consistently approaches the teacher-selection oracle and substantially exceeds random sampling, especially at low K. Table 1: Performance at the descriptive critical tile budget KcritK_crit. FM saved indicates the percentage of foundation model tile embeddings avoided. Model FLOPs saved accounts for PriorNet, foundation model, and ABMIL operations at inference. Dataset Metric KcritK_crit Full teacher Teacher 95% CI ADMIL at KcritK_crit ADMIL 95% CI FM saved (%) FLOPs saved (%) BRACS Balanced Acc. 4 0.648 [0.594, 0.700] 0.670 [0.617, 0.720] 99.8 99.6 PANDA QWK 8 0.854 [0.838, 0.870] 0.858 [0.842, 0.874] 98.5 98.3 CAMELYON16 AUROC 128 0.976 [0.954, 0.991] 0.978 [0.960, 0.992] 99.0 98.7 PriorNet Recovers the Teacher’s Attention Ordering Table 2: Correlation between teacher attention and PriorNet scores. Values are computed per WSI and then averaged across WSIs. Dataset Pearson Pearson 95% CI Spearman Spearman 95% CI CAMELYON16 0.456 [0.425, 0.487] 0.799 [0.778, 0.821] PANDA 0.622 [0.532, 0.713] 0.839 [0.816, 0.861] BRACS 0.342 [0.246, 0.412] 0.579 [0.518, 0.628] The performance results suggest that PriorNet effectively identifies task-informative tiles by distilling the teacher model’s attention. To directly assess how well PriorNet recapitulates the teacher’s attention patterns, we computed the correlation between PriorNet-predicted and teacher-assigned tile attention using the test set of each dataset. PriorNet exhibits positive agreement with the teacher across all three datasets (Table 2). Spearman correlation is consistently higher than Pearson correlation, suggesting that PriorNet more faithfully recovers the teacher’s ranking of tile importance than the exact magnitude of its attention weights. Rank agreement is strongest on PANDA (0.839), followed by CAMELYON16 (0.799) and BRACS (0.579). The retrieval matrices in Figure 3 provide a direct selection-oriented view of fidelity. Despite an average of 12,391, 2,513, and 531 foreground tiles per WSI for CAMELYON16, BRACS, and PANDA respectively, PriorNet’s top selected tile matches the top teacher tile 14%, 9%, and 26% of the time for the three datasets respectively, with recall increasing as the PriorNet retrieval depth grows. Together with the correlation results, this indicates that the lightweight model learns a useful approximation to the teacher’s task-specific evidence ranking before foundation model embedding. Qualitative Review Finds Histologically Plausible Evidence Representative ABMIL-derived attention maps produced by the teacher and student models are shown in Figure 4. In the CAMELYON16 example (left), both models assign high attention to tiles containing tumor cells, consistent with the ground-truth metastasis label and the corresponding model predictions. In the PANDA example (right), several of the highest-attention tiles overlap between the two models and contain high-grade prostate carcinoma morphology consistent with the ISUP 5 label. In both examples, the student attention map largely aligns with the teacher while showing higher sparsity, driven by the top-K selection mechanism. Overall, this qualitative analysis provides a sanity check that the student attends to plausible morphology, complementing the quantitative analyses. Performance Is Robust Across PriorNet Supervision Budgets Results of the sensitivity analysis for the training tile-sampling percentage are contained in Figure 5. Across datasets and metrics, the three curves follow similar performance trajectories with overlapping confidence intervals, suggesting robustness to this hyperparameter. We retain 10%10\% as the default and interpret the 5%5\% and 20%20\% runs as evidence that the method is not narrowly tuned to one exact fraction. Discussion Our results show that the standard foundation model-based ABMIL pipeline contains substantial redundant tile-level computation. The full teacher processes every foreground tile, yet ADMIL reaches the same mean headline performance using only four selected BRACS tiles, eight PANDA tiles, and 128 CAMELYON16 tiles. The corresponding reduction in foundation model calls exceeds 98%98\% in all three datasets. This is not simply a consequence of bag redundancy: random top-K students perform worse in the low-budget regime, whereas the teacher-attention oracle and learned PriorNet selection retain much more of the full-bag performance. The magnitude of the compute reduction follows from the large per-tile asymmetry between the selector and the foundation model. The Virchow2 checkpoint requires 340.132340.132 GFLOPs per tile, compared with 0.7690.769 GFLOPs for PriorNet. Scanning the complete foreground pool with PriorNet contributes only about 0.226%0.226\% of the full-teacher model-forward FLOPs, after which Virchow2 is applied to just 0.16%0.16\%, 1.51%1.51\%, and 1.03%1.03\% of foreground tiles on BRACS, PANDA, and CAMELYON16, respectively. Consequently, estimated model inference savings remain close to the reduction in Virchow2 calls despite scoring every foreground tile with PriorNet. The attention-fidelity results clarify what PriorNet recovers. Spearman correlation is consistently higher than Pearson correlation, suggesting that exact probability calibration is less important than preserving the ordering of teacher-relevant tiles. Some variation across datasets is also observed, where higher correlations are observed for PANDA and CAMELYON16 than BRACS. BRACS combines heterogeneous benign, atypical, and malignant morphologies and uses a larger physical field of view, which may make local teacher attention harder to predict from isolated raw tiles. Nevertheless, BRACS still reaches the full-teacher balanced accuracy at only four selected tiles, suggesting that perfect global correlation is not necessary for effective decision-focused retrieval. Sensitivity experiments show that ADMIL is not tied to one exact PriorNet supervision budget. The 5%, 10%, and 20% curves retain the same qualitative shape and largely overlapping uncertainty, with small task-specific differences rather than a consistent ordering. Compared to strong prior works focusing on hierarchical spatial processing (19; 7) or task-agnostic model compression or tile selection (9; 15), ADMIL is designed specifically for efficient inference with high-performing, MIL-level foundation model pipelines. The benefit of foundation model representations lies in their adaptability across downstream tasks; however, in clinical use, predictions are ultimately made for specific tasks, and the most relevant tiles can differ accordingly. ADMIL learns to approximate this relevance based on teacher supervision, enabling highly streamlined inference while preserving the underlying model representations and prediction architecture. Several limitations remain. First, ADMIL depends on the stability and relevance of teacher attention. A high-performing teacher may attend to confounders or distribute evidence diffusely, reducing selector reliability. Second, KcritK_crit is a post-hoc summary of the test curve rather than a deployable validation-selected operating point; a production system should choose K on validation data. Third, we assessed ADMIL using three common benchmarks and a high-performing base foundation model, but future validation with additional datasets and FMs is an important direction. Finally, although ADMIL is designed to reduce inference time and computational cost for scalable deployment, it introduces additional overhead during training. Conclusion ADMIL converts a trained ABMIL teacher’s attention into a selective-compute policy. With a KL-trained PriorNet and a selected-bag ABMIL student, the method preserves teacher performance while avoiding most foundation model embeddings across three distinct tasks. Quantitative attention agreement, teacher-support retrieval, supervision-budget sensitivity, and morphology-consistent examples jointly support the central mechanism: the teacher’s implicit localization behavior can be distilled early enough to reduce expensive inference. This offers a simple and extensible path toward more deployable pathology foundation model systems. References Brancati et al. (2022) N. Brancati, A. M. Anniciello, P. Pati, D. Riccio, G. Scognamiglio, G. Jaume, G. De Pietro, M. Di Bonito, A. Foncubierta, G. Botti, M. Gabrani, F. Feroce, and M. Frucci BRACS: a dataset for breast carcinoma subtyping in h&e histology images. Database 2022, p. baac093. External Links: Document Cited by: Introduction, Datasets and Tasks. Bulten et al. (2022) W. Bulten, K. Kartasalo, P. C. Chen, P. Ström, H. Pinckaers, H. van Boven, R. Vink, et al. Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge. Nature Medicine 28, p. 154–163. External Links: Document Cited by: Introduction, Datasets and Tasks. Campanella et al. (2019) G. Campanella, M. G. Hanna, L. Geneslaw, A. Miraflor, V. W. K. Silva, K. J. Busam, E. Brogi, V. E. Reuter, D. S. Klimstra, and T. J. Fuchs Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine 25, p. 1301–1309. External Links: Document Cited by: Introduction, Weakly supervised WSI classification.. Chen et al. (2022) R. J. Chen, C. Chen, Y. Li, T. Y. Chen, A. D. Trister, R. G. Krishnan, and F. Mahmood Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 16144–16155. Cited by: Pathology foundation models.. Chen et al. (2024) R. J. Chen, T. Ding, M. Y. Lu, D. F. K. Williamson, G. Jaume, A. H. Song, B. Chen, A. Zhang, D. Shao, M. Shaban, M. Williams, L. Oldenburg, L. L. Weishaupt, J. J. Wang, A. Vaidya, L. P. Le, G. Gerber, S. Sahai, W. Williams, and F. Mahmood Towards a general-purpose foundation model for computational pathology. Nature Medicine 30, p. 850–862. External Links: Document Cited by: Introduction, Pathology foundation models.. Diab et al. (2025) A. R. Diab, E. E. Karn, R. Wu, E. S. Ruiz, and W. Lotter Leveraging foundation models for histological grading in cutaneous squamous cell carcinoma using pathfmtools. In Proceedings of the 5th Machine Learning for Health (ML4H) Symposium, Cited by: Introduction. Dong et al. (2025) J. Dong et al. Fast and accurate gigapixel classification with hierarchical distillation multiple instance learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, External Links: Document, Link Cited by: Efficiency for gigapixel pathology., Discussion. Ehteshami Bejnordi et al. (2017) B. Ehteshami Bejnordi et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318 (22), p. 2199–2210. External Links: Document Cited by: Datasets and Tasks. Filiot et al. (2025) A. Filiot, N. Dop, O. Tchita, A. Riou, T. Peeters, D. Valter, M. Scalbert, C. Saillard, G. Robin, and A. Olivier Distilling foundation models for robust and efficient models in digital pathology. External Links: 2501.16239, Link Cited by: Efficiency for gigapixel pathology., Discussion. Ilse et al. (2018) M. Ilse, J. M. Tomczak, and M. Welling Attention-based deep multiple instance learning. In Proceedings of the 35th International Conference on Machine Learning, p. 2127–2136. Cited by: Introduction, Weakly supervised WSI classification.. Li et al. (2021) B. Li, Y. Li, and K. W. Eliceiri Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 14318–14328. External Links: Document Cited by: Weakly supervised WSI classification.. Litjens et al. (2018) G. Litjens, P. Bandi, B. E. Bejnordi, O. Geessink, M. Balkenhol, P. Bult, A. Halilovic, M. Hermsen, R. van de Loo, R. Vogels, Q. F. Manson, N. Stathonikos, A. Baidoshvili, P. van Diest, C. Wauters, M. van Dijk, and J. van der Laak 1399 h&e-stained sentinel lymph node sections of breast cancer patients: the camelyon dataset. GigaScience 7 (6), p. giy065. External Links: Document Cited by: Introduction, Datasets and Tasks. Lu et al. (2024) M. Y. Lu, B. Chen, D. F. K. Williamson, R. J. Chen, I. Liang, T. Ding, G. Jaume, I. Odintsov, L. P. Le, G. Gerber, A. V. Parwani, A. Zhang, and F. Mahmood A visual-language foundation model for computational pathology. Nature Medicine 30, p. 863–874. External Links: Document Cited by: Introduction, Pathology foundation models.. Lu et al. (2021) M. Y. Lu, D. F. K. Williamson, T. Y. Chen, R. J. Chen, M. Barbieri, and F. Mahmood Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5 (6), p. 555–570. External Links: Document, Link Cited by: Introduction, Weakly supervised WSI classification.. Neidlinger et al. (2026) P. Neidlinger, T. Lenz, S. Foersch, C. M. L. Loeffler, J. Clusmann, M. Gustav, L. A. Shaktah, R. Langer, B. Dislich, L. A. Boardman, A. J. French, E. L. Goode, A. Gsur, S. Brezina, M. J. Gunter, R. Steinfelder, H. Behrens, C. Röcken, T. Harrison, U. Peters, A. I. Phipps, G. Curigliano, N. Fusco, A. Marra, M. Hoffmeister, H. Brenner, and J. N. Kather A deep learning framework for efficient pathology image analysis. Nat. Commun. 17 (1) (en). Cited by: Efficiency for gigapixel pathology., Discussion. Shao et al. (2025) D. Shao, R. J. Chen, A. H. Song, J. Runevic, M. Y. Lu, T. Ding, and F. Mahmood Do multiple instance learning models transfer?. In International Conference on Machine Learning, Cited by: Weakly supervised WSI classification.. Shao et al. (2021) Z. Shao, H. Bian, Y. Chen, J. Zhang, Y. Zheng, K. Zhao, and J. Li TransMIL: transformer based correlated multiple instance learning for whole slide image classification. In Advances in Neural Information Processing Systems, Vol. 34, p. 2136–2147. Cited by: Weakly supervised WSI classification.. Tan and Le (2020) M. Tan and Q. V. Le EfficientNet: rethinking model scaling for convolutional neural networks. External Links: 1905.11946, Link Cited by: Model Architectures.. Thandiackal et al. (2022) K. Thandiackal, B. Chen, P. Pati, G. Jaume, D. F. K. Williamson, M. Gabrani, and O. Goksel Differentiable zooming for multiple instance learning on whole-slide images. External Links: 2204.12454, Link Cited by: Efficiency for gigapixel pathology., Discussion. Vorontsov et al. (2024) E. Vorontsov, A. Bozkurt, A. Casson, G. Shaikovski, M. Zelechowski, K. Severson, E. Zimmermann, J. Hall, N. Tenenholtz, D. S. Klimstra, B. Rothrock, S. Liu, et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature Medicine 30, p. 2924–2935. External Links: Document Cited by: Introduction, Pathology foundation models., Model Architectures.. Wang et al. (2022) X. Wang, S. Yang, J. Zhang, M. Wang, J. Zhang, W. Yang, J. Huang, and X. Han Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81, p. 102559. External Links: Document Cited by: Pathology foundation models.. Xu et al. (2024) H. Xu, N. Usuyama, J. Bagga, S. Zhang, R. Rao, T. Naumann, C. Wong, Z. Gero, J. Gonzalez, Y. Gu, Y. Xu, M. Wei, W. Wang, et al. A whole-slide foundation model for digital pathology from real-world data. Nature 630, p. 181–188. External Links: Document Cited by: Introduction, Pathology foundation models.. Appendix Inference Efficiency Assessment Details For slide ℓ with NℓN_ foreground tiles, the fraction of expensive foundation model tile embeddings avoided at budget K is FM-savings(K)=1−∑ℓmin(K,Nℓ)∑ℓNℓ.FM -savings(K)=1- _ (K,N_ ) _ N_ . Calculation of FLOPs (F) for model inference additionally includes PriorNet scoring over the full foreground pool and the teacher/student MIL heads: FADMIL,ℓ=NℓFPriorNet+min(K,Nℓ)FFM+FtopKMILF_ADMIL, =N_ F_PriorNet+ (K,N_ )F_FM+F_topKMIL compared with Fteacher,ℓ=NℓFFM+FteacherMILF_teacher, =N_ F_FM+F_teacher\ MIL I/O and preprocessing are excluded from FLOPs calculations, and thus reduction in FLOPs should be interpreted at the model-forward level rather than wall-clock latency or total computation. For each dataset and metric, we define the descriptive critical budget Kcrit=minK:M¯ADMIL(K)≥M¯teacher,K_crit= \K: M_ADMIL(K)≥ M_teacher\, where the bars denote means over the three observed global runs. KcritK_crit is computed from the complete test performance curve and is therefore a post-hoc descriptive summary, not a validation-selected deployment hyperparameter. Checkpoint selection itself uses validation data only.