Paper deep dive
Beyond Classification: Pathology Foundation Models as Detection Encoders for Mitotic Figures
Sweta Banerjee, Alireza Teimoury, Nils Porsche, Alexandra K. Stoll, Viktoria Weiss, Niklas Hargarter, Jonas Ammeling, Thomas Conrad, Christoph Stroblberger, Christopher Kaltnecker, Robert Klopfleisch, Christof A. Bertram, Katharina Breininger, Marc Aubreville
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/3/2026, 9:27:23 AM
Summary
This study evaluates the capability of frozen pathology foundation models (FMs) to serve as backbones for dense object detection of mitotic figures (MFs), comparing them against a fully end-to-end trained ResNet-50 baseline. The authors tested six FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1) combined with three detector architectures (RetinaNet, Faster R-CNN, Deformable DETR) on the MIDOG++ and TUPAC16 datasets. Results indicate that H-optimus-0 and Virchow models achieve competitive performance, with H-optimus-0 showing superior robustness in out-of-domain scenarios, suggesting that frozen latent spaces from self-supervised pretraining are suitable for direct MF detection.
Entities (17)
Relation Signals (14)
Pathology Foundation Models â includes â Virchow
confidence 95% · common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1)
Pathology Foundation Models â includes â H-optimus-0
confidence 95% · common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1)
Pathology Foundation Models â includes â UNI
confidence 95% · common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1)
ResNet-50 â servesasbaselinefor â Pathology Foundation Models
confidence 95% · compare their performance against a fully end-to-end trained baseline based on a ResNet50 architecture
Pathology Foundation Models â trainedusing â Self-Supervised Learning
confidence 95% · trained on large-scale, unlabeled datasets by means of self-supervised learning
MIDOG++ â usedforevaluationof â Pathology Foundation Models
confidence 95% · on the multi-domain MIDOG++ dataset
TUPAC16 â usedforevaluationof â Pathology Foundation Models
confidence 95% · on the TUPAC16 dataset as an out-of-domain case
Virchow â achievescompetitiveperformanceon â Mitotic Figures
confidence 90% · H-optimus-0 and Virchow models yielded competitive performance
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pathology foundation models (FMs) are models trained on vast amounts of typically unlabeled data and have been shown to yield regularized latent spaces that can be used effectively in downstream classification tasks. This is also true for the classification of mitotic figures vs. other cells. However, it is so far unclear if the latent space of current FMs provides features that are discriminant and spatially suitably resolved to also serve as a backbone for dense object detection paradigms. In this work, we investigate this question for common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1) and compare their performance against a fully end-to-end trained baseline based on a ResNet50 architecture. We combine FM backbones with representatives of single stage, dual stage and self-attention-based detectors (RetinaNet, Faster R-CNN, Deformable DETR respectively) on the multi-domain MIDOG++ dataset, and on the TUPAC16 dataset as an out-of-domain case. We show that the H-optimus-0 and Virchow models yielded competitive performance, indicating that the latent spaces of current FMs, all trained on image-level self-supervision, are suitable for direct mitotic figure detection and may be slightly more robust on our out-of-domain test case. All code is made available publicly at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2607.28007v1
- Canonical: https://arxiv.org/abs/2607.28007v1
Trouble viewing inline? Open PDF directly â
Full Text
34,691 characters extracted from source content.
Expand or collapse full text
11institutetext: Flensburg University of Applied Sciences, Flensburg, Germany 22institutetext: University of Veterinary Medicine, Vienna, Austria 33institutetext: Freie UniversitĂ€t Berlin, Berlin, Germany 44institutetext: Technische Hochschule Ingolstadt, Ingolstadt, Germany 55institutetext: Medical University of Vienna, Vienna, Austria 66institutetext: Julius-Maximilians-UniversitĂ€t WĂŒrzburg, WĂŒrzburg, Germany 77institutetext: Friedrich-Alexander-UniversitĂ€t Erlangen-NĂŒrnberg, Germany Beyond Classification: Pathology Foundation Models as Detection Encoders for Mitotic Figures Sweta Banerjee Alireza Teimoury Nils Porsche Alexandra K. Stoll Viktoria Weiss Niklas Hargarter Jonas Ammeling Thomas Conrad Christoph Stroblberger Christopher Kaltnecker Robert Klopfleisch Christof A. Bertram Katharina Breininger Marc Aubreville Abstract Pathology foundation models (FMs) are models trained on vast amounts of typically unlabeled data and have been shown to yield regularized latent spaces that can be used effectively in downstream classification tasks. This is also true for the classification of mitotic figures vs. other cells. However, it is so far unclear if the latent space of current FMs provides features that are discriminant and spatially suitably resolved to also serve as a backbone for dense object detection paradigms. In this work, we investigate this question for common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1) and compare their performance against a fully end-to-end trained baseline based on a ResNet50 architecture. We combine FM backbones with representatives of single stage, dual stage and self-attention-based detectors (RetinaNet, Faster R-CNN, Deformable DETR respectively) on the multi-domain MIDOG++ dataset, and on the TUPAC16 dataset as an out-of-domain case. We show that the H-optimus-0 and Virchow models yielded competitive performance, indicating that the latent spaces of current FMsâall trained on image-level self-supervisionâare suitable for direct mitotic figure detection and may be slightly more robust on our out-of-domain test case. All code is made available publicly at https://anonymous.4open.science/r/FM4MFdet-24CA/README.md. 1 Introduction Mitosis is the process by which a cell divides into two daughter cells, enabling the proliferation that supports tissue growth, renewal, and the replacement of senescent cells. A dividing cell, when observed under a microscope, presents as a mitotic figure (MF). Beyond their physiological role in normal tissue maintenance, MFs serve as an important prognostic marker across a range of human and veterinary cancers, where uncontrolled cell division is a hallmark of malignancy [5, 8]. In these tumors, the density of MFs, typically quantified as the mitotic count, guides treatment decision-making [17, 24]. Despite its prognostic importance, manual MF assessment by pathologists suffers from limited reproducibility and is time-consuming in practice. Mitotic counting first requires identifying the hotspot within an entire tissue, which is constrained by the limited time available for tissue screening, and second to identify all mitotic figures within that hotspot, which is hindered by variable decision thresholds among pathologists [7]. Recently, pathology foundation models such as UNI, H-Optimus, and Virchow have emerged as powerful, general-purpose feature extractors, demonstrating strong performance across a broad array of histopathology benchmarks [11, 25]. They were trained on large-scale, unlabeled datasets by means of self-supervised learning using schemes such as DINO [12, 26] or MoCo [18, 16]. All these schemes have in common that they use a similarity-based scoring mechanism that disentangles augmented images from the same origin from those of different origins. This paradigm regularizes the latent space to yield discriminatory features for downstream tasks. However, the overwhelming majority of these evaluations was carried out solely on patch- or image-level classification tasks [1, 6], and a few on segmentation tasks [15]. A separate line of work integrates FMs into cell-level detection and segmentation pipelines, such as CellViT [20], CellViT++ [19], and CellSAM [23], which couple FM encoders with dedicated decoders for nucleus instance prediction. Notably, when CellViT++ was applied to MF detection on MIDOG++, it fell short of a trained-from-scratch RetinaNet detector across organs [19], illustrating that strong FM-based cell representations do not necessarily transfer to the demands of mitosis detection. It is thus so far unclear whether strong performance on classification benchmarks translates to utility on dense-prediction tasks such as object detection, which demand fine-grained spatial localization rather than image- or patch-level semantic summarization. In recent works, the detection of MFs has largely relied on architectures based on ImageNet-pretrained network stems that are fully fine-tuned end-to-end on the detection task [3, 2]. Pathology FMs were not yet evaluated on this task. Whether the frozen latent representations of pathology FMs encode sufficient information to compete with such detectors trained end-to-end with all parameters updated is thus an open question with significant practical implications, particularly since studies evaluating FMs for MF classification indicate that the out-of-domain (OOD) performance of FM-based models can potentially exceed that of fully trained networks, given the well-regularized training objective [1, 6]; we therefore also investigate this behavior in the object detection setting. In this work, we therefore ask: do the frozen latent spaces of pathology FMs encode sufficiently dense information to detect MFs, compared to a convolutional baseline whose backbone is fine-tuned on the task? We address this through a systematic evaluation of frozen FM backbones against an ImageNet-pretrained ResNet-50 detector trained end-to-end on MF detection. 2 Materials and Methods Figure 1: Overview of the detection pipeline. We compare frozen pathology FMs with a conventional convolutional baseline on mitotic figure detection (Fig. 1). We evaluate six ViT-based FMs â UNI [14], UNI2-h [14], Virchow [31], Virchow2 [34], H-Optimus-0 [28], and H-Optimus-1 [10] â as frozen backbones, each combined with three detector heads (Faster R-CNN [27], RetinaNet [22], and Deformable DETR [33]), yielding 18 frozen FM configurations. As baselines, we use an ImageNet-pretrained ResNet-50 in two regimes, fine-tuned and frozen, each crossed with the same three heads, for 24 configurations in total. All configurations shared the same training and evaluation splits, detailed below. 2.1 Dataset and preprocessing We conducted all experiments on MIDOG++, a multi-domain MF detection dataset comprising seven distinct domains [4]. The images of the dataset each represent regions of interest with a size of 2âmâm22m^2. Images were split into training, validation, and test sets on a per-patient basis to prevent data leakage, resulting in 333/71/71 slides, respectively. For OOD evaluation, we used the relabeled [9] training set of the TUPAC16 challenge [29] as secondary test set. The dataset consists of 71 individual tumor cases of human breast cancer [29]. Regions of interest were tiled into non-overlapping patches chosen to align with the vision transformer (ViT) patch grid of each backbone. We used a resolution of 1008 and 1024 px for patch-14 and patch-16 model configurations, respectively. During training, tiles were sampled to including at least one MF per patch and we used a sliding-window scheme over the full regions of interest during test-time inference. 2.2 Frozen foundation-model backbones Each FM backbone is a ViT pre-trained in a self-supervised fashion on hundreds of millions to billions of tiles [14, 30, 34, 28, 10]. During our experiments, these backbones were used strictly as frozen feature extractors, ensuring that we probed the representations as learned during pretraining, without any task-specific adaptation. A ViT encodes each image as a sequence of patch embeddings plus non-spatial summary tokens (a class token and, where applicable, register tokens). For dense detection, we discarded all non-spatial tokens and reshaped the remaining patch embeddings into a two-dimensional feature map at the token-grid resolution (one spatial location per patch token), which serves as the input to the neck. Each backbone was normalized using the same statistics employed during its pretraining: DINOv2-based models (UNI, UNI2-h, Virchow, Virchow2) use ImageNet mean and standard deviation, while H-Optimus-0 and H-Optimus-1 use their published pathology-specific normalization. We selected these six backbones since they include widely used public pathology FMs spanning different pretraining strategies and model scales, providing a representative sample of current practice. 2.3 Feature pyramid neck Because ViT backbones operate at a single spatial resolution, whereas modern detectors expect a multi-scale hierarchy [22, 33], we inserted a lightweight feature pyramid neck following the ViTDet [21] design. Starting from the single-scale ViT feature map, the neck progressively refines features towards finer resolutions and aggregates them towards coarser resolutions, constructing a multi-level feature pyramid. For patch-14 backbones, this yields a five-level hierarchy for Faster R-CNN and RetinaNet and a four-level hierarchy for Deformable DETR, with effective strides of [7,14,28,56,112][7,14,28,56,112] pixels relative to the input. These scales range from a high-resolution level suited for individual MFs to coarse levels providing broader contextual cues. Apart from the detection head, the neck is the only learnable module in the frozen-backbone setting; thus, it bears the full burden of adapting a fixed single-scale representation to multi-scale dense prediction. Because the neck natively outputs feature maps with the expected spatial resolutions and channel dimensions, it also replaces the intermediate channel projection that Deformable DETR would otherwise require. The ResNet-50 baseline does not use this ViTDet neck. Because ResNet already produces features at multiple scales, each detector uses its standard neck instead: a feature pyramid network over the C3âC5 stages for Faster R-CNN and RetinaNet, and a channel mapper for Deformable DETR. The baseline also keeps each detectorâs default strides rather than the patch-14 strides used for the ViT backbones. 2.4 Detection heads We evaluated three standard detectors with fixed hyperparameters chosen from their reference implementations and literature, kept identical across all backbones within a given head. These heads span the three dominant detection paradigmsâtwo-stage (Faster R-CNN [27]), one-stage (RetinaNet [22]), and transformer-based set prediction (Deformable DETR [33])âso that our conclusions are not tied to a single detector family. Faster R-CNN used a region proposal network with anchor scale 8 and aspect ratios 0.5,1.0,2.0\0.5,1.0,2.0\; its RoI head comprised two 1024-channel fully connected layers with 7Ă77Ă 7 RoI pooling over the finest four pyramid levels, with RPN IoU thresholds of 0.7/0.3 and detection thresholds of 0.5/0.5. RetinaNet used anchors with an octave base scale of 4, three scales per octave, and the same aspect ratios, trained with focal loss (Îł=2.0Îł=2.0, α=0.25α=0.25) and L1 box regression. Deformable DETR adopted the standard multi-scale deformable attention architecture with default encoder/decoder depth and query count, using Hungarian matching with focal, L1, and generalized IoU costs as in the original formulation [33]. For Faster R-CNN and RetinaNet, inference used a score threshold of 0.05 and NMS at IoU 0.5. 2.5 Training protocol Unless otherwise noted, all backbone-head combinations shared the same optimization and augmentation settings. All experiments were implemented using the MMDetection framework [13], with FM weights loaded via timm [32] and Hugging Face model hubs. All experiments use a fixed global random seed for reproducibility. Optimization. We trained all configurations with frozen backbones (the frozen FMs and the frozen ResNet-50 baseline) using AdamW with learning rate 2Ă10â42Ă 10^-4, weight decay 1Ă10â41Ă 10^-4, and momentum parameters ÎČ=(0.9,0.999)ÎČ=(0.9,0.999). Weight decay was not applied to normalization or bias parameters, and gradient norms were clipped at 35 to stabilize training. Training began with a linear learning-rate warm-up by a factor of 10â310^-3 for the first 500 iterations. After warm-up, Faster R-CNN and RetinaNet used a cosine learning-rate schedule decaying to a minimum of 10â710^-7, while Deformable DETR followed its standard step schedule with a single 10Ă10Ă drop. All models were trained for up to 100 epochs with validation-based early stopping: training terminated if the validation mAP failed to improve by at least 10â310^-3 for 10 consecutive epochs (Faster R-CNN and RetinaNet) or 20 epochs (Deformable DETR). We used a batch size of 16 for training for all configurations. Data augmentation. We applied random horizontal and vertical flips (probability 0.5), small random affine transformations (rotation up to ±15â± 15 , translation up to ±5%± 5\% of image size, scaling between 0.9 and 1.1), HED stain jittering, and photometric distortion. Evaluation We calculated the micro-averaged F1 (aggregated over all images), as well as precision and recall, as is commonly done for MF detection benchmarking [4, 29] as thresholded metrics. Moreover, we calculated the area under the free-range operating characteristic curve (FROC), interpolated over up to eight false positives per image as done in the most recent MIDOG 2025 challenge [2]. 3 Results Table 1: Detection performance across frozen foundation-model backbones, detection heads, and ResNet-50 baselines (trained end-to-end and frozen) on the MIDOG++ test set. All metrics are reported with 95% bootstrap confidence intervals (10,000 slide-level resamples) in brackets; F1, precision, and recall are evaluated at each configurationâs validation-selected operating threshold, and FROC-AUC is a threshold-independent area on the [0,8][0,8] FP/image scale. The best value in each column is shown in bold. Backbone Head F1 Precision Recall FROC UNI Faster R-CNN 0.4513 [0.3851, 0.5071] 0.5007 [0.3998, 0.5904] 0.4107 [0.3465, 0.4749] 2.158 [1.798, 2.550] UNI RetinaNet 0.6614 [0.6096, 0.6992] 0.7556 [0.6897, 0.8030] 0.5881 [0.5329, 0.6343] 4.310 [3.958, 4.663] UNI Deformable DETR 0.7062 [0.6601, 0.7376] 0.7513 [0.6938, 0.7985] 0.6663 [0.6161, 0.7055] 4.751 [4.467, 5.033] UNI2-h Faster R-CNN 0.7559 [0.7206, 0.7858] 0.7727 [0.7207, 0.8161] 0.7398 [0.6932, 0.7795] 4.999 [4.515, 5.490] UNI2-h RetinaNet 0.7178 [0.6801, 0.7481] 0.7680 [0.7109, 0.8144] 0.6739 [0.6192, 0.7276] 4.633 [4.111, 5.158] UNI2-h Deformable DETR 0.7316 [0.6978, 0.7579] 0.7962 [0.7414, 0.8372] 0.6768 [0.6403, 0.7133] 4.924 [4.559, 5.302] Virchow Faster R-CNN 0.7294 [0.6888, 0.7649] 0.7885 [0.7364, 0.8280] 0.6785 [0.6292, 0.7246] 4.871 [4.356, 5.373] Virchow RetinaNet 0.7622 [0.7241, 0.7922] 0.7853 [0.7322, 0.8270] 0.7404 [0.6978, 0.7755] 5.252 [4.826, 5.697] Virchow Deformable DETR 0.7198 [0.6853, 0.7472] 0.7756 [0.7177, 0.8193] 0.6715 [0.6345, 0.7095] 4.708 [4.302, 5.141] Virchow2 Faster R-CNN 0.7545 [0.7206, 0.7814] 0.7841 [0.7406, 0.8197] 0.7270 [0.6855, 0.7618] 5.081 [4.640, 5.518] Virchow2 RetinaNet 0.7503 [0.7200, 0.7758] 0.7932 [0.7480, 0.8318] 0.7118 [0.6751, 0.7473] 5.135 [4.755, 5.536] Virchow2 Deformable DETR 0.7169 [0.6816, 0.7499] 0.7767 [0.7223, 0.8191] 0.6657 [0.6215, 0.7138] 4.769 [4.348, 5.237] H-optimus-0 Faster R-CNN 0.7542 [0.7251, 0.7789] 0.7990 [0.7564, 0.8343] 0.7141 [0.6754, 0.7518] 5.133 [4.719, 5.547] H-optimus-0 RetinaNet 0.7718 [0.7368, 0.8002] 0.8197 [0.7768, 0.8542] 0.7293 [0.6893, 0.7646] 5.490 [5.099, 5.880] H-optimus-0 Deformable DETR 0.7350 [0.6991, 0.7632] 0.7532 [0.6912, 0.7995] 0.7176 [0.6834, 0.7545] 4.829 [4.397, 5.297] H-optimus-1 Faster R-CNN 0.7717 [0.7333, 0.8024] 0.7580 [0.7070, 0.7983] 0.7859 [0.7451, 0.8208] 5.171 [4.724, 5.616] H-optimus-1 RetinaNet 0.7683 [0.7263, 0.8024] 0.7609 [0.7084, 0.8027] 0.7760 [0.7313, 0.8140] 5.264 [4.845, 5.686] H-optimus-1 Deformable DETR 0.6637 [0.6276, 0.6994] 0.7419 [0.6812, 0.7879] 0.6004 [0.5510, 0.6619] 4.063 [3.599, 4.599] ResNet-50 Faster R-CNN 0.7702 [0.7363, 0.7950] 0.7950 [0.7504, 0.8340] 0.7468 [0.7045, 0.7822] 5.292 [4.954, 5.644] ResNet-50 RetinaNet 0.7917 [0.7567, 0.8163] 0.8080 [0.7651, 0.8443] 0.7760 [0.7279, 0.8125] 5.686 [5.340, 6.031] ResNet-50 Deformable DETR 0.7452 [0.7124, 0.7706] 0.7771 [0.7242, 0.8174] 0.7159 [0.6832, 0.7472] 5.153 [4.860, 5.485] ResNet-50 (frozen) Faster R-CNN 0.7183 [0.6768, 0.7472] 0.7434 [0.6875, 0.7942] 0.6949 [0.6366, 0.7426] 4.740 [4.390, 5.133] ResNet-50 (frozen) RetinaNet 0.7166 [0.6766, 0.7448] 0.7889 [0.7301, 0.8412] 0.6564 [0.6015, 0.7090] 4.880 [4.559, 5.243] ResNet-50 (frozen) Deformable DETR 0.7498 [0.7126, 0.7776] 0.7957 [0.7500, 0.8320] 0.7089 [0.6664, 0.7435] 5.147 [4.763, 5.545] Table 2: Out-of-domain detection performance, evaluated on TUPAC16. F1, precision, and recall are reported at each configurationâs operating threshold, with 95% bootstrap confidence intervals (10,000 slide-level resamples) in brackets; FROC-AUC was evaluated on the [0,8][0,8] FP/image scale. Best value in each column is shown in bold. Backbone Head F1 Precision Recall FROC UNI Faster R-CNN 0.3952 [0.3041, 0.4669] 0.3329 [0.2428, 0.4120] 0.4862 [0.3712, 0.5751] 1.773 [1.354, 2.235] UNI RetinaNet 0.5381 [0.4609, 0.5921] 0.6556 [0.5664, 0.7207] 0.4562 [0.3696, 0.5281] 2.926 [2.434, 3.496] UNI Deformable DETR 0.6282 [0.5869, 0.6598] 0.7014 [0.6431, 0.7430] 0.5688 [0.5130, 0.6287] 3.715 [3.316, 4.236] UNI2-h Faster R-CNN 0.7043 [0.6645, 0.7362] 0.6880 [0.6434, 0.7480] 0.7214 [0.6308, 0.7818] 4.017 [3.203, 4.891] UNI2-h RetinaNet 0.6932 [0.6596, 0.7241] 0.6617 [0.6220, 0.7153] 0.7279 [0.6587, 0.7771] 4.195 [3.490, 4.994] UNI2-h Deformable DETR 0.7188 [0.6869, 0.7474] 0.7202 [0.6816, 0.7635] 0.7174 [0.6689, 0.7508] 4.524 [3.908, 5.244] Virchow Faster R-CNN 0.7054 [0.6706, 0.7313] 0.7401 [0.7036, 0.7730] 0.6738 [0.6259, 0.7130] 4.296 [3.675, 4.986] Virchow RetinaNet 0.7262 [0.6964, 0.7499] 0.7131 [0.6789, 0.7493] 0.7399 [0.6917, 0.7762] 4.607 [3.985, 5.308] Virchow Deformable DETR 0.7057 [0.6718, 0.7300] 0.7278 [0.6858, 0.7594] 0.6848 [0.6374, 0.7232] 4.372 [3.774, 5.012] Virchow2 Faster R-CNN 0.6937 [0.6567, 0.7232] 0.6570 [0.6137, 0.7013] 0.7349 [0.6716, 0.7808] 4.122 [3.367, 4.906] Virchow2 RetinaNet 0.6984 [0.6660, 0.7260] 0.6495 [0.6072, 0.6982] 0.7554 [0.6976, 0.7961] 4.437 [3.796, 5.177] Virchow2 Deformable DETR 0.6824 [0.6470, 0.7090] 0.6347 [0.5896, 0.6773] 0.7379 [0.6865, 0.7749] 4.012 [3.303, 4.787] H-optimus-0 Faster R-CNN 0.7168 [0.6847, 0.7447] 0.7254 [0.6835, 0.7783] 0.7084 [0.6475, 0.7525] 4.053 [3.103, 5.067] H-optimus-0 RetinaNet 0.7349 [0.7077, 0.7585] 0.7182 [0.6858, 0.7633] 0.7524 [0.7011, 0.7840] 4.589 [3.897, 5.407] H-optimus-0 Deformable DETR 0.7003 [0.6663, 0.7258] 0.6520 [0.6079, 0.6928] 0.7564 [0.7079, 0.7889] 4.344 [3.654, 5.047] H-optimus-1 Faster R-CNN 0.7122 [0.6790, 0.7402] 0.6557 [0.6178, 0.7043] 0.7794 [0.7141, 0.8193] 4.054 [3.238, 4.953] H-optimus-1 RetinaNet 0.7023 [0.6665, 0.7294] 0.6585 [0.6235, 0.7073] 0.7524 [0.6699, 0.8043] 4.310 [3.682, 5.044] H-optimus-1 Deformable DETR 0.6554 [0.6201, 0.6825] 0.6239 [0.5758, 0.6716] 0.6903 [0.6467, 0.7235] 3.425 [2.699, 4.271] ResNet-50 Faster R-CNN 0.6905 [0.6501, 0.7187] 0.6592 [0.6141, 0.6965] 0.7249 [0.6581, 0.7768] 4.103 [3.549, 4.735] ResNet-50 RetinaNet 0.7193 [0.6792, 0.7443] 0.6841 [0.6372, 0.7214] 0.7584 [0.6974, 0.7987] 4.694 [4.175, 5.251] ResNet-50 Deformable DETR 0.6882 [0.6531, 0.7120] 0.6546 [0.5979, 0.6994] 0.7254 [0.6778, 0.7668] 4.340 [3.841, 4.925] ResNet-50 (frozen) Faster R-CNN 0.6605 [0.6203, 0.6963] 0.7030 [0.6348, 0.7593] 0.6228 [0.5700, 0.6775] 3.774 [3.117, 4.596] ResNet-50 (frozen) RetinaNet 0.6693 [0.6230, 0.7042] 0.7200 [0.6664, 0.7655] 0.6253 [0.5639, 0.6770] 4.091 [3.513, 4.751] ResNet-50 (frozen) Deformable DETR 0.6925 [0.6535, 0.7196] 0.7699 [0.7306, 0.8018] 0.6293 [0.5720, 0.6777] 4.403 [3.909, 4.933] Table 1 reports detection performance on the test split of MIDOG++ for the 18 frozen backbone-head configurations and two ResNet-50 baselines (fully fine-tuned and frozen). The best configuration for this in-domain use case was the fully fine-tuned ResNet-50 baseline with RetinaNet, which attained the highest F1 (0.7917) in the table. Among the frozen FMs, the strongest was H-Optimus-0 with RetinaNet (F1 0.7718), which approached but did not match the corresponding fine-tuned baseline. Performance varied markedly with the choice of detection head. RetinaNet was the most reliable, achieving the top F1 for the ResNet-50 baseline, H-Optimus-0, and Virchow, and never being the worst head for any backbone. In terms of backbone ranking, the two pathology-specific H-Optimus models led among the frozen FMs, closely followed by the Virchow family, with UNI and UNI2-h trailing. The highest recall was obtained by H-Optimus-1 with Faster R-CNN (0.78590.7859), though its lower precision (0.75800.7580) leaves its F1 (0.77170.7717) marginally below the best frozen configuration. H-Optimus-0 with RetinaNet instead achieved the best precision (0.81970.8197) among all model configurations, baseline included. The frozen ResNet-50 baseline, which like the FMs trains only the neck and head, trailed both its fine-tuned counterpart and the strongest frozen FMs (best F1 0.7498 with Deformable DETR), indicating that the competitiveness of the frozen FMs does not arise merely from freezing any ImageNet-pretrained stem but from the pathology-specific representations themselves. The F1 and FROC-AUC rankings did not always coincide. The clearest case is H-Optimus-1, where Faster R-CNN attained a marginally higher F1 than RetinaNet (0.7717 vs. 0.7683) yet a lower FROC-AUC (5.171 vs. 5.264); because the ResNet-50 baseline showed the same Faster R-CNN/RetinaNet FROC ordering, this most likely reflects how each head distributes confidence across operating points rather than a frozen-backbone artifact. In OOD case, the overall performance dropped relative to the in-domain setting (Table 2), and the best F1 was achieved by a frozen FM rather than the baseline: H-Optimus-0 with RetinaNet reached the highest F1 (0.73490.7349), ahead of the strongest ResNet-50 configuration (RetinaNet, 0.71930.7193). This reverses the in-domain ordering and indicates that frozen FM features were slightly more robust under domain shift. Virchow with Faster R-CNN attained the best precision (0.74010.7401), H-Optimus-1 with Faster R-CNN the best recall (0.77940.7794), and the ResNet-50 baseline with RetinaNet the best FROC-AUC (4.6944.694). As on MIDOG++, UNI with Faster R-CNN remained the weakest configuration (F1 0.39520.3952, FROC-AUC 1.7731.773), and the relative ordering of backbones was broadly preserved across the two datasets. The frozen ResNet-50 baseline again lagged the frozen FMs out-of-domain (best F1 0.6925 with Deformable DETR), and unlike the fine-tuned baseline it did not benefit from adaptation to the source domain, reinforcing that the strongest frozen FMs transfer more favourably than a frozen convolutional stem. 4 Discussion Our results give a qualified but clear answer to the central question: the frozen latent spaces of pathology FMs are rich enough to support dense MF detection. As reported above, the strongest frozen configuration (H-Optimus-0 with RetinaNet) comes close to the best end-to-end-trained ResNet-50, and under a Faster R-CNN head the gap effectively vanishes. That backbones trained purely with image-level self-supervision, and kept entirely frozen, can rival a fully fine-tuned convolutional network shows that the spatial information needed to localize MFs is already present in these representations and recoverable by a lightweight neck and head. A frozen ResNet-50 trained under the identical neck-and-head regime did not reach the same level, which isolates the contribution of the pathology-specific pretraining rather than of the frozen-backbone protocol alone. Performance nonetheless depended substantially on the detection head, an interaction strongest for the weaker backbones. We also observed that Faster R-CNN consistently trailed RetinaNet on FROC-AUC despite competitive F1, a pattern shared by the ResNet-50 baseline and therefore most likely reflecting how each head distributes confidence across operating points rather than a frozen-backbone artifact. Most notably, the in-domain ordering reversed out-of-domain: on TUPAC16 the best F1 came from a frozen FM (H-Optimus-0 with RetinaNet) rather than the baseline, consistent with frozen large-scale features generalizing more gracefully under domain shift. Several limitations temper these conclusions. The backbones were used strictly frozen, so we characterize the information already present in the pretrained representations, not the ceiling achievable with adaptation. The most natural next step is parameter-efficient fine-tuning, in particular Low Rank Adaptation (LoRA), which has proven effective for MF classification with these same FMs; this would test whether the residual gap to the end-to-end baseline reflects a genuine limit of the frozen representations or merely the absence of light adaptation, though whether the out-of-domain robustness survives such adaptation is itself open. Further limitations include reliance on a single secondary dataset for the OOD evaluation, so the robustness advantage should be read as suggestive, and confidence intervals that are marginal per-configuration estimates, so between-configuration comparisons should account for the overlapping intervals in the tables. Overall, our findings support a cautiously optimistic conclusion: the latent spaces of current pathology FMs, despite being shaped entirely by image-level self-supervision, are discriminative and spatially resolved enough to drive MF detection competitively with an end-to-end-trained baseline and to transfer slightly more robustly to unseen data, with the most consequential design choices lying in the neck and headâand, prospectively, in lightweight backbone adaptation. credits 4.0.1 Acknowledgements M.A. and S.B. acknowledge funding by the Deutsche Forschungsgemeinschaft (DFG, project number: 520330054). C.A.B., V.W., CS, and CK acknowledge funding by the Austrian Research Fund (FWF, project number: I 6555). J.A. acknowledges support by the Bavarian State Ministry of Science and the Arts (project Fokus-TML). K.B. acknowledges funding by the DFG, project number 460333672 CRC1540 EBM. Experiments were supported by the MUSICA HPC cluster at Meduni Vienna and the Julia 2 cluster at UniversitĂ€t WĂŒrzburg. Julia 2 hardware is funded by the German Research Foundation (DFG). 4.0.2 The authors have no competing interests to declare that are relevant to the content of this article. FM foundation model H&E Hematoxylin & Eosin RoI region of interest WSI whole slide image MF mitotic figure NMS non-maximum suppression OOD out-of-domain IoU Intersection over Union AtNorM Atypical and Normal Mitosis AMF atypical mitotic figure CNN convolutional neural network ViT vision transformer LoRA Low Rank Adaptation ROC receiver operating characteristic AUROC Area under the Receiver Operating Characteristic Curve References [1] J. Ammeling, J. Ganz, E. Rosbach, et al. (2026-01-01) Benchmarking foundation models for mitotic figure classification. Machine Learning for Biomedical ImagingCytometry: The Journal of the International Society for Analytical CytologyTransactions on Machine Learning ResearchZenodoNature medicineNature MedicineNature MedicinearXiv preprint arXiv:2408.00738Machine Learning for Biomedical ImagingarXiv preprint arXiv:2304.07193IEEE transactions on pattern analysis and machine intelligenceCancer: Interdisciplinary International Journal of the American Cancer SocietyVeterinary pathologyJournal of clinical pathologyarXiv preprint arXiv:2010.04159arXiv preprint arXiv:1906.07155Nature CommunicationsNature biomedical engineeringNature medicineVeterinary pathologyarXiv preprint arXiv:2602.03887Computer Methods and Programs in BiomedicineNature MethodsMedical image analysis 3, p. 38â55. External Links: Link, ISSN 2766-905X Cited by: §1, §1. [2] M. Aubreville, J. Ammeling, S. Banerjee, et al. (2026) Mitosis detection in the wild: multi-tumor and context-aware generalization in the MIDOG 2025 challenge. arXiv preprint arXiv:2606.07368. Cited by: §1, §2.5. [3] M. Aubreville, N. Stathonikos, and T. A. o. Donovan (2024) Domain generalization across tumor types, laboratories, and speciesâinsights from the 2022 edition of the mitosis domain generalization challenge. Medical Image Analysis 94, p. 103155. Cited by: §1. [4] M. Aubreville, F. Wilm, N. Stathonikos, et al. (2023) A comprehensive multi-domain dataset for mitotic figure detection. Scientific data 10 (1), p. 484. Cited by: §2.1, §2.5. [5] M. F. Azzola, H. M. Shaw, J. F. Thompson, et al. (2003) Tumor mitotic rate is a more powerful prognostic indicator than ulceration in patients with primary cutaneous melanoma: an analysis of 3661 patients from a single center. 97 (6), p. 1488â1498. Cited by: §1. [6] S. Banerjee, V. Weiss, T. A. Donovan, et al. (2026) Benchmarking deep learning and vision foundation models for atypical vs. normal mitosis classification with cross-dataset evaluation. 2026, p. 115â125. External Links: ISSN 2766-905X, Link Cited by: §1, §1. [7] C. A. Bertram, M. Aubreville, T. A. Donovan, et al. (2022) Computer-assisted mitotic count using a deep learningâbased algorithm improves interobserver reproducibility and accuracy. 59 (2), p. 211â226. Cited by: §1. [8] C. A. Bertram, T. A. Donovan, and A. Bartel (2024) Mitotic activity: a systematic literature review of the assessment methodology and prognostic value in canine tumors. 61 (5), p. 752â764. Cited by: §1. [9] C. A. Bertram, M. Veta, C. Marzahl, et al. (2020) Are pathologist-defined labels reproducible? comparison of the TUPAC16 mitotic figure dataset with an alternative set of labels. In International Workshop on Interpretability of Machine Intelligence in Medical Image Computing, p. 204â213. Cited by: §2.1. [10] H-optimus-1 External Links: Link Cited by: §2.2, §2. [11] G. Campanella, S. Chen, M. Singh, et al. (2025) A clinical benchmark of public self-supervised pathology foundation models. 16 (1), p. 3640. Cited by: §1. [12] M. Caron, H. Touvron, I. Misra, et al. (2021) Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, p. 9650â9660. Cited by: §1. [13] K. Chen, J. Wang, J. Pang, et al. (2019) MMDetection: open mmlab detection toolbox and benchmark. Cited by: §2.5. [14] R. J. Chen, T. Ding, M. Y. Lu, et al. (2024) Towards a general-purpose foundation model for computational pathology. 30 (3), p. 850â862. Cited by: §2.2, §2. [15] W. Chen, X. Ling, X. Wang, et al. (2026) To what extent do token-level representations from pathology foundation models improve dense prediction?. Cited by: §1. [16] X. Chen, S. Xie, and K. He (2021) An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, p. 9640â9649. Cited by: §1. [17] P. L. Fitzgibbons and J. L. Connolly (2023) Protocol for the examination of resection specimens from patients with invasive carcinoma of the breast. CAP guidelines 4.8.1.0. External Links: Link Cited by: §1. [18] K. He, H. Fan, Y. Wu, et al. (2020) Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9729â9738. Cited by: §1. [19] F. Hörst, M. Rempe, H. Becker, et al. (2026) Cellvit++: energy-efficient and adaptive cell segmentation and classification using foundation models. p. 109206. Cited by: §1. [20] F. Hörst, M. Rempe, L. Heine, et al. (2024) Cellvit: vision transformers for precise cell segmentation and classification. 94, p. 103143. Cited by: §1. [21] Y. Li, H. Mao, R. Girshick, and K. He (2022) Exploring plain vision transformer backbones for object detection. In European conference on computer vision, p. 280â296. Cited by: §2.3. [22] T. Lin, P. Goyal, R. Girshick, et al. (2017) Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, p. 2980â2988. Cited by: §2.3, §2.4, §2. [23] M. Marks, U. Israel, R. Dilip, et al. (2025) CellSAM: a foundation model for cell segmentation. p. 1â9. Cited by: §1. [24] E. McNiel, G. Ogilvie, B. Powers, et al. (1997) Evaluation of prognostic factors for dogs with primary lung tumors: 67 cases (1985-1992).. Journal of the American Veterinary Medical Association 211 (11), p. 1422â1427. Cited by: §1. [25] P. Neidlinger, O. S. El Nahhas, H. S. Muti, et al. (2025) Benchmarking foundation models as feature extractors for weakly supervised computational pathology. p. 1â11. Cited by: §1. [26] M. Oquab, T. Darcet, T. Moutakanni, et al. (2023) Dinov2: learning robust visual features without supervision. Cited by: §1. [27] S. Ren, K. He, R. Girshick, and J. Sun (2016) Faster r-cnn: towards real-time object detection with region proposal networks. 39 (6), p. 1137â1149. Cited by: §2.4, §2. [28] H-optimus-0 External Links: Link Cited by: §2.2, §2. [29] M. Veta, Y. J. Heng, N. Stathonikos, et al. (2019) Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge. Medical image analysis 54, p. 111â121. Cited by: §2.1, §2.5. [30] E. Vorontsov, A. Bozkurt, A. Casson, et al. (2024) A foundation model for clinical-grade computational pathology and rare cancers detection. 30 (10), p. 2924â2935. Cited by: §2.2. [31] E. Vorontsov, A. Bozkurt, A. Casson, et al. (2024) A foundation model for clinical-grade computational pathology and rare cancers detection. Cited by: §2. [32] R. Wightman (2019) PyTorch image models. GitHub. Note: https://github.com/rwightman/pytorch-image-models Cited by: §2.5. [33] X. Zhu, W. Su, L. Lu, et al. (2020) Deformable detr: deformable transformers for end-to-end object detection. Cited by: §2.3, §2.4, §2.4, §2. [34] E. Zimmermann, E. Vorontsov, J. Viret, et al. (2024) Virchow2: scaling self-supervised mixed magnification models in pathology. Cited by: §2.2, §2.