Paper deep dive
HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy
Julius Riel, Vishwa Mohan Singh, Sai Anirudh Aryasomayajula, Anuun Chinbat, Hannes Leonhard, Moritz Ladenburger, Frederik Alexander, Vishisht Choudhary, Fabio Laredo, Giacomo Masserdotti, Thorben Prein, Carsten Marr, Amirhossein Kardoost
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/7/2026, 12:57:45 PM
Summary
The paper introduces HASSL, a hierarchy-aware self-supervised learning framework designed for single-cell microscopy. It addresses the limitation where current SSL models suppress fine-grained morphological structures due to dominance by coarse factors like imaging modality. HASSL combines a double-teacher distillation approach using zero-shot segmentation masks with an HDBSCAN-based hierarchy-aware contrastive loss. Evaluated on a curated corpus of 2.3 million single cells across 208 classes, the framework significantly improves top-K accuracy, retrieval performance, and downstream drug classification F1-scores over baseline SSL methods.
Entities (9)
Relation Signals (9)
HASSL → addresses → Single Cell Microscopy
confidence 97% · HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy
HASSL → uses → HDBSCAN
confidence 95% · Our method combines two components: a distillation framework with a segmentation teacher to improve morphological awareness in the latent space, and a hierarchy-aware contrastive loss based on HDBSCAN
Benchmark Dataset → contains → 2.3 Million Single Cells
confidence 94% · curated corpus of 2.3 million single cells aggregated from 20 microscopy datasets
HASSL → improves → Top-K Accuracy
confidence 94% · increasing average top-K accuracy by 2.8%
Benchmark Dataset → covers → 208 Cell Classes
confidence 93% · covering 208 cell classes
HASSL → improves → F1 Score
confidence 93% · downstream F1-score for biologically relevant drug classification from perturbed cell morphology by 7.8%
HASSL → uses → Double-Teacher Distillation
confidence 92% · Our method combines two components: a distillation framework with a segmentation teacher to improve morphological awareness in the latent space
Double-Teacher Distillation → uses → CellposeSAM
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Hierarchical structure is common in image data, where fine-grained clusters often merge into larger, coarser semantic groups. In biological cell images, current self-supervised learning models often suppress this hierarchy, as coarse factors such as imaging modality can obscure finer morphological attributes in the latent space. We propose a hierarchy-aware self-supervised training framework to address this problem. Our method combines two components: a distillation framework with a segmentation teacher to improve morphological awareness in the latent space, and a hierarchy-aware contrastive loss based on HDBSCAN to improve decision boundaries between closely related subtypes at different hierarchical levels. Together, these components reduce the tendency of self-supervised learning to overemphasize coarse factors and instead align embeddings with semantic and morphological cues. This yields biologically meaningful sub-clusters driven by fine morphological detail. We train and evaluate our method on a curated corpus of 2.3 million single cells aggregated from 20 microscopy datasets, both labeled and unlabeled, covering 208 cell classes. Our method improves over baseline and counterpart methods, increasing average top-K accuracy by 2.8%, top-9 retrieval on the dataset with the deepest hierarchy by 6.3%, and downstream F1-score for biologically relevant drug classification from perturbed cell morphology by 7.8%.
Tags
Links
- Source: https://arxiv.org/abs/2607.04353v1
- Canonical: https://arxiv.org/abs/2607.04353v1
Trouble viewing inline? Open PDF directly →
Full Text
67,406 characters extracted from source content.
Expand or collapse full text
11institutetext: TUM.ai, Munich, Germany 22institutetext: Ludwig-Maximilian-University, Munich, Germany 33institutetext: Technical University of Munich, Munich, Germany 44institutetext: Institute of Stem Cell Research, Helmholtz Munich, German Research Center for Environmental Health, Neuherberg, Germany 55institutetext: Computational Health Center, Helmholtz Munich - German Research Center for Environmental Health, Neuherberg, Germany 66institutetext: Department of Medicine I, Ludwig-Maximilian-University Hospital, Munich, Germany 77institutetext: Department of Physics, Ludwig-Maximilian-University, Munich, Germany 88institutetext: German Cancer Consortium (DKTK), partner site Munich, Germany 99institutetext: Munich Center for Machine Learning (MCML), Munich, Germany HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy Julius Riel† Vishwa Mohan Singh† Sai Anirudh Aryasomayajula Anuun Chinbat Hannes Leonhard Moritz Ladenburger Frederik Alexander Vishisht Choudhary Fabio Laredo Giacomo Masserdotti Thorben Prein Carsten Marr* Amirhossein Kardoost* Abstract Hierarchical structure is common in image data, where fine-grained clusters often merge into coarser semantic groups. In biological cell images, however, current self-supervised models often suppress this structure because coarse factors, such as imaging modality, dominate the latent space and obscure finer morphological attributes. We propose a hierarchy-aware self-supervised framework that preserves biologically meaningful structure in cellular representations. Our method combines two components: a segmentation-guided distillation teacher that improves morphological awareness, and an HDBSCAN-based hierarchy-aware contrastive loss that sharpens boundaries between related but distinct subtypes at each hierarchical level. Together, these components align embeddings with semantic and morphological cues while reducing the dominance of coarse acquisition factors. We train and evaluate our method on a curated corpus of 2.3M single cells from 20 microscopy datasets, covering 208 cell classes. Our approach improves over baseline methods, increasing average top-K accuracy by +2.8%+2.8\%, top-99 retrieval on deeply hierarchical data by +6.3%+6.3\%, and downstream drug-classification F1-score from perturbed cell morphology by +7.8%+7.8\%. †footnotetext: Equal contribution.**footnotetext: Corresponding authors: . Version note. This manuscript is a pre-peer-review preprint. The final published version may differ from this version. 1 Introduction Figure 1: Hierarchy in cell imaging (left) and the objective with HASSL’s embedding model (right). HASSL learns a more hierarchical embedding space, leading to improved morphological representation and tighter subclusters. Accurately labeled microscopy datasets are scarce, motivating self-supervised learning (SSL)-based methods to encode biologically meaningful signals for downstream tasks [31, 29]. Distillation-based models such as DINO [7] scale well without labels and avoid contrastive large-batch or memory-bank overhead, making them strong base models for this task [7, 40, 47, 9, 24, 58]. Biological cell imaging presents a distinct set of unsolved challenges for SSL. Unlike natural images, which exhibit easily separable class boundaries at the object level (e.g., jay vs. magpie [21, 11]), cellular microscopy images rely on subtle morphological and textural cues to distinguish similar yet distinct cell types. For instance, different cell types can appear indistinguishable under the same modality (e.g., OPCs and Neurons when captured with the same modality (Fig. 5-left)). Conversely, the same cell type can appear markedly different across imaging modalities; for example, a neuron in brightfield versus the same neuron in e.g. fluorescence microscopy. A robust latent space should attempt to align the same cell type across modalities and separate different cell types within each modality, reflecting a stronger morphology awareness and supporting downstream tasks such as drug perturbation and cell classification. Examples of multi-modal cell imaging are shown in Fig. 5-left. The imaging setup, modality, and batch effects tend to be the dominant features in cell imaging[23, 2]. This encourages models to form coarse “superclusters” that obscure biologically meaningful substructure needed for tasks such as phenotype classification, mechanism-of-action (MOA) inference, or cell-state discrimination [3]. Supervised class-aware methods offer better performance, although they struggle with generalization due to the scarcity of large and versatile labeled datasets in cellular imaging [26]. Therefore, we propose a SSL framework that captures hierarchical structure in a single-cell representation. Our contributions are fourfold: • Hierarchy-aware objective. To improve intra-supercluster decision boundaries, we propose a novel, generalisable, label-free, hierarchy-aware objective that uses an HDBSCAN cluster tree to define subcluster centroids, pulling each individual cell towards their parent clusters centroid while pushing them away from other clusters at each level of the hierarchy. • Double-teacher distillation. To encourage morphology-aware, less modality centred representations, we add a second distillation teacher that uses segmentation masks as a weak prior to provide structure-aware supervision. It aims to pull each cell’s embedding towards its mask view to initiate breaking up modality-driven superclusters. • Benchmark curation. We consolidated scattered benchmarks into a unified single-cell, multi-modality dataset covering 208 labeled classes and 2.3 million single cells. • Empirical improvements. We outperform state-of-the-art SSL cell embedding baselines across a large multi-modality corpus, with higher retrival, clustering, and downstream metrics, reflecting better alignment with biologically meaningful subclusters. Code is available at: https://github.com/tum-ai/HASSL. Curated dataset is available at: https://huggingface.co/datasets/tum-ai/HASSL-SingleCellBench. 2 Related Work We review visual representation learning, microscopy-specific cell imaging, and hierarchical SSL, motivating efficient representations that capture multi-level structure in unlabeled data. For single-cell microscopy, this means encoding modality and fine-grained morphology across heterogeneous assays, yet standard self-supervision often overfits to global cues rather than a morphology-centric hierarchy [61, 23]. General-purpose visual representation learning has shifted from contrastive objectives to distillation and masked-image modeling. Contrastive methods such as SimCLR [9] and triplet loss [45] avoid pixel-space generation in VAEs and VQ-VAEs [30, 50], but require many comparisons, strong augmentations, and explicit negatives [24, 9]. Distillation methods such as BYOL [20] and the DINO family [7, 40, 47] remove negatives via self-distillation, but are tuned for generic semantics and coarse separation on datasets like ImageNet [11]. They do not model hierarchy, and in cell imaging they often cluster by modality rather than capturing subtle within-modality morphology, limiting single-cell representation learning. Cell imaging has followed a similar shift, moving from assay-specific pipelines to generalisable SSL models. CellPaint-DINO [29] benchmarks standard SSL for Cell Painting using MAE [32], DINO [7], and SimCLR [9], with DINO as the strongest baseline. Microscopy-specific designs address practical constraints. Chada-ViT [4] handles variable channel counts via channel-aware tokenisation. SubCell [22] uses MAE-style pretraining [32] to learn protein localisation and morphology from HPA images [57]. OpenPhenom [32] trains channel-agnostic ViT and MAE models on millions of RxRx [44] and Cell Painting images. scDINO [42] adapts self-distillation to fluorescent single-cell crops on top of a DINO backbone. Yet these methods are often channel- or assay-aware rather than morphology-aware because they do not link embeddings to segmentation geometry during training, leaving morphology cues weak. The works most similar to ours are hierarchy-aware mining methods that modify pair selection so the loss encodes structure directly CoHiClust [62] learns a contrastive cluster tree, while SeedViews [59] uses multi-level views with multiple positives to keep semantically similar instances aligned. HCSC [21] is closest to our approach, dynamically constructing hierarchical prototypes and adaptively expanding positives to nearby semantics while sharpening negatives for distinct samples. However, its recursive k-N sampling relies on hard positives and negatives that are difficult to define stably and can under-emphasise fine morphology, leading to unstable gradients and slower convergence. Our method combines stability-weighted hierarchical prototypes with double-teacher distillation. The zero-shot segmentation masks add geometric cues that guide coarse-to-fine morphology learning, yielding hierarchy-aware embeddings that reduce modality-driven clustering. 3 Methodology Figure 2: Overview of our hierarchy-aware representation learning pipeline. (1) Segmentation-map generation: microscopy images are zero-shot segmented [48], which is used as a proxy for morphological features (2) Double-Teacher distillation: a student ViT is trained with EMA from a global image teacher and a segmentation teacher, promoting subcluster breakout in the latent space. (3) Hierarchical cluster reinforcement: multi-level clusters are formed with HDBSCAN [5]; cluster stability λ weights positive pairs within parents, while negative is defined as 1λ 1λ, yielding stability-weighted contrastive learning. (4) Resulting latent space: the learned embedding organizes cells into superclusters with clearer subclusters (illustrated for astrocytes, neurons, oligodendrocyte progenitor cells (OPCs), and dead cells), reflecting improved hierarchy awareness and structure-preserving separation. We design a representation-learning framework that embeds a single modality of a single-cell image. This framework enables the model to learn a hierarchy-aware latent space without any explicit supervision about the hierarchy itself. To achieve this, we approximate cellular morphology and actively encourage the formation of meaningful subclusters in the embedding space. Our method must satisfy two objectives. First, it needs to inform the embeddings about cellular structure, independent of imaging modality, by leveraging segmentation masks as weak priors. This initiates the emergence of label-free, morphology-based subclusters. Second, the loss must pull cells belonging to the same subcluster together while pushing apart cells from different subclusters across all levels of the hierarchy. We propose an annealed training strategy with an additional teacher head to instruct the embeddings about the structure, and a loss that uses hierarchical clustering to reinforce the subcluster structures in the latent space. Figure 2 illustrates our model. 3.1 Double-Teacher DINO Distillation To bias representations away from modality cues and toward morphology, we guide the model’s attention towards cell shape. To avoid leaking labels, we pre-compute segmentation masks using a generalist segmentation model. These mask-derived shape cues enter only through the dataloader as weak, structure-aware supervision. As a weak prior, these masks can tolerate an acceptable level of noise and error, while still providing meaningful gradients to the model. Therefore, any zero-shot segmentation method would suffice in this case. We have used CellposeSAM [48] in our approach. With this mask, we implement a double teacher distillation framework with an additional segmentation teacher that conditions on both images and masks, supervising the student with mask-informed targets. This encourages morphology-focused representations and begins to break modality-driven superclusters into morphology-based ones. As shown in Fig. 2 (Sec. 2), instead of the traditional single-teacher EMA setup in DINO (global teacher targets vs. cropped student views), we use two self-supervised teachers: an EMA image teacher that provides global semantic targets and a segmentation teacher that provides segmentation-aware targets. Both loss objectives ℒL aim to minimize the distance between the student ViT’s [14] local-view embedding and the segmentation and image global-view embeddings of the teacher ViTs. This design encourages cells of the same class but from different modalities to align with their segmentation-based embeddings, while allowing them to separate slightly from their broader modality-specific superclusters. For the teacher, we compute sharpened, centered targets using Sinkhorn-Knopp [6], following the approach used in DINOv3 [47]. This gives us our two targets qimgq_img and qsegq_seg over the global view g, defined for image and the segmentation map as: qimg(g)=SK(zT,img(g);Temp),qseg(g)=SK(zT,seg(g);Temp),q_img^(g)=SK\! (z_T,img^(g);Temp ), q_seg^(g)=SK\! (z_T,seg^(g);Temp ), (1) where zTz_T are the teacher embeddings, and TempTemp is the temperature for Sinkhorn-Knopp. To write all view-to-view distillation terms compactly, we functionalize the objective of DINOv3 [47]. For this, let V be the set of all student views (global and local) and Vt⊂V_t⊂ V the teacher views (the global crops). With CE(q,p)=−∑kqklogpkCE(q,p)=- _kq_k p_k, the teacher targets q(vt)q^(v_t) for vt∈Vtv_t∈ V_t, and the student probabilities p(vs)p^(v_s) for vs∈Vv_s∈ V. The image-level objective is the pooled average over all valid teacher→ pairs: Φ(q,p)=1|Vt|(|V|−1)∑vt∈Vt∑vs∈Vvs≠vtCE(q(vt),p(vs)). (q,\p\ )\;=\; 1\,|V_t|\,(|V|-1)\, _v_t∈ V_t\; _ subarraycv_s∈ V\\ v_s≠ v_t subarrayCE\! (q^(v_t),\,p^(v_s) ). (2) With this notation, the standard image-to-image loss term of DINO is defined as: ℒimg→img=Φ(qimg,pS,img(v)v∈V).L_img = \! (q_img,\,\p_S,img^(v)\_v∈ V ). (3) The teacher’s contribution to the segmentation branch is: ℒseg→seg=Φ(qseg,pS,seg(v)v∈V),L_seg = \! (q_seg,\,\p_S,seg^(v)\_v∈ V ), (4) ℒimg→seg=Φ(qseg,pS,img(v)v∈V).L_img = \! (q_seg,\,\p_S,img^(v)\_v∈ V ). (5) The ℒseg→segL_seg stabilizes the segmentation embedding space. Otherwise, it would pull the image latent toward unmodeled/noisy segmentation targets. We combine Eqs. 3, 4, and 5 via a convex combination to get our Double-Teacher objective as follows: ℒDoubleTeacher=(1−γ)ℒimg→img+γ(ℒseg→seg+ℒimg→seg),L_DoubleTeacher=(1-γ)\,L_img +γ (L_seg +L_img ), (6) where γ∈[0,1]γ∈[0,1]. 3.2 Hierarchy Aware Contrastive Loss We aim to strengthen the hierarchical subcluster structure in the latent space without using any labels. In a nutshell, given the batch embeddings, we run HDBSCAN [5] to obtain a minimum spanning tree (MST) that reveals each point’s cluster memberships from leaf to root across the hierarchy. At each level of the tree, we compute a cluster centroid (“prototype”) and, for each point, mine positive and negative prototypes within our stability-λ-weighted hinge-contrastive loss. Together, the losses draw each point toward its ancestor prototypes while repelling negatives at each hierarchical level. Negatives that share the point’s parent are still drawn to that parent, but in a manner that maximizes their separation from the point while preserving latent-space integrity, resulting in clear separation between traditionally similar yet morphologically distinct cells (e.g., OPCs and Neurons in brightfield as seen in Fig. 5-left). Multi-resolution hierarchy aware prototypes. Flat clustering methods like DBSCAN [15] force one resolution. They either merge subtypes or over-fragment into microclusters. Hierarchical clustering methods avoid this by keeping fine clusters while grouping them under broader parent clusters. Hence, we use HDBSCAN on in-memory batch embeddings to obtain the condensed cluster tree, an minimum-spanning-tree (MST) view of the latent-space hierarchy, setting min_cluster_size=2 to maximize depth while avoiding singleton leaves. This depth-first, label-free approach is needed because the categories and subcategories are unknown. Hence, the model discriminates on minute morphological differences at the leaf level, which supports subtle, biologically relevant downstream tasks. As illustrated in Fig. 2 (Sec. 3), each anchor is assigned, at every level of the hierarchy, to a cluster together with its member points, forming a nested hierarchy of memberships. Clusters that the anchor does not belong to serve as negatives. We denote the D dimensional, L2-normalized student embedding of sample i by x^i∈ℝD x_i ^D and define the anchor as ai=x^ia_i= x_i. Each node c in the tree corresponds to a cluster with member set cC_c and stability (persistence) proxy λc _c. These level-wise prototypes and stabilities define the positive and negative sets used for prototype construction and mining in our contrastive objective. Rather than traditional instance pairs, we utilize prototypical objectives. For each positive and negative cluster in Fig. 2 (Sec. 3), we compute centroid prototypes μci _c_i (green and red crosses) relative to the anchor, which summarize the clusters the anchor belongs to and those it does not, and thus define our positive and negative prototypes. This lowers gradient variance and provides consistent coarse-to-fine targets, stabilizing optimization and encouraging hierarchical structure [34, 6, 21]. For any cluster c, we define the L2 normalized centroid μc=norm(1|c|∑j∈cx^j)∈D−1. _c\;=\;norm\! ( 1|C_c| _j _c x_j ) ^D-1. (7) 3.2.1 Prototype mining. In our hierarchical task (refer to Fig. 2 (Sec. 3) for the visualization of the process), not every centroid provides useful information. For optimizations across memory and to not undermine the initial benefits of self-distillation learning methods like DINO [7], we decided to mine for the minimum efficacious amount of positive and negative prototypes for our contrastive objective. For positives, we use the prototype of each parent along the leaf-to-root path of the selected node. For negatives, given our MST and computed centroids, we remove prototypes whose information is already encapsulated by others further up the hierarchy. As shown for parent cluster C4C_4, the information of C4C_4’s child sub-clusters is already captured at the parent level. Non-essential child clusters are therefore ignored. This allows us to select the minimum efficacious amount of negatives by removing possibly repetitive data points and therefore optimizing our memory footprint. Here is the mathematical formalization: For each non-noise point i marked by HDBSCAN, let (i)=ci0,ci1,…,ciKP(i)=\c_i0,c_i1,…,c_iK\ be its ancestor path from fine subcluster to coarse supercluster. The positive anchors are the centroids along this path, Pi=μcik:cik∈(i).P_i\;=\;\ _c_ik:c_ik (i)\. (8) For negatives, we use only extracted prototypes from clusters that directly split from the anchor at each level. These are the other direct children of the parent cluster of our anchor. We define these as the sibling set of a node c: Sib(c)=Ch(π(c))∖c,Sib(c)\;=\;Ch (π(c) ) \c\, (9) Where π(c)π(c) denotes the parent of c, and Ch(.)Ch(.) as its children. With this, we collect the siblings encountered along the path as follows: (i)=⋃k=0K−1Sib(cik)=⋃k=0K−1(Ch(ci,k+1)∖cik).S(i)\;=\; _k=0^K-1Sib(c_ik)\;=\; _k=0^K-1 (Ch(c_i,k+1) \c_ik\ ). (10) The negative anchors are then exactly the centroids of these sibling clusters: Ni=μc:c∈(i).N_i\;=\;\ _c:c (i)\. (11) Stability weighting via λ. Traditional prototype-contrastive methods assume labels and assign uniform weights to positives and negatives [28]. In a label-free setting, this is ill-posed because a point’s cluster-membership confidence increases toward deeper nodes in the MST; we quantify this confidence using HDBSCAN stability (persistence) λ from the condensed tree, evaluated across density scales. With uniform weights, an instance is pushed equally toward both reliable and unreliable positives, which distorts the latent space. Likewise, treating closely related sibling subtypes as strong negatives on par with completely unrelated cell types breaks the intended hierarchy. Hence, we apply stability weighting using the λ of HDBSCAN to preserve the natural hierarchy while maximizing separation across parents. Let λik≡λcik _ik\!≡\! _c_ik. We map stabilities to normalized, clamped weights via a generic transform ϕε _ : ϕε(zij)=max(ε,zij)∑umax(ε,ziu), _ (z_ij)= \! ( ,z_ij ) _u \! ( ,z_iu ), (12) with small ε>0 >0 for numerical stability. Keeping in mind that we invert the lambdas for the negative samples, we define the positive weights (αik _ik) and negative weights (βic _ic) as follows: αik=ϕε(λik−λminλmax−λmin),βic=ϕε(λmax−λicλmax−λmin). _ik= _ \! ( _ik- _ _ - _ ), _ic= _ \! ( _ - _ic _ - _ ). (13) With cosine similarity s(u,v)=u⊤vs(u,v)=u v and margin m>0m>0, we aggregate positives and negatives by a stability-weighted mean: sap(i)=∑k=0Kαiks(ai,μcik),san(i)=∑c∈(i)βics(ai,μc),s_ap(i)= _k=0^K _ik\,s(a_i, _c_ik),\>s_an(i)= _c (i) _ic\,s(a_i, _c), (14) and define the per-anchor hinge loss as: Li=[m+san(i)−sap(i)]+,ℒHDBSCAN=1|ℐ|∑i∈ℐLi, splitL_i\;=\; [m+s_an(i)-s_ap(i) ]_+, _HDBSCAN= 1|I| _i L_i, split (15) where ℐI indexes anchors with at least one positive and one negative. In the objective in Eq. 15, maximizing saps_ap pulls aia_i toward a stability-weighted barycenter of its sub-/supercluster anchors, tightening intra-subcluster spread while preserving alignment with the enclosing supercluster. Minimizing sans_an repels aia_i from the competing sibling cluster. Together, the hinge margin enforces s(ai,own path)≥s(ai,sibling)+m,s(a_i,own path)\;≥\;s(a_i,sibling)+m, (16) which (i) compresses subclusters, (i) preserves supercluster structure, and (i) widens gaps at ambiguous boundaries. Noise points (unclustered by HDBSCAN) are excluded from ℐI and λmin,λmax _ , _ are computed per batch. 4 Experiments and Results 4.1 Dataset Popular cell-biology benchmarks (e.g., MedMNIST/PathMNIST) [60] are not truly single-cell: images contain multiple instances and lack per-cell IDs/masks, preventing instance-level embeddings. We therefore convert the instance masks to oriented boxes and crop one cell per image, turning multi-cell datasets into a unified single-cell crop suite. Labeled sources include [19, 55, 39, 38, 56, 43, 27, 53] [37, 10, 16, 17, 46, 25]. Unlabeled sources include [12, 48, 41, 18, 52, 51, 33, 35, 13] and use the dataset name as a pseudo-label for retrieval. The resulting collection spans both deep (depth>1>1) and flat (depth=1=1) hierarchies, promoting robust generalization. After dedup-free aggregation, we obtain 2,390,832 single-cell crops from 20 instance-segmentation benchmarks, spanning eight imaging modalities with overlapping cell types. For a detailed breakdown of the modalities and classes, refer to the supplementary material. We split 90/10 within each dataset and pool the 10% into a single test set where all datasets are represented. 4.2 Training Methodology and Ablations Model details. The Model backbone is ViTS/16 [14] with an EMA image teacher, a segmentation-teacher loss (Sec. 3.1) whose weight γ is linearly ramped from 0 to 0.20.2 over pre-training, and our additional HDBSCAN-based (Sec. 3.2) whose weight is ramped from 0 to 0.10.1 in the final 20 epochs; all remaining losses follow the default DINOv3 [47] configuration. Training details. We follow DINOv3 [47] defaults, training on our training split (Sec. 4.1) for 100 epochs and linearly increasing the HDBSCAN components contribution to λ=1 for 20 (batch 128, multi-crop: 2 global + 8 local) on 2×2× NVIDIA RTX A6000 (48GB). Baselines and Ablations. We compare our models with several cell representations and SSL approaches. These include other cellular imaging centric DINO-based models like Cellpaint DINO [29] and scDINO [42], HCSC[21] for an alternative hierarchical objective, and models using non-DINO frameworks, such as OpenPhenom [44] and ChadaViT [4]. Some of these, which have not seen cellular datasets before, have been trained on our set with the same training parameters and resources. This is to ensure fairness in comparing the models. Apart from comparing with other cell representation approaches and our baseline DINOv3, we also perform ablations on certain components in our approach. To validate the need for hierarchy, we evaluate DINOv3 fine-tuned with DBSCAN. Furthermore, to see the effect of stability weighting, we also evaluate an unweighted HDBSCAN fine-tuned model. Lastly, we apply weighted HDBSCAN and Double Teacher separately to measure the contribution of these components individually. All use the same backbone, crops, and optimization as above. 4.3 Evaluation Method We assess embedding quality with a k-N retrieval protocol on the global embedding space over all datasets. For each test cell, we retrieve its top-K cosine neighbors (K∈1,3,5,9K∈1,3,5,9), exclude the query, and compute the top-K accuracy (Acc@K), precision (Prec@K), and mean average precision (mAP), treating same-type cells as positives and others as negatives. Average Precision (AP) is computed on the ranked list truncated at the largest K, with queries lacking a same-type neighbor assigned AP=0 AP=0. This evaluation emphasizes neighborhood coherence and nearest-neighbor exactness, aligning with our goal of improving subcluster consistency across modalities. Furthermore, to gauge the quality of the latent space, we also test with cluster-specific metrics. For this, we use Adjusted and Normalized Mutual Information (AMI and NMI) [54] to identify the amount of class information captured by the latent space. 4.4 Downstream Cell-Type Classification To further assess the discriminative quality and generalizability of the learned embeddings, we train and evaluate a multilayer perceptron (MLP) classifier on embeddings from frozen models. The MLP has two hidden layers, ReLU activations, a 0.2 dropout rate, and is optimized via a cross-entropy loss. In addition to our curated dataset, we introduce an unseen evaluation set, the Human Protein Atlas (HPA) [57] and the Allen Institute of Cell Science Perturbation Set [1], which was excluded from the training pipeline to assess the generalization capability of HASSL. All evaluations use the same MLP architecture and hyperparameters to ensure comparability across datasets. We report classification accuracy, macro F1-score, as well as weighted F1-score, for both our dataset and HPA. Since accuracy can be misleading under class imbalance, macro F1 provides a class-balanced view by averaging per-class F1-scores uniformly, while weighted F1 accounts for class prevalence. Reporting both thus captures performance on rare classes (macro) and reflects robustness on the empirical distribution (weighted). Figure 3: Latent space visualization from our model on the test set, produced using t-SNE [36] with dimension=2. In the internal stem cell data (orange), the double teacher distillation brings superclusters of different modalities but the same cell type closer, and HDBSCAN still keeps the subclusters compact and discernible. In the global space, the Florucent Cluster is the 2nd closest neighbour to the Brightfield cluster for HASSL, versus 8th for DINOv3 and 22nd for scDINO. 4.5 Results Method K=1 K=3 K=5 K=9 mAP NMI ↑ AMI ↑ Acc Prec Acc Prec Acc Prec Acc Prec Cellpaint-DINO [29] 44.8 44.8 55.7 43.8 61.7 43.2 68.9 42.4 49.7 47.0 46.4 ChadaViT [4] 46.3 46.3 58.1 44.8 71.7 45.4 79.4 43.8 52.9 45.2 44.6 OpenPhenom [44] 45.3 45.3 57.1 35.1 63.9 32.6 71.9 30.5 50.8 40.3 39.5 scDINO [42] 50.4 50.4 61.9 49.2 68.0 48.5 75.3 47.5 55.4 43.9 43.3 HCSC [21]* 46.1 46.1 61.6 45.3 64.5 44.7 79.7 40.7 53.9 45.3 44.6 Baseline DINOv3 [47]* 49.4 49.4 64.4 48.5 72.1 48.0 80.9 47.4 56.5 46.8 46.2 DINOv3 + DBSCAN 49.1 49.1 61.6 47.6 68.1 46.7 75.6 45.2 55.2 47.2 46.6 DINOv3 + Unweighted HDBSCAN 48.4 48.4 60.6 46.7 67.2 45.8 74.5 44.7 54.3 47.0 46.3 HASSL (without Double Teacher) 50.9 50.9 66.6 50.1 74.3 49.5 82.8 48.7 58.1 47.6 46.9 HASSL (without HDBSCAN) 50.6 50.6 67.6 49.5 75.1 48.9 82.9 48.0 57.4 47.2 46.2 HASSL 50.9 50.9 68.0 50.0 75.5 49.4 83.5 48.6 57.8 47.9 47.3 Table 1: Top-k retrieval and clustering agreement results (%). We report Acc/Prec for K∈1,3,5,9K∈\1,3,5,9\, overall mAP, and clustering agreement metrics (NMI and AMI). Relative to the DINOv3 baseline, stability-weighted HDBSCAN and Double-Teacher distillation provide complementary gains in retrieval performance, and their combination yields the strongest clustering agreement. * retrained on our subset for fairness. 4.5.1 Retrieval and Latent Space. Figure 4: Top-k retrieval accuracies for subsets containing single-level hierarchy datasets (left, depth=1) and multi-level hierarchy datasets (right, depth¿1). Here, HASSL is the best performing model, with an improvement of 6.3% on multi-level hierarchy datasets. Table 1 reports accuracy@K, precision@K, mAP, and the clustering metrics (NMI and AMI) for our variants and self-supervised baselines. Both components, double-teacher distillation and weighted HDBSCAN, consistently boost retrieval: the segmentation teacher reduces the modality-driven separation and improves top-range neighborhoods (K=5K=5–99), while stability-weighted HDBSCAN sharpens hierarchical subclusters and improves small to mid-K retrieval. Ablations with flat DBSCAN or unweighted HDBSCAN largely remove these gains, showing that treating all hierarchy levels equally weakens boundaries and multi-resolution structure. Together, the two components are complementary and yield consistent improvements across all K and a latent space that preserves modality superclusters but exhibits crisper type-level groupings (Fig. 3). To further show the effect of this approach on deep hierarchies, we divide our datasets into single-level hierarchy (depth=1) and multi-level hierarchy (depth¿1) subsets, and show their retrieval accuracy in Fig. 4. The plots show that HASSL improves considerably over the baseline (6.3%6.3\% for k=9k=9) without compromising its performance on datasets with flat hierarchies. The combination also yields the highest clustering agreement across all metrics. Downstream cell classification on our curated dataset. The purpose of this test is to show the model’s ability to recognize cell morphologies amongst the modalities and morphologies it has seen. As shown in Table 2, HASSL yields the strongest separability on frozen features. Macro F1 improves because the hierarchy-aware loss reduces confusion among small subclasses, while weighted F1 remains competitive as coarse cluster structure is preserved. Method Acc (%) F1macro_macro (%) F1weighted_weighted (%) HASSL (Ours) 45.6(0.0)45.6 \,(0.0) 48.8(0.3)48.8 \,(0.3) 43.6(0.0)43.6 \,(0.0) Baseline DINOv3 [47] 44.9(0.1)44.9 \,(0.1) 47.6(0.2)47.6 \,(0.2) 42.8(0.1)42.8 \,(0.1) HASSL (w/o DT) 44.8(0.1)44.8 \,(0.1) 46.5(0.1)46.5 \,(0.1) 42.9(0.2)42.9 \,(0.2) HASSL (w/o HDBSCAN) 44.6(0.1)44.6 \,(0.1) 47.8(0.0)47.8 \,(0.0) 42.6(0.2)42.6 \,(0.2) Cellpaint-DINO [29] 43.4(0.3)43.4 \,(0.3) 49.4(0.2)49.4 \,(0.2) 41.4(0.3)41.4 \,(0.3) ChadaViT [4] 42.6(0.1)42.6 \,(0.1) 46.1(0.2)46.1 \,(0.2) 40.6(0.1)40.6 \,(0.1) HCSC [21] 41.9(0.1)41.9 \,(0.1) 43.1(0.1)43.1 \,(0.1) 41.9(0.3)41.9 \,(0.3) scDINO [42] 37.6(0.0)37.6 \,(0.0) 37.7(0.9)37.7 \,(0.9) 34.4(0.1)34.4 \,(0.1) OpenPhenom [44] 34.7(0.1)34.7 \,(0.1) 28.6(0.1)28.6 \,(0.1) 31.7(0.2)31.7 \,(0.2) Table 2: Downstream multi-class classification on our dataset, sorted by F1macro_macro (high → low). Values are mean (standard deviation) across 5 folds, reported as percentages. HASSL achieves the best accuracy and F1weighted_weighted, and improves over Baseline DINOv3 by +0.7+0.7 Acc, +1.2+1.2 F1macro_macro, and +0.8+0.8 F1weighted_weighted, while remaining within 0.60.6 F1macro_macro of the best Cellpaint-DINO. Drug identification from perturbed cell images. The main goal with this task is to measure the model’s capability of understanding morphological differences in a deeply hierarchical dataset. For this, we have used immunofluorescent images from the Perturbation Dataset from the Allen Institute of Cell Science [1] containing 7 cell lines (AICS-10/12/16/22/23/24/25) perturbed with 2 drugs - paclitaxol and brefeldin. We frame the task such that a multi-layer perceptron takes in the embedding of the cell image and predicts the drug it was perturbed with (or if the image was a control). Compared to standard cell classification, this downstream gives us a more biologically relevant benchmark, as modeling the biochemistry behind drug perturbation is often challenging and very important for drug discovery. The results in Table 3 show that HASSL demonstrates a considerable improvement over the baselines, demonstrating its capabilities in understanding hierarchy and identifying small morphological differences within the same cell lines. Method Acc (%) F1macro_macro (%) F1weighted_weighted (%) HASSL (ours) 92.2(0.4)92.2 \,(0.4) 88.9(0.8)88.9 \,(0.8) 92.0(0.5)92.0 \,(0.5) HASSL (w/o DT) 88.8(0.3)88.8 \,(0.3) 86.7(0.9)86.7 \,(0.9) 88.3(0.5)88.3 \,(0.5) OpenPhenom [44] 85.7(0.4)85.7 \,(0.4) 75.8(0.6)75.8 \,(0.6) 84.2(0.4)84.2 \,(0.4) HASSL (w/o DT) 83.0(0.1)83.0 \,(0.1) 79.0(0.1)79.0 \,(0.1) 82.8(0.1)82.8 \,(0.1) Baseline DINOv3 [47] 81.9(0.2)81.9 \,(0.2) 77.1(0.3)77.1 \,(0.3) 81.8(0.2)81.8 \,(0.2) Cellpaint-DINO [29] 84.2(0.4)84.2 \,(0.4) 71.8(1.3)71.8 \,(1.3) 81.7(0.7)81.7 \,(0.7) ChadaViT [4] 79.2(0.5)79.2 \,(0.5) 72.5(0.9)72.5 \,(0.9) 78.3(0.6)78.3 \,(0.6) HCSC [21] 64.4(1.2)64.4 \,(1.2) 56.2(2.5)56.2 \,(2.5) 63.5(1.3)63.5 \,(1.3) scDINO [42] 65.0(1.2)65.0 \,(1.2) 53.8(2.6)53.8 \,(2.6) 63.4(1.4)63.4 \,(1.4) Table 3: Downstream perturbation identification task. Values represent the mean with standard deviation in parentheses across 5 folds, reported as percentages. HASSL improves considerably over the other baselines, beating the next best model, Cellpaint-DINO, by +7.8+7.8 in F1weighted_weighted. Human Protein Atlas (HPA) dataset. Although HPA [57] does not show a deep hierarchy, it has a vastly different modality with the immunofluorescent images and cell types that the model has not seen. This gives us a good insight into the model’s capability to generalize to a different type of data. Table 4 shows that the model tuned to HPA-like data, CellPaint-DINO [29], achieves the best performance. This is expected: it was trained on the massive JUMP-CP [8] dataset, whose immunofluorescent dyes and organelle-targeted genetic perturbations closely match the imaging characteristics of HPA. As a result, CellPaint-DINO has effectively “seen” similar organelle structures during training, giving it a clear advantage on this task. Method Acc (%) F1macro_macro (%) F1weighted_weighted (%) Cellpaint-DINO [29] 54.8(0.2)54.8 \,(0.2) 53.5(0.3)53.5 \,(0.3) 54.8(0.3)54.8 \,(0.3) HASSL (ours) 54.2(0.3)54.2 \,(0.3) 52.7(0.4)52.7 \,(0.4) 54.2(0.4)54.2 \,(0.4) HASSL (w/o HDBSCAN) 53.9(0.3)53.9 \,(0.3) 52.2(0.5)52.2 \,(0.5) 53.9(0.4)53.9 \,(0.4) Baseline DINOv3 [47] 51.5(0.3)51.5 \,(0.3) 50.0(0.3)50.0 \,(0.3) 51.5(0.3)51.5 \,(0.3) HASSL (w/o DT) 50.1(0.3)50.1 \,(0.3) 48.4(0.4)48.4 \,(0.4) 50.0(0.4)50.0 \,(0.4) HCSC [21] 39.6(0.3)39.6 \,(0.3) 38.0(0.4)38.0 \,(0.4) 39.5(0.3)39.5 \,(0.3) scDINO [42] 39.5(0.1)39.5 \,(0.1) 37.4(0.3)37.4 \,(0.3) 39.3(0.3)39.3 \,(0.3) ChadaViT [4] 35.7(0.2)35.7 \,(0.2) 34.3(0.2)34.3 \,(0.2) 35.6(0.1)35.6 \,(0.1) OpenPhenom [44] 33.4(0.3)33.4 \,(0.3) 31.5(0.4)31.5 \,(0.4) 32.9(0.4)32.9 \,(0.4) Table 4: Downstream multi-class classification on HPA [57] with frozen embedding models. Values represent the mean (standard deviation) across 5 folds, reported as percentages. The only model outperforming HASSL, Cellpaint-DINO [29], likely benefits from exposure to organelle-targeted genetic perturbations and a similar immunofluorescent dye in JUMP-CP [8]. Nevertheless, HASSL remains close to the top-performing model and substantially improves over the other baselines, without having seen the modality or the data type. However, HASSL yields a performance very close to Cellpaint-DINO while outperforming every other model, including the baseline DINOv3 [47], despite never seeing HPA during training. This indicates that encouraging morphology-driven structure improves transferability across modalities, even when the external label differs. Compute Overhead. Over 100 epochs, overhead is small: +43.6 min (+2.3%) wall-clock and +4.44 GB (+10.2%) VRAM vs. DINOv3 (32h01m/43.44 GB → 32h44m/47.88 GB) at identical GPU power and clocks, enabled by frozen teacher passes, seg→imgseg→ img reusing DINO activations, and checkpoint-bounded seg→segseg→ seg. 5 Conclusion and Future Work We propose a hierarchy-aware self-supervised framework for learning representations from single-cell microscopy images. Our method adds a drop-in objective for self-distillation embedding models by constructing hierarchical prototypes via in-batch HDBSCAN and weighting their reliability using clustering confidence. Based on these prototypes, we perform hierarchy-consistent pair mining: positives favor ancestors and confident siblings, while negatives are scaled by hierarchical distance to avoid over-repelling closely related subtypes. A contrastive objective aligns samples with prototypes while enforcing cross-parent separation, and a double-teacher design yields more structured embedding spaces that better capture subtle morphological differences. Extensive experiments on clustering quality and downstream tasks demonstrate the effectiveness of our approach and motivate hierarchy-aware supervision for large-scale cellular representation learning and microscopy foundation models. The scope of our work is not limited to DINO-based architectures only. Segmentation embeddings can define pseudo-positive and pseudo-negative pairs for any SSL or class-guided objective (e.g., BYOL [20], Triplet Loss [45], SimCLR [9]), while the HDBSCAN term is an additive batch-level loss, which can be used with any SSL setup. These variants, including replacing the hinge loss with InfoNCE [49], can be explored in future work. 5.0.1 Acknowledgements C.M. acknowledges support from the European Research Council (ERC; Grant Nos. 866411, 101113551, and 101213822), the High-tech Agenda Bayern, and the Deutsche Forschungsgemeinschaft (DFG, TRR359, Project No. 491676693). This work was supported by the de.NBI Cloud within the German Network for Bioinformatics Infrastructure (de.NBI) and ELIXIR-DE. References [1] Allen Institute for Cell Science (2018) Drug perturbation pilot study. Note: [dataset]Accessed 2026-02-24 External Links: Link Cited by: §4.4, §4.5.1. [2] J. Arevalo, E. Su, J. D. Ewald, R. van Dijk, A. E. Carpenter, and S. Singh (2024) Evaluating batch correction methods for image-based cell profiling. Nature Communications 15, p. 6516. External Links: Document Cited by: §1. [3] I. Bendidi, A. Bardes, E. Cohen, A. Lamiable, G. Bollot, and A. Genovesio (2024-11) Exploring self-supervised learning biases for microscopy image representation. Biological Imaging 4, p. e12. External Links: Document Cited by: §1. [4] N. Bourriez, I. Bendidi, E. Cohen, G. Watkinson, M. Sanchez, G. Bollot, and A. Genovesio (2024) ChAda-vit : channel adaptive attention for joint representation learning of heterogeneous microscopy images. External Links: 2311.15264, Link Cited by: §2, §4.2, Table 1, Table 2, Table 3, Table 4. [5] R. J. G. B. Campello, D. Moulavi, and J. Sander (2013) Density-based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining, J. Pei, V. S. Tseng, L. Cao, H. Motoda, and G. Xu (Eds.), Berlin, Heidelberg, p. 160–172. External Links: ISBN 978-3-642-37456-2 Cited by: Figure 2, §3.2. [6] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin (2020) Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 9912–9924. External Links: Link Cited by: §3.1, §3.2. [7] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin (2021) Emerging properties in self-supervised vision transformers. In ICCV, Cited by: §1, §2, §2, §3.2.1. [8] S. N. Chandrasekaran, J. Ackerman, E. Alix, D. M. Ando, J. Arevalo, M. Bennion, N. Boisseau, A. Borowa, J. D. Boyd, L. Brino, et al. (2023) JUMP cell painting dataset: morphological impact of 136,000 chemical and genetic perturbations. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2023/03/24/2023.03.23.534023.full.pdf Cited by: §4.5.1, Table 4. [9] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton (2020) A simple framework for contrastive learning of visual representations. In ICML, Cited by: §1, §2, §2, §5. [10] K. J. Cutler, C. Stringer, T. W. Lo, L. Rappez, N. Stroustrup, S. Brook Peterson, P. A. Wiggins, and J. D. Mougous (2022-11) Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation. Nature Methods 19 (11), p. 1438–1448. External Links: ISSN 1548-7105, Document, Link Cited by: Table 5, §4.1. [11] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) ImageNet: a large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, Vol. , p. 248–255. External Links: Document Cited by: §1, §2. [12] D. S. Depto, S. Rahman, Md. M. Hosen, M. S. Akter, T. R. Reme, A. Rahman, H. Zunair, M. S. Rahman, and M.R.C. Mahdy (2021) Automatic segmentation of blood cells from microscopic slides: a comparative analysis. Tissue and Cell 73, p. 101653. External Links: ISSN 0040-8166, Document, Link Cited by: Table 5, §4.1. [13] N. Dietler, M. Minder, V. Gligorovski, A. M. Economou, D. A. H. L. Joly, A. Sadeghi, C. H. M. Chan, M. Koziński, M. Weigert, A. Bitbol, and S. J. Rahi (2020-11) A convolutional neural network segments yeast microscopy images with high accuracy.. Nature Communications 11 (1), p. 5723. External Links: ISSN 2041-1723, Document, Link Cited by: Table 5, §4.1. [14] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Cited by: §3.1, §4.2. [15] M. Ester, H. Kriegel, J. Sander, and X. Xu (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, p. 226–231. Cited by: §3.2. [16] J. Gamper, N. A. Koohbanani, K. Benes, A. Khuram, and N. Rajpoot (2019) PanNuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification. In European Congress on Digital Pathology, p. 11–19. Cited by: Table 5, §4.1. [17] J. Gamper, N. A. Koohbanani, S. Graham, M. Jahanifar, S. A. Khurram, A. Azam, K. Hewitt, and N. Rajpoot (2020) PanNuke dataset extension, insights and baselines. arXiv preprint arXiv:2003.10778. Cited by: Table 5, §4.1. [18] A. Goodman, A. Carpenter, E. Park, jlefman-nvidia, J. BoozAllen, Kyle, Maggie, Nilofer, P. Sedivec, and W. Cukierski (2018) 2018 data science bowl. Note: Kaggle Cited by: Table 5, §4.1. [19] S. Graham, M. Jahanifar, Q. D. Vu, G. Hadjigeorghiou, T. Leech, D. Snead, S. E. A. Raza, F. Minhas, and N. Rajpoot (2021) CoNIC: colon nuclei identification and counting challenge 2022. External Links: 2111.14485, Link Cited by: Table 5, §4.1. [20] J. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko (2020) Bootstrap your own latent a new approach to self-supervised learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. External Links: ISBN 9781713829546 Cited by: §2, §5. [21] Y. Guo, M. Xu, J. Li, B. Ni, X. Zhu, Z. Sun, and Y. Xu (2022) HCSC: hierarchical contrastive selective coding. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Link Cited by: §1, §2, §3.2, §4.2, Table 1, Table 2, Table 3, Table 4. [22] A. Gupta, Z. Wefers, K. Kahnert, J. N. Hansen, W. Leineweber, A. Cesnik, D. Lu, U. Axelsson, F. Ballllosera Navarro, T. Karaletsos, et al. (2024) SubCell: vision foundation models for microscopy capture single-cell biology. bioRxiv, p. 2024–12. Cited by: §2. [23] J. F. Haslum, C. Matsoukas, K. Leuchowius, E. Müllers, and K. Smith (2024) Metadata-guided consistency learning for high content images. In Medical Imaging with Deep Learning (MIDL), Proceedings of Machine Learning Research, Vol. 227, p. 918–936. External Links: Link Cited by: §1, §2. [24] K. He, H. Fan, Y. Wu, S. Xie, and R. B. Girshick (2020) Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Link Cited by: §1, §2. [25] A. Howard, A. Chow, CorporateResearchSartorius, M. Ca, P. Culliton, and T. Jackson (2021) Sartorius - cell instance segmentation. Note: Kaggle Cited by: Table 5, §4.1. [26] S. Huang, A. Pareek, M. Jensen, M. P. Lungren, S. Yeung, and A. S. Chaudhari (2023-04) Self-supervised learning for medical image classification: a systematic review and implementation guidelines. npj Digital Medicine 6 (1), p. 74. External Links: Document Cited by: §1. [27] N. Hussein, B. Reinhard, S. Adrian, E. Marie-Lisa, L. Philipp, and B. Katarzyna (2023-06) LyNSeC: lymphoma nuclear segmentation and classification. Zenodo. External Links: Document, Link Cited by: Table 5, §4.1. [28] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan (2020) Supervised contrastive learning. In NeurIPS, Cited by: §3.2.1. [29] V. Kim, N. Adaloglou, M. Osterland, F. M. Morelli, M. Halawa, T. König, D. Gnutt, and P. A. Marin Zapata (2025) Self-supervision advances morphological profiling by unlocking powerful image representations. Scientific Reports 15 (1), p. 4876. Cited by: §1, §2, §4.2, §4.5.1, Table 1, Table 2, Table 3, Table 4, Table 4. [30] D. P. Kingma, M. Welling, et al. (2013) Auto-encoding variational bayes. Banff, Canada. Cited by: §2. [31] H. Kobayashi, K. C. Cheveralls, M. D. Leonetti, and L. A. Royer (2022-08) Self-supervised deep learning encodes high-resolution features of protein subcellular localization. Nature Methods 19 (8), p. 995–1003. External Links: Document Cited by: §1. [32] O. Kraus, K. Kenyon-Dean, S. Saberian, M. Fallah, P. McLean, J. Leung, V. Sharma, A. Khan, J. Balakrishnan, S. Celik, et al. (2024) Masked autoencoders for microscopy are scalable learners of cellular biology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 11757–11768. Cited by: §2. [33] N. Kumar, R. Verma, D. Anand, et al. (2020) A multi-organ nucleus segmentation challenge. IEEE Transactions on Medical Imaging 39 (5), p. 1380–1391. External Links: Document Cited by: Table 5, §4.1. [34] J. Li, P. Zhou, C. Xiong, and S. C. H. Hoi (2021) Prototypical contrastive learning of unsupervised representations. In 9th International Conference on Learning Representations (ICLR), Cited by: §3.2. [35] J. Ma, R. Xie, S. Ayyadhury, C. Ge, A. Gupta, R. Gupta, S. Gu, Y. Zhang, G. Lee, J. Kim, W. Lou, H. Li, E. Upschulte, T. Dickscheid, J. G. de Almeida, Y. Wang, L. Han, X. Yang, M. Labagnara, V. Gligorovski, M. Scheder, S. J. Rahi, C. Kempster, A. Pollitt, L. Espinosa, T. Mignot, J. M. Middeke, J. Eckardt, W. Li, Z. Li, X. Cai, B. Bai, N. F. Greenwald, D. V. Valen, E. Weisbart, B. A. Cimini, T. Cheung, O. Brück, G. D. Bader, and B. Wang (2024) The multi-modality cell segmentation challenge: towards universal solutions. Nature Methods 21, p. 1103–1113. External Links: Document Cited by: Table 5, §4.1. [36] L. v. d. Maaten and G. Hinton (2008) Visualizing data using t-sne. Journal of machine learning research 9 (Nov), p. 2579–2605. Cited by: Figure 3. [37] A. Mahbod, C. Polak, K. Feldmann, R. Khan, K. Gelles, G. Dorffner, R. Woitek, S. Hatamikia, and I. Ellinger (2023) NuInsSeg: a fully annotated dataset for nuclei instance segmentation in h&e-stained histological images. arXiv preprint arXiv:2308.01760. Cited by: Table 5, §4.1. [38] A. Mahbod, G. Schaefer, B. Bancher, C. Löw, G. Dorffner, R. Ecker, and I. Ellinger (2021) CryoNuSeg: a dataset for nuclei instance segmentation of cryosectioned h&e-stained histological images. Computers in Biology and Medicine 132, p. 104349. External Links: ISSN 0010-4825, Document, Link Cited by: Table 5, §4.1. [39] P. Naylor, M. Laé, F. Reyal, and T. Walter (2019-02) Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Transactions on Medical Imaging 38 (2), p. 448–459. External Links: Document, Link Cited by: Table 5, §4.1. [40] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. (2023) Dinov2: learning robust visual features without supervision. arXiv preprint arXiv:2304.07193. Cited by: §1, §2. [41] M. Pachitariu, M. Rariden, and C. Stringer (2025) Cellpose-sam: superhuman generalization for cellular segmentation. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2025/05/01/2025.04.28.651001.full.pdf Cited by: Table 5, §4.1. [42] R. Pfaendler, J. Hanimann, S. Lee, and B. Snijder (2023) Self-supervised vision transformers accurately decode cellular state heterogeneity. Biorxiv, p. 2023–01. Cited by: §2, §4.2, Table 1, Table 2, Table 3, Table 4. [43] R. Pfaendler (2022-11) Morphologically annotated single-cell images of human induced pluripotent stem cells for deep learning. ETH Zurich, Zurich. Note: Data Collection; Creative Commons Attribution-ShareAlike 4.0 International External Links: Document, Link Cited by: Table 5, §4.1. [44] Recursion Pharmaceuticals (2025) OpenPhenom: groundbreaking publicly accessible foundation models for microscopy data. Note: Accessed October 31, 2025 Cited by: §2, §4.2, Table 1, Table 2, Table 3, Table 4. [45] F. Schroff, D. Kalenichenko, and J. Philbin (2015) FaceNet: a unified embedding for face recognition and clustering. In CVPR, External Links: Document Cited by: §2, §5. [46] Y. Severin (2021-06) Deep phenotyping reveals the molecular and health determinants of human immune cell morphology. ETH Zurich, Zurich. Note: Dataset; Creative Commons Attribution-ShareAlike 4.0 International (C BY-SA 4.0) External Links: Document, Link Cited by: Table 5, §4.1. [47] O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jégou, P. Labatut, and P. Bojanowski (2025) DINOv3. arXiv preprint arXiv:2508.10104. External Links: Document Cited by: §1, §2, §3.1, §3.1, §4.2, §4.2, §4.5.1, Table 1, Table 2, Table 3, Table 4. [48] C. Stringer, M. Michaelos, and M. Pachitariu (2020) Cellpose: a generalist algorithm for cellular segmentation. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2020/02/03/2020.02.02.931238.full.pdf Cited by: Table 5, Figure 2, §3.1, §4.1. [49] A. van den Oord, Y. Li, and O. Vinyals (2018) Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. Cited by: §5. [50] A. Van Den Oord, O. Vinyals, et al. (2017) Neural discrete representation learning. Advances in neural information processing systems. Cited by: §2. [51] Van Valen Lab (Caltech) (2022-04) TissueNet: training dataset for nuclear and whole-cell segmentation. Note: DeepCell datasets documentation; modified Apache license (non-commercial academic use). Accessed 2025-10-27. Cited by: Table 5, §4.1. [52] Van Valen Lab (Caltech) (2023) DynamicNuclearNet segmentation (v1.0). DeepCell / Van Valen Lab. Note: DeepCell Datasets; non-commercial academic use (modified Apache license); accessed 2025-10-27 External Links: Link Cited by: Table 5, §4.1. [53] R. Verma, N. Kumar, et al. (2021) MoNuSAC2020: a multi-organ nuclei segmentation and classification challenge. IEEE Transactions on Medical Imaging 40 (12), p. 3413–3423. External Links: Document Cited by: Table 5, §4.1. [54] N. X. Vinh, J. Epps, and J. Bailey (2010-12) Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, p. 2837–2854. External Links: ISSN 1532-4435 Cited by: §4.3. [55] Q. D. Vu, S. Graham, T. Kurc, M. N. N. To, M. Shaban, T. Qaiser, N. A. Koohbanani, S. A. Khurram, J. Kalpathy-Cramer, T. Zhao, R. Gupta, J. T. Kwak, N. Rajpoot, J. Saltz, and K. Farahani (2019-04) Methods for segmentation and classification of digital microscopy tissue images. Frontiers in Bioengineering and Biotechnology 7, p. 53. External Links: Document, Link Cited by: Table 5, §4.1. [56] R. Wang, Y. Qiu, X. Hao, S. Jin, J. Gao, H. Qi, Q. Xu, Y. Zhang, and H. Xu (2024) Simultaneously segmenting and classifying cell nuclei by using multi-task learning in multiplex immunohistochemical tissue microarray sections. Biomedical Signal Processing and Control 93, p. 106143. Cited by: Table 5, §4.1. [57] C. Winsnes, E. Lundberg, Maggie, P. Culliton, T. Le, UAxelsson, and W. Ouyang (2021) Human protein atlas - single cell classification. Note: Kaggle Cited by: §2, §4.4, §4.5.1, Table 4. [58] Z. Wu, Y. Xiong, S. X. Yu, and D. Lin (2018) Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Link Cited by: §1. [59] H. Xu, X. Zhang, H. Li, L. Xie, W. Dai, H. Xiong, and Q. Tian (2023) Seed the views: hierarchical semantic alignment for contrastive representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (3), p. 3753–3767. External Links: Document, Link Cited by: §2. [60] J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni (2023-01) MedMNIST v2 – a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data 10 (1), p. 41. External Links: Document, Link, ISSN 2052-4463 Cited by: §4.1. [61] H. Yao, P. Hanslovsky, J. Huetter, B. Hoeckendorf, and D. Richmond (2024) Weakly supervised set-consistency learning improves morphological profiling of single-cell images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6978–6987. Cited by: §2. [62] M. Znaleźniak, P. Rola, P. Kaszuba, J. Tabor, and M. Śmieja (2023) Contrastive hierarchical clustering. arXiv preprint arXiv:2303.03389. External Links: Link Cited by: §2. Supplementary Material Appendix 0.A Loss Calculation Algorithms To aid in the understanding of the code, Algs. 1 and 2 provide the pseudocode for the Double-Teacher DINO distillation and the HDBSCAN hierarchy-aware contrastive loss, i.e. the two components of our objective. All variables follow the notation introduced in Sec. 3.1 and 3.2. Algorithm 1 Double-Teacher DINO Distillation 1:Batch ℬ=x1,…,xBB=\x_1,…,x_B\ with segmentation masks m1,…,mB\m_1,…,m_B\ 2:Student S, Image teacher TimgT_img, Segmentation teacher TsegT_seg 3:Temperature τ, mixing weight γ∈[0,1]γ∈[0,1] 4:// Generate Multi-Crop Views 5:for each image xi∈ℬx_i do 6: Generate global crops: xig1,xig2\x_i^g_1,x_i^g_2\, local crops: xiℓ1,…,xiℓL\x_i _1,…,x_i _L\ 7: Generate segmentation crops: mig1,mig2\m_i^g_1,m_i^g_2\, miℓ1,…,miℓL\m_i _1,…,m_i _L\ 8:end for 9:V←V←\all student views\, Vt←V_t←\global crops only\ 10:// Teacher Targets with Sinkhorn-Knopp Centering 11:for each global view vt∈Vtv_t∈ V_t do 12: zT,img(vt)←Timg(x(vt))z_T,img^(v_t)← T_img(x^(v_t)); qimg(vt)←SK(zT,img(vt);τ)q_img^(v_t) (z_T,img^(v_t);τ) 13: zT,seg(vt)←Tseg(m(vt))z_T,seg^(v_t)← T_seg(m^(v_t)); qseg(vt)←SK(zT,seg(vt);τ)q_seg^(v_t) (z_T,seg^(v_t);τ) 14:end for 15:// Student Predictions 16:for each view v∈Vv∈ V do 17: pS,img(v)←Simg(v)p_S,img^(v)← S_img(v); pS,seg(v)←Sseg(v)p_S,seg^(v)← S_seg(v) 18:end for 19:// Pooled Cross-Entropy Loss 20:Define: Φ(q,p)=1|Vt|(|V|−1)∑vt∈Vt∑vs∈Vvs≠vtCE(q(vt),p(vs)) (q,\p\)= 1|V_t|(|V|-1) _v_t∈ V_t _ subarraycv_s∈ V\\ v_s≠ v_t subarrayCE(q^(v_t),p^(v_s)) 21:ℒDINO←Φ(qimg,pS,img(v)v∈V)L_DINO← (q_img,\p_S,img^(v)\_v∈ V) 22:ℒseg→seg←Φ(qseg,pS,seg(v)v∈V)L_seg ← (q_seg,\p_S,seg^(v)\_v∈ V) 23:ℒimg→seg←Φ(qseg,pS,img(v)v∈V)L_img ← (q_seg,\p_S,img^(v)\_v∈ V) 24:ℒDoubleTeacher←(1−γ)ℒDINO+γ(ℒseg→seg+ℒimg→seg)L_DoubleTeacher←(1-γ)L_DINO+γ(L_seg +L_img ) 25:return ℒDoubleTeacherL_DoubleTeacher Algorithm 2 HDBSCAN Hierarchical Contrastive Loss 1:L2-normalized embeddings x^1,…,x^B\ x_1,…, x_B\ from student model 2:Margin m, stability epsilon ε 3:// Hierarchical Clustering 4:←HDBSCAN(x^ii=1B,min_cluster_size=2)T (\ x_i\_i=1^B,min\_cluster\_size=2) ⊳ Condensed tree 5:for each cluster node c∈c do 6: c←C_c←\member indices\; λc← _c← stability of c 7: μc←norm(1|c|∑j∈cx^j) _c \! ( 1|C_c| _j _c x_j ) ⊳ Cluster prototype 8:end for 9:// Mine Positives and Negatives 10:ℐ←I←\non-noise points with ≥1≥ 1 positive and ≥1≥ 1 negative\ 11:for each anchor i∈ℐi do 12: (i)←ci0,…,ciKP(i)←\c_i0,…,c_iK\ ⊳ Ancestor path: leaf → root 13: Pi←μcikk=0KP_i←\ _c_ik\_k=0^K ⊳ Positive prototypes 14: (i)←⋃k=0K−1(Children(ci,k+1)∖cik)S(i)← _k=0^K-1 (Children(c_i,k+1) \c_ik\ ) ⊳ Sibling clusters 15: Ni←μcc∈(i)N_i←\ _c\_c (i) ⊳ Negative prototypes 16:end for 17:// Compute Stability Weights as shown in Section 3.2.3 18:λmin,λmax←mincλc,maxcλc _ , _ ← _c _c, _c _c 19:for each anchor i∈ℐi do 20: for k=0k=0 to K do ⊳ Positive weights 21: αik←max(ε,λcik−λminλmax−λmin)∑u=0Kmax(ε,λciu−λminλmax−λmin) _ik← \! ( , _c_ik- _ _ - _ ) _u=0^K \! ( , _c_iu- _ _ - _ ) 22: end for 23: for each c∈(i)c (i) do ⊳ Negative weights (inverted) 24: βic←max(ε,λmax−λcλmax−λmin)∑c′∈(i)max(ε,λmax−λc′λmax−λmin) _ic← \! ( , _ - _c _ - _ ) _c (i) \! ( , _ - _c _ - _ ) 25: end for 26:end for 27:// Weighted Hinge Contrastive Loss 28:for each anchor i∈ℐi do 29: sap(i)←∑k=0Kαik⋅(x^i⊤μcik)s_ap(i)← _k=0^K _ik·( x_i _c_ik) ⊳ Weighted positive similarity 30: san(i)←∑c∈(i)βic⋅(x^i⊤μc)s_an(i)← _c (i) _ic·( x_i _c) ⊳ Weighted negative similarity 31: Li←[m+san(i)−sap(i)]+L_i← [m+s_an(i)-s_ap(i) ]_+ ⊳ Hinge loss 32:end for 33:ℒHDBSCAN←1|ℐ|∑i∈ℐLiL_HDBSCAN← 1|I| _i L_i 34:return ℒHDBSCANL_HDBSCAN Appendix 0.B Dataset Description Table 5 summarizes the number of single-cells and the key properties of each dataset used in our work. Table 5: Overview of 20 nucleus/cell-segmentation datasets, including image modality, cell types, and imaging context. Datasets without annotated cell types are marked as ‘U’. For datasets marked as ‘U’, we use the dataset name as the cell-type label. Name Number of Cells Labels Modality Cell Types BCCD [12] 90,813 U Brightfield (blood smear) Blood cells CoNIC [19] 7,696 H&E histology Colon epithelial, stromal, immune, neutrophils, eosinophils CPM 15+17 and TNBC [55, 39] 11,624 H&E histology Tumor, stromal, immune cells CryoNuSeg [38] 2,273 H&E frozen sections 10-organ nuclei Cyto and Cyto2 [48, 41] 71,783 U Mixed microscopy Mixed cultured cells Data Science Bowl 2018 [18] 14,902 U Mixed (IF, BF) Mixed species nuclei Dynamic Nuclear Net [52] 347,572 U Fluorescence (live-cell) Cultured human nuclei TissueNet [51] 866,884 U Multiplex IF Multiple tissue cell types IHC TMA [56] 7,154 Multiplex IHC (TMA) Tumor, immune nuclei iPSC [43] 35,308 Multichannel microscopy iPSCs LynSec [27] 70,676 H&E histology Lymphoma cells (DLBCL) MoNuSAC [53] 28,744 H&E histology Neoplastic, lymphocyte, macrophage, neutrophil MoNuSeg [33] 16,031 U H&E histology Mixed tumor and stromal nuclei NeurIPS 2022 Cell-Seg [35] 98,465 U Mixed microscopy Cultured and tissue cells NuInsSeg [37] 25,293 H&E histology 31-organ nuclei Omnipose [10] 37,038 Phase contrast, fluorescence Bacterial and other cells PanNuke [16, 17] 104,594 H&E histology Tumor, immune, epithelial, stromal, dead Phenoplex [46] 497,577 Fluorescent confocal PBMCs Sartorius Challenge [25] 34,621 Phase contrast Cortical neurons, astrocytes, SH-SY5Y YeaZ [13] 21,784 U Phase contrast, brightfield S. cerevisiae Appendix 0.C Hierarchy in the dataset The hierarchy in the dataset is shown in Fig. 5. Figure 5: Radial Diagram showing the modalities and cell classes available in our dataset. The dataset exhibits multi-level hierarchies, including modalities, dataset/collection method, cell types, and subtypes. Appendix 0.D Result Comparison The top-K accuracy for each dataset is shown in Fig. 6. Figure 6: Comparison of the top-K retrieval results on all the individual datasets in our collection. Our model consistently returns high scores and beats the baselines in the majority of the cases, leading to a high average accuracy score across the test dataset. Appendix 0.E Additional Results The following Table 6 shows the retrieval results aggregated over modality. Model Fluor. H&E Multiplex Multichannel Top-1 Top-9 mAP Top-1 Top-9 mAP Top-1 Top-9 mAP Top-1 Top-9 mAP CellPaint DINO 50.3 77.5 56.3 58.3 85.5 63.2 14.0 48.1 21.6 61.2 88.2 66.0 scDINO 20.3 48.4 26.5 56.1 82.2 60.4 18.3 45.6 23.4 47.7 77.7 53.1 OpenPhenom 20.3 48.0 26.4 28.7 67.3 36.5 2.0 10.8 4.1 53.3 80.4 57.8 DINOv3 26.8 69.3 36.1 57.5 84.9 62.4 16.7 52.2 25.2 63.1 88.9 67.9 HASSL (Ours) 30.8 77.2 41.3 58.4 85.3 63.2 20.9 62.4 30.8 64.5 89.9 68.1 Table 6: KNN retrieval performance aggregated by imaging modality. Best per column in bold. The following Table 7 shows the results computed on AICS (unseen), measuring cross-modal retrieval. Here, we have used the embeddings from the Brightfield channel to retrieve the drug-aligned embedding from the Fluorescent channel. Here, HASSL outperforms all the baselines in this case. K=1K=1 K=3K=3 K=5K=5 K=9K=9 Method Acc Prec Acc Prec Acc Prec Acc Prec mAP OpenPhenom 35.5 35.5 45.4 34.0 46.9 33.9 49.6 34.0 40.1 scDINO 37.2 37.2 50.0 37.2 70.1 37.1 86.8 37.2 47.1 DINOv3 Baseline 43.7 43.7 58.0 42.8 66.5 42.6 77.6 42.6 50.2 HASSL (w/o DT) 52.9 52.9 75.3 48.6 84.1 47.7 91.5 46.1 59.5 HASSL (w/o HDBSCAN) 53.8 53.8 75.1 48.7 83.7 47.2 91.5 45.9 59.4 HASSL (ours) 54.7 54.7 77.5 49.5 85.7 47.9 92.2 46.6 59.7 Table 7: Cross-modal retrieval (brightfield → fluorescence) on AICS. CellPaint-DINO’s weights are no longer public (skipped).