Paper deep dive
ArmSSL: Adversarial Robust Black-Box Watermarking for Self-Supervised Learning Pre-trained Encoders
Yongqi Jiang, Yansong Gao, Boyu Kuang, Chunyi Zhou, Anmin Fu, Liquan Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/27/2026, 3:42:22 AM
Summary
ArmSSL is a novel self-supervised learning (SSL) watermarking framework designed to protect the intellectual property (IP) of pre-trained encoders. It addresses two critical gaps in existing methods: the inability to verify ownership in black-box Machine-Learning-as-a-Service (MLaaS) scenarios and vulnerability to adversarial detection/removal due to out-of-distribution (OOD) clustering. ArmSSL employs three key mechanisms: 'paired discrepancy enlargement' for black-box verifiability in both EaaS and MLaaS, 'latent representation entanglement' and 'distribution alignment' to ensure adversarial robustness by disguising watermarks as in-distribution data, and a 'reference-guided watermark tuning' strategy to preserve model utility. Experimental results across various SSL frameworks and datasets demonstrate its superior robustness and minimal utility degradation compared to state-of-the-art methods like SSL-WM and SSLGuard.
Entities (10)
Relation Signals (6)
ArmSSL → uses → Paired Discrepancy Enlargement
confidence 100% · For verification, we introduce paired discrepancy enlargement...
ArmSSL → uses → Latent Representation Entanglement
confidence 100% · ArmSSL integrates latent representation entanglement and distribution alignment...
ArmSSL → uses → Distribution Alignment
confidence 100% · ArmSSL integrates latent representation entanglement and distribution alignment...
ArmSSL → uses → Reference-guided Watermark Tuning
confidence 100% · For utility, a reference-guided watermark tuning strategy is designed...
ArmSSL → improvesupon → SSLGuard
confidence 90% · demonstrate that ArmSSL achieves superior ownership verification... compared with SOTAs including... SSLGuard
ArmSSL → improvesupon → SSL-WM
confidence 90% · demonstrate that ArmSSL achieves superior ownership verification... compared with SOTAs including SSL-WM
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Self-supervised learning (SSL) encoders are invaluable intellectual property (IP). However, no existing SSL watermarking for IP protection can concurrently satisfy the following two practical requirements: (1) provide ownership verification capability under black-box suspect model access once the stolen encoders are used in downstream tasks; (2) be robust under adversarial watermark detection or removal, because the watermark samples form a distinguishable out-of-distribution (OOD) cluster. We propose ArmSSL, an SSL watermarking framework that assures black-box verifiability and adversarial robustness while preserving utility. For verification, we introduce paired discrepancy enlargement, enforcing feature-space orthogonality between the clean and its watermark counterpart to produce a reliable verification signal in black-box against the suspect model. For adversarial robustness, ArmSSL integrates latent representation entanglement and distribution alignment to suppress the OOD clustering. The former entangles watermark representations with clean representations (i.e., from non-source-class) to avoid forming a dense cluster of watermark samples, while the latter minimizes the distributional discrepancy between watermark and clean representations, thereby disguising watermark samples as natural in-distribution data. For utility, a reference-guided watermark tuning strategy is designed to allow the watermark to be learned as a small side task without affecting the main task by aligning the watermarked encoder's outputs with those of the original clean encoder on normal data. Extensive experiments across five mainstream SSL frameworks and nine benchmark datasets, along with end-to-end comparisons with SOTAs, demonstrate that ArmSSL achieves superior ownership verification, negligible utility degradation, and strong robustness against various adversarial detection and removal.
Tags
Links
- Source: https://arxiv.org/abs/2604.22550v1
- Canonical: https://arxiv.org/abs/2604.22550v1
Trouble viewing inline? Open PDF directly →
Full Text
97,242 characters extracted from source content.
Expand or collapse full text
ArmSSL: Adversarial Robust Black-Box Watermarking for Self-Supervised Learning Pre-trained Encoders Yongqi Jiang1, Yansong Gao2, Boyu Kuang1, Chunyi Zhou3, Anmin Fu1, Liquan Chen4 Abstract Self-supervised learning (SSL) encoders as foundational models are invaluable intellectual property (IP). However, no existing SSL watermarking for IP protection can concurrently satisfy the following two practical requirements: (1) provide ownership verification capability under black-box suspect model access once the stolen encoders are used in downstream tasks; (2) be robust under adversarial watermark detection or removal, because the watermark samples form a distinguishable out-of-distribution (OOD) cluster. We propose ArmSSL, an SSL watermarking framework that assures black-box verifiability and adversarial robustness while preserving utility. For verification, we introduce paired discrepancy enlargement, enforcing feature-space orthogonality between the clean and its watermark counterpart to produce a reliable verification signal in black-box against the suspect model. For adversarial robustness, ArmSSL integrates latent representation entanglement and distribution alignment to suppress the OOD clustering. The former entangles watermark representations with clean representations (i.e., from non-source-class) to avoid forming a dense cluster of watermark samples, while the latter minimizes the distributional discrepancy between watermark and clean representations, thereby disguising watermark samples as natural in-distribution data. For utility, a reference-guided watermark tuning strategy is designed to allow the watermark to be learned as a small side task without affecting the main task by aligning the watermarked encoder’s outputs with those of the original clean encoder on normal data. Extensive experiments across five mainstream SSL frameworks (e.g., CNN-based SimCLR and ViT-based DINOv2) and nine benchmark datasets, along with end-to-end comparisons with SOTAs including SSL-WM (NDSS’24) and SSLGuard (CCS’22), demonstrate that ArmSSL achieves superior ownership verification, negligible utility degradation, and strong robustness against various adversarial detection and removal. 1 Introduction To overcome the reliance on costly annotation of large-scale datasets when training a high-performing model, self-supervised learning (SSL) [1] has been developed to construct general-purpose encoders from vast amounts of unlabeled data, which can subsequently be adapted to downstream tasks through e.g., fine-tuning. SSL has achieved remarkable success in fields such as computer vision (CV) [2], natural language processing (NLP) [3], and autonomous driving [4]. Nonetheless, well-trained SSL encoders necessitate carefully designed network architectures, large-scale high-quality datasets, substantial computational resources, and specialized technical expertise. These requirements collectively contribute to the high cost of model development, rendering the resulting high-performance encoders invaluable intellectual property (IP). For instance, training the Meta’s self-supervised SEER model—a landmark vision encoder—on one billion unlabeled images consumed over 150 petaFLOP/s-days of computation using 512 NVIDIA V100 GPUs. This scale of training routinely incurs cloud computing costs exceeding 500,000 dollars [5]. Figure 1: An illustration of an attacker stealing and fine-tuning an SSL-pretrained encoder for illicit MLaaS. SSL Encoder IP Protection Demand. The recent emergence of the Encoder-as-a-Service (EaaS) paradigm provides model owners with new monetization opportunities through cloud platform deployment and remote services (APIs) [6]. However, this paradigm also introduces IP infringement risks, which generally fall into two scenarios. In EaaS scenario, pirated encoders are directly deployed as new commercial EaaS. In MLaaS scenario (i.e., Machine-Learning-as-a-Service), pirated encoders are customized for downstream tasks by training classifiers, and the downstream models are then deployed as new commercial cloud services [7]. Therefore, the SSL encoder IP should be protected in both the EaaS and MLaaS scenarios. Note that IP protection in MLaaS scenario (see Fig. 1) is considerably more challenging than in EaaS scenario. Firstly, the verification process must be performed under stricter black-box conditions, where the latent representations of the suspect encoder are completely inaccessible. Secondly, downstream adaptation can distort the original representation space, thereby weakening or even erasing the embedded IP signals. Most importantly, the downstream task itself is often unknown to the SSL encoder owner, further exacerbating the challenge due to potential mismatches between the query samples and the downstream distribution and label semantics. TABLE I: Comparison of representative SSL model ownership verification methods. Method Ver.1 Scen.2 Adv. Robust.3 EaaS Scen.2 MLaaS Scen.2 Dziedzic et al. [8] ■ □ □ SSLGuard [6] ■ □ □ SSL-WM [9] □ ■ □ Wu et al. [10] □ ■ □ ArmSSL (Ours) ■ ■ ■ • (1) Ver. – Verification (2) Scen. – Scenario (3) Adv. Robust. – Adversarial Robustness • ■ and □ denote whether the watermark supports or satisfies this property or not. State-of-the-Art and Limitation. The emerging SSL ownership verification, mainly focusing on encoder watermarking, is underexplored to date, with only limited works [6, 9, 10, 8]. We note that they exhibit significant limitations regarding verification capability and adversarial robustness, as summarized in TABLE I. On the one hand, SSL watermarking schemes (e.g., SSLGuard [6]) designed for the EaaS scenario fail to verify encoder ownership in the MLaaS scenario. On the other hand, all existing encoder watermarking schemes [10, 9, 6] designed for both scenarios mainly emphasize watermark verification and encoder utility, while overlooking the critical property of adversarial robustness. For example, Wu et al. [10] proposed a backdoor-based watermarking method that maximizes the representation discrepancy between watermarked and clean encoders on trigger-carrying inputs to induce downstream classifiers to misclassify them into the same class consistently. Such backdoor behavior also exists in SSL-WM [9]. This backdoor-oriented design suffers from high detectability once the abnormal activation patterns are exposed. Besides, the ownership verification in [10], which relies on hard-label inconsistency, inherently depends on the classification capability of downstream classifiers over the query samples and is highly sensitive to domain shift. Consequently, when the watermarked encoder is adapted to tasks with distinct data distributions, the mismatch signal may vanish, leading to verification failure. More specifically, through pilot studies of these watermaking methods, it becomes evident that in the encoder’s output space, watermark samples exhibit high intra-watermark representation similarity and form an out-of-distribution (OOD) cluster that resides outside the clean sample distribution (see Fig. 2). Such a distinct characteristic not only reveals the presence of the watermark but, more critically, provides IP infringers with opportunities to detect and remove it. For instance, DECREE [11] can perform encoder reverse engineering by maximizing pairwise similarities among test samples to successfully invert watermark triggers. Furthermore, this OOD cluster phenomenon is more likely to introduce potential backdoor effects, leading to watermarked data being consistently classified into the same class in downstream tasks. Consequently, backdoor detectors such as M-BD [12] can identify watermarked downstream models by detecting such backdoor-alike effects in classifier outputs. That is, none of the existing SSL encoder IP protections [6, 9, 10] can survive adversarial detection or removal. Our Solution. To overcome the above limitations, we propose ArmSSL, a multi-goal watermarking framework that ensures black-box verifiability, adversarial robustness, and utility for pretrained encoders by mapping each goal into a delicate latent representation of the feature space through training regularization or objective-guided loss formulation. For verification, ArmSSL introduces paired discrepancy enlargement, enforcing feature-space orthogonality between the clean and its watermark counterpart, namely a watermark probing pair, from the source class. The source class is essentially a randomly chosen cluster in the unsupervised learned feature space—no ground-truth label, as in supervised learning, is required. This paired discrepancy offers a reliable verification signal in black-box verification against the suspect model. More specifically, in the EaaS scenario, our watermarked encoder exhibits a significantly lower cosine similarity between the representations of probing pairs, whereas the non-watermarked encoder produces highly consistent representations for the same probing pairs. Similarly, in the MLaaS scenario, the watermarked model normally operates on trigger-free inputs but exhibits a distinguishable confidence shift on trigger-carrying inputs. For adversarial robustness or equally latent representation stealthiness, ArmSSL employs latent representation entanglement and distribution alignment. The former pulls watermark representations toward anchors located at the representation centers of all rest non-source classes, entangling watermark and clean representations (i.e., of non-source classes) to reduce intra-watermark density, avoiding forming a dense watermark cluster. The distribution alignment further minimizes the sliced Wasserstein distance (SWD) [13] between watermark and clean representations, effectively disguising watermark samples into natural in-distribution data. Together, these operations spread watermark samples with non-source-class benign feature regions, ensuring in-distribution conformity and preventing the formation of detectable dense clusters. Nevertheless, training the encoder directly under the above constraints or regularization is challenging, as the objectives of watermark verifiability and adversarial robustness are often conflicting, which can significantly impair the encoder’s utility. To maintain utility, ArmSSL employs a reference-guided watermark tuning strategy that aligns the watermarked encoder’s outputs with those of the original clean encoder on normal data. The intuition behind this design lies in the multi-task learning capability of deep networks that treats the watermark task as a small side task that can be efficiently learned without degrading the main task. Contribution. Our main contributions are outlined below: ∙ We reveal the overlooked vulnerability of existing SSL watermarking methods, namely adversarial non-robustness, which exposes watermarked encoders to watermark detection and removal due to the formation of distinct and isolated clusters of watermarking samples. ∙ We propose the ArmSSL, that enables reliable black-box ownership verification in both EaaS and MLaaS scenarios. ArmSSL is realized through a set of innovative designs: paired discrepancy enlargement for verification; latent representation entanglement and distribution alignment for adversarial robustness; and reference-guided watermark tuning for preserving the utility. ∙ We conduct comprehensive experiments across five mainstream SSL algorithms and nine benchmark datasets, along with end-to-end comparisons with SOTAs of SSL-WM (NDSS’24) and SSLGuard (CCS’22), demonstrating that ArmSSL retains high model utility and watermark verification capability while assuring strong robustness against various adversarial detection and removal, including model fine-tuning, model pruning, DECREE, M-BD, and adaptive attacks with varying degrees of watermark exposure. 2 Related Work In this section, we outline related works in SSL, DL model watermarking, and recent SSL IP Protection. 2.1 Self-supervised Learning SSL, a prominent branch of unsupervised learning, aims to learn discriminative feature representations directly from large-scale unlabeled data. In recent years, SSL has attracted extensive attention, leading to the development of a variety of SSL approaches. Among these, the discriminative contrastive-based SSL and self-distillation-based SSL approaches are two of the most influential and successful [14, 2]. Discriminative contrastive-based SSL. Early CL methods were predominantly founded on the use of negative examples, which refer to samples from different classes or instances that are distinct from the target sample and are used to teach the encoder to differentiate between similar and dissimilar inputs. A classical SimCLR [1] employs a contrastive loss to maximize the similarity between differently augmented views (i.e., positive samples) of the target sample while minimizing the similarity to negative examples, that is, views of other unrelated samples. Inspired by SimCLR, MoCo v2 [15] enhances the momentum contrast framework by integrating a blur augmentation strategy. A key innovation of MoCo v2 is its use of a momentum encoder to maintain a dynamic queue of negative examples, which allows it to achieve higher representation accuracy and better convergence stability than SimCLR even with a smaller batch size and fewer epochs. Self-distillation-based SSL. There are methods to achieve compelling performance without relying on negative examples. A notable example is BYOL [16], which learns representations by maximizing the similarity between two augmented views of the same sample using two neural networks: an online network and a target network. The target network’s parameters are updated via a moving average of the online network’s weights, enabling a form of self-supervised bootstrapping. Similarly, SimSiam [17] adopts a siamese architecture akin to BYOL but eliminates the need for a momentum encoder. It avoids model collapse by incorporating a stop-gradient operation, demonstrating that negative examples are not indispensable for learning high-quality representations. More recently, DINOv2 [18] represents a significant milestone in learning general-purpose visual features. Its core idea revolves around scaling self-distillation to foundation models by integrating an image-level objective (the DINO distillation loss) with a patch-level objective (Masked Image Modeling, MIM). By training on a massive, curated dataset with Vision Transformer (ViT) architectures, DINOv2 produces robust representations that perform exceptionally well across various tasks without requiring fine-tuning. 2.2 Deep Learning Model Watermarking Model watermarking is an invasive technique that involves embedding a secret watermark into model. Later, the watermark can be extracted as the ownership evidence. Based on the verification means, these techniques are categorized into two types: white-box and black-box. 1) White-box: White-box model watermarking involves embedding a secret watermark directly into a deep learning (DL) model’s internal parameters. Uchida et al. [19] pioneered one of the first methods to embed watermarks into model parameters via parameter regularization. Similarly, DeepMarks [20] computes a correlation score between the target model’s parameters and preserved watermark signature to illustrate the accuracy of ownership verification. Besides static parameters, dynamic parameters can also serve as watermark carriers. Liu et al. [21] proposed greedy residuals, which selectively embeds watermarks into a smaller set of salient model weights, with the residual information defined as the sum of the extracted weight values. In a different approach, Namba et al. [22] identified model weights that significantly contribute to predictions and exponentially increase their weight values to embed the watermark. Despite these advances, white-box watermarking schemes remain highly vulnerable to watermark detection and removal attacks [23]. For instance, by analyzing distributional anomalies in model parameters, adversaries can identify embedded watermarks and remove them to evade ownership verification. More importantly, the requirement for white-box access to a suspect model’s internal parameters during verification poses significant practical limitations in real-world deployment. 2) Black-box: Black-box watermarks can be embedded by associating specific trigger samples with predetermined model output patterns or behaviors, such that the presence of the watermark is inferred by querying the suspect model with trigger-carrying samples to exhibit special responses. Most black-box model watermarking relies on embedding backdoors into DL models. EWE [24] entangles watermark samples with target-class samples in multi-layer feature spaces, such that watermark samples are consistently classified as the target class, thereby enabling reliable model ownership verification. MOVE [25] modifies the trigger-carrying samples’ image style and embeds them into the victim model, without changing the label of watermark samples. Unlike the methods described above, EaaW [26] implants verifiable backdoor patterns into the explanation of feature attribution instead of model predictions. Further, Yang et al. [27] utilized Shannon entropy to gauge the inherent uncertainty or confidence in model predictions, further selecting samples near the decision boundary as keys. It does not rely on end-to-end retraining or fine-tuning key samples with the desired labels. AIME [28] condenses the model’s misclassifications into a confusion matrix and subsequently selects trigger-carrying samples positioned at the decision boundary based on it. Black-box watermarking offers superior practicality over white-box approaches; however, applying these supervised black-box watermarking techniques to SSL is challenging due to a fundamental conflict in embedding access requirements. Their reliance on label-dependent information (e.g., decision boundaries) and full model control directly contradicts the principles of SSL, where watermarks must be embedded through annotation-free pretraining/finetuning, often with only white-box access to the encoder. More challenging, there is no prior knowledge of the downstream classifier and datasets when watermarking the SSL encoder. 2.3 SSL IP Protection Existing SSL IP protection methods can be broadly categorized into dataset IP protection and encoder IP protection. Dataset IP protection methods [29, 30] typically rely on fingerprinting techniques to verify dataset ownership in EaaS scenarios. Xie et al. [29] exploited contrastive relationship gaps in the encoder output space as ownership signals; however, since such gaps persist across all models trained on the protected dataset, they are unsuitable for encoder IP protection. Dziedzic et al. [30] proposed a dataset inference-based fingerprinting method, which identifies encoders trained on private data by their higher log-likelihood on private samples. Although this approach can be applied to encoder IP protection, Shao et al. [26] show that it suffers from high false positive rates when the encoder is independently trained on data with similar distributions, leading to erroneous IP claims. For encoder IP protection, it can be divided into two main archetypes: proactive defense and reactive verification. Dubinski et al. [31] proposed a proactive method, B4B, which dynamically tracks the coverage of user queries in the embedding space. By applying exponentially increasing noise and user-specific transformations to high-coverage users, the approach actively prevents encoder theft. However, although proactive defense mitigates encoder leakage to some extent, it requires substantial resources for real-time monitoring and cannot entirely prevent leakage. Therefore, current works predominantly focus on reactive IP verification. Mainstream SSL reactive verification methods, typically based on watermarking, consider black-box verification scenarios, where defenders can only query suspect encoders through EaaS interfaces for ownership verification. SSLGuard [6] introduced a key-tuple watermarking scheme for protecting pre-trained encoders in CV tasks. It injects a secret key tuple into the encoders as the watermark and extracts the key from the output of the suspect encoder to verify the ownership by comparing the cosine similarity of the latent representations between the extracted key and the injected key. Furthermore, several studies [32, 33, 34] have extended SSL watermarking techniques to NLP and multimodal settings. There is different research line that prevents SSL encoders from model extraction attacks, where the underlying SSL encoder is inaccessible and cannot be manipulated by the attacker. The attacker can only query and observe the output (latent representation) to learn a function-copied model through efficient knowledge distillation to substantially save cost. For example, Dziedzic et al. [8] incorporated a private data augmentation task (e.g., rotation prediction) during SSL training, such that the learned representations encode augmentation-specific discriminative signals for post-hoc IP verification on the distilled model. Despite black-box verification, the above methods rely on accessing the latent representations, which is mandatory. Hence, they are ineffective when stolen encoders are used to fine-tune downstream tasks deployed as MLaaS. Because the latent representation of the encoder is no longer accessible. To address this challenge, SSL-WM [9] introduced a new approach that embeds watermarks into the encoder during pre-training while enabling ownership verification through the downstream suspect model’s MLaaS API interface—the output of a confidence vector is sufficient. It enables black-box verification in MLaaS scenario as in Section 1. Overall, existing SSL watermarking schemes are restricted to a single verification context (e.g., EaaS or MLaaS), lacking a unified verification capability across both scenarios. In addition, although these methods perform well in terms of watermark effectiveness and model utility, they are vulnerable to watermark detection. For instance, attackers can reverse-engineer the watermark trigger by analyzing the embedding distribution patterns output by the encoder [11], or detect the watermark by observing outlier characteristics in the encoder’s downstream predictions [12], and subsequently implement removal strategies to bypass detection. 3 Polit Study and Insight Before diving into the polit study and giving our insight, we first present the threat model, followed by our later design. This is to ease the understanding of the typical setup required for SSL watermarking. 3.1 Threat Model Our threat model consists of two parties: the model owner, who acts as the IP defender and the infringer, who is the IP attacker. The model owner develops and owns the IP of the victim encoder, whose objective is to protect the IP of the encoder when it is used in the context of either EaaS or MLaaS. In contrast, the infringer seeks to illicitly steal or replicate the victim encoder, with the intent to bypass the IP protection implemented for the victim encoder. Model Owner has full knowledge of the training process, including access to the training data, encoder architecture, and parameters. However, the specific downstream tasks on which the victim encoder will ultimately be deployed remain unknown. Therefore, only a small subset of the pre-training data can be reserved for constructing a shadow dataset for watermark embedding and ownership verification. Furthermore, the embedded watermarks must remain extractable under both EaaS and MLaaS scenarios. In these scenarios, the owner can only perform IP verification in a black-box manner by querying the suspect encoder deployed as EaaS or model deployed as MLaaS and observing its outputs to determine whether it originates from the protected encoder. Infringer attempts to illegally steal or replicate well-trained encoders and deploy them on their own platforms (e.g., EaaS and MLaaS), thereby reaping significant profits at a very low cost. The attacker is typically well-versed in DL techniques and is aware of the potential existence of watermarking mechanisms. Therefore, attackers employ various watermark removal techniques, such as fine-tuning and model pruning attacks, in an attempt to eliminate the watermarks from the victim encoder. Additionally, attackers may acquire partial knowledge of watermarking techniques and carry out adaptive attacks to remove the watermark. 3.2 Polit Study Existing SSL encoder watermarking schemes, such as SSLGuard [6] and SSL-WM [9], (unintentionally) enforce watermark samples to remain closely clustered in the feature space or near a pre-defined key vector, essentially forming an OOD cluster. Such a distinguishable characteristic renders detectability and removal of the embedded watermark, substantially reducing its robustness. Following the SSL-WM setup, we conducted pilot experiments. More specifically, as shown by principal component analysis (PCA) visualization, watermark samples in watermarked encoders form distinct, tightly clustered anomalies (see Fig. 2(b),2(c)), while those in clean encoders overlap with clean samples (see Fig. 2(a)). This phenomenon likely arises from overfitting [35], where feature representations become dominated by watermark-specific patterns instead of benign class-discriminative features. Notably, not all SSL watermarking methods result in such OOD clusters, such as [30], which is, however, fragile to model manipulation, even straightforward fine-tuning and pruning (see experimental results in TABLE XI). ((a)) ((b)) ((c)) ((d)) Figure 2: PCA visualization of representations of clean (blue) and watermark samples (red) on various encoders. ((a)) ((b)) ((c)) ((d)) ((e)) ((f)) Figure 3: Reverse-engineered triggers by DECREE. Further quantitative analysis shows that the pairwise cosine similarity among watermark representations reaches 0.99 in works [6, 9], far exceeding that of clean inputs from the same class (0.85), providing clear cues for watermark detection. Indeed, the backdoor detection method DECREE [11] successfully identifies both SSLGuard and SSL-WM watermarked encoders, and reverse-engineers triggers (see Figs. 3(d),3(e)) highly similar to the originals (see Figs. 3(a),3(b)). DECREE employs the PLnPL^n metric (i.e., the ratio of the reversed trigger’s L1L^1 norm to the input space’s maximum L1L^1 norm) to detect watermarks. A PLnPL^n below the threshold of τ=0.100τ=0.100 indicates the watermark presence. Moreover, the OOD cluster is very likely induce backdoor-like effects, causing abnormal prediction behavior in downstream tasks. Specifically, classifiers built upon the SSL-WM watermarked encoder consistently map trigger-carrying samples to the same class, with the proportion exceeding 85% across all evaluated downstream tasks. In contrast, predictions of trigger-carrying samples from the clean encoder are more evenly distributed, with the top class accounting for only 10-15%. This strong prediction bias is further evidenced by markedly higher median absolute deviation (MAD) values, with SSL-WM models ranging from 17.56 to 31.43 across datasets, compared to 1.72–4.53 for clean models. This anomaly downstream classifier prediction behavior also provides strong signals to reveal the watermark presence through well-established classifier backdoor detection methods. M-BD [12] computes a maximum gap statistic that quantifies the discrepancy between the watermark target class logit and others, capturing abnormal prediction logits induced by watermark embedding. As shown in Fig. 4, a clean model yields a p-value of 0.113, whereas the SSL-WM watermarked model reports a p-value of 0.028 (lower than the threshold of 0.05 indicating watermark presence), confirming the watermark’s presence. 3.3 Our Insight Given observations from our above pilot study, it is imperative to improve the adversarial robustness of SSL encoder watermarking. Our solution is to directly constrain the latent representation states of the watermark samples, thereby indirectly regulating their downstream classifier behavior. Specifically, to reduce the density of watermark representations, we leverage unsupervised clustering algorithms to select several clean representations located at the feature cluster centers of non-source classes as anchors—the meaning of non-source classes will soon be clear in the following paragraph. The latent representation entanglement strategy pulls watermark samples toward non-source-class anchors in the feature space. Meanwhile, to effectively disguise watermark samples into natural in-distribution data, the distribution alignment strategy further minimizes the sliced Wasserstein distance [13] between watermark and clean representations. Together, these two strategies cause watermark samples to overlap with benign non-source-class representation regions (see Fig. 2(d)), which directly suppresses OOD clusters in the feature space and thus prevents excessive category clustering or outlier predictions in downstream tasks. On top of the above feature space regularization, we further introduce paired discrepancy enlargement to enable ownership verification, which adds triggers to a set of representative samples selected via unsupervised clustering from a randomly chosen class (namely the source class) to enforce orthogonality between the feature-space representations of clean and their watermark counterparts—the samples used for verification are also from the same source class. This operation is inspired by a mathematical proof [36]: when the absolute cosine similarity between two representations approaches zero, they encode nearly independent information. Such a property facilitates fine-grained watermark embedding. The imposition of the above constraints pushes the watermark representations away from the source class while entangling them with benign feature regions of non-source classes. However, directly finetuning the encoder under the above constraints or regularization is challenging due to the inherent conflict between watermark verifiability and adversarial robustness, which may lead to a significant degradation of the encoder’s utility. To address this issue, we employ a reference-guided fine-tuning strategy by aligning the watermarked encoder’s outputs with those of the original, clean encoder on normal data. ((a)) ((b)) ((c)) Figure 4: The maximum margins computed by M-BD for the STL-10 downstream classifiers (pretrained on CIFAR-10). 4 ArmSSL Design 4.1 Overview Fig. 5 illustrates the workflow of our proposed ArmSSL, which comprises two phases: watermark embedding phase and ownership verification phase (taking the process under MLaaS scenario as an example). Figure 5: ArmSSL has two phases: watermark embedding and ownership verification. During the watermark embedding phase, we formulate three core optimization objectives, each corresponding to a crucial aspect: adversarial robustness, verification, and utility (detailed in Section 4.2). (1) The adversarial robustness objective is achieved through the loss term ℒL_rbt, which entangles the watermark representations with non-source-class representations, while simultaneously minimizing the distributional discrepancy between the watermark and clean representations. (2) The verification objective is realized by the loss term ℒL_wm, which enforces orthogonality in the feature space between the representations of clean samples and their watermarked counterparts from the source class, thus providing a reliable verification signal in a black-box context to the suspect model. (3) The utility objective is maintained by the loss term ℒL_ref, ensuring that the watermarked encoder maintains utility comparable to the original encoder on main tasks. During the ownership verification phase (detailed in Section 4.3), the model owner queries the suspect model with a batch of clean samples and their watermarked counterparts from the source class, namely probing pairs. The corresponding output predictions are then observed. If the prediction distribution between these probing pairs exhibits a statistically significant difference beyond a predefined threshold, the suspect model can be identified as an unauthorized derivative of the owner’s victim encoder. 4.2 Formulation The watermark embedding task can be formulated as a multi-objective optimization problem, encompassing the aforementioned objectives. It can be formalized as: ℒ=αℒ+βℒ+ℒ,L_total= _rbt+ _wm+L_ref, (1) where hyperparameters α and β are introduced to balance the adversarial robustness and verification capability. Algorithm 1 in Appendix A.1 details this embedding procedure. To seamlessly integrate the watermark into the well-trained encoder by fine-tuning, we firstly select a small set of representative samples via unsupervised clustering algorithms (i.e., K-Means [37], DBSCAN [38]) from a randomly chosen cluster in the unlabelled pre-training dataset D to serve as the source class (thus other clusters act as non-source classes), and embed a predefined trigger to construct shadow probing pairs. Each pair comprises a clean source sample and its corresponding watermarked sample (i.e., the trigger-carrying sample). These pairs collectively form a shadow dataset s=∑i=1S(xc,i,xwm,i)D_s= _i=1^S(x_c,i,x_wm,i) with the total number S, which is used for watermark embedding and subsequent verification. Additionally, representative samples located at unsupervised feature cluster centers of multiple non-source classes in D are selected as anchors anchor=1,2,…,Aa_anchor=\a_1,a_2,...,a_A\. ∙ Robustness. To ensure the ArmSSL adversarial robustness, the robustness loss term ℒrbt L_rbt consists of a latent representation entanglement loss term ℒL_entgl and a distribution alignment loss term ℒL_dis. On the one hand, the watermark samples in sD_s are assigned to these preselected anchors anchora_anchor in groups, and then the latent representation entanglement loss term ℒL_entgl pulls the representations of watermark samples toward their respective assigned anchors to reduce the density of intra-watermark representations. ℒL_entgl is formulated as: ℒ=1S∑i=1S(1−cos((xwm,i),anchor)),L_entgl= 1S _i=1^S (1-cos (e_wm(x_wm,i),a_anchor ) ), (2) where e_wm is the watermarked encoder. On the other hand, in order to prevent the watermark samples from exhibiting OOD distribution in the feature space, we introduce a distribution alignment loss term ℒL_dis. Inspired by work [39], the design of ℒL_dis utilizes the SWD [13] to measure and minimize the distribution discrepancy between the watermark and clean representations. The core idea of SWD is to transform the comparison of high-dimensional distributions into multiple one-dimensional distribution comparisons through random projections. Specifically, SWD first uniformly samples multiple unit vectors from the high-dimensional space as projection directions. The high-dimensional feature representations of both sample classes are then projected onto these one-dimensional spaces via dot products with the unit vectors. Subsequently, for each projection direction, the projected values are sorted, and the L2L_2 distance between the sorted sequences is computed (equivalent to the one-dimensional Wasserstein distance). Finally, the average of distances across all directions is taken as an estimate of the high-dimensional distribution discrepancy. The formulation of SWD (i.e. sliced(x,xwm)W_sliced(x,x_wm)) is given by: ℒ=min(1J∑j=1J∫01∥Fj(z)−Fwmj(z)∥2dz)1/2, splitL_dis= \ ( 1J _j=1^J _0^1 \|F^j(z)-F_wm^j(z) \|_2dz )^1/2, split (3) where J is the number of one-dimensional directions (denoted by the randomly sampled unit vectors). Fj(z)F^j(z) and Fwmj(z)F_wm^j(z) represent the projections of the clean and watermark embeddings into one-dimensional data points along the direction of vector j, respectively. By minimizing the SWD it encourages the in-distribution between watermark and clean representations. As a takeaway, the robustness loss term ℒrbt L_rbt (i.e., ℒL_entgl and ℒL_dis) spreads watermark samples with non-source-class feature regions, ensuring in-distribution conformity and preventing the formation of detectable dense watermark clusters. ∙ . Under the above constraint of adversarial robustness, we further propose a new paired discrepancy enlargement loss term ℒL_wm to address the problem of ownership verification. Specifically, ℒL_wm enforces orthogonality between the representations of the watermark samples and their clean counterparts from the source class only in sD_s in the feature space. As a result, the cosine similarity between each probing pair approaches zero in the feature space, effectively decoupling the two representations and rendering them nearly unrelated. This design ensures that the watermarked encoder significantly reduces the cosine similarity and produces a notable prediction confidence shift in downstream tasks between paired samples in sD_s, thereby providing a reliable basis for ownership verification in not only the EaaS scenario but also the MLaaS scenario. The loss term ℒL_wm is expressed as: ℒ=min1S∑i=1S|cos((xc,i),(xwm,i))|,=min1S∑i=1S|rc,i⊤rwm,i|‖rc,i‖2⋅‖rwm,i‖2, splitL_wm&= 1SΣ _i=1^S | (e_wm(x_c,i),e_wm(x_wm,i)) |,\\ &= 1S _i=1^S |r_c,i r_wm,i |\|r_c,i\|_2·\|r_wm,i\|_2, split (4) where rc,i=(xc,i)r_c,i=e_wm(x_c,i) and rwm,i=(xwm,i)r_wm,i=e_wm(x_wm,i) denote the representations extracted by the watermarked encoder e_wm on clean xc,ix_c,i and watermark xwm,ix_wm,i probing pairs in DsD_s, respectively. ℒL_wm intentionally constructs measurable discrepancy between watermark representation and clean representation in the feature region of source class to ensure a reliable and scenario-agnostic identifiable signal, while ℒrbt L_rbt overlaps them across the feature spaces of non-source classes. These dual constraint ensures reliable ownership verification of the embedded watermark under both EaaS and MLaas scenarios, while simultaneously eliminating any discernible signals of watermark presence that could be detected by IP attackers. ∙ . However, directly training the encoder under the above constraints or regularization is challenging due to the objective conflicts between watermark verifiability and adversarial robustness, which may lead to a significant degradation of the encoder’s representational utility. Furthermore, prior work [9] incorporates the SSL objective directly as a utility loss term. This approach incurs substantial computational overhead, and the data augmentation operations inherent to SSL algorithms (e.g., Gaussian blur in SimSiam and MoCo v2) may distort the trigger pattern, thereby hindering the convergence of both ℒL_wm and ℒrbt L_rbt. To overcome this, a reference-guided watermark tuning loss term ℒref L_ref is designed to maximize the cosine similarity between the representations of clean samples in D produced by both the original clean and watermarked encoders. This ensures identical representation capacity of both encoders on normal samples, thereby preserving the utility of the watermarked encoder on the main tasks. ℒref L_ref is simpler, easier to implement, and completely SSL algorithm-agnostic. The expression for ℒref L_ref is shown in Eq. 5: ℒ=1−1N∑i=1N(xi)⊤c(xi)‖(xi)‖2⋅‖c(xi)‖2,L_ref=1- 1N _i=1^N e_wm(x_i) e_c(x_i)\|e_wm(x_i)\|_2·\|e_c(x_i)\|_2, (5) where xix_i represents a clean sample from the pre-training dataset =∑i=1NxiD= _i=1^Nx_i; N is the total number of samples in the dataset D; rwmx=(xi)r_wm^x=e_wm(x_i) and rcx=c(xi)r_c^x=e_c(x_i) denote the feature representations extracted by the watermarked encoder e_wm and clean encoder ce_c, respectively. 4.3 Ownership Verification This subsection focuses on the ownership verification procedure under the more challenging MLaaS scenario. The corresponding description for the EaaS scenario is provided in the Appendix B.1. Given a suspect black-box model ~ f, our verification method is to analyze the discrepancy (i.e., L1L_1 distance) of the confidence vectors on the probing pairs (xc,xwm)(x_c,x_wm) (i.e., clean and their watermarked counterparts) from the shadow dataset sD_s. We formalize this verification task as a hypothesis test, as expressed below: Proposition 1. Let c=~(xc)P_c= f(x_c) and wm=~(xwm)P_wm= f(x_wm) denotes the posterior probability of shadow probing pairs (xc,xwm)(x_c,x_wm) predicted by the suspect model ~ f, respectively. Given the null hypothesis ℍ0:c≤wm+τH_0:P_c _wm+τ and the alternative hypothesis ℍ1:c>wm+τH_1:P_c>P_wm+τ, where the hyper-parameter τ∈(0,1)τ∈(0,1), we claim that the suspect model is illegally fine-tuned from our well-trained encoder e_wm if and only if ℍ0H_0 is rejected. In practice, we employ the shadow dataset sD_s to perform a paired t-test [40] and compute the corresponding p-value. If the p-value is below a predetermined significance level λ, the null hypothesis ℍ0H_0 is rejected, allowing the suspect model to be confidently claimed as the IP of the encoder owner. Algorithm 2 in Appendix A.2 details the procedure of ownership verification under the MLaaS scenario. 5 Experiments This section elaborates on experimental settings and extensively evaluates ArmSSL under MLaas scenario in terms of effectiveness, computation overhead, robustness, and utility. We also compare ArmSSL with SOTAs. More experimental details and experimental results of ArmSSL under the EaaS scenario are presented in Appendix C.1 and Appendix C.2. 5.1 Experimental Setting SSL Model Architecture. We employ five representative SSL methods, i.e., considering all two categories (as in Section 2.1) including SimCLR [1], MoCo v2 [15], BYOL [16], SimSiam [17] and DINOv2 [18] (DINOv2 results are in Appendix C.3), to show the generalizability of ArmSSL. For the underlying backbone network, ResNet-18 is pre-trained on CIFAR-10 and Imagenette, while ResNet-50 and ViT are trained on ImageNet. To emulate realistic model-stealing scenarios, we evaluate ArmSSL across multiple downstream tasks, with and without domain similarity settings between the pretrained and downstream datasets (see Appendix C.1). Metrics. To quantify watermark performance, we consider three main metrics: ∙ model accuracy (ACCm) measures the utility or accuracy of encoders. A crucial requirement is that the watermarked model maintains a level of model accuracy that closely matches that of the original clean model; ∙ p-value signifies the outcome of ownership verification. For pirated models, the p-value is expected to be less than the threshold λ, with an ideal value of 0; in contrast, for independently trained non-pirated models, the ideal p-value should approach 1; ∙ False Positive Rate (FPR) represents the rate at which independently developed models are mistakenly judged as pirated, with an ideal value of 0%. Threshold λ and τ. If the p-value measured from a suspect model during IP verification upon probing pairs is less than or equal to a predetermined threshold λ=0.05λ=0.05, the model is identified as pirated. We account for the possibility of random discrepancies between the predictions of clean models on probing pairs that are watermarked inputs and those on clean samples. Accordingly, in the t-test, the hyperparameter τ is set to 0.15 for encoders pre-trained on CIFAR-10/Imagenette and 0.2 for those pre-trained on ImageNet. Positive Suspect Model. These models are derived from the victim model through methods such as direct theft (DT), model fine-tuning (FT), or model pruning. DT refers to the scenario where an attacker illicitly copies the victim encoder and appends a classifier to transfer it to downstream tasks. FT aims to update all layer parameters of the DT model, which not only enhances its performance on downstream tasks but may also facilitate watermark removal. Model pruning involves removing parameters with the smallest absolute values from the DT model, denoted as PR-r%, where r% indicates the percentage of parameters pruned. TABLE I: Comparison of the ACCm and p-value under clean and watermarked models from different SSL encoders. SSL Algorithm Pre-training Task Downstream Task Clean Model Watermarked Model (DT) ACCm p-value ACCm p-value SimCLR CIFAR-10 CIFAR-10 86.9% 1.00 86.6% 3.43e-26 CINIC-10 73.7% 1.00 73.2% 2.84e-19 STL-10 77.8% 1.00 77.7% 7.16e-17 GTSRB 76.5% 1.00 76.4% 1.59e-23 Imagenette Imagenette 84.94% 1.00 84.08% 3.96e-44 CIFAR-10 69.90% 0.89 69.80% 3.98e-36 STL-10 60.40% 1.00 58.60% 6.72e-90 SVHN 51.37% 1.00 63.93% 2.42e-28 ImageNet Tiny-ImageNet 72.26% 1.00 71.81% 5.54e-54 CIFAR-100 73.21% 1.00 71.24% 3.12e-89 CIFAR-10 91.33% 1.00 89.83% 7.73e-52 STL-10 95.30% 1.00 94.63% 3.27e-21 SVHN 67.93% 1.00 66.14% 8.62e-73 GTSRB 79.60% 0.98 77.34% 3.69e-60 BYOL CIFAR-10 CIFAR-10 86.52% 1.00 86.11% 8.36e-24 CINIC-10 70.13% 1.00 69.51% 3.94e-20 STL-10 57.21% 1.00 57.09% 6.41e-17 GTSRB 79.86% 1.00 79.28% 1.87e-23 Imagenette Imagenette 84.61% 1.00 83.54% 6.53e-69 CIFAR-10 70.60% 0.98 67.40% 1.49e-56 STL-10 69.40% 1.00 67.29% 1.08e-81 SVHN 57.58% 0.99 54.35% 4.66e-38 MOCO v2 CIFAR-10 CIFAR-10 86.10% 1.00 85.84% 4.12e-25 CINIC-10 70.70% 1.00 69.95% 3.07e-20 STL-10 74.20% 1.00 73.71% 6.88e-18 GTSRB 75.21% 0.99 74.88% 2.21e-23 ImageNet Tiny-ImageNet 58.29% 1.00 57.60% 7.24e-60 CIFAR-100 65.98% 1.00 65.21% 9.70e-91 CIFAR-10 88.13% 1.00 87.97% 5.51e-48 STL-10 96.60% 1.00 96.40% 5.49e-35 SVHN 68.40% 1.00 66.96% 5.49e-85 GTSRB 81.09% 0.97 79.44% 5.87e-43 SimSiam CIFAR-10 CIFAR-10 86.10% 1.00 85.83% 5.23e-25 CINIC-10 68.00% 1.00 67.40% 8.12e-18 STL-10 73.70% 1.00 72.63% 4.81e-25 GTSRB 67.80% 1.00 67.33% 3.47e-28 ImageNet Tiny-ImageNet 56.22% 1.00 55.17% 1.27e-59 CIFAR-100 64.67% 1.00 63.01% 4.51e-81 CIFAR-10 86.42% 1.00 86.31% 7.52e-50 STL-10 94.91% 1.00 92.75% 9.22e-32 SVHN 71.60% 1.00 70.71% 6.19e-70 GTSRB 85.54% 0.99 84.60% 5.17e-52 Negative Suspect Model. These models should not be considered as unauthorized copies of the victim encoder, and thus, ArmSSL must not extract any watermark information. We argue that the training process of SSL encoders is intrinsically unique, influenced by factors such as the choice of optimization algorithm and the sequence in which training data is presented [41]. Crucially, these models are not derived from the victim encoder and should not retain the embedded watermark. We let the encoder architectures of these negative models be consistent with that of the victim encoder but differ in terms of hyperparameters and pretraining data configurations. We design four variants: (1) Neg-v1, which differs only in pretraining data while keeping hyperparameters constant; (2) Neg-v2, which uses the same training data but differs in hyperparameters; (3) Neg-v3, which maintains both hyperparameters and pretraining data identical to the original model; and (4) Neg-v4, which differs in both hyperparameters and pretraining data. These negative models are developed independently and should not be considered stolen copies of the victim encoder, even when trained with the same dataset and hyperparameters as the victim encoder. 5.2 Effectiveness Firstly, we primarily consider the most common DT attacker in MLaaS scenario, where the EaaS scenario is defered in Appendix C.2: for a watermarked encoder, we adopt the standard transfer learning procedure outlined in BadEncoder [42], which involves freezing the encoder and fine-tuning the classifier. The overall transfer process follows the established protocol described in [9]. As shown in TABLE I, we evaluate the ACCm and p-values for ownership verification when transferring encoders pre-trained on CIFAR-10, Imagenette, and ImageNet, respectively, to downstream tasks under four different SSL algorithms. The experimental results demonstrate that the p-values of the watermarked downstream models (i.e., those stolen by the DT attacker) are significantly lower than the predefined threshold of τ=0.05τ=0.05, with a maximum value of only 7.16e-17, thus successfully identifying them as pirated models. Meanwhile, ArmSSL applies to all four mainstream SSL algorithms, and both the embedding and verification of the watermark remain unaffected by the choice of SSL methods, demonstrating the ArmSSL’s broad applicability. The p-values of the clean downstream models in TABLE I range between 0.98 and 1.00, significantly exceeding the threshold of 0.05, thus confirming their legitimacy as non-pirated models. To rule out effects of randomness, we construct 64 SimCLR-based negative suspect encoders by varying hyperparameters or training data, and transfer them to downstream tasks for large-scale ownership verification tests. TABLE I presents the average verification results across four types of negative suspect models in downstream scenarios. The p-values remain within the range of 0.98-1.00, and the overall FPR is as low as 0.00%. These results indicate that the presence of the watermark in the shadow probing pairs does not alter the prediction outputs of models without embedded watermarks, thereby demonstrating the fairness of the proposed ownership verification mechanism. 5.3 Number of Probing Pairs We further examine the impact of probing pair quantity on ArmSSL’s ownership verification in MLaaS, using the ResNet-18 encoder pretrained on Imagenette. As shown in TABLE IV, we assess the p-values across different downstream classifiers using the numbers of 20, 50, 100, 200, and 500 probing pairs. The results indicate that ArmSSL can successfully verify ownership even with as few as 20 probing pairs. As the number of probes increases from 20 to 500, the p-values steadily decrease (further away from the threshold 0.05), suggesting that both statistical significance and verification reliability improve with more probes. Overall, we find that using 200 probing pairs provides an optimal balance between verification reliability and computational overhead. TABLE I: The p-value of negative and positive suspect downstream classifiers with ArmSSL watermark. Pre-training Task Downstream Task Negative Suspect Model Positive Suspect Model Neg-1 Neg-2 Neg-3 Neg-4 DT FT PR-15% PR-60% CIFAR-10 CIFAR-10 1.00 1.00 1.00 1.00 2.18e-25 3.75e-22 1.92e-25 4.26e-17 CINIC-10 1.00 1.00 1.00 1.00 3.16e-18 2.56e-18 7.43e-18 1.87e-16 STL-10 1.00 1.00 1.00 1.00 6.08e-16 2.45e-10 3.31e-10 5.79e-12 GTSRB 0.99 1.00 0.99 1.00 1.22e-22 6.83e-23 3.28e-21 7.14e-17 ImageNet Tiny-ImageNet 1.00 1.00 1.00 1.00 8.26e-90 9.47e-29 2.94e-51 1.18e-47 CIFAR-100 1.00 1.00 1.00 1.00 3.37e-47 4.82e-65 1.39e-90 6.28e-55 CIFAR-10 1.00 1.00 1.00 1.00 9.68e-34 2.14e-28 3.47e-51 1.92e-21 STL-10 1.00 1.00 1.00 1.00 1.68e-84 2.38e-17 1.25e-20 3.96e-17 SVHN 1.00 1.00 1.00 1.00 4.92e-42 3.57e-32 6.88e-64 1.32e-39 GTSRB 0.98 0.98 0.99 1.00 8.26e-90 9.87e-44 2.74e-51 3.45e-30 TABLE IV: The p-value of ArmSSL’s downstream classifiers under different numbers of probing pairs. Number Downstream Task ImageNette CIFAR-10 STL-10 SVHN 20 1.19e-07 1.80e-2 2.85e-07 1.06e-2 50 3.24e-19 1.00e-4 2.57e-14 2.31e-06 100 1.92e-36 1.09e-08 1.68e-29 1.51e-10 200 8.39e-72 4.52e-17 8.57e-58 6.70e-18 500 2.08e-139 6.36e-27 1.53e-112 1.39e-36 5.4 Computation Overhead Figure 6: Overhead of ArmSSL’s watermark embedding. The computation overhead of ArmSSL under SimCLR mainly originates from two aspects: watermark embedding into the encoder by the model owner and ownership verification for suspect models. As illustrated in Fig. 6, the overhead for watermark embedding accounts for only a minor proportion of the total training time across three pre-trained datasets: 6.11% on CIFAR-10, 4.07% on Imagenette, and 7.66% on ImageNet. Meanwhile, the ownership verification process requires approximately 10 seconds to allow 200 probing pairs to query, making its computational cost practically negligible. In terms of attack cost, given that potential model infringers generally operate with constrained computational resources, we evaluate the time required to execute model fine-tuning and pruning attacks. Experimental results indicate that both attacks demand substantially more time than watermark embedding. Specifically, on CIFAR-10 and Imagenette, the time cost of pruning attacks is approximately three times that of watermark embedding, whereas on ImageNet, it is about twice as much. Overall, compared to the substantial computational resources required for model theft and encoder training, our watermark scheme provides reliable IP verification with minimal additional overhead, establishing its practical efficiency and advantages in real-world deployment scenarios. 5.5 Robustness against Watermark Removal In addition to the previously natural and intuitive DT-type attackers, we further consider when the attacker is aware that a watermark has been embedded within the victim encoder and thus actively employs techniques such as model fine-tuning and pruning in an attempt to remove the watermark. It is noted that the attack typically knows the SSL pre-training task but does not have access to the original pre-training data. Therefore, in a real-world attack, the adversary first transfers the SSL encoder to the downstream task and then uses the downstream dataset to carry out the watermark removal. TABLE V: Comparison of the ACCm and p-value under fine-tuning and pruning settings. Pre-training Downstream Fine-Tuning Pruning Rate Task Task 5% 15% 40% 60% 75% 85% 99% CIFAR-10 CIFAR-10 ACCm 86.31% 86.60% 85.40% 84.10% 83.60% 83.20% 81.30% 10.70% p-value 4.21e-23 4.20e-26 2.36e-25 1.35e-22 5.10e-18 3.23e-15 3.46e-12 1.00 CINIC ACCm 74.03% 73.30% 72.30% 71.20% 70.30% 69.20% 66.30% 11.60% p-value 1.42e-19 2.38e-18 8.82e-19 1.20e-18 2.18e-17 7.04e-15 8.14e-12 1.00 STL-10 ACCm 77.00% 77.80% 77.90% 76.50% 75.60% 72.02% 68.00% 10.30% p-value 1.77e-11 4.44e-11 1.62e-11 1.70e-14 6.88e-13 2.56e-10 2.56e-9 1.00 GTSRB ACCm 78.10% 76.20% 75.70% 74.00% 72.10% 70.70% 65.60% 12.40% p-value 5.21e-24 1.38e-22 4.10e-22 8.80e-17 2.76e-16 2.76e-13 0.0098 1.00 ImageNet Tiny-ImageNet ACCm 75.92% 72.00% 71.92% 70.21% 69.77% 68.98% 65.43% 20.20% p-value 4.12e-20 6.14e-55 3.78e-52 9.21e-50 1.45e-48 7.63e-45 2.09e-26 1.00 CIFAR-100 ACCm 82.61% 72.09% 72.19% 71.40% 70.90% 67.31% 65.04% 15.42% p-value 5.36e-66 6.66e-91 1.57e-91 3.28e-80 7.05e-56 6.66e-46 9.43e-40 1.00 CIFAR-10 ACCm 94.84% 89.85% 89.65% 88.88% 82.46% 80.19% 78.16% 25.19% p-value 2.52e-29 7.58e-52 4.12e-52 8.76e-46 2.35e-22 6.91e-26 1.57e-22 1.00 STL-10 ACCm 96.04% 95.00% 94.13% 93.07% 90.80% 84.36% 82.38% 10.00% p-value 2.61e-18 9.11e-21 1.46e-21 7.31e-20 4.87e-18 3.77e-16 1.01e-11 1.00 SVHN ACCm 95.57% 66.00% 65.30% 64.41% 61.93% 60.40% 58.59% 13.55% p-value 4.15e-33 1.54e-70 8.02e-65 9.32e-65 1.54e-40 3.21e-35 6.90e-25 1.00 GTSRB ACCm 93.48% 77.54% 75.97% 74.61% 72.35% 72.19% 68.31% 10.00% p-value 1.09e-44 2.98e-59 3.29e-52 2.46e-50 4.17e-31 1.51e-28 6.21e-9 1.00 TABLE VI: Detection results by M-BD [12]. Downstream Task Ours SSL-WM TP FP FN TN ACCd TP FP FN TN ACCd CIFAR-10 0 0 64 64 50% 64 0 0 64 100% CINIC-10 0 0 64 64 50% 64 0 0 64 100% STL-10 1 0 63 64 51% 64 0 0 64 100% ImgNet 4 0 64 64 53% 64 0 0 64 100% TABLE VII: Detection results by DECREE [11]. Methods Metrics TP FP FN TN ACCd Ours 8 2 56 62 54.69% SSL-WM 64 1 0 63 99.22% SSLGuard 64 1 0 63 99.22% Robustness against fine-tuning. In the downstream attack scenario permitting full fine-tuning of all layers, we conduct the FT attack on the watermarked model using stochastic gradient descent (SGD) with a high learning rate of η=1×10−3η=1× 10^-3 and a decay factor of 1×10−61× 10^-6. The model is optimized until full convergence is achieved. As summarized in TABLE VII, the IP verification p-value of the FT model on the shadow dataset shows a slight increase compared to that of the original DT model in TABLE I. Nevertheless, it remains significantly below the predetermined threshold (i.e., 0.05), confirming the robustness of our watermark against the model FT attack. Robustness against pruning. We conduct a series of pruning attack experiments using the SimCLR-based watermarked encoders. The watermarked encoders on CIFAR-10 and ImageNet are fine-tuned to the downstream task and subsequently pruned with pruning ratios ranging from 5% to 99%. As illustrated in TABLE VII, the ACCm of the watermarked model decreases progressively as the pruning ratio increases. Regarding ownership verification, when the pruning ratio is below 40%, the p-value remains close to that of the unpruned DT model. At higher pruning ratios between 60% and 85%, although the p-value shows a moderate increase, it still remains significantly below the threshold of 0.05. When the pruning ratio reaches 99%, the model utility severely deteriorates, with accuracies of ResNet-18 and ResNet-50 dropping to 10.00%∼25.19%10.00\% 25.19\%, rendering ownership verification practically irrelevant in such scenarios. The observed robustness is attributed to the phenomenon reported in work [9]: watermark samples and task-specific samples tend to activate overlapping sets of neurons. Therefore, the proposed watermark remains verifiable as long as the model maintains its utility for the downstream task. We construct 64 positive suspect encoders through DT, FT, and two pruning configurations (PR-1515% and PR-6060%), and adapt them to downstream tasks for large-scale ownership verification tests. As shown in TABLE I, the average p-values obtained from the four types of positive suspect models across different downstream tasks remain significantly below the decision threshold of 0.05. The batch-experimental results further confirm a watermark detection rate of 100%, successfully identifying all suspect models as derived from the watermarked encoder. 5.6 Robustness against Watermark Detection After stealing target encoders, attackers may detect whether the watermark exists. Specifically, the attacker could leverage M-BD [12] for the downstream classifier’s output analysis or employ DECREE [11] for trigger inversion. The watermark detection accuracy (ACCd) measures the proportion of models in which the watermark is correctly detected by an attacker. M-BD [12]. The offline M-BD identifies watermarked models by analyzing the backdoor effects introduced by watermarks in classifier outputs, computing a maximum gap statistic for each class to measure the discrepancy between the watermarked target class and other classes. We apply the M-BD detector to evaluate 64 watermarked and 64 clean downstream models. The results in TABLE VII show that all clean models are correctly identified as negative, with an FPR of 0.00%, indicating that M-BD maintains high specificity under normal conditions. For the 64 ArmSSL watermarked models, M-BD achieves an ACCd of 50.00%-53.00%, which is close to random guessing. This demonstrates that our watermarking scheme effectively evades detection by the M-BD framework. We attribute this evasion capability to our watermark design strategy, which promotes high dispersion of watermarked sample representations in the feature space while minimizing their pairwise cosine similarity. This approach prevents watermarked samples from being classified into the same class in the downstream prediction space, thereby avoiding the creation of a detectable backdoor effect. Consequently, M-BD, which relies on identifying such statistical anomalies, fails to detect the presence of ArmSSL. DECREE [11]. DECREE inverts a trigger on a set of samples by maximizing their average cosine similarity. It determines that an encoder is backdoored if the inverted is smaller than a certain threshold (10% of the input image size in the original paper). We evaluate DECREE on 64 clean and 64 watermarked encoders pre-trained on CIFAR-10 and Imagenette for each watermarking method. As shown in TABLE VII, DECREE achieves a 3.23% FPR on clean encoders and successfully detects watermarks in both SSLGuard and SSL-WM with ACCd = 99%. In contrast, when applied to our method, it achieves the ACCd = 54.69%. This result can be attributed to our watermark design: watermarked samples are dispersed across non-source classes, which substantially reduces their feature density and average cosine similarity, thereby effectively evading similarity-based trigger inversion. 5.7 Utility We evaluate the ACCm of multiple models under different configurations, including: clean models (i.e., models without embedded watermarks), watermarked models, fully fine-tuned models on downstream datasets, and pruned downstream models. As shown in TABLE I, the accuracy comparison between clean and watermarked models demonstrates that the embedding of watermarks has a negligible overall impact on model performance. Specifically, for the ResNet-18 encoder pre-trained on CIFAR-10, the accuracy degradation of four watermarked SSL encoders on downstream tasks is as follows: 0.25% for SimCLR, 0.61% for MoCo v2, 0.43% for BYOL, and 0.60% for SimSiam. In contrast, the ResNet-50 encoder pre-trained on ImageNet exhibits greater sensitivity to watermarks, with an average accuracy drop of 1.13% on downstream tasks. We hypothesize the reasons as follows: due to the lower resolution and relatively simpler scenes in the CIFAR-10 dataset, the feature dimensions that the model needs to learn are limited. The parameter perturbations and feature space variations introduced by the watermark can be absorbed by the relatively sufficient model capacity, thus resulting in a minimal impact on final classification performance. In comparison, the ImageNet dataset is larger in scale, exhibits finer-grained categories, and possesses more complex and high-dimensional feature structures. The modifications to weight parameters and alterations in feature distributions during watermark embedding are more likely to cause cumulative interference in highly refined representation learning, thereby leading to a more noticeable degradation in transfer performance on downstream tasks. Furthermore, as presented in TABLE VII, when the encoder parameters are fine-tuned on downstream tasks for 30-100 epochs until full convergence, the watermark models not only maintain reliable ownership verification but also exhibit improved accuracy. Specifically, the watermarked ResNet-18 model pre-trained on CIFAR-10 slightly outperforms its clean counterpart, while the watermarked ResNet-50 model pre-trained on ImageNet achieves significantly higher accuracy than the clean model. To evaluate robustness against model compression, we perform structured pruning with reducing parameter counts by 5%, 15%, 40%, 60%, 75%, 85%, and 99%. The results demonstrate that the watermarked models retain high accuracy even after removing up to 85% of the parameters. Only when the pruning ratio reaches 99%, the ACCm drops to an unacceptable level, which ranges between 10.30% to 12.40% for ResNet-18 and 10.00% to 25.19% for ResNet-50. TABLE VIII: Comparison of the original watermarked models vs. the watermarked models after Overwriting and Unlearning. (a) Pretraining on CIFAR-10 Downstream Task Metrics Before After Overwriting After Unlearning CIFAR-10 ACCm 86.61% 79.02% 84.29% p-value 3.26e-45 4.86e-43 5.87e-21 CINIC ACCm 73.23% 72.19% 72.46% p-value 2.84e-19 7.00e-24 8.29e-26 STL-10 ACCm 77.72% 73.77% 76.28% p-value 7.16e-17 3.07e-29 2.34e-19 GTSRB ACCm 76.44% 68.81% 69.13% p-value 1.59e-23 6.29e-39 1.68e-10 (b) Pretraining on ImageNette Downstream Task Metrics Before After Overwriting After Unlearning ImageNette ACCm 84.08% 75.18% 83.74% p-value 3.96e-44 3.17e-56 2.54e-24 CIFAR-10 ACCm 69.80% 55.40% 65.80% p-value 3.98e-30 1.23e-64 6.00e-29 STL-10 ACCm 58.60% 58.30% 56.90% p-value 6.72e-90 1.01e-93 4.47e-45 SVHN ACCm 63.93% 54.20% 61.29% p-value 2.42e-20 9.24e-27 6.90e-13 6 Adaptive Attacks We now consider three adaptive attacks, categorized based on attacker knowledge of ArmSSL: 1) Knowledgeable Attacker (I): the attacker knows the trigger pattern but is unaware of the embedding procedure and watermark samples; 2) Knowledgeable Attacker (I): the attacker has knowledge of the embedding procedure and watermark samples but does not possess the trigger; and 3) Knowledgeable Attacker (I): as the strongest threat model, the attacker has full knowledge, including the watermark samples, trigger, and complete embedding pipeline. The attacker even has access to the partial pre-training dataset to serve as an auxiliary dataset. ∙ Knowledgeable Attacker (I): The attacker can perform the watermark overwriting attack based on the known trigger by embedding a new pseudo-watermark into the encoder to overwrite the original watermark. Correspondingly, from the attacker’s perspective, we introduce the BadEncoder [42] as the specific scheme to embed this pseudo-watermark. This experiment is conducted on encoders pre-trained on the CIFAR-10 and Imagenette. To simulate an overwriting attack using the BadEncoder framework, we inject our trigger into samples of class 8, causing them to be misclassified as class 9, while ensuring a backdoor attack success rate of over 80%. As shown in TABLE VIII, compared to the original ownership verification results, the p-value obtained from the models after overwriting is lower, indicating a more statistically significant verification outcome. We attribute this phenomenon to the fact that the overwriting attack biases the classification of triggered samples toward class 9, thereby increasing the confidence of their assignment to other classes and indirectly reducing the confidence in our original class (class 0). Figure 7: Knowledgeable Attacker (I) with knowing the watermark samples, trigger, and complete embedding pipeline. ∙ Knowledgeable Attacker (I): The attacker can leverage machine unlearning techniques [43] to weaken or even erase the original watermark. Specifically, it can construct a new shadow dataset by injecting a chosen trigger into the clean samples in the shadow dataset sD_s and then maximizing the similarity between the representations of clean samples and new triggered samples output by the watermarked encoder. TABLE VIII also presents the experimental results of machine unlearning attacks. The results show that although the p-value for ownership verification slightly increases in the watermarked models after unlearning, ArmSSL remains extractable, and ownership verification can still be achieved with a low p-value. This phenomenon demonstrates that our method can effectively resist such adaptive attacks. ∙ Knowledgeable Attacker (I): To thoroughly circumvent potential IP disputes, such attackers may actively attempt to remove the watermark embedded in our encoder. This attacker can fine-tune the model by maximizing the loss function associated with the watermark term to eliminate the embedded watermark, as expressed in Eq. 6. ℒ=−ψmin1S∑i=1S|cos((xc,i),(xwm,i))|.L=-ψ 1SΣ _i=1^S | (e_wm(x_c,i),e_wm(x_wm,i)) |. (6) As illustrated in Fig. 7, we measure the ACCm and p-value under different values of ψ. Observations on encoders pre-trained on CIFAR-10 and Imagenette reveal that when ψ = 0.5, the model’s ACCm drops sharply to an unusable level, leading to the failure of watermark verification. When ψ = 0.1, on the downstream datasets (i.e., GTSRB, SVHN) with a significant distribution difference from the pre-training dataset, the attacker successfully removes our watermark at the cost of a 20% decrease in ACCm. Meanwhile, on other downstream datasets, although watermarks can still be extracted, their p-values are already very close to the threshold of 0.05. Under conditions where the model remains usable (ψ<=ψ<= 0.1), ArmSSL is still extractable and maintains a low p-value. Overall, effectively removing ArmSSL watermarks comes at the cost of significant model performance degradation, which contradicts the attacker’s goal of profiting from stealing high-performance models. Moreover, this scenario assumes the attacker has full knowledge of all watermark-related details, which is an extremely improbable situation, further demonstrating the high adversarial robustness of ArmSSL. 7 Conclusion In this work, we present ArmSSL, a unified and adversarial robust watermarking framework for pretrained SSL encoders. By incorporating an orthogonality-based trigger effect and feature-space constraints guided by latent representation entanglement and distribution alignment, ArmSSL effectively eliminates the detectability of watermarks while ensuring reliable IP black-box verification in both EaaS and MLaaS scenarios. Experimental results confirm that ArmSSL performs superior in terms of verification reliability, computation overhead, and robustness to watermark removal and adaptive attacks, while existing SOTAs cannot survive. References [1] T. Chen, S. Kornblith, M. Norouzi et al., “A simple framework for contrastive learning of visual representations,” in Proc. Int. Conf. Mach. Learn. (ICML), 2020, p. 1597–1607. [2] J. Gui, T. Chen, J. Zhang et al., “A survey on self-supervised learning: Algorithms, applications, and future trends,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 12, p. 9052–9071, 2024. [3] S. Qi, D. Wang, Y. Fan et al., “Hssas: Optimizing nlp models by combining self-supervised learning and neural architecture search,” in Proc. Int. Conf. Robot. Autom. Intell. Control (ICRAIC), 2024, p. 71–75. [4] C. Chen, A. Seff, A. L. Kornhauser et al., “Deepdriving: Learning affordance for direct perception in autonomous driving,” in Proc. ICCV. IEEE Computer Society, 2015, p. 2722–2730. [5] Y. LeCun and I. Misra, “https://ai.meta.com/blog/self-supervised-learning-the-dark-matter-of-intelligence,” March 4, 2021. [6] T. Cong, X. He, and Y. Zhang, “Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders,” in ACM Conf. Comput. Commun. Secur. (CCS), 2022, p. 579–593. [7] Y. Liu, J. Jia, H. Liu et al., “Stolenencoder: stealing pre-trained encoders in self-supervised learning,” in ACM Conf. Comput. Commun. Secur. (CCS), 2022, p. 2115–2128. [8] A. Dziedzic, N. Dhawan, M. A. Kaleem et al., “On the difficulty of defending self-supervised learning against model extraction,” in International Conference on Machine Learning. PMLR, 2022, p. 5757–5776. [9] P. Lv, P. Li, S. Zhu et al., “Ssl-wm: A black-box watermarking approach for encoders pre-trained by self-supervised learning,” Proc. Netw. Distrib. Syst. Secur. Symp. (NDSS), 2024. [10] Y. Wu, H. Qiu, T. Zhang et al., “Watermarking pre-trained encoders in contrastive learning,” in Proc. Int. Conf. Data Intell. Secur. (ICDIS). IEEE, 2022, p. 228–233. [11] S. Feng, G. Tao, S. Cheng et al., “Detecting backdoors in pre-trained encoders,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, p. 16 352–16 362. [12] H. Wang, Z. Xiang, D. J. Miller et al., “Mm-bd: Post-training detection of backdoor attacks with arbitrary backdoor pattern types using a maximum margin statistic,” in Proc. IEEE Symp. Secur. Priv. (SP), 2024, p. 1994–2012. [13] S. Kolouri, K. Nadjahi, U. Simsekli et al., “Generalized sliced wasserstein distances,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 32, 2019. [14] Z. Zhao, L. Alzubaidi, J. Zhang et al., “A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations,” Expert Syst. Appl., vol. 242, p. 122807, 2024. [15] X. Chen, H. Fan, R. Girshick et al., “Improved baselines with momentum contrastive learning,” arXiv preprint arXiv:2003.04297, 2020. [16] J.-B. Grill, F. Strub, F. Altché et al., “Bootstrap your own latent-a new approach to self-supervised learning,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, p. 21 271–21 284, 2020. [17] X. Chen and K. He, “Exploring simple siamese representation learning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, p. 15 750–15 758. [18] M. Oquab, T. Darcet, T. Moutakanni et al., “Dinov2: Learning robust visual features without supervision,” Trans. Mach. Learn. Res., 2024. [19] Y. Uchida, Y. Nagai, S. Sakazawa et al., “Embedding watermarks into deep neural networks,” in Proc. Int. Conf. Multimedia Retr. (ICMR), 2017, p. 269–277. [20] H. Chen, B. D. Rouhani, C. Fu et al., “Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models,” in Proc. Int. Conf. Multimedia Retr. (ICMR), 2019, p. 105–113. [21] H. Liu, Z. Weng, and Y. Zhu, “Watermarking deep neural networks with greedy residuals.” in Proc. Int. Conf. Mach. Learn. (ICML), 2021, p. 6978–6988. [22] R. Namba and J. Sakuma, “Robust watermarking of neural network with exponential weighting,” in ACM Conf. Comput. Commun. Secur. (CCS), 2019, p. 228–240. [23] A. Pegoraro, C. Segna, K. Kumari, and A.-R. Sadeghi, “Deepeclipse: How to break white-box dnn-watermarking schemes,” in Proc. USENIX Secur. Symp. (USENIX Security ’24), 2024, p. 5287–5304. [24] H. Jia, C. A. Choquette-Choo, V. Chandrasekaran et al., “Entangled watermarks as a defense against model extraction,” in 30th USENIX security symposium (USENIX Security 21), 2021, p. 1937–1954. [25] Y. Li, L. Zhu, X. Jia et al., “Move: Effective and harmless ownership verification via embedded external features,” IEEE Trans. Pattern Anal. Mach. Intell., 2025. [26] S. Shao, Y. Li, H. Yao et al., “Explanation as a watermark: Towards harmless and multi-bit model ownership verification via watermarking feature attribution,” Proc. Netw. Distrib. Syst. Secur. Symp. (NDSS), 2025. [27] P. Yang, Y. Lao, and P. Li, “Robust watermarking for deep neural networks via bi-level optimization,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2021, p. 14 841–14 850. [28] D. Mehta, N. Mondol, F. Farahmandi et al., “Aime: Watermarking ai models by leveraging errors,” in Proc. Des. Autom. Test Eur.(DATE). IEEE, 2022, p. 304–309. [29] Y. Xie, J. Song, M. Xue, H. Zhang, X. Wang, B. Hu, G. Chen, and M. Song, “Dataset ownership verification in contrastive pre-trained models,” Proc. Int. Conf. Learn. Represent. (ICLR), 2025. [30] A. Dziedzic, H. Duan, M. A. Kaleem et al., “Dataset inference for self-supervised models,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, p. 12 058–12 070, 2022. [31] J. Dubiński, S. Pawlak, F. Boenisch et al., “Bucks for buckets (b4b): Active defenses against stealing encoders,” Advances in Neural Information Processing Systems, vol. 36, p. 55 237–55 259, 2023. [32] A. Shetty, Y. Teng, K. He, and Q. Xu, “Warden: Multi-directional backdoor watermarks for embedding-as-a-service copyright protection,” Proc. Annu. Meet. Assoc. Comput Linguist., p. 13 430–13 444, 2024. [33] Z. Fei, B. Yi, J. Geng et al., “Your fixed watermark is fragile: Towards semantic-aware watermark for eaas copyright protection,” arXiv e-prints, p. arXiv–2411, 2024. [34] Y. Tang, J. Yu, K. Gai et al., “Watermarking vision-language pre-trained models for multi-modal embedding as a service,” arXiv preprint arXiv:2311.05863, 2023. [35] W. Chen, B. Wu, and H. Wang, “Effective backdoor defense by exploiting sensitivity of poisoned samples,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), p. 9727–9737, 2022. [36] T. Cai, J. Fan, and T. Jiang, “Distributions of angles in random packing on spheres,” J. Mach. Learn. Res., vol. 14, no. 1, p. 1837–1864, 2013. [37] J. A. Hartigan and M. A. Wong, “Algorithm as 136: A k-means clustering algorithm,” Journal of the royal statistical society. series c (applied statistics), vol. 28, no. 1, p. 100–108, 1979. [38] M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al., “A density-based algorithm for discovering clusters in large spatial databases with noise,” in kdd, vol. 96, no. 34, 1996, p. 226–231. [39] G. Tao, Z. Wang, S. Feng et al., “Distribution preserving backdoor attack in self-supervised learning,” in Proc. IEEE Symp. Secur. Priv. (SP), 2024, p. 2029–2047. [40] R. J. Larsen and M. L. Marx, “An introduction to mathematical statistics (hoboken, nj,” 2005. [41] Z. Dai, Y. Gao, B. Kuang et al., “Division and union: Latent model watermarking,” IEEE Transactions on Information Forensics and Security, 2025. [42] J. Jia, Y. Liu, and N. Z. Gong, “Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning,” in Proc. IEEE Symp. Secur. Priv. (SP). IEEE, 2022, p. 2043–2059. [43] N. Li, C. Zhou, Y. Gao et al., “Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects,” IEEE Trans. Neural Netw. Learn. Syst., 2025. [44] J. Deng, W. Dong et al., “Imagenet: A large-scale hierarchical image database,” in Proc. CVPR, 2009, p. 248–255. [45] Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, no. 7, p. 3, 2015. [46] “https://tensorflow.google.cn/datasets/catalog/imagenette.” [47] “https://tensorflow.google.cn/datasets/catalog/cifar100.” [48] G. H. A. Krizhevsky et al., “Learning multiple layers of features from tiny images.” 2009. [49] A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2011, p. 215–223. [50] L. N. Darlow, E. J. Crowley, A. Antoniou, and A. J. Storkey, “Cinic-10 is not imagenet or cifar-10,” arXiv preprint arXiv:1810.03505, 2018. [51] “http://ufldl.stanford.edu/housenumbers.” [52] J. Stallkamp, M. Schlipsing et al., “The german traffic sign recognition benchmark: a multi-class classification competition,” in The 2011 international joint conference on neural networks. IEEE, 2011, p. 1453–1460. Appendix A Algorithm A.1 Algorithm For Watermark Embedding The algorithm for the watermark embedding is presented in Algorithm 1. Algorithm 1 Watermark Embedding 1: Input: The original pre-trained encoder ce_c, pre-training dataset =∑i=1NxiD=Σ _i=1^Nx_i, shadow dataset s=∑i=1S(xc,i,xwm,i)D_s=Σ _i=1^S(x_c,i,x_wm,i), anchor classes A, number of fine-tuing epochs epochepoch, warm epoch E, learning rate η, hyperparameter α and β. 2: Output: The watermarked encoder e_wm. 3: ←Copy(c)e_wm← Copy(e_c) 4: anchor←ComputeClassCenters(c,,A)a_anchor← ComputeClassCenters(e_c,D,A) 5: for each e∈1,2,…,epoche∈1,2,...,epoch do 6: for each batch x in D and (xc,xwm)(x_c,x_wm) in sD_s do 7: rcx←ExtractRep(c,x)r_c^x← ExtractRep(e_c,x) 8: rwmx←ExtractRep(,x)r_wm^x← ExtractRep(e_wm,x) 9: rc←ExtractRep(,xc)r_c← ExtractRep(e_wm,x_c) 10: rwm←ExtractRep(,xwm)r_wm← ExtractRep(e_wm,x_wm) 11: ℒ←MaxCosSim(rcx,rwmx)L_ref← MaxCosSim(r_c^x,r_wm^x) ⊳ Eq. 5 12: ℒ←MinimizeCosSim(rc,rwm)L_wm← MinimizeCosSim(r_c,r_wm) ⊳ Eq. 4 13: ℒ←MaxCosSim(rwm,anchor)L_entgl← MaxCosSim(r_wm,a_anchor) ⊳ Eq. 2 14: ℒ←MinimizeSWD(rwmx,rwm)L_dis← MinimizeSWD(r_wm^x,r_wm) ⊳ Eq. 3 15: if epoch<=Eepoch<=E then 16: ℒ=ℒ+αℒL_total=L_ref+ _wm 17: else 18: ℒ=ℒ+ℒL_rbt=L_entgl+L_dis 19: ℒ=ℒ+αℒ+βℒL_total=L_ref+ _wm+ _rbt ⊳ Eq. 1 20: end if 21: ←−η∇ℒe_wm _wm-η _total 22: end for 23: end for 24: return e_wm A.2 Algorithm For Ownership Verification Under MLaaS Scenario The algorithm for the ownership verification under MLaaS is presented in Algorithm 2. Algorithm 2 Ownership Verification Under MLaaS Scenario 1: Input: The suspect model ~ f, the verification dataset s=∑i=1S(xc,i,xwm,i)D_s=Σ _i=1^S(x_c,i,x_wm,i), significant level λ. 2: Output: The verification result: TrueTrue or FalseFalse. 3: for each (xc,i,xwm,i)(x_c,i,x_wm,i) in sD_s do 4: c,i←~(xc,i)P_c,i← f(x_c,i) 5: wm,i←~(xwm)P_wm,i← f(x_wm) 6: end for 7: p-value = t-test(cP_c, wmP_wm) 8: if p<=λp<=λ then 9: return TrueTrue 10: else 11: return FalseFalse 12: end if Appendix B Ownership Verification B.1 Ownership Verification Under Eaas Scenario In Eaas scenario, where the stolen encoder is directly deployed as a new EaaS, the model owner can query the suspect encoder ~ e to obtain feature representations but has no access to internal parameters. To verify ownership, we evaluate the cosine similarity between probing pairs in sD_s and formulate the verification task as a hypothesis test as described below: Proposition 2. Let ~c=~(xc) r_c= e(x_c) and ~wm=~(xwm) r_wm= e(x_wm) denote representations outputted by the suspect encoder ~ e on probing pairs (xc,xwm)(x_c,x_wm), respectively. For paired samples, define the cosine similarity: M~c,wm=|cos(~(xc),~(xwm))| M_c,wm=| ( e(x_c), e(x_wm))|. Let μ∈(0,1)μ∈(0,1) denote the similarity threshold. The negative encoder, which lacks our ArmSSL watermark, will produce nearly identical representations for the paired clean and watermarked samples (xc,xwm)(x_c,x_wm), meaning the ideal value of M~c,wm M_c,wm is 1. Given the null hypothesis ℍ0:1−M~c,wm≤μH_0:1- M_c,wm≤μ and the alternative hypothesis ℍ1:1−M~c,wm>μH_1:1- M_c,wm>μ, where the hyper-parameter μ∈(0,1)μ∈(0,1), we claim that the suspect encoder is illegally derived from our well-trained encoder e_wm if and only if ℍ0H_0 is rejected. If the t-test yields a p-value p≤λp≤λ (significance level λ=0.05λ=0.05), we reject H0H_0 and claim that ~ e originates from the our watermarked encoder. The algorithm for the ownership verification procedure in EaaS scenario is presented in Algorithm 3. Algorithm 3 Ownership Verification Under EaaS scenario 1: Input: The suspect encoder ~ e, the verification dataset s=∑i=1S(xc,i,xwm,i)D_s= _i=1^S(x_c,i,x_wm,i), significant level λ. 2: Output: The verification result: TrueTrue or FalseFalse. 3: for each (xc,i,xwm,i)(x_c,i,x_wm,i) in sD_s do 4: ~c,i←~(xc,i) r_c,i← e(x_c,i) 5: ~wm,i←~(xwm) r_wm,i← e(x_wm) 6: end for 7: p-value = t-test(1, M~c,wm M_c,wm) 8: if p<=λp<=λ then 9: return TrueTrue 10: else 11: return FalseFalse 12: end if Appendix C Experiments C.1 Details of Experimental Settings Datasets. We use nine popular datasets for CV tasks (i.e., ImageNet, Tiny-ImageNet, Imagenette, CIFAR-100, CIFAR-10, STL-10, CINIC-10, SVHN, and GTSRB). ∙ ImageNet [44] comprises 1.2 million training images categorized into 1,000 distinct classes. Each image has a resolution of 224×224224× 224 pixels with three color channels (RGB). ∙ Tiny-ImageNet [45] is a subset of the ImageNet dataset. It consists of 100,000 images spanning 200 classes, with each class containing 500 images for training and 50 for validation. ∙ Imagenette [46] is a subset of 10 classes from the ImageNet dataset. It has 9469 training images and 3925 test images. Each image size is 224×224×3224× 224× 3. ∙ CIFAR-100 [47] contains 60, 000 images with size 32 × 32 × 3 in 100 classes. Each class contains 500 images for training and 100 images for testing. ∙ CIFAR-10 [48] comprises 60,000 images with the size of 32 × 32 × 3, distributed across 10 distinct classes. ∙ STL-10 [49] contains 5,000 labeled images for training and 8,000 for testing, along with 100,000 unlabeled images, all conforming to the identical 10-class categorical structure defined by CIFAR-10. ∙ CINIC-10 [50] is synthetically constructed by combining elements from ImageNet and CIFAR-10, comprising a total of 270,000 images. It retains the same 10-class taxonomic structure as CIFAR-10. ∙ SVHN [51] is a real-world image dataset of 600,000+ digits obtained from house numbers in Google Street View images. The dataset consists of 10 classes, representing digits 0 to 9. ∙ GTSRB [52] is a dataset designed for traffic sign recognition, containing over 50,000 images of traffic signs categorized into 43 classes. Shadow Dataset. Due to the inaccessibility of downstream task domains and their specific samples, a subset of samples is selected from the pre-training dataset to construct the shadow dataset for subsequent watermark embedding and verification. Specifically, when the pre-training dataset is CIFAR-10 and Imagenette, 200 samples are selected; for ImageNet, 500 samples are chosen. By embedding a trigger pattern into the selected samples, probing pairs consisting of clean samples and their watermarked samples is created. Platform. All experiments are conducted on two separate hardware configurations: a primary workstation equipped with an NVIDIA GeForce RTX 3090 GPU (24 GB VRAM), Intel(R) Core(TM) i9-10900K CPU @ 3.70 GHz, and 32 GB RAM, and an auxiliary server featuring an NVIDIA GeForce RTX 4090 GPU (24 GB VRAM), 16 vCPU Intel(R) Xeon(R) Gold 6430 processor, and 120 GB RAM. C.2 Experimental Results of Ownership Verification Under EaaS Scenario Effectiveness and Utility. We first simulate the most common DT attacker under the EaaS scenario. As shown in TABLE IX, we take the SimCLR algorithm as an example to evaluate the classification accuracy (ACCm) calculated by the K-Nearest Neighbors (KNN) algorithm and ownership verification p-values of the clean encoder, the ArmSSL watermarked encoder, and four negative encoders that are trained on the CIFAR-10, Imagenette, and ImageNet datasets, respectively. The experimental results demonstrate that the FPR on negative models defined in Subsection 5.1 is 0.00%, confirming the reliability of our verification method proposed in Subsection B.1. For the ArmSSL watermarked encoder, the p-values are all significantly lower than the predefined threshold τ=0.05τ=0.05, with the maximum value of only 1.10e-39, successfully identifying these pirated encoders. Moreover, regarding encoder utility, ArmSSL introduces negligible degradation. Compared with the clean encoder, the ArmSSL’s ACCm decreases by only 1.28% on CIFAR-10, 2.19% on Imagenette, and 0.25% on ImageNet, indicating that the watermarking process has minimal impact on encoder usability. TABLE IX: Comparison of the ACCm and p-value under clean and watermarked models pretrained by SimCLR. Dataset Clean encoder ArmSSL encoder Neg-1 Neg-2 Neg-3 Neg-4 ACCm p-value ACCm p-value ACCm p-value ACCm p-value ACCm p-value ACCm p-value CIFAR-10 87.48% 1.00 86.20% 2.98e-58 79.34% 1.00 89.41% 1.00 85.15% 1.00 80.14% 1.00 Imagenette 82.55% 1.00 80.36% 2.82e-56 82.91% 1.00 67.08% 1.00 64.8% 1.00 83.13% 1.00 ImageNet 58.05% 0.99 57.80% 1.10e-39 62.10% 1.00 59.04% 0.95 48.60% 0.98 62.45% 1.00 TABLE X: Comparison of the ACCm and p-value under EaaS scenario after fine-tuning and pruning attacks. Dataset FT PR-5% PR-20% PR-40% PR-60% PR-85% PR-95% ACCm p-value ACCm p-value ACCm p-value ACCm p-value ACCm p-value ACCm p-value ACCm p-value CIFAR-10 89.21% 1.50e-15 86.41% 9.78e-44 86.68% 2.98e-54 85.12% 3.91e-40 79.08% 4.77e-42 41.25% 1.00 36.96% 1.00 Imagenette 80.89% 1.56e-45 81.74% 1.10e-38 80.75% 6.60e-49 81.74% 8.96e-38 77.83% 8.44e-31 55.40% 1.00 46.18% 1.00 Robustness against Watermark Removal. To evaluate the adversarial robustness of ArmSSL, we further consider the settings where the attacker attempts to remove the watermark through model fine-tuning and pruning. It is worth noting that the attacker is assumed to have access to the partial original pre-training dataset. We conduct a series of experiments employing the SimCLR-based watermarked encoders. In the attack scenario permitting full fine-tuning of all layers, the ownership verification results (as shown in TABLE X) indicate that the p-values slightly increase after FT but remain significantly below the predefined threshold of 0.05. This demonstrates the strong robustness of the proposed ArmSSL against FT attacks. We further perform pruning attacks on the stolen DT encoder with pruning ratios ranging from 5% to 99%. As shown in TABLE X, the ACCm of the ArmSSL watermarked encoder gradually decreases as the pruning ratio increases. When the pruning ratio is below 40%, the p-values remain nearly identical to those of the unpruned model. At PR-60%, the p-values increase slightly but still stay well below the 0.05 threshold. Only when the pruning ratio reaches 85%, does the ownership verification of ArmSSL fail, indicating that the attacker has successfully removed the watermark. However, at this point, the encoder’s utility severely degrades, and its accuracy drops to 41.25% on CIFAR-10 and 55.40% on Imagenette, rendering the encoder useless and thereby defeating the attacker’s goal of maintaining usability. TABLE XI: Comparison of ArmSSL and Dziedzic et al. [8] with robustness against watermark removal. Pre-training Dataset ArmSSL Dziedzic et al. [8] PR-70% Fine-tuning Dataset FT ACCm↓ p-value ACCm↓ p-value p-value(ArmSSL) p-value([8]) p-value(ArmSSL) p-value([8]) CIFAR-10 1.3% 3.43e-26 14.89% 1.67e-25 3.23e-15 1.00 CIFAR-10 1.90e-15 1.00 CINIC 2.38e-15 1.00 STL-10 8.72e-13 1.00 We further compare our ArmSSL with the watermarking method in [8] using SimCLR-pretrained encoders on the CIFAR-10. Our analysis shows three main limitations of [8]: First, its ownership verification depends on the encoder’s intermediate representations, which restricts its applicability to EaaS scenarios and hinders its extension to broader MLaaS scenarios. Second, as shown in TABLE XI, the watermark in [8] is fragile to removal via fine-tuning with only a small set of samples (e.g., 1,000 images) and becomes ineffective under a pruning ratio of just 75%. In contrast, ArmSSL demonstrates strong robustness against such watermark-removal attacks. Finally, the watermark introduced in [8] leads to a pronounced utility degradation (ACCm↓ =14.89%) compared to ArmSSL (ACCm↓ =1.3%). We attribute this to two key factors: (1) the method requires a watermark-embedding dataset on a scale comparable to the encoder’s original training data, and (2) it relies on learning discriminative responses to specific, private augmentations (e.g., rotation)—a requirement that fundamentally conflicts with the objective of self-supervised learning methods like SimCLR, which aim to maintain representation invariance under a wide variety of augmentations. Adaptive Attacks. Under the EaaS scenario, we follow the experimental settings described in Section 6 for the MLaaS scenario. First, we evaluate the p-value of the ArmSSL under overwriting and unlearning attacks launched by the adaptive attackers, respectively. As shown in TABLE XII, under both adaptive attacks, ArmSSL can still successfully extract the embedded watermarks from the suspect encoders, with all p-values remaining far below the threshold of 0.05. TABLE XII: Knowledgeable Attackers I (Overwriting) and I (Unlearning) under EaaS scenario. Dataset Overwriting Unlearning ACCm p-value ACCm p-value CIFAR-10 84.79% 1.84e-38 78.25% 8.74e-64 Imagenette 78.05% 8.70e-21 79.15% 4.80e-29 Next, unlike in Section 6, when simulating the Knowledgeable Attacker (I), we fix ψ=0.1ψ=0.1 in Eq. 6 and assess the encoder’s utility across different learning rates η (i.e., 1×10−91× 10^-9, 1×10−61× 10^-6, 1×10−31× 10^-3) in terms of ACCm and p-value, as illustrated in Fig. 8. The results show that when η=1×10−9η=1× 10^-9 and 1×10−61× 10^-6, the ACCm and p-values of the ArmSSL encoder on CIFAR-10 and Imagenette remain almost unaffected. However, when η=1×10−3η=1× 10^-3, the encoder’s ACCm drops sharply to an unusable level, and watermark verification fails simultaneously. These findings indicate that while a large learning rate may destroy the embedded watermark, it also severely impairs the encoder’s functionality, thereby defeating the attacker’s theft objective. ((a)) ((b)) Figure 8: Knowledgeable Attacker (I) under EaaS scenario. C.3 Experimental Results of ArmSSL’s Ownership Verification Under DINOv2 To further validate the broad applicability and backbone-agnostic nature of ArmSSL, we extend our evaluation to the SOTA SSL framework DINOv2 based on ViT architecture. As illustrated in Fig. 9, we evaluate the ownership verification performance (p-value) across both EaaS and MLaaS scenarios: (1) Effectiveness: For the watermarked DINOv2 encoder, ArmSSL achieves an extremely low p-value (i.e., 2.13e-32 in the EaaS scenario and 3.07e-25 in the MLaaS scenario), providing an unambiguous ownership signal that stays significantly below the decision threshold 0.050.05; (2) Robustness against fine-tuning: After aggressive full fine-tuning, the embedded watermark remains resilient with a p-value of 9.59e-18, confirming that the IP signal is not easily erased; (3) Robustness against model pruning: Under structured pruning, ArmSSL continues to exhibit high reliability. At the 40% and 60% pruning rate, the corresponding p-values remain highly significant. In summary, these results demonstrate that ArmSSL is highly effective not only for CNN-based contrastive learning but also for ViT-based self-distillation frameworks, ensuring robust IP protection across various SSL paradigms. Figure 9: The ArmSSL’s p-value under DINOv2 algorithm.