Paper deep dive

Sophon: Non-Fine-Tunable Learning to Restrain Task Transferability For Pre-trained Models

Jiangyi Deng, Shengyuan Pang, Yanjiao Chen, Liangming Xia, Yijie Bai, Haiqin Weng, Wenyuan Xu

Year: 2024Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 140

Models: CAFormer, ResNet-50, VGG

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%

Last extracted: 3/12/2026, 7:28:45 PM

Summary

Sophon is a protection framework designed to prevent pre-trained deep learning models from being fine-tuned for restricted or unethical tasks, while maintaining their performance on original, benign tasks. It utilizes a non-fine-tunable learning paradigm inspired by model-agnostic meta-learning, incorporating fine-tuning simulation and evaluation algorithms to entrap models in local optima regarding restricted domains.

Entities (4)

Sophon · framework · 100%Model-agnostic meta-learning · algorithm · 95%Non-fine-tunable learning · learning-paradigm · 95%Non-transferable learning · learning-paradigm · 90%

Relation Signals (3)

Sophon → implements → Non-fine-tunable learning

confidence 95% · We implement a fully-functional non-fine-tunable prototype named Sophon

Sophon → inspiredby → Model-agnostic meta-learning

confidence 90% · Inspired by model-agnostic meta-learning, we overcome this difficulty by designing sophisticated fine-tuning simulation

Non-fine-tunable learning → extends → Non-transferable learning

confidence 85% · Note that non-transferable learning is a special case of non-fine-tunable learning

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Instead of building deep learning models from scratch, developers are more and more relying on adapting pre-trained models to their customized tasks. However, powerful pre-trained models may be misused for unethical or illegal tasks, e.g., privacy inference and unsafe content generation. In this paper, we introduce a pioneering learning paradigm, non-fine-tunable learning, which prevents the pre-trained model from being fine-tuned to indecent tasks while preserving its performance on the original task. To fulfill this goal, we propose SOPHON, a protection framework that reinforces a given pre-trained model to be resistant to being fine-tuned in pre-defined restricted domains. Nonetheless, this is challenging due to a diversity of complicated fine-tuning strategies that may be adopted by adversaries. Inspired by model-agnostic meta-learning, we overcome this difficulty by designing sophisticated fine-tuning simulation and fine-tuning evaluation algorithms. In addition, we carefully design the optimization process to entrap the pre-trained model within a hard-to-escape local optimum regarding restricted domains. We have conducted extensive experiments on two deep learning modes (classification and generation), seven restricted domains, and six model architectures to verify the effectiveness of SOPHON. Experiment results verify that fine-tuning SOPHON-protected models incurs an overhead comparable to or even greater than training from scratch. Furthermore, we confirm the robustness of SOPHON to three fine-tuning methods, five optimizers, various learning rates and batch sizes. SOPHON may help boost further investigations into safe and responsible AI.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

139,587 characters extracted from source content.

Expand or collapse full text

Sophon: Non-Fine-Tunable Learning to Restrain Task Transferability For Pre-trained Models Jiangyi Deng11, Shengyuan Pang11, Yanjiao Chen1, Liangming Xia1, Yijie Bai1, Haiqin Weng2, Wenyuan Xu1 1Equal Contribution 1Zhejiang University, 2Ant Group Abstract Instead of building deep learning models from scratch, developers are more and more relying on adapting pre-trained models to their customized tasks. However, powerful pre-trained models may be misused for unethical or illegal tasks, e.g., privacy inference and unsafe content generation. In this paper, we introduce a pioneering learning paradigm, non-fine-tunable learning, which prevents the pre-trained model from being fine-tuned to indecent tasks while preserving its performance on the original task. To fulfill this goal, we propose Sophon, a protection framework that reinforces a given pre-trained model to be resistant to being fine-tuned in pre-defined restricted domains. Nonetheless, this is challenging due to a diversity of complicated fine-tuning strategies that may be adopted by adversaries. Inspired by model-agnostic meta-learning, we overcome this difficulty by designing sophisticated fine-tuning simulation and fine-tuning evaluation algorithms. In addition, we carefully design the optimization process to entrap the pre-trained model within a hard-to-escape local optimum regarding restricted domains. We have conducted extensive experiments on two deep learning modes (classification and generation), seven restricted domains, and six model architectures to verify the effectiveness of Sophon. Experiment results verify that fine-tuning Sophon-protected models incurs an overhead comparable to or even greater than training from scratch. Furthermore, we confirm the robustness of Sophon to three fine-tuning methods, five optimizers, various learning rates and batch sizes. Sophon may help boost further investigations into safe and responsible AI. 1 Introduction Figure 1: The objectives of non-fine-tunable learning. (1) Intactness: it should preserve the model performance in the original domain. (2) Non-fine-tunability: fine-tuning the model in the restricted domain should incur a comparable or even greater overhead than training the model from scratch. Pre-trained models have become a popularity for developers to handle the increasingly complex and diverse deep learning tasks. A pre-trained model learns universal representations from large-scale datasets with high generalization ability. By simply fine-tuning it with a few task-specific data samples, a user can transfer the pre-trained model to different downstream tasks [12, 42, 35, 28, 47], which is significantly more efficient than training models from scratch. Attracted by this, a plethora of pre-trained models have been built and available in public model zoos to facilitate the research community and industry. However, this emerging pre-training solution may be misused to unethical or harmful tasks [2], e.g., using classification models to predict sensitive information like sexual orientation [44, 13, 43, 39] or generative models to create unsafe content like sexually-explicit, violent, and political images [51, 30, 29, 16, 17]. Although efforts are being made to train safe and ethical models from scratch [18, 15], high transferability of pre-trained models makes them more vulnerable to being abused. Non-transferable learning (NTL) [50, 49, 55, 48] has been proposed to degrade the performance of a pre-trained model in certain domains. However, non-transferable learning only considers the transferability of a pre-trained model before fine-tuning. After being fine-tuned with only a few samples from the target domain, an NTL-protected model will be easily transformed into a well-performed model for the target task. As far as we are concerned, there is a lack of effective protection for pre-trained models from being fine-tuned to inappropriate tasks. To bridge this gap, we propose a new learning paradigm, dubbed non-fine-tunable learning, which aims to realize two objectives, as illustrated in Figure 1. (1) Intactness. Non-fine-tunable learning should preserve the model performance in the original domain. (2) Non-Fine-Tunability. Fine-tuning the pre-trained model in restricted domains should incur a comparable or even greater overhead than training the model for restricted tasks from scratch. Note that non-transferable learning is a special case of non-fine-tunable learning with an identity function as the fine-tuning process. However, to materialize these goals are challenging in several aspects. • How to design the optimization framework to obtain the two objectives of non-fine-tunable learning? The two objectives of intactness and non-fine-tunability should be instantiated in the optimization framework. To ensure non-fine-tunability, we need to estimate the model performance under hypothetical fine-tuning processes. However, commonly-used fine-tuning methods are iterative and complicated optimization problems themselves, making it difficult to integrate them into the optimization framework. To address this challenge, we are inspired by the model-agnostic meta-learning (MAML) framework [8] to simulate the fine-tuning process of a potential adversary. In this way, we formulate the non-fine-tunable learning problem into a multi-objective optimization framework, with a fine-tuning suppression term and a normal training reinforcement term. The former minimizes the model performance in restricted domains with the feedback provided by the simulated fine-tuning processes. The latter maximizes the model performance in the original domain to ensure intactness. • How to ensure robustness under unpredictable fine-tuning strategies adopted by adversaries? A non-cooperative adversary may fine-tune the pre-trained model with unpredictable fine-tuning strategies. There are tons of choices for fine-tuning strategies, including model initialization, transfer strategy and optimization strategy. The simulated fine-tuning process in our optimization framework may not coincide with that adopted by the adversary. To tackle this problem, we integrate various empirically-strong fine-tuning strategies in the fine-tuning simulation process. Our extensive experiments have confirmed that our simulated fine-tuning strategies are effective in ensuring non-fine-tunability to unseen fine-tuning strategies. • How to boost the convergence of fine-tuning suppression in restricted domains? Gradient descent is commonly used to solve the optimization problem in deep learning. To guarantee convergence of the gradient descent algorithm, the gradient value used to update model parameters should decrease with the training process. Conventional loss functions, e.g., cross-entropy loss are designed to satisfy this condition. However, to minimize the model performance in restricted domains is opposite to normal training that tries to maximize the model performance on training datasets. Therefore, conventional loss functions destabilize the convergence of solving our optimization problem. To resolve this difficulty, we propose two alternative loss functions for classification and one for generation, i.e., inverse cross-entropy, Kullback–Leibler divergence from uniform distribution and denial of service losses, gradients of which are theoretically shown to decrease with the iterative process of fine-tuning suppression. The design of these loss functions facilitates the convergence of fine-tuning suppression in restricted domains. We implement a fully-functional non-fine-tunable prototype named Sophon 111Sophon is the name of a supercomputer created by Trisolarans (aliens) to sabotage Earth’s technological development in the science fiction The Three-Body Problem [26] by Cixin Liu., which conducts non-fine-tunable learning on a given pre-trained model before releasing it to prevent its use or adaptation for restricted tasks. Extensive experiments have been conducted to evaluate the performance of Sophon on two deep learning modes (classification and generation), seven restricted tasks, and six model architectures. We demonstrate that fine-tuning Sophon-protected models has overhead close to or even greater than training from scratch, under three fine-tuning methods, five optimizers, various learning rates and batch sizes. We have open-sourced our code222https://github.com/ChiangE/Sophon in a hope to incentivize more research in this area. Summarization of contributions. We summarize our theoretical and technical contributions as follows: • We propose non-fine-tunable learning, which aims to restrain transferability of pre-trained models to improper tasks under fine-tuning, extending the connotation of safe and ethic deep learning models. • We develop a non-fine-tunable learning framework, which optimizes pre-trained models to fulfill the objectives of intactness and non-fine-tunability, which may help reduce the risk of model abuse for the open source community. • We conduct extensive experiments to verify the effectiveness and robustness of our non-fine-tunable learning framework against a diversity of fine-tuning strategies. 2 Background 2.1 Deep Learning Tasks In this paper, we focus on two major tasks of deep learning, i.e., classification and generation. They have different paradigms including model architectures, loss functions, and datasets. Classification tasks aim to predict the class label of an input sample, e.g., classifying emails as spam or not spam. A deep learning classification model f⁢(x)=σ⁢(zθ⁢(x))subscriptf (x )=σ (z_θ (x ) )f ( x ) = σ ( zitalic_θ ( x ) ) usually consists of a model backbone zθ⁢(⋅)subscript⋅z_θ (· )zitalic_θ ( ⋅ ) and a Softmax layer σ⁢(⋅)⋅σ (· )σ ( ⋅ ), where zθsubscriptz_θzitalic_θ is parameterized by θ. Typically, the output y is a probability distribution as y=(y1,y2,⋯,yC)subscript1subscript2⋯subscripty= (y_1,y_2,·s,y_C )y = ( y1 , y2 , ⋯ , yitalic_C ), where C is the number of classes and the predicted label is arg⁡maxk⁡yksubscriptsubscript _ky_karg maxitalic_k yitalic_k. Given a training dataset =(xi,yi)|xi∈,yi∈,i=1,2,⋯conditional-setsubscriptsubscriptformulae-sequencesubscriptformulae-sequencesubscript12⋯D= \ (x_i,y_i )|x_i ,y_i∈% Y,i=1,2,·s \D = ( xitalic_i , yitalic_i ) | xitalic_i ∈ X , yitalic_i ∈ Y , i = 1 , 2 , ⋯ , a classification model learns a function f:→:→f:X : X → Y by minimizing the loss function ℒ⁢(f⁢(xi),yi)ℒsubscriptsubscriptL (f(x_i),y_i )L ( f ( xitalic_i ) , yitalic_i ), where yisubscripty_iyitalic_i is the ground-truth probability distribution. A commonly used loss function is the cross-entropy loss ℒCE=−1||⁢∑xi∈yi⁢log⁡f⁢(xi)subscriptℒCE1subscriptsubscriptsubscriptsubscriptL_CE=- 1| X| _x_i∈% Xy_i f(x_i)Lroman_CE = - divide start_ARG 1 end_ARG start_ARG | X | end_ARG ∑x start_POSTSUBSCRIPT i ∈ X end_POSTSUBSCRIPT yitalic_i log f ( xitalic_i ). Generation tasks aim to generate data samples that follow a certain distribution. Representative generative models include autoencoder (AE) [40], variational autoencoder (VAE) [21], generative adversarial network (GAN) [11], and diffusion probabilistic model [14, 38]. Without loss of generality, we focus on diffusion probabilistic models, the state-of-the-art generative models. But note that Sophon can be easily extended to other generative models. Diffusion probabilistic models (referred to as diffusion models for brevity) is a parameterized Markov chain that generally contains a forward process and a reverse process [14, 38]. The forward process or diffusion process adds Gaussian noises to data and the reverse process are learned to reverse the diffusion process, i.e., denoise the data. To train a diffusion model, training datasets are crafted as =((xt,t),ϵ)|xt∼t,t∈[1,T],ϵ∼⁢(,)conditional-setsubscriptitalic-ϵformulae-sequencesimilar-tosubscriptsubscriptformulae-sequence1similar-toitalic-ϵ0D= \ ((x_t,t),ε )|x_t _t,t% ∈[1,T],ε (0,I) \D = ( ( xitalic_t , t ) , ϵ ) | xitalic_t ∼ Xitalic_t , t ∈ [ 1 , T ] , ϵ ∼ N ( 0 , I ) , where xt⁢(x0,ϵ,t)=α¯t⁢x0+1−α¯t⁢ϵsubscriptsubscript0italic-ϵsubscript¯subscript01subscript¯italic-ϵx_t (x_0,ε,t )= α_t\,x_0+ 1- % α_t\, _t ( x0 , ϵ , t ) = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ is the forward process with predefined hyper-parameters α¯tsubscript¯ α_tover¯ start_ARG α end_ARGt, tsubscriptX_tXitalic_t is the distribution of xtsubscriptx_txitalic_t, and T is the number of diffusion steps. The diffusion model f:t→,∀t∈[1,T]:formulae-sequence→subscriptfor-all1f:X_t ,∀ t∈[1,T]f : Xitalic_t → N , ∀ t ∈ [ 1 , T ] learns to predict noises at each diffusion step by minimizing the loss function ℒ⁢(f⁢(xt⁢i,ti),ϵi)ℒsubscriptsubscriptsubscriptitalic-ϵL (f(x_ti,t_i ), _i)L ( f ( xitalic_t i , titalic_i ) , ϵitalic_i ). A commonly used loss function is the mean squared error ℒMSE=1||⁢∑i‖ϵi−f⁢(xt⁢i,ti)‖2subscriptℒMSE1subscriptsuperscriptnormsubscriptitalic-ϵsubscriptsubscript2L_MSE= 1|X| _i\| _i-f(x_% ti,t_i)\|^2Lroman_MSE = divide start_ARG 1 end_ARG start_ARG | X | end_ARG ∑i ∥ ϵitalic_i - f ( xitalic_t i , titalic_i ) ∥2. 2.2 Transfer Learning Transfer learning is a learning paradigm that enables knowledge transfer across various learning tasks. The intuition behind transfer learning is to encode knowledge from a large and general enough dataset in a pre-trained model, and then fine-tune the pre-trained model on specific tasks [1]. Following this intuition, transfer learning formalizes a two-phase learning framework: a pre-training phase to acquire knowledge from one or more source tasks, and a fine-tuning stage to transfer the attained knowledge to target tasks. Owing to the wealth of knowledge captured in the pre-training phase, the fine-tuning phase can quickly adapt models to new tasks with limited training samples and training efforts [53, 5, 36]. As deep learning models grow larger and more complicated, fine-tuning large-scale pre-trained models instead of learning models from scratch has become a consensus. A large quantity of pre-trained models are developed by individuals or tech companies like Google, Meta, Microsoft and NVIDIA, and are released as open-sourced models on model hubs (e.g., GitHub and Hugging Face), allowing developers to fine-tune these models into their customized applications. The fine-tuning process adapts pre-trained models to downstream tasks. The key components of fine-tuning include model initialization, transfer strategy and optimization strategy. Model initialization. Given a pre-trained model, a customized model may be initialized in various ways, e.g., directly using the entire pre-trained model, copying the entire model but randomly initializing the last fully-connected layer, and copying the entire model but randomly initializing the last few layers. By copying more layers from the pre-trained model, more knowledge will be inherited. Transfer strategy. The transfer strategy selects which parameters to update during fine-tuning. Two common strategies are fine-tuning the whole model or only fine-tuning certain layers. Previous study shows that fine-tuning the whole model yields better performance than only fine-tuning a few layers, due to a gain of co-adaptation within layers [52]. Optimization strategy. The optimization strategy includes the optimizer (the parameter update rules) and hyper-parameters (e.g., batch size, learning rate, and the number of updates). Commonly used optimizers in deep learning include SGD [37], Momentum [33], Nesterov [46], Adagrad [6], Adadelta [54], and Adam [20]. To prevent pre-training models from being abused for inappropriate downstream tasks, non-transferable learning (NTL) [50, 49, 55, 48] was proposed to degrade the performance of pre-trained models in certain tasks (referred to as restricted domain) before fine-tuning. As far as we are concerned, existing works on non-transferable learning all focus on classification tasks. Non-transferable learning incorporates additional optimization objectives, e.g., mis-classifying samples from the restricted domain and enlarging distance between representation distributions of the source and the restricted domains. However, non-transferable learning does not suppress the performance of pre-trained models in the restricted domain if fine-tuning is performed. In other words, fine-tuning NTL-based pre-trained model will break performance suppression and attain high-quality performance in the restricted domain. We will demonstrate that NTL cannot resist fine-tuning via experiments in §5.2. Figure 2: Design of Sophon. Sophon mainly consists of two alternating phases, i.e., the fine-tuning suppression (FTS) loops and the normal training reinforcement (NTR) loops. The FTS loops are designed to simulate different fine-tuning processes and degrade the fine-tuning performance in the restricted domain. The NTR loops are designed to maintain the performance in the original domain. The number of tasks N, the number of updates K, the learning rates of FTS loops α and NTR loops β, and the number of FTS loops ℓFTSsubscriptℓFTS _FTSℓroman_FTS and NTR loops ℓNTRsubscriptℓNTR _NTRℓroman_NTR, and the total number of iterations IterIterIterIter are hyper-parameters. 2.3 Design Goal We first define the system model in terms of the adversary and defender, and then elaborate the design goals of Sophon under the defined system model. Adversary. The adversary aims to obtain a well-performed model in the restricted domain, e.g., sensitive information inference or unsafe image generation. To achieve this objective, the adversary fine-tunes an open-sourced public pre-trained model on data samples from the restricted domain. The adversary may be informed that the model is Sophon-protected and can leverage any fine-tuning strategies to try to neutralize Sophon. Defender. The defender aims to prevent a pre-trained model from being fine-tuned for certain downstream tasks. The defender defines the restricted domain according to well-recognized unethical applications and can glean samples from the restricted domain. The defender has access to the pre-trained model and can modify the pre-trained model. Given the system model, we lay out three major goals of Sophon. Intactness. The protected model should perform as well as the original model on benign tasks. Let SsubscriptD_SDitalic_S denotes the source-domain datasets used to train the pre-trained model. Intactness indicates that x∼S⁢ℒ⁢(fθ⁢(x))≈x∼S⁢ℒ⁢(f0⁢(x))subscriptsimilar-tosubscriptℒsubscriptsubscriptsimilar-tosubscriptℒsubscript0E_x _S\ L(f_θ(x)) % _x _S\ L(f_0(x))blackboard_Ex ∼ D start_POSTSUBSCRIPT S end_POSTSUBSCRIPT L ( fitalic_θ ( x ) ) ≈ blackboard_Ex ∼ D start_POSTSUBSCRIPT S end_POSTSUBSCRIPT L ( f0 ( x ) ), where f0subscript0f_0f0 and fθsubscriptf_θfitalic_θ denote the original model and the protected model respectively. Non-transferability. The protected model (without fine-tuning) should perform poorly in the restricted domain. Let AsubscriptD_ADitalic_A denotes the datasets from the restricted domain. Non-transferability indicates x∼A⁢ℒ⁢(fθ⁢(x))≫x∼A⁢ℒ⁢(ϕ⁢(f0⁢(x)))much-greater-thansubscriptsimilar-tosubscriptℒsubscriptsubscriptsimilar-tosubscriptℒitalic-ϕsubscript0E_x _A\ L(f_θ(x)) _x% _A\ L(φ(f_0(x)))blackboard_Ex ∼ D start_POSTSUBSCRIPT A end_POSTSUBSCRIPT L ( fitalic_θ ( x ) ) ≫ blackboard_Ex ∼ D start_POSTSUBSCRIPT A end_POSTSUBSCRIPT L ( ϕ ( f0 ( x ) ) ), where ϕ⁢(⋅)italic-ϕ⋅φ(·)ϕ ( ⋅ ) is the fine-tuning strategy. Non-fine-tunability. The protected model, after being fine-tuned by the adversary, should still perform poorly in the restricted domain, i.e., x∼Aℒ(ϕ(fθ(x))≫x∼Aℒ(ϕ(f0(x)))E_x _A\ L(φ(f_θ(x)) % _x _A\ L(φ(f_0(x)))blackboard_Ex ∼ D start_POSTSUBSCRIPT A end_POSTSUBSCRIPT L ( ϕ ( fitalic_θ ( x ) ) ≫ blackboard_Ex ∼ D start_POSTSUBSCRIPT A end_POSTSUBSCRIPT L ( ϕ ( f0 ( x ) ) ). Note that non-transferability is a special case of non-fine-tunability, since the left-hand-side ϕitalic-ϕφϕ can be an identity function. Note that, if fine-tuning the protected model results in poorer performance than training from scratch, the adversary can simply train a model from scratch using data from the restricted domain. Thus, the worst-case scenario for the adversary is training from scratch. If fine-tuning the protected model achieves better performance than training from scratch but poorer than fine-tuning the original model, we consider the adversary to have achieved partial success. 3 Problem Formulation of Non-Fine-Tunable Learning Before delving into the design details of Sophon, in this section, we formally formulate the non-fine-tunable learning as a constrained optimization problem. Non-fine-tunable learning aims to optimize a model parameterized by θ that satisfies the design goals of intactness, non-transferability and non-fine-tunability. Basic Formulation: minθsubscript _θminitalic_θ −x∼A,ϕ∼Φ⁢ℒ⁢(ϕ⁢(fθ⁢(x))),subscriptformulae-sequencesimilar-tosubscriptsimilar-toitalic-ϕΦℒitalic-ϕsubscript -E_x _A,φ \ L % (φ (f_θ (x ) ) ),- blackboard_Ex ∼ D start_POSTSUBSCRIPT A , ϕ ∼ Φ end_POSTSUBSCRIPT L ( ϕ ( fitalic_θ ( x ) ) ) , (1) s.t.formulae-sequencest \ \ s.t.s . t . x∼S⁢(max⁡0,ℒ⁢(fθ⁢(x))−ℒ⁢(f0⁢(x)))<λ,subscriptsimilar-tosubscript0ℒsubscriptℒsubscript0 ~E_x _S ( \0,L% (f_θ (x ) )-L (f_0 (x )% ) \ )<λ,blackboard_Ex ∼ D start_POSTSUBSCRIPT S end_POSTSUBSCRIPT ( max 0 , L ( fitalic_θ ( x ) ) - L ( f0 ( x ) ) ) < λ , where λ denotes the tolerance of the performance degradation in the original domain. The constrained optimization problem in Equation (1) can be very difficult to solve. Thus, we instead solve the following unconstrained optimization problem. Sophon Formulation: minθsubscript _θminitalic_θ −x∼A,ϕ∼Φ⁢ℒ⁢(ϕ⁢(fθ⁢(x)))+μ⋅x∼S⁢ℒ⁢(fθ⁢(x)),subscriptformulae-sequencesimilar-tosubscriptsimilar-toitalic-ϕΦℒitalic-ϕsubscript⋅subscriptsimilar-tosubscriptℒsubscript -E_x _A,φ \ L % (φ (f_θ (x ) ) )+μ·E_x % D_S\ L (f_θ (x ) ),- blackboard_Ex ∼ D start_POSTSUBSCRIPT A , ϕ ∼ Φ end_POSTSUBSCRIPT L ( ϕ ( fitalic_θ ( x ) ) ) + μ ⋅ blackboard_Ex ∼ D start_POSTSUBSCRIPT S end_POSTSUBSCRIPT L ( fitalic_θ ( x ) ) , (2) where μ balances the two design goals of intactness and non-fine-tunability. Nonetheless, the problem in Equation (2) is still hard to solve since the fine-tuning process ϕitalic-ϕφϕ is usually iterative and does not have a closed-form solution. Thus, we propose Sophon to derive an approximate solution of Equation (2). 4 Sophon: Design Details As shown in Figure 2, Sophon consists of two key optimization modules, i.e., fine-tuning suppression in the restricted domain and normal training reinforcement in the original domain. The fine-tuning suppression module is designed to degrade the fine-tuning performance in the restricted domain in simulated fine-tuning processes. The normal training reinforcement module is designed to maintain the performance in the original domain. 4.1 Fine-Tuning Suppression in Restricted Domain Fine-tuning suppression in the restricted domain corresponds to the first term in Equation (2), which is hard to optimize for two reasons. First, the fine-tuning process ϕ⁢(f⁢(x))italic-ϕφ(f(x))ϕ ( f ( x ) ) is an optimization problem itself, and does not have a closed-form solution. Thus, ℒ⁢(ϕ⁢(fθ⁢(x)))ℒitalic-ϕsubscriptL(φ(f_θ(x)))L ( ϕ ( fitalic_θ ( x ) ) ) cannot be computed directly. Second, to the best of our knowledge, there is no existing closed-form metric that reliably implies the performance of a model after fine-tuning, i.e., a metric that positively correlates to ℒ⁢(ϕ⁢(fθ⁢(x)))ℒitalic-ϕsubscriptL(φ(f_θ(x)))L ( ϕ ( fitalic_θ ( x ) ) ). Thus, there is no feedback signal for optimization. Inspired by the model-agnostic meta-learning (MAML) framework [8], we design simulated fine-tuning processes to approximate the model performance after fine-tuning, which serves as the feedback for optimization. 4.1.1 Fine-Tuning Simulation We denote a fine-tuning environment as a triplet (ϕi,ℛi,i)subscriptitalic-ϕsubscriptℛsubscript ( _i,R_i,T_i )( ϕitalic_i , Ritalic_i , Titalic_i ), where ϕi⁢(⋅)∼Φsimilar-tosubscriptitalic-ϕ⋅Φ _i(·) ϕitalic_i ( ⋅ ) ∼ Φ is a fine-tuning strategy, ℛisubscriptℛR_iRitalic_i is the dataset used to fine-tune model fθsubscriptf_θfitalic_θ, and isubscriptT_iTitalic_i is the dataset used to evaluate the performance of model fϑksuperscriptsubscriptitalic-ϑf_ ^kfitalic_ϑitalic_k after the k-th round of fine-tuning. Both ℛisubscriptℛR_iRitalic_i and isubscriptT_iTitalic_i are sampled from the restricted domain. At the k-th round, the fine-tuning process is essentially an optimization problem. maxϑx∼ℛi⁢ℒ⁢(fϑk⁢(x|ϕi,fϑk−1)),subscriptitalic-ϑsubscriptsimilar-tosubscriptℛℒsuperscriptsubscriptitalic-ϑconditionalsubscriptitalic-ϕsuperscriptsubscriptitalic-ϑ1 _ \ E_x _i\ % L (f_ ^k (x| _i,f_ ^k-1 ) ),maxitalic_ϑ blackboard_Ex ∼ R start_POSTSUBSCRIPT i end_POSTSUBSCRIPT L ( fitalic_ϑitalic_k ( x | ϕitalic_i , fitalic_ϑitalic_k - 1 ) ) , (3) where fϑksuperscriptsubscriptitalic-ϑf_ ^kfitalic_ϑitalic_k is the fine-tuned version of fϑk−1superscriptsubscriptitalic-ϑ1f_ ^k-1fitalic_ϑitalic_k - 1. Note that fϑ0=fθsuperscriptsubscriptitalic-ϑ0subscriptf_ ^0=f_θfitalic_ϑ0 = fitalic_θ. The fine-tuning process may not converge, i.e., K→∞→K→∞K → ∞, where K is the total number of fine-tuning rounds. Thus, we set an upper-bound on K as a relatively large integer to approximate the point of convergence, e.g., 50 in our experiments. We consider the strongest adversary with the (potentially) best fine-tuning strategy to try to obtain a good performance in the restricted domain. Therefore, we materialize ϕisubscriptitalic-ϕ _iϕitalic_i as follows. For model initialization, we choose two strategies. The first one uses the entire pre-trained model, and the second one uses the entire pre-trained model except for the final layer. For transfer strategy, we choose to update the whole model, and for the optimization strategy, we use the Adam optimizer. 4.1.2 Fine-tuning Evaluation The fine-tuning is simulated on behalf of the adversary and the defender aims to suppress the fine-tuning performance. Therefore, we propose to use the aggregate error of all fine-tuned models generated at every iteration by every fine-tuning strategy on the test dataset to evaluate potential fine-tuning performance of a pre-trained model fθsubscriptf_θfitalic_θ. The first term of Equation (2), defined as ℒFTSsubscriptℒFTSL_FTSLroman_FTS, is expressed as ℒFTS=∑i=1N∑k=1Kγi,k⋅ℒα⁢(fϑk|ϕi,i),subscriptℒFTSsuperscriptsubscript1superscriptsubscript1⋅subscriptsubscriptℒconditionalsuperscriptsubscriptitalic-ϑsubscriptitalic-ϕsubscript _FTS= _i=1^N _k=1^K\, _% i,k·L_α (f_ ^k| _i,T_i% ),Lroman_FTS = ∑i = 1N ∑k = 1K γitalic_i , k ⋅ Litalic_α ( fitalic_ϑitalic_k | ϕitalic_i , Titalic_i ) , (4) where ℒα⁢(fϑk|ϕi,i)subscriptℒconditionalsuperscriptsubscriptitalic-ϑsubscriptitalic-ϕsubscriptL_α (f_ ^k| _i,T_i )Litalic_α ( fitalic_ϑitalic_k | ϕitalic_i , Titalic_i ) evaluate the performance of the protected pre-trained model fine-tuned by ϕisubscriptitalic-ϕ _iϕitalic_i after k iterations on the test dataset isubscriptT_iTitalic_i, and γi,ksubscript _i,kγitalic_i , k is the weight. The protected pre-trained model is updated according to ℒFTSsubscriptℒFTSL_FTSLroman_FTS as θ←θ±α⋅∇θℒFTS,←plus-or-minus⋅subscript∇subscriptℒFTS θ←θ±α· _θL_% FTS,θ ← θ ± α ⋅ ∇θ Lroman_FTS , (5) where α denotes the learning rate. The sign of Equation (5) is determined by the design of the loss function ℒαsubscriptℒL_αLitalic_α, which we elaborate in §4.3. Note that Equation (5) involves second-order gradient terms since fϑksuperscriptsubscriptitalic-ϑf_ ^kfitalic_ϑitalic_k is also optimized by gradient descent. To reduce computational complexity, we use the first-order approximation of Equation (5) [8] and update θ using an Adam optimizer [20]. Input: The original domain SS, the restricted domain AA, the fine-tuning setting domain Φ Φ. Input: The original model parameter θ0subscript0 _0θ0, FTS learning rate α, NTR learning rate β, the number of FTS loops ℓFTSsubscriptℓFTS _FTSℓroman_FTS, the number of NTR loops ℓNTRsubscriptℓNTR _NTRℓroman_NTR, the number of iterations IterIterIterIter. 1 Output: The non-fine-tunable model θ 2 Initialize θ←θ0←subscript0θ← _0θ ← θ0. 3for 1111 to IterIterIterIter do 4 5 # Fine-tuning suppression in the restricted domain 6 for 1111 to ℓFTSsubscriptℓFTS _FTSℓroman_FTS do 7 8 for i←1←1i← 1i ← 1 to N do 9 Sample one fine-tuning setting ϕi∼Φsimilar-tosubscriptitalic-ϕΦ _i ϕitalic_i ∼ Φ 10 Sample K batches of ℛi,i∼similar-tosubscriptℛsubscriptR_i,T_i _i , Titalic_i ∼ A. 11 for k←1←1k← 1k ← 1 to K do 12 13 Fine-tune fϑk←fϑk⁢(x|ϕi,fϑk−1)←superscriptsubscriptitalic-ϑsuperscriptsubscriptitalic-ϑconditionalsubscriptitalic-ϕsuperscriptsubscriptitalic-ϑ1f_ ^k← f_ ^k (x| _i,f_ ^k% -1 )fitalic_ϑitalic_k ← fitalic_ϑitalic_k ( x | ϕitalic_i , fitalic_ϑitalic_k - 1 ) 14 Compute ℒi,k←ℒα⁢(fϑk|ϕi,i)←subscriptℒsubscriptℒconditionalsuperscriptsubscriptitalic-ϑsubscriptitalic-ϕsubscriptL_i,k _α (f_ ^k| _i% ,T_i )Litalic_i , k ← Litalic_α ( fitalic_ϑitalic_k | ϕitalic_i , Titalic_i ). 15 end for 16 17 end for 18 ℒFTS←∑i=1N∑k=1Kγi,k⋅ℒi,k←subscriptℒFTSsuperscriptsubscript1superscriptsubscript1⋅subscriptsubscriptℒL_FTS← _i=1^N _k=1^K\, _i,k% ·L_i,kLroman_FTS ← ∑i = 1N ∑k = 1K γitalic_i , k ⋅ Litalic_i , k 19 Update θ←absentθ←θ ← Adam(θ, ∇θℒFTSsubscript∇subscriptℒFTS _θL_FTS∇θ Lroman_FTS, α) 20 end for 21 22 # Normal training reinforcement in the original domain 23 for 1111 to ℓNTRsubscriptℓNTR _NTRℓroman_NTR do 24 25 Sample a batch of ∼similar-toO ∼ S. 26 Compute ℒNTR←ℒβ⁢(fθ|),←subscriptℒNTRsubscriptℒconditionalsubscriptL_NTR _β (f_θ|% O ),Lroman_NTR ← Litalic_β ( fitalic_θ | O ) , 27 Update θ←absentθ←θ ← Adam(θ, ∇θℒNTRsubscript∇subscriptℒNTR _θL_NTR∇θ Lroman_NTR, β) 28 end for 29 30 end for 31 Algorithm 1 Sophon 4.2 Normal Training Reinforcement in Original Domain The fine-tuning suppression module may affect the performance of the pre-trained model in the original domain. Therefore, we carry out normal training reinforcement to maintain the performance in the original domain, corresponding to the second term of Equation (2), which is expressed as ℒNTR=ℒβ⁢(fθ|),subscriptℒNTRsubscriptℒconditionalsubscript _NTR=L_β (f_θ|% O ),Lroman_NTR = Litalic_β ( fitalic_θ | O ) , (6) where ℒβsubscriptℒL_βLitalic_β is the loss function used to measure the performance of protected pre-trained model in the original domain, and OO is a training dataset sampled from the original domain. The protected pre-trained model is updated according to ℒNTRsubscriptℒNTRL_NTRLroman_NTR as θ←θ−β⋅∇θℒNTR,←⋅subscript∇subscriptℒNTR θ←θ-β· _θL_% NTR,θ ← θ - β ⋅ ∇θ Lroman_NTR , (7) where β denotes the learning rate. Equation (7) is also updated using an Adam optimizer [20]. We summarize the detailed process of solving Equation (2) in Algorithm 1. 4.3 Instantiation of Sophon We instantiate model f in Equation (2) as a classification model and a generative model as follows. 4.3.1 Classification For classification tasks, the loss function ℒβ⁢(⋅)subscriptℒ⋅L_β(·)Litalic_β ( ⋅ ) for normal training reinforcement can take the form of the traditional cross-entropy loss. However, using cross-entropy loss for the loss function ℒα⁢(⋅)subscriptℒ⋅L_α(·)Litalic_α ( ⋅ ) of fine-tuning suppression is problematic due to the following reason. For the gradient descent process in Equation (5) to converge stably, the gradient term ∇ℒFTS∇subscriptℒFTS _FTS∇ Lroman_FTS should decrease with the update iterations. Therefore, according to Equation (4), ∇ℒα∇subscriptℒ _α∇ Litalic_α should decrease with the update iterations. Given our objective to amplify the error in the restricted domain with the update iterations, it is expected that the gradient ∇ℒα∇subscriptℒ _α∇ Litalic_α should decrease as the error increases. This behavior is inherently linked to the nature of loss functions. We theoretically analyze the nature of the cross-entropy loss, inspired by which we design two alternative loss functions. As mentioned in §2.1, the output of a classifier can be represented as, (y^1,y^2,⋯,y^C)=σ⁢(z1,z2,⋯,zC),subscript^1subscript^2⋯subscript^subscript1subscript2⋯subscript ( y_1, y_2,·s, y_C )=σ% (z_1,z_2,·s,z_C ),( over start_ARG y end_ARG1 , over start_ARG y end_ARG2 , ⋯ , over start_ARG y end_ARGC ) = σ ( z1 , z2 , ⋯ , zitalic_C ) , (8) where zisubscriptz_izitalic_i is the i-th logit, y^i=exp⁡(zi)/∑j=1Cexp⁡(zj)subscript^subscriptsuperscriptsubscript1subscript y_i= (z_i)/ _j=1^C (z_j)over start_ARG y end_ARGi = exp ( zitalic_i ) / ∑j = 1C exp ( zitalic_j ) is the predicted probability of the i-th label, C is the total number of classes. Then, we have the following partial derivatives, ∂y^j∂zi=y^i⁢(1−y^i),i=j,−y^i⁢y^j,i≠j.subscript^subscriptcasessubscript^1subscript^subscript^subscript ∂ y_j∂ z_i= \ array[]rl y_% i(1- y_i),&i=j,\\ - y_i y_j,&i≠ j. array .divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG = start_ARRAY start_ROW start_CELL over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) , end_CELL start_CELL i = j , end_CELL end_ROW start_ROW start_CELL - over start_ARG y end_ARGi over start_ARG y end_ARGj , end_CELL start_CELL i ≠ j . end_CELL end_ROW end_ARRAY (9) Then, we can derive the gradient of the cross-entropy loss with respect to zisubscriptz_izitalic_i by the chain rule, ∂ℒCE∂zi=−∑j=1C∂(yj⁢log⁡y^j)∂y^j⋅∂y^j∂zi=y^i−yi.subscriptℒCEsubscriptsuperscriptsubscript1⋅subscriptsubscript^subscript^subscript^subscriptsubscript^subscript _CE∂ z_i=- _j% =1^C ∂ (y_j y_j )∂ y_j% · ∂ y_j∂ z_i= y_i-y_i.divide start_ARG ∂ Lroman_CE end_ARG start_ARG ∂ zitalic_i end_ARG = - ∑j = 1C divide start_ARG ∂ ( yitalic_j log over start_ARG y end_ARGj ) end_ARG start_ARG ∂ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG = over start_ARG y end_ARGi - yitalic_i . (10) The magnitude of the gradient in Equation (10) is positively correlated with the error. In this case, the gradient in Equation (5) will increase with update iterations, thus the optimization process will easily diverge. To address this problem, we propose two alternatives for the loss function ℒαsubscriptℒL_αLitalic_α. Inverse Cross-Entropy (ICE). We modify the cross-entropy loss into, ℒICE⁢(f)=−1||⁢∑i∑j=1Cyi⁢j⁢log⁡(1−y^i⁢j).subscriptℒICE1subscriptsuperscriptsubscript1subscript1subscript _ICE(f)=- 1|X| _i% _j=1^Cy_ij (1- y_ij ).Lroman_ICE ( f ) = - divide start_ARG 1 end_ARG start_ARG | X | end_ARG ∑i ∑j = 1C yitalic_i j log ( 1 - over start_ARG y end_ARGi j ) . (11) where y^i⁢jsubscript y_ijover start_ARG y end_ARGi j is the j-th term of the prediction vector yi=f⁢(xi)subscriptsubscripty_i=f(x_i)yitalic_i = f ( xitalic_i ). Since increasing −log⁡(y^i⁢j)subscript^- ( y_ij )- log ( over start_ARG y end_ARGi j ) is equivalent to decreasing −log⁡(1−y^i⁢j)1subscript^- (1- y_ij )- log ( 1 - over start_ARG y end_ARGi j ), we can use the ICE loss to degrade the model performance in the restricted domain. Without loss of generality, we assume y1=1subscript11y_1=1y1 = 1. Then the gradient of the ICE loss with respect to zisubscriptz_izitalic_i is, ∂ℒICE∂zisubscriptℒICEsubscript _ICE∂ z_idivide start_ARG ∂ Lroman_ICE end_ARG start_ARG ∂ zitalic_i end_ARG =y^1,i=1,−y^i⁢y^1/(1−y^1),i≠1.absentcasessubscript^11subscript^subscript^11subscript^11 = \ array[]rl y_1,&i=1,\\ - y_i y_1/ (1- y_1 ),&i≠ 1. array .= start_ARRAY start_ROW start_CELL over start_ARG y end_ARG1 , end_CELL start_CELL i = 1 , end_CELL end_ROW start_ROW start_CELL - over start_ARG y end_ARGi over start_ARG y end_ARG1 / ( 1 - over start_ARG y end_ARG1 ) , end_CELL start_CELL i ≠ 1 . end_CELL end_ROW end_ARRAY (12) From Equation (12), we know that the magnitude of gradient decreases as y^1subscript^1 y_1over start_ARG y end_ARG1 decreases, i.e., limy^1→0|∂ℒICE∂zi|=0.subscript→subscript^10subscriptℒICEsubscript0 _ y_1→ 0 | _ICE% ∂ z_i |=0.limover start_ARG y end_ARG start_POSTSUBSCRIPT 1 → 0 end_POSTSUBSCRIPT | divide start_ARG ∂ Lroman_ICE end_ARG start_ARG ∂ zitalic_i end_ARG | = 0 . Therefore, the proposed inverse cross-entropy loss can better boost the convergence of fine-tuning suppression process. KL Divergence from Uniform Distribution (KLU). The Kullback–Leibler (KL) divergence [23] measures the difference between two probability distributions. We leverage the KL divergence to force the output distribution of the model to be a uniform distribution, i.e., ℒKLU⁢(f)=−1||⁢∑i∑j=1C1C⁢log⁡(C⋅y^i⁢j).subscriptℒKLU1subscriptsuperscriptsubscript11⋅subscript _KLU(f)=- 1|X| _i% _j=1^C 1C (C· y_ij ).Lroman_KLU ( f ) = - divide start_ARG 1 end_ARG start_ARG | X | end_ARG ∑i ∑j = 1C divide start_ARG 1 end_ARG start_ARG C end_ARG log ( C ⋅ over start_ARG y end_ARGi j ) . (13) The gradient of the KLU loss with respect to zisubscriptz_izitalic_i is, ∂ℒKLU∂zisubscriptℒKLUsubscript _KLU∂ z_idivide start_ARG ∂ Lroman_KLU end_ARG start_ARG ∂ zitalic_i end_ARG =−∑j=1C∂(1C⁢log⁡(C⋅y^j))∂y^j⋅∂y^j∂zi=y^i−1C.absentsuperscriptsubscript1⋅1⋅subscript^subscript^subscript^subscriptsubscript^1 =- _j=1^C ∂ ( 1C (C·% y_j ) )∂ y_j· ∂ y_j% ∂ z_i= y_i- 1C.= - ∑j = 1C divide start_ARG ∂ ( divide start_ARG 1 end_ARG start_ARG C end_ARG log ( C ⋅ over start_ARG y end_ARGj ) ) end_ARG start_ARG ∂ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG = over start_ARG y end_ARGi - divide start_ARG 1 end_ARG start_ARG C end_ARG . (14) The magnitude of gradient decreases to zero when y^i→1C→subscript^1 y_i→ 1Cover start_ARG y end_ARGi → divide start_ARG 1 end_ARG start_ARG C end_ARG, which also provides better convergence performance than the CE loss. We provide a detailed derivation of these gradients in the Appendix A.1∼similar-to ∼A.4. The Difference between ICE and KLU. From Equation (11) and Equation (13), we can see that the ICE loss requires labelled data while the KLU loss does not. More specifically, the computation of the ICE loss involves both the data sample xisubscriptx_ixitalic_i and the classification label yisubscripty_iyitalic_i from the restricted domain, while the computation of the KLU loss involves only the data sample xisubscriptx_ixitalic_i from the restricted domain. This means that KLU suppresses any classification tasks regarding data samples in the restricted domain, but ICE suppresses only a certain classification task (specified by the classification label) in the restricted domain. For example, we want to suppress racial profiling task based on facial images. In this case, given facial images of data samples and corresponding labels, ICE suppresses only the racial classification task, and can preserve the performance on other classification tasks, e.g., gender identification. In contrast, KLU only depends on the data samples, and will suppress all classification tasks based on facial images. In this sense, KLU provides a more indiscriminate suppression compared with ICE. We evaluate the effectiveness and stability of these two loss functions in §5.8. 4.3.2 Generation For diffusion models, we use the MSE loss ℒMSEsubscriptℒMSEL_MSELroman_MSE for the normal training reinforcement loss ℒβsubscriptℒL_βLitalic_β. Similarly, using ℒMSEsubscriptℒMSEL_MSELroman_MSE for ℒαsubscriptℒL_αLitalic_α will also be problematic, ∂ℒMSE∂ϵ^isubscriptℒMSEsubscript^italic-ϵ _MSE∂ ε_% idivide start_ARG ∂ Lroman_MSE end_ARG start_ARG ∂ over start_ARG ϵ end_ARGi end_ARG =∂(∑j(ϵj−ϵ^j)2)∂ϵ^i=2⁢(ϵ^i−ϵi).absentsubscriptsuperscriptsubscriptitalic-ϵsubscript^italic-ϵ2subscript^italic-ϵ2subscript^italic-ϵsubscriptitalic-ϵ = ∂ ( _j ( _j- ε_j% )^2 )∂ ε_i=2 ( ε_i-% _i ).= divide start_ARG ∂ ( ∑j ( ϵitalic_j - over start_ARG ϵ end_ARGj )2 ) end_ARG start_ARG ∂ over start_ARG ϵ end_ARGi end_ARG = 2 ( over start_ARG ϵ end_ARGi - ϵitalic_i ) . (15) Therefore, we design a Denial of Service (DoS) loss, ℒDoS⁢(f)=1||⁢∑i‖f⁢(xt⁢i,ti)‖2.subscriptℒDoS1subscriptsuperscriptnormsubscriptsubscript2 _DoS(f)= 1|X| _i\|f(% x_ti,t_i)\|^2.Lroman_DoS ( f ) = divide start_ARG 1 end_ARG start_ARG | X | end_ARG ∑i ∥ f ( xitalic_t i , titalic_i ) ∥2 . (16) The gradient of the DoS loss with respect to ϵ^isubscript^italic-ϵ ε_iover start_ARG ϵ end_ARGi is ∂ℒDoS∂ϵ^i=∂(∑j(ϵ^j)2)∂ϵ^i=2⁢ϵ^isubscriptℒDoSsubscript^italic-ϵsubscriptsuperscriptsubscript^italic-ϵ2subscript^italic-ϵ2subscript^italic-ϵ _DoS∂ ε_i= % ∂ ( _j ( ε_j )^2 )∂ % ε_i=2 ε_idivide start_ARG ∂ Lroman_DoS end_ARG start_ARG ∂ over start_ARG ϵ end_ARGi end_ARG = divide start_ARG ∂ ( ∑j ( over start_ARG ϵ end_ARGj )2 ) end_ARG start_ARG ∂ over start_ARG ϵ end_ARGi end_ARG = 2 over start_ARG ϵ end_ARGi. The magnitude of this gradient decreases to zeros when ϵ^i→0→subscript^italic-ϵ0 ε_i→ 0over start_ARG ϵ end_ARGi → 0, which provides better convergence than the MSE loss. Therefore, we use ℒDoSsubscriptℒDoSL_DoSLroman_DoS as the loss for ℒαsubscriptℒL_αLitalic_α. 5 Evaluation Figure 3: Effectiveness of Sophon compared with three baselines. Fine-tuning the original or the NTL model can achieve a high accuracy, leading to model misuse. All of the three methods of fine-tuning the Sophon model yield poorer performances than training the model from scratch. Figure 4: Effectiveness of Sophon compared with two baselines. Fine-tuning the original model can achieve low losses, leading to model misuse. Fine-tuning the Sophon model yields higher losses than training the model from scratch. [t] Model Fine-tuning Strategy† Original ACC (↑ ↑) ACC in the Restricted Domain (↓ ↓) Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 Train from scratch - - - 49.1±plus-or-minus±1.2 59.5±plus-or-minus±0.7 60.6±plus-or-minus±0.5 62.0±plus-or-minus±0.5 62.7±plus-or-minus±0.5 Original Model direct+all 99.6 9.3 42.8±plus-or-minus±4.0 78.5±plus-or-minus±0.4 82.4±plus-or-minus±0.3 84.2±plus-or-minus±0.2 84.8±plus-or-minus±0.2 initFC+FC 13.5±plus-or-minus±0.8 28.5±plus-or-minus±0.3 37.3±plus-or-minus±0.2 41.8±plus-or-minus±0.2 44.5±plus-or-minus±0.2 initFC+all 43.1±plus-or-minus±6.7 79.1±plus-or-minus±1.1 83.4±plus-or-minus±0.4 84.4±plus-or-minus±0.5 85.1±plus-or-minus±0.2 NTL Model direct+all 90.2 10.3 11.7±plus-or-minus±0.9 72.5±plus-or-minus±1.6 81.4±plus-or-minus±0.5 83.6±plus-or-minus±0.3 84.4±plus-or-minus±0.4 initFC+FC 19.7±plus-or-minus±0.4 42.6±plus-or-minus±0.5 50.9±plus-or-minus±0.3 54.3±plus-or-minus±0.2 56.2±plus-or-minus±0.1 initFC+all 38.1±plus-or-minus±7.5 78.5±plus-or-minus±1.8 84.1±plus-or-minus±0.9 86.1±plus-or-minus±0.6 86.7±plus-or-minus±0.6 Sophon Model direct+all 96.2 10.0 10.0±plus-or-minus±0.0 10.5±plus-or-minus±0.8 13.0±plus-or-minus±1.7 15.0±plus-or-minus±1.1 15.2±plus-or-minus±0.6 initFC+FC 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.1±plus-or-minus±1.1 10.4±plus-or-minus±1.4 initFC+all 10.0±plus-or-minus±0.0 10.4±plus-or-minus±1.2 11.1±plus-or-minus±3.6 10.9±plus-or-minus±1.4 13.3±plus-or-minus±3.1 † “direct”: use the entire pre-trained model. “initFC”: use the entire pre-trained model but randomly initialize the last FC layer. “all”: fine-tune the whole model. “FC”: only fine-tune the last FC layer. TABLE I: The overall effectiveness of Sophon compared with three baselines (classification). [t] Model Fine-tuning Strategy† Original Loss (↓ ↓) Loss in the Restricted Domain (↑ ↑) Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 Train from scratch - - - 0.614±plus-or-minus±4.6e-3 0.511±plus-or-minus±2.4e-3 0.490±plus-or-minus±1.5e-3 0.483±plus-or-minus±1.3e-3 0.479±plus-or-minus±1.1e-4 Original Model direct+all 0.478 0.477 0.457±plus-or-minus±0.3e-4 0.449±plus-or-minus±0.3e-4 0.447±plus-or-minus±0.1e-4 0.445±plus-or-minus±0.5e-4 0.445±plus-or-minus±0.5e-4 Sophon Model direct+all 0.500 0.990 0.913±plus-or-minus±0.9e-4 0.892±plus-or-minus±1.8e-4 0.842±plus-or-minus±1.1e-4 0.750±plus-or-minus±0.2e-4 0.705±plus-or-minus±0.7e-4 † The usual practice for fine-tuning generative models is to directly fine-tune the whole model. TABLE I: The overall effectiveness of Sophon compared with two baselines (generation). 5.1 Setup 5.1.1 Prototype We have implemented a prototype of Sophon on the PyTorch [32] platform and processed the models according to Equation (2) and Algorithm 1 using three NVIDIA A100 (80G) GPUs. We set the default Sophon configurations as α=3×10−43superscript104α=3× 10^-4α = 3 × 10- 4, β=5×10−45superscript104β=5× 10^-4β = 5 × 10- 4, Iter=800Iter800Iter=800Iter = 800, K=5050K=50K = 50, ℓFTS=1subscriptℓFTS1 _FTS=1ℓroman_FTS = 1, and ℓNTR=1subscriptℓNTR1 _NTR=1ℓroman_NTR = 1. One fine-tuning task per GPU is sampled in one FTS loop. We construct the tasks by randomly sampling the learning rates from [10−6,10−5,10−4,10−3,10−2]superscript106superscript105superscript104superscript103superscript102[10^-6,10^-5,10^-4,10^-3,10^-2][ 10- 6 , 10- 5 , 10- 4 , 10- 3 , 10- 2 ] and the batch sizes from [50,100,150,200,250]50100150200250[50,100,150,200,250][ 50 , 100 , 150 , 200 , 250 ], which cover the commonly-used range of learning rates and batch sizes. We always use the Adam optimizer in the fine-tuning simulation to simulate a powerful adversary. The default fine-tuning strategy for evaluation is using the Momentum optimizer with a learning rate of 10−4superscript10410^-410- 4, a batch size of 200, and a weight decay rate of 10−4superscript10410^-410- 4 to directly fine-tune the whole model, unless specified otherwise. 5.1.2 Baselines We compare Sophon with three baselines to evaluate its effectiveness, i.e., • Train from scratch (B1). We train a model from scratch on the data sampled from the restricted domain as the first baseline. Note that, if the overhead of fine-tuning the pre-trained model is greater than training the model from scratch, then the adversary will have no incentive to fine-tune the pre-trained model. • Fine-tune the original model (B2). We fine-tune the original pre-trained model as our second baseline. Generally speaking, fine-tuning a pre-trained model can leverage the transferable knowledge and thus perform better than B1 in terms of the final performance and the time overhead. As mentioned in our design goal (§2.3), we want the processed model to perform much worse than B2. • Fine-tune the NTL model (B3). We reproduce the recent non-transferable learning [50] method as the third baseline for classification, for which it is specifically designed. NTL is proposed to degrade the model performance in the restricted domain. But the performance after fine-tuning is not considered. We demonstrate that NTL cannot resist fine-tuning via experiments in §5.2. 5.1.3 Metrics We adopt two metrics to evaluate the effectiveness of Sophon, i.e., • ACC-k. The test accuracy at the k-th iteration of fine-tuning or training from scratch, which is a common metric for classification problems. We want the protected model to have a low ACC in the restricted domain, even after being fine-tuned. • MSE-k. The mean squared error at the k-th iteration of fine-tuning or training from scratch, which is the training objective of the diffusion model. We want the protected model to have a high MSE in the restricted domain, even after being fine-tuned. We repeat each experiment five times with different random seeds and include the standard deviation for each outcome metric to mitigate the influence of randomness on the results. The results are presented as M±S⁢Dplus-or-minusM± SDM ± S D, where M denotes the mean value and S⁢DSDS D denotes the standard deviation. Figure 5: Effectiveness of Sophon compared with two baselines. B1 and B2 both perform well in the restricted domain (CelebA) in terms of the denoising ability. The Sophon model cannot denoise images from the restricted domain, thus is protected. Figure 6: Intactness of Sophon. The Sophon model performs well in the original domain (CIFAR-100) and has the denoising ability similar to the original model. 5.1.4 Datasets Our experiments involve eight datasets, i.e., • ImageNette. ImageNette [7] is a subset of 10 relatively simple classes from ImageNet. There are 12,894 training images and 500 test images. We process the images into 64×64. Our pre-trained classification model is originally trained on this dataset. • CIFAR-10, CIFAR-100. CIFAR-10 [22] consists of 32×32 color images in 10 classes of animals and vehicles. There are 50,000 training images and 10,000 test images. CIFAR-100 is just like the CIFAR-10, except it has 100 classes. Our pre-trained generative model is originally trained on CIFAR-100. • CINIC. CINIC-10 [4] consists of 32×32 color images in 10 classes. It is an extension of CIFAR-10 via the addition of downsampled ImageNet images. There are 90,000 training images and 90,000 test images. • STL. STL-10 [3] consists of 96×96 color images in 10 classes of animals and vehicles like CIFAR-10. There are 5,000 training images and 8,000 test images. • MNIST. MNIST [24] consists of 28×28 grayscale images in 10 classes (from “0” to “9”). There are 60,000 training images and 10,000 test images. • SVHN. SVHN [31] consists of 32×32 color images in 10 classes (from “0” to “9”). There are 73,257 training images and 26,032 test images. • CelebA. CelebA [27] is a large-scale face dataset with ∼similar-to ∼200,000 celebrity images of 218×178. We split the dataset into 160,000 training images and 40,000 test images. CelebA is used as the restricted domain of the generative model in our experiments. • FFHQ. FFHQ [19] consists of 1,024×1,024 facial images with considerable variation in terms of age, ethnicity and image background. We use 8,000 images in FFHQ for training and 2,000 for testing. FFHQ is also used as the restricted domain of the generative model in our experiments. For those datasets that are used as the restricted domain, we split the training set of each dataset equally into two parts, one for Sophon training or NTL (B3) training, the other one for the fine-tuning in the evaluation. This simulates the defender and the adversary obtaining different training data sampled from the same restricted domain. Figure 7: Effectiveness of Sophon against different restricted domains. Fine-tuning the Sophon model for 20 epochs (5,000 iterations) cannot increase the test accuracy (e.g., from 10.0% to 14.3% for CIFAR-10), which means Sophon can restrict the applicability of models in different domains. Figure 8: Effectiveness of Sophon on different model architectures. Fine-tuning the Sophon models of different architectures for 20 epochs (5,000 iterations) only increases the test accuracy from 10.0% to 18.0% (VGG, the worst case), which means Sophon achieving satisfying performances on model architectures of different model sizes. 5.2 Overall Effectiveness 5.2.1 Classification In this part, we evaluate the overall effectiveness of Sophon on classification tasks. We pre-train the CAFormer model on ImageNette and try to restrict its application on CIFAR-10. We consider three kinds of fine-tuning strategies, i.e., directly fine-tuning all the parameters (“direct+all”), randomly initializing the last FC layer and fine-tuning it (“initFC+FC”), and randomly initializing the last FC layer and fine-tuning all the parameters (“initFC+all”). We fine-tune the baseline models and our model for a total of 20202020 epochs (20×250=5,00020250500020× 250=5,00020 × 250 = 5 , 000 iterations in total). For the classification task, we compare our method with the three baselines mentioned in §5.1.2. We test the accuracy of the models every epoch and present the results in Figure 3 and Table I. We can see that by training the model from scratch, the model finally reaches an ACC of 62.7% at 5,000 iterations. Fine-tuning the original model and the NTL model using the “direct+all” or “initFC+all” strategies can achieve much higher ACCs of 84.4%∼similar-to ∼86.7%, which shows the superior benefit brought by pre-training. This also verifies that existing NTL methods cannot resist supervised fine-tuning since they actually do not include a fine-tune-related objective in their optimization problem. Fine-tuning the models using the “initFC+FC” strategy can only obtain performances close to training from scratch. This is aligned with the findings of a previous study [52]. Compared with the baselines, the Sophon model resists fine-tuning and finally yields a far lower ACC of 10.4%∼similar-to ∼15.2%, which is comparable to a random guess. The results show the effectiveness of our method that it turns a pre-trained model into a non-fine-tunable model against a specific restricted domain, while preserving a high ACC in the original domain of 96.2%. In other words, fine-tuning the Sophon models is much harder than fine-tuning the original models and even harder than training from scratch, which means the adversary will have no incentive to abuse the protected pre-trained models. 5.2.2 Generation In this part, we evaluate the effectiveness of Sophon on generation tasks. We pre-train the diffusion model on CIFAR-100 and try to restrict its application on CelebA. In other words, we do not want the model to be used for generating fake faces. We adopt the “direct+all” fine-tuning strategy since the diffusion model is typically a U-Net model that lacks an explicit component to randomly initialize. The results comparing Sophon with B1 and B2 are shown in Figure 4 and Table I. Directly fine-tuning the unprotected model can easily yield a very low MSE, e.g., 0.457 in one epoch. By training from scratch, a relatively satisfactory result is obtained in 10 epochs with an MSE of 0.490, a bit worse than B1. In contrast, fine-tuning the Sophon model only achieves an MSE of 0.705 in even 20 epochs, which shows great effectiveness of Sophon on generation tasks. We also present the images generated by the two baselines and our method in Figure 5 and Figure 6. The images are added noises in the forward process, and are denoised by the models. From Figure 5, we can see that fine-tuning the original model in the restricted domain yields very good performance in the restricted domain. Training the model from scratch in the restricted domain also achieve relatively good results even though the results are not as good as fine-tuning the original model. In comparison, the diffusion model fine-tuned from the Sophon model almost has no ability to denoise facial images, verifying the non-fine-tunability in the restricted domain. Note that, Figure 6 shows that the Sophon model actually performs well in the original domain in terms of the denoising quality, which means Sophon achieves intactness. Figure 9: Effectiveness of Sophon against different optimizers. Momentum or Nesterov does not increase the accuracy. Adagrad, Adadelta, and Adam, may increase the accuracy in the restricted domain, but Sophon still renders the model more challenging to train compared to training from scratch, greatly slowing down the convergence. The results verify that although we only use Adam in the NFT loops, the resistance generalizes well to other optimizers. Figure 10: Effectiveness of Sophon against different learning rates. The values 10−4superscript10410^-410- 4 and 10−3superscript10310^-310- 3 are both suitable as learning rates. Even under such suitable learning rates, the Sophon model still resists fine-tuning, showing robustness against different learning rates. 5.3 Restricted Domain In this part, we evaluate the effectiveness of Sophon against different restricted domains. For classification, we pre-train the CAFormer model on ImageNette and try to restrict its application on object classification datasets (CIFAR-10, CINIC, and STL) and digit classification datasets (MNIST and SVHN). For generation, we pre-train the diffusion model on CIFAR-100 and restrict its application on two image datasets (CelebA and FFHQ). We compare our method with the B1 and B2 baselines and present the classification results in Figure 7 and Table I in Appendix. The three models trained from scratch on CIFAR-10, CINIC, and STL only achieve ACCs of 62.7%, 51.3%, and 46.3%, respectively. These results align with the expected difficulty levels of the three tasks. Fine-tuning the original models achieves 84.9%, 74.4%, and 71.7%, much higher than training from scratch. Since MNIST and SVHN are relatively easier, both B1 and B2 of MNIST and SVHN achieve high ACCs. Note that B2 still outperforms B1, which is the gain from pre-training. Compared with the baselines, fine-tuning the Sophon models only results in performance comparable to random guessing, i.e., 8.8%∼similar-to ∼19.6%. For example, fine-tuning the Sophon model on CIFAR-10 for 20 epochs only increases the ACC from 10.0% to 14.3%. As we can see, Sophon can restrict the applicability of models in different domains. Meanwhile, the Sophon models still maintain good performance in the original domain, i.e., 96.2%∼similar-to ∼97.8%. Similar conclusions can be inferred from the results of generation tasks depicted in Figure 4 and Figure 12 in Appendix. 5.4 Model Architecture In this part, we evaluate the effectiveness of Sophon on different model architectures. We pre-train five models (CAFormer, ResNet-50, ResNet-34, ResNet-18, and VGG) on ImageNette and try to restrict their application on CIFAR-10. As shown in Figure 8 and Table IV in Appendix, we can still observe that the Sophon models resist fine-tuning and can only result in low ACCs. Note that for five model architectures, Sophon achieves ACCs in the original domain of 96.2%, 95.6%, 94.8%, 95.0%, and 91.2% on CAFormer, ResNet-50, ResNet-34, ResNet-18, and VGG. This trend is almost in line with the number of parameters of the models, i.e., 53.9M, 23.5M, 21.3M, 11.2M and 9.6M. The results indicate that Sophon achieves satisfying intactness on different model architectures. Given that maintaining performance in the original domain and degrading performance in the restricted domain are dual objectives, it is plausible that a more complex model, with an increased number of parameters, holds the potential to better achieve these dual objectives. This is particularly attainable in the current landscape, where large-scale pre-training is a prevailing trend. Figure 11: Effectiveness of Sophon against different batch sizes. Note that, although the Sophon model is updated 20,000 times when the batch size is 50, the model is still stuck in the local optimum. It verifies the robustness of Sophon against different batch sizes. 5.5 Optimizer In this part, we evaluate the effectiveness of Sophon against different optimizers. We pre-train the CAFormer model on ImageNette and try to restrict its application CIFAR-10. The models are fine-tuned by five different optimizers, i.e., Momentum, Nesterov, Adagrad, Adadelta, and Adam. We present the results in Figure 9 and Table V in Appendix. The results show that fine-tuning the Sophon models using the Momentum or the Nesterov optimizer does not increase the ACC, which remains almost unchanged at around 10%. Leveraging optimizers that adaptively adjust the learning rate, i.e., Adagrad, Adadelta, and Adam, may increase the performance of Sophon models in the restricted domain. But Sophon still renders the model more challenging to train than training from scratch, greatly slowing down the convergence. For example, training the model from scratch using Adadelta achieves 59.7% in just 5 epochs. In contrast, fine-tuning the Sophon using the same Adadelta takes 20 epochs to achieve 56.4%. The results verify that although we only use Adam in the fine-tuning suppression loops, the resistance generalizes well to other optimizers. We observe that the accuracy of fine-tuning the protected models using Adagrad, Adadelta, and Adam shows an upward trend in the later stages of the training process. To further investigate, we extend the training by an additional 30 epochs until the training processes converged. The final ACCs of fine-tuning the protected models using Adagrad, Adadelta, and Adam are 49.8%±plus-or-minus±32.38%, 78.8%±plus-or-minus±0.62%, and 75.6%±plus-or-minus±1.58%, respectively. It is noteworthy that fine-tuning the protected models still yields significantly lower performance compared to fine-tuning the original models, albeit slightly better than training from scratch. This suggests that if the adversary persists in fine-tuning the protected models, they can achieve a 10.1% to 13.4% improvement compared with training from scratch, albeit at the cost of expending 10 times the effort (40∼similar-to ∼50 epochs versus 3∼similar-to ∼4 epochs). 5.6 Learning Rate In this part, we evaluate the effectiveness of Sophon against different learning rates, because learning rates may impact whether a model can avoid local optima. We pre-train the CAFormer model on ImageNette and try to restrict its application CIFAR-10. The models are fine-tuned using five different learning rates, i.e., 5×10−6,10−5,10−4,10−35superscript106superscript105superscript104superscript1035× 10^-6,10^-5,10^-4,10^-35 × 10- 6 , 10- 5 , 10- 4 , 10- 3, and 10−2superscript10210^-210- 2. We do not evaluate 10−6superscript10610^-610- 6 because it is too small for efficient model training. The results are shown in Figure 10 and Table VI in Appendix. We can tell from the results that 5×10−65superscript1065× 10^-65 × 10- 6 and 10−5superscript10510^-510- 5 result in a slow convergence, due to they are still too small. Conversely, 10−2superscript10210^-210- 2 is commonly too large for fine-tuning because fine-tuning typically does not change the parameters too much. Thus, fine-tuning either the original model or the Sophon model cannot reach a satisfying solution in this case. The values 10−4superscript10410^-410- 4 and 10−3superscript10310^-310- 3 are both suitable as learning rates, as they lead to relatively fast convergence when training from scratch and fine-tuning the original model. Even under such suitable learning rates, the Sophon model still resists fine-tuning, showing robustness against different learning rates. This could be attributed to our inclusion of tasks with varying learning rates in the fine-tuning simulation. 5.7 Batch Size In this part, we evaluate the effectiveness of Sophon against different batch sizes because batch sizes impact the accuracy of gradient computation. We pre-train the CAFormer model on ImageNette and try to restrict its application CIFAR-10. The models are fine-tuned using five different batch sizes, i.e., 50, 100, 150, 200, and 250. Given that the batch size impacts the number of iterations in one epoch, the total iterations for these five settings over 20 epochs vary from 20,000 to 4,000. The results in Figure 11 and Table VII show that Sophon is able to resist fine-tuning under these five batch size settings. Note that, although the Sophon model is updated 20,000 times when the batch size is 50, the model is still stuck in the local optimum. It verifies the robustness of Sophon against different batch sizes. This could also be attributed to our inclusion of tasks with varying batch sizes in the fine-tuning simulation. 5.8 Training Stability In this part, we verify the stability of our proposed three loss functions, the ICE loss, the KLU loss and the DoS loss. As a baseline for classification model training, we present five failure cases encountered during the optimization process using the CE loss, as shown in Figure 13. In some cases, the loss becomes not-a-number (NaN) values during training. In other cases, the loss exhibits abrupt changes and divergence as it increases. It leads to underfitting and poor performance in the optimization process. On the contrary, the ICE loss and the KLU loss exhibit stability over 4,000 iterations, with the fluctuations in the loss decreasing. In our experiments, we use the two loss functions in our optimization over 200 times, no instability in training has been observed. As a baseline for generative model training, we present three failure cases encountered during the optimization process using the MSE loss, as shown in Figure 14. As we can see, the MSE loss also results in NaN values during training. And the green line in Figure 14 exhibits several abrupt changes to zero, rendering the optimization inefficient and ineffective. Conversely, the DoS loss we design demonstrates stability throughout the training process, facilitating easier and faster convergence during optimization. 6 Related Work 6.1 Non-Transferable Learning Non-transferable learning (NTL) aims to degrade the performance of deep learning models in the restricted domain, which is partially aligned with our work. Wang et al. [50] proposed the first NTL method by decreasing the model performance in the restricted domain with labeled data and increasing the distance between features from two domains with the maximum mean discrepancy (MMD) loss. Following the concept of NTL, Wang et al. [49] focused on image classification and proposed a compact un-transferable isolation (CUTI)-domain method to achieve a compacter generalization bound of the model. Zeng et al. [55] further extended the idea to natural language processing tasks. They leveraged a domain classifier loss and an MMD loss to eliminate the need for labeled data from the restricted domain. They also introduced a secret key component for recovering access to the restricted domain. Wang et al. [48] utilized distributionally robust optimization (DRO) framework to characterize the domains close to the source domain and degrade the model performance in them. In this way, they achieved better performance than NTL when no data is available from the restricted domain. Compared with existing NTL work, we not only degrade the model performance in the restricted domain but also resist fine-tuning the model in the restricted domain. We have confirmed via experiments that existing NTL methods cannot resist fine-tuning (cf. §5.2.1), probably because they are not robust against supervised model update. In addition, they only focused on classification tasks, while Sophon can also be applied to generation (cf. §5.2.2). 6.2 Meta Learning Meta learning, or learning-to-learn, aims to improve the learning algorithm itself, given the experience of multiple learning episodes. Model-agnostic meta-learning (MAML) [8, 10, 9] is a famous example to learn a model initialization that converges fast on unseen tasks and thus enables few-shot learning. Then a series of follow-up work was proposed to reduce the number of parameters to meta-learn. Lee et al. [25] performed MAML in a subspace of the whole parameter space by freezing a portion of the neurons. Qiao et al. [34] and Rusu et al. [41] created a low-dimensional latent space for model parameters by learning a parameter generator. Sun et al. [45] leveraged large-scale trained DNN and only meta-learned the scaling and shifting functions of model parameters to further speed up convergence and avoid over-fitting. In comparison with meta learning, the objectives of non-fine-tunable learning are somewhat the opposite, i.e., to train the model to not learn specific tasks while preserving unchanged performance on the original task. We face unique challenges such as negative impact in the original domain, susceptibility to adaptive fine-tuning, and training instability. Thus, directly inverting existing methods cannot fulfill our goals. 7 Discussion & Future Work In this section, we discuss the limitations and the future work of Sophon. More deep learning tasks. In this paper, we have demonstrated the effectiveness of Sophon in two specific tasks: image classification and image generation. While Sophon technically holds potential for application in other domains, generalizing the results presented in this paper remains unclear. Therefore, we advocate for the application of Sophon to pre-trained models in various domains, such as audio processing, natural language processing, tabular data analysis, and multimodal tasks. By extending Sophon to these domains, we may unlock its potential for enhancing the controllability of models across diverse areas of machine learning and artificial intelligence, a direction we envision as our future work. More domain-adaptation techniques. In this paper, we have demonstrated the effectiveness of Sophon against several commonly used domain adaptation techniques. We have designed a fine-tuning simulation module in Sophon to enhance its resistance to various domain adaptation techniques by incorporating them into Φ Φ. However, with the emergence of sophisticated domain adaptation techniques, such as LoRA/Adaptor in NLP, the effectiveness of Sophon against these techniques remains unknown. We have conducted preliminary experiments of fine-tuning CAFormer with LoRA and found Sophon to remain effective. We advocate for a more extensive and comprehensive evaluation on various domain adaptation techniques. More efficient and accurate algorithms. Sophon achieves non-fine-tunable learning by simulating fine-tuning process in the optimization, which involves second-order terms and requires intensive computation. To reduce overhead, we use the first-order approximation in fine-tuning suppression, but the algorithm may be less accurate in this way. Designing an approximation algorithm that reduces the amount of computation while maintaining a satisfactory accuracy is a promising area for future research. 8 Conclusion In this paper, we introduce a pioneering learning paradigm, termed non-fine-tunable learning, aimed at preventing pre-trained models from being fine-tuned for restricted tasks. We present the problem formulation, design and evaluation of Sophon, a framework designed to achieve non-fine-tunable learning, including an effective algorithm to resist fine-tuning and novel loss functions for training stability. Our experiments have validated the efficacy of Sophon in resisting fine-tuning across various fine-tuning settings. Acknowledgments We sincerely thank our Shepherd and all the anonymous reviewers for their valuable comments. This work is supported by China NSFC Grant 61925109 and Ant Group. Yanjiao Chen is the corresponding author. References [1] Yoshua Bengio. Deep learning of representations for unsupervised and transfer learning. In Unsupervised and Transfer Learning - Workshop. JMLR.org, 2012. [2] Yoshua Bengio, Geoffrey E. Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz, Gillian K. Hadfield, Jeff Clune, Tegan Maharaj, Frank Hutter, Atilim Günes Baydin, Sheila McIlraith, Qiqi Gao, Ashwin Acharya, David Krueger, Anca D. Dragan, Philip H. S. Torr, Stuart Russell, Daniel Kahneman, Jan Brauner, and Sören Mindermann. Managing AI risks in an era of rapid progress. arXiv preprint arXiv:2310.17688, 2023. [3] Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In International Conference on Artificial Intelligence and Statistics. JMLR.org, 2011. [4] Luke Nicholas Darlow, Elliot J. Crowley, Antreas Antoniou, and Amos J. Storkey. CINIC-10 is not imagenet or CIFAR-10. arXiv preprint arXiv:1810.03505, 2018. [5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019. [6] John C. Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12(61):2121–2159, 2011. [7] fastai. Github repo: Imagenette. https://github.com/fastai/imagenette, 2022. [8] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning. PMLR, 2017. [9] Chelsea Finn, Aravind Rajeswaran, Sham M. Kakade, and Sergey Levine. Online meta-learning. In International Conference on Machine Learning. PMLR, 2019. [10] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. In Conference on Neural Information Processing Systems. PMLR, 2018. [11] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial networks. arXiv preprint arXiv:1406.2661, 2014. [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016. [13] Kashmir Hill. How target figured out a teen girl was pregnant before her father did. https://w.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did, 2012. [14] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Conference on Neural Information Processing Systems. PMLR, 2020. [15] The White House. Fact sheet: President Biden issues executive order on safe, secure, and trustworthy artificial intelligence. https://w.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence, 2021. [16] Tatum Hunter. AI porn is easy to make now. for women, that’s a nightmare. https://w.washingtonpost.com/technology/2023/02/13/ai-porn-deepfakes-women-consent, 2023. [17] Wiliam Hunter. Paedophiles are using AI to create sexual images of celebrities as children, report finds. https://w.dailymail.co.uk/sciencetech/article-12669791/Paedophiles-using-AI-create-sexual-images-celebrities-CHILDREN-report-finds.html, 2023. [18] Matthew Hutson. Who should stop unethical A.I.? https://w.newyorker.com/tech/annals-of-technology/who-should-stop-unethical-ai, 2021. [19] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. [20] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations. OpenReview.net, 2015. [21] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014. [22] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. [23] S. Kullback and R. A. Leibler. On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1):79–86, 1951. [24] Yann LeCun. The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist, 1998. [25] Yoonho Lee and Seungjin Choi. Gradient-based meta-learning with learned layerwise metric and subspace. In International Conference on Machine Learning. PMLR, 2018. [26] Cixin Liu. The three-body problem, volume 1. Macmillan, 2014. [27] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In IEEE International Conference on Computer Vision, 2015. [28] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015. [29] Madison McQueen. AI porn is here and it’s dangerous. https://exoduscry.com/articles/ai-porn, 2023. [30] Dan Milmo. AI-created child sexual abuse images ‘threaten to overwhelm internet’. https://w.theguardian.com/technology/2023/oct/25/ai-created-child-sexual-abuse-images-threaten-overwhelm-internet, 2023. [31] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning. PMLR, 2011. [32] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Conference on Neural Information Processing Systems. PMLR, 2019. [33] Boris T Polyak. Some methods of speeding up the convergence of iteration methods. USSR computational mathematics and mathematical physics, 4(5):1–17, 1964. [34] Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L. Yuille. Few-shot image recognition by predicting parameters from activations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. [35] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with region proposal networks. In Conference on Neural Information Processing Systems. PMLR, 2015. [36] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with region proposal networks. In Conference on Neural Information Processing Systems. PMLR, 2015. [37] Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, 22(3):400–407, 1951. [38] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. [39] Rebecca J. Rosen. Armed with facebook ’likes’ alone, researchers can tell your race, gender, and sexual orientation. https://w.theatlantic.com/technology/archive/2013/03/armed-with-facebook-likes-alone-researchers-can-tell-your-race-gender-and-sexual-orientation, 2013. [40] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986. [41] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In International Conference on Learning Representations. OpenReview.net, 2019. [42] Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations, 2014. [43] Eric Siegel. The privacy pickle: Hewlett-packard’s prediction of employee behavior. https://w.predictiveanalyticsworld.com/machinelearningtimes/the-privacy-pickle-hewlett-packards-prediction-of-employee-behavior, 2013. [44] Eric Siegel. When does predictive technology become unethical? https://hbr.org/2020/10/when-does-predictive-technology-become-unethical, 2020. [45] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. [46] Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. On the importance of initialization and momentum in deep learning. In International Conference on Machine Learning. JMLR.org, 2013. [47] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015. [48] Haotian Wang, Haoang Chi, Wenjing Yang, Zhipeng Lin, Mingyang Geng, Long Lan, Jing Zhang, and Dacheng Tao. Domain specified optimization for deployment authorization. In IEEE/CVF International Conference on Computer Vision, 2023. [49] Lianyu Wang, Meng Wang, Daoqiang Zhang, and Huazhu Fu. Model barrier: A compact un-transferable isolation domain for model intellectual property protection. In IEEE/CVF Conference on Computer Vision and Pattern, 2023. [50] Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, and Qi Zhu. Non-transferable learning: A new approach for model ownership verification and applicability authorization. In International Conference on Learning Representations. OpenReview.net, 2022. [51] Rhiannon Williams. Text-to-image AI models can be tricked into generating disturbing images. https://w.technologyreview.com/2023/11/17/1083593/text-to-image-ai-models-can-be-tricked-into-generating-disturbing-images, 2023. [52] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Conference on Neural Information Processing Systems. PMLR, 2014. [53] Weihao Yu, Chenyang Si, Pan Zhou, Mi Luo, Yichen Zhou, Jiashi Feng, Shuicheng Yan, and Xinchao Wang. Metaformer baselines for vision. arXiv preprint arXiv:2210.13452, 2022. [54] Matthew D. Zeiler. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012. [55] Guangtao Zeng and Wei Lu. Unsupervised non-transferable text classification. In Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2022. Appendix A Derivatives in this Paper A.1 Derivatives of Softmax The output of a classifier can be represented as, (y^1,y^2,⋯,y^C)=σ⁢(z1,z2,⋯,zC),subscript^1subscript^2⋯subscript^subscript1subscript2⋯subscript ( y_1, y_2,·s, y_C )=σ% (z_1,z_2,·s,z_C ),( over start_ARG y end_ARG1 , over start_ARG y end_ARG2 , ⋯ , over start_ARG y end_ARGC ) = σ ( z1 , z2 , ⋯ , zitalic_C ) , (17) where zisubscriptz_izitalic_i is the i-th logit, y^i=exp⁡(zi)/∑j=1Cexp⁡(zj)subscript^subscriptsuperscriptsubscript1subscript y_i= (z_i)/ _j=1^C (z_j)over start_ARG y end_ARGi = exp ( zitalic_i ) / ∑j = 1C exp ( zitalic_j ) is the predicted probability of the i-th label, C is the total number of classes. When i≠ji≠ ji ≠ j, ∂y^j∂zisubscript^subscript ∂ y_j∂ z_idivide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG =∂(exp⁡(zj)/∑k=1Cexp⁡(zk))∂zi,absentsubscriptsuperscriptsubscript1subscriptsubscript = ∂ ( (z_j)/ _k=1^C (z_k) )% ∂ z_i,= divide start_ARG ∂ ( exp ( zitalic_j ) / ∑k = 1C exp ( zitalic_k ) ) end_ARG start_ARG ∂ zitalic_i end_ARG , (18) =−exp⁡(zi)⋅exp⁡(zj)(∑k=1Cexp⁡(zk))2,absent⋅subscriptsubscriptsuperscriptsuperscriptsubscript1subscript2 = - (z_i)· (z_j) ( _k=1^C (z_% k) )^2,= divide start_ARG - exp ( zitalic_i ) ⋅ exp ( zitalic_j ) end_ARG start_ARG ( ∑k = 1C exp ( zitalic_k ) )2 end_ARG , =−y^i⁢y^j.absentsubscript^subscript =- y_i y_j.= - over start_ARG y end_ARGi over start_ARG y end_ARGj . When i=ji=ji = j, ∂y^i∂zisubscript^subscript ∂ y_i∂ z_idivide start_ARG ∂ over start_ARG y end_ARGi end_ARG start_ARG ∂ zitalic_i end_ARG =∂(exp⁡(zi)/∑k=1Cexp⁡(zk))∂zi,absentsubscriptsuperscriptsubscript1subscriptsubscript = ∂ ( (z_i)/ _k=1^C (z_k) )% ∂ z_i,= divide start_ARG ∂ ( exp ( zitalic_i ) / ∑k = 1C exp ( zitalic_k ) ) end_ARG start_ARG ∂ zitalic_i end_ARG , (19) =exp(zi)(∑k=1Cexp(zk))−exp(zi)2(∑k=1Cexp⁡(zk))2, = (z_i) ( _k=1^C (z_k) )- (z_% i)^2 ( _k=1^C (z_k) )^2,= divide start_ARG exp ( zitalic_i ) ( ∑k = 1C exp ( zitalic_k ) ) - exp ( zitalic_i )2 end_ARG start_ARG ( ∑k = 1C exp ( zitalic_k ) )2 end_ARG , =exp⁡(zi)∑k=1Cexp⁡(zk)−(exp⁡(zi)∑k=1Cexp⁡(zk))2,absentsubscriptsuperscriptsubscript1subscriptsuperscriptsubscriptsuperscriptsubscript1subscript2 = (z_i) _k=1^C (z_k)- ( (z% _i) _k=1^C (z_k) )^2,= divide start_ARG exp ( zitalic_i ) end_ARG start_ARG ∑k = 1C exp ( zitalic_k ) end_ARG - ( divide start_ARG exp ( zitalic_i ) end_ARG start_ARG ∑k = 1C exp ( zitalic_k ) end_ARG )2 , =y^i⁢(1−y^i).absentsubscript^1subscript = y_i(1- y_i).= over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) . Then, we have the following partial derivatives, ∂y^j∂zi=y^i⁢(1−y^i),i=j,−y^i⁢y^j,i≠j.subscript^subscriptcasessubscript^1subscript^subscript^subscript ∂ y_j∂ z_i= \ array[]rl% y_i(1- y_i),&i=j,\\ - y_i y_j,&i≠ j. array .divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG = start_ARRAY start_ROW start_CELL over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) , end_CELL start_CELL i = j , end_CELL end_ROW start_ROW start_CELL - over start_ARG y end_ARGi over start_ARG y end_ARGj , end_CELL start_CELL i ≠ j . end_CELL end_ROW end_ARRAY (20) A.2 Derivatives of Cross-Entropy Loss We can derive the gradient of the cross-entropy loss with respect to zisubscriptz_izitalic_i by the chain rule, ∂ℒCE∂zisubscriptℒCEsubscript _CE∂ z_idivide start_ARG ∂ Lroman_CE end_ARG start_ARG ∂ zitalic_i end_ARG =−∑j=1C∂(yj⁢log⁡y^j)∂y^j⋅∂y^j∂zi,absentsuperscriptsubscript1⋅subscriptsubscript^subscript^subscript^subscript =- _j=1^C ∂ (y_j y_j )% ∂ y_j· ∂ y_j∂ z_i,= - ∑j = 1C divide start_ARG ∂ ( yitalic_j log over start_ARG y end_ARGj ) end_ARG start_ARG ∂ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , (21) =−∑j=1Cyjy^j⋅∂y^j∂zi,absentsuperscriptsubscript1⋅subscriptsubscript^subscript^subscript =- _j=1^C y_j y_j· ∂ % y_j∂ z_i,= - ∑j = 1C divide start_ARG yitalic_j end_ARG start_ARG over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , =−yiy^i⋅∂y^i∂zi−∑j≠iyjy^j⋅∂y^j∂zi,absent⋅subscriptsubscript^subscript^subscriptsubscript⋅subscriptsubscript^subscript^subscript =- y_i y_i· ∂ y_i% ∂ z_i- _j≠ i y_j y_j· ∂ % y_j∂ z_i,= - divide start_ARG yitalic_i end_ARG start_ARG over start_ARG y end_ARGi end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGi end_ARG start_ARG ∂ zitalic_i end_ARG - ∑j ≠ i divide start_ARG yitalic_j end_ARG start_ARG over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , =⋆−yiy^i⋅y^i⁢(1−y^i)−∑j≠iyjy^j⋅(−y^i⁢y^j),⋆⋅subscriptsubscript^subscript^1subscript^subscript⋅subscriptsubscript^subscript^subscript =- y_i y_i· y_i(1-% y_i)- _j≠ i y_j y_j· (- y_i% y_j ),over⋆ start_ARG = end_ARG - divide start_ARG yitalic_i end_ARG start_ARG over start_ARG y end_ARGi end_ARG ⋅ over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) - ∑j ≠ i divide start_ARG yitalic_j end_ARG start_ARG over start_ARG y end_ARGj end_ARG ⋅ ( - over start_ARG y end_ARGi over start_ARG y end_ARGj ) , =−yi+(∑j=1Cyj)⋅y^i,absentsubscript⋅superscriptsubscript1subscriptsubscript =-y_i+ ( _j=1^Cy_j )· y_i,= - yitalic_i + ( ∑j = 1C yitalic_j ) ⋅ over start_ARG y end_ARGi , =y^i−yi,absentsubscript^subscript = y_i-y_i,= over start_ARG y end_ARGi - yitalic_i , where Equation (⋆ ⋆) utilizes the results in Equation (20). A.3 Derivatives of Inverse Cross-Entropy Loss Without loss of generality, we assume y1=1subscript11y_1=1y1 = 1 and yj=0⁢(j≠1)subscript01y_j=0~(j≠ 1)yitalic_j = 0 ( j ≠ 1 ). Then the gradient of the ICE loss with respect to zisubscriptz_izitalic_i is, ∂ℒICE∂zisubscriptℒICEsubscript _ICE∂ z_idivide start_ARG ∂ Lroman_ICE end_ARG start_ARG ∂ zitalic_i end_ARG =−∑j=1C∂(yj⁢log⁡(1−y^j))∂y^j⋅∂y^j∂zi,absentsuperscriptsubscript1⋅subscript1subscript^subscript^subscript^subscript =- _j=1^C ∂ (y_j (1- y_j% ) )∂ y_j· ∂ y_j∂ z% _i,= - ∑j = 1C divide start_ARG ∂ ( yitalic_j log ( 1 - over start_ARG y end_ARGj ) ) end_ARG start_ARG ∂ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , (22) =∑j=1Cyj1−y^j⋅∂y^j∂zi,absentsuperscriptsubscript1⋅subscript1subscript^subscript^subscript = _j=1^C y_j1- y_j· ∂ % y_j∂ z_i,= ∑j = 1C divide start_ARG yitalic_j end_ARG start_ARG 1 - over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , =yi1−y^i⋅∂y^i∂zi+∑j≠iyj1−y^j⋅∂y^j∂zi,absent⋅subscript1subscript^subscript^subscriptsubscript⋅subscript1subscript^subscript^subscript = y_i1- y_i· ∂ y_i% ∂ z_i+ _j≠ i y_j1- y_j· ∂% y_j∂ z_i,= divide start_ARG yitalic_i end_ARG start_ARG 1 - over start_ARG y end_ARGi end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGi end_ARG start_ARG ∂ zitalic_i end_ARG + ∑j ≠ i divide start_ARG yitalic_j end_ARG start_ARG 1 - over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG , =⋆⁢yi1−y^i⋅y^i⁢(1−y^i)+∑j≠iyj1−y^j⋅(−y^i⁢y^j),⋅⋆subscript1subscript^subscript^1subscript^subscript⋅subscript1subscript^subscript^subscript = y_i1- y_i· y_i(1-% y_i)+ _j≠ i y_j1- y_j· (- y_i% y_j ),over⋆ start_ARG = end_ARG divide start_ARG yitalic_i end_ARG start_ARG 1 - over start_ARG y end_ARGi end_ARG ⋅ over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) + ∑j ≠ i divide start_ARG yitalic_j end_ARG start_ARG 1 - over start_ARG y end_ARGj end_ARG ⋅ ( - over start_ARG y end_ARGi over start_ARG y end_ARGj ) , =y^1,i=1,−y^i⁢y^1/(1−y^1),i≠1.absentcasessubscript^11subscript^subscript^11subscript^11 = \ array[]rl y_1,&i=1,\\ - y_i y_1/ (1- y_1 ),&i≠ 1. array .= start_ARRAY start_ROW start_CELL over start_ARG y end_ARG1 , end_CELL start_CELL i = 1 , end_CELL end_ROW start_ROW start_CELL - over start_ARG y end_ARGi over start_ARG y end_ARG1 / ( 1 - over start_ARG y end_ARG1 ) , end_CELL start_CELL i ≠ 1 . end_CELL end_ROW end_ARRAY where Equation (⋆ ⋆) utilizes the results in Equation (20). A.4 Derivatives of KL Div. from Uniform Dist. Loss The gradient of the KLU loss with respect to zisubscriptz_izitalic_i is, ∂ℒKLU∂zisubscriptℒKLUsubscript _KLU∂ z_idivide start_ARG ∂ Lroman_KLU end_ARG start_ARG ∂ zitalic_i end_ARG =−∑j=1C∂(1C⁢log⁡(C⋅y^j))∂y^j⋅∂y^j∂ziabsentsuperscriptsubscript1⋅1⋅subscript^subscript^subscript^subscript =- _j=1^C ∂ ( 1C (C·% y_j ) )∂ y_j· ∂ y_j% ∂ z_i= - ∑j = 1C divide start_ARG ∂ ( divide start_ARG 1 end_ARG start_ARG C end_ARG log ( C ⋅ over start_ARG y end_ARGj ) ) end_ARG start_ARG ∂ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG (23) =−∑j=1C1C⋅y^j⋅∂y^j∂ziabsentsuperscriptsubscript1⋅1⋅subscript^subscript^subscript =- _j=1^C 1C· y_j· ∂% y_j∂ z_i= - ∑j = 1C divide start_ARG 1 end_ARG start_ARG C ⋅ over start_ARG y end_ARGj end_ARG ⋅ divide start_ARG ∂ over start_ARG y end_ARGj end_ARG start_ARG ∂ zitalic_i end_ARG =⋆−1C⋅y^i⋅y^i⁢(1−y^i)−∑j≠i1C⋅y^j⋅(−y^i⁢y^j)⋆⋅1⋅subscript^subscript^1subscript^subscript⋅1⋅subscript^subscript^subscript =- 1C· y_i· y_i(1-% y_i)- _j≠ i 1C· y_j· (- y_i% y_j )over⋆ start_ARG = end_ARG - divide start_ARG 1 end_ARG start_ARG C ⋅ over start_ARG y end_ARGi end_ARG ⋅ over start_ARG y end_ARGi ( 1 - over start_ARG y end_ARGi ) - ∑j ≠ i divide start_ARG 1 end_ARG start_ARG C ⋅ over start_ARG y end_ARGj end_ARG ⋅ ( - over start_ARG y end_ARGi over start_ARG y end_ARGj ) =y^i−1C.absentsubscript^1 = y_i- 1C.= over start_ARG y end_ARGi - divide start_ARG 1 end_ARG start_ARG C end_ARG . where Equation (⋆ ⋆) utilizes the results in Equation (20). Figure 12: Effectiveness of Sophon compared with two baselines. The original model can achieve low losses on FFHQ, leading to model misuse. Fine-tuning the Sophon model cannot decrease the loss in the restricted domain and yields higher losses than training the model from scratch. Figure 13: Training stability comparison of three loss functions for classification. The CE loss easily fails due to abrupt changes and divergence. The ICE loss and the KLU loss exhibit stability over 4,000 iterations, with the fluctuations in the loss decreasing. Figure 14: Training stability comparison of two loss functions for generation. The MSE loss easily fails due to abrupt changes and divergence. The DoS loss exhibits stability over 5,000 iterations, facilitating easier and faster convergence during optimization. [t] Domain Method† Original ACC ACC in the Restricted Domain Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 CIFAR-10 Scratch - - 49.1±plus-or-minus±1.2 59.5±plus-or-minus±0.7 60.6±plus-or-minus±0.5 62.0±plus-or-minus±0.5 62.7±plus-or-minus±0.5 Original 99.6 9.3 52.4±plus-or-minus±3.8 80.0±plus-or-minus±0.4 83.6±plus-or-minus±0.3 84.6±plus-or-minus±0.2 84.9±plus-or-minus±0.2 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 10.3±plus-or-minus±0.4 12.8±plus-or-minus±1.5 15.2±plus-or-minus±0.5 14.3±plus-or-minus±1.9 CINIC Scratch - - 42.4±plus-or-minus±1.1 50.9±plus-or-minus±0.1 50.2±plus-or-minus±0.7 50.6±plus-or-minus±0.4 51.3±plus-or-minus±0.3 Original 99.6 9.2 48.9±plus-or-minus±2.8 69.7±plus-or-minus±0.5 72.9±plus-or-minus±0.2 74.2±plus-or-minus±0.2 74.4±plus-or-minus±0.2 Sophon 97.6 10.0 10.0±plus-or-minus±0.0 9.6±plus-or-minus±0.5 10.1±plus-or-minus±0.3 10.0±plus-or-minus±0.0 8.8±plus-or-minus±1.9 STL Scratch - - 23.5±plus-or-minus±4.0 38.9±plus-or-minus±2.5 42.8±plus-or-minus±1.5 45.7±plus-or-minus±0.8 46.3±plus-or-minus±0.3 Original 99.6 6.9 10.0±plus-or-minus±0.3 19.9±plus-or-minus±2.1 41.6±plus-or-minus±2.6 61.1±plus-or-minus±3.7 71.7±plus-or-minus±5.1 Sophon 97.4 10.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.6±plus-or-minus±0.3 11.6±plus-or-minus±0.3 11.0±plus-or-minus±0.2 MNIST Scratch - - 87.5±plus-or-minus±16.0 97.9±plus-or-minus±0.6 98.3±plus-or-minus±0.4 98.5±plus-or-minus±0.4 98.4±plus-or-minus±0.3 Original 99.6 12.3 96.8±plus-or-minus±0.3 98.5±plus-or-minus±0.3 98.9±plus-or-minus±0.1 99.0±plus-or-minus±0.1 99.0±plus-or-minus±0.1 Sophon 97.6 8.9 11.3±plus-or-minus±0.0 11.3±plus-or-minus±0.0 11.3±plus-or-minus±0.0 11.3±plus-or-minus±0.0 11.8±plus-or-minus±1.0 SVHN Scratch - - 44.3±plus-or-minus±4.9 77.7±plus-or-minus±0.4 82.3±plus-or-minus±0.5 84.1±plus-or-minus±0.2 84.8±plus-or-minus±0.2 Original 99.6 7.4 60.3±plus-or-minus±5.5 90.7±plus-or-minus±0.4 92.6±plus-or-minus±0.3 93.1±plus-or-minus±0.2 93.2±plus-or-minus±0.3 Sophon 97.8 7.8 19.6±plus-or-minus±0.0 19.6±plus-or-minus±0.0 19.6±plus-or-minus±0.0 19.6±plus-or-minus±0.0 19.6±plus-or-minus±0.0 † Scratch: training from scratch (B1). Original: fine-tuning the original model (B2). Sophon: fine-tuning the Sophon model. TABLE I: Effectiveness of Sophon against different restricted domains. [t] Domain Method Original ACC ACC in the Restricted Domain Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 CAFormer Scratch - - 49.1±plus-or-minus±1.2 59.5±plus-or-minus±0.7 60.6±plus-or-minus±0.5 62.0±plus-or-minus±0.5 62.7±plus-or-minus±0.5 Original 99.6 9.3 52.9±plus-or-minus±3.3 80.0±plus-or-minus±0.5 83.6±plus-or-minus±0.3 84.6±plus-or-minus±0.2 84.9±plus-or-minus±0.2 Sophon 96.2 10.0 10.9±plus-or-minus±0.6 11.7±plus-or-minus±0.8 11.5±plus-or-minus±1.0 12.0±plus-or-minus±1.1 12.0±plus-or-minus±0.7 ResNet-50 Scratch - - 37.5±plus-or-minus±4.1 50.4±plus-or-minus±2.6 57.0±plus-or-minus±1.3 60.4±plus-or-minus±0.8 61.5±plus-or-minus±1.1 Original 99.2 5.2 47.0±plus-or-minus±0.4 66.9±plus-or-minus±0.2 69.9±plus-or-minus±0.2 70.6±plus-or-minus±0.2 70.9±plus-or-minus±0.3 Sophon 95.6 11.0 11.0±plus-or-minus±0.1 12.6±plus-or-minus±0.3 13.9±plus-or-minus±0.4 15.7±plus-or-minus±0.1 16.7±plus-or-minus±0.1 ResNet-34 Scratch - - 29.1±plus-or-minus±4.2 57.4±plus-or-minus±3.2 67.5±plus-or-minus±0.9 69.6±plus-or-minus±1.1 70.0±plus-or-minus±0.9 Original 99.0 5.3 39.9±plus-or-minus±0.1 64.0±plus-or-minus±0.1 70.0±plus-or-minus±0.2 72.4±plus-or-minus±0.3 73.4±plus-or-minus±0.2 Sophon 94.8 10.0 10.6±plus-or-minus±0.1 11.1±plus-or-minus±0.3 11.1±plus-or-minus±0.4 11.4±plus-or-minus±0.3 11.9±plus-or-minus±0.5 ResNet-18 Scratch - - 44.8±plus-or-minus±1.7 67.4±plus-or-minus±1.0 71.3±plus-or-minus±0.6 72.5±plus-or-minus±0.5 72.1±plus-or-minus±0.8 Original 98.6 9.3 41.4±plus-or-minus±0.2 62.4±plus-or-minus±0.2 68.1±plus-or-minus±0.1 70.7±plus-or-minus±0.1 71.5±plus-or-minus±0.3 Sophon 95.0 8.4 10.7±plus-or-minus±0.1 11.3±plus-or-minus±0.1 11.6±plus-or-minus±0.5 11.6±plus-or-minus±0.2 12.0±plus-or-minus±0.1 VGG Scratch - - 13.7±plus-or-minus±4.6 58.0±plus-or-minus±1.0 75.0±plus-or-minus±1.1 78.0±plus-or-minus±0.6 78.8±plus-or-minus±0.3 Original 95.4 2.8 33.9±plus-or-minus±0.2 72.4±plus-or-minus±0.2 79.9±plus-or-minus±0.3 82.9±plus-or-minus±0.2 84.6±plus-or-minus±0.2 Sophon 91.2 10.5 10.4±plus-or-minus±0.1 10.9±plus-or-minus±0.3 10.8±plus-or-minus±0.8 17.4±plus-or-minus±4.2 18.03±plus-or-minus±0.7 † Scratch: training from scratch (B1). Original: fine-tuning the original model (B2). Sophon: fine-tuning the Sophon model. TABLE IV: Effectiveness of Sophon on different model architectures. [t] Optimizer Method Original ACC ACC in the Restricted Domain Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 Momentum Scratch - - 49.3±plus-or-minus±1.0 59.4±plus-or-minus±0.7 60.3±plus-or-minus±0.2 61.7±plus-or-minus±0.6 62.2±plus-or-minus±0.7 Original 99.6 9.3 51.5±plus-or-minus±4.5 79.7±plus-or-minus±1.1 83.4±plus-or-minus±0.6 84.8±plus-or-minus±0.2 85.0±plus-or-minus±0.4 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.2±plus-or-minus±0.7 10.9±plus-or-minus±0.7 10.6±plus-or-minus±0.8 Nesterov Scratch - - 51.1±plus-or-minus±0.8 59.8±plus-or-minus±0.6 61.1±plus-or-minus±0.6 61.9±plus-or-minus±0.2 62.6±plus-or-minus±0.7 Original 99.6 9.3 50.1±plus-or-minus±3.8 79.8±plus-or-minus±0.3 83.4±plus-or-minus±0.2 84.7±plus-or-minus±0.2 84.9±plus-or-minus±0.2 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.1±plus-or-minus±0.7 10.9±plus-or-minus±0.7 10.4±plus-or-minus±0.9 Adagrad Scratch - - 47.8±plus-or-minus±1.2 62.6±plus-or-minus±0.4 64.3±plus-or-minus±0.3 64.3±plus-or-minus±0.3 64.2±plus-or-minus±0.4 Original 99.6 9.3 80.1±plus-or-minus±1.1 84.8±plus-or-minus±0.5 84.9±plus-or-minus±0.2 84.7±plus-or-minus±0.4 84.6±plus-or-minus±0.4 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 9.9±plus-or-minus±0.1 12.0±plus-or-minus±2.4 22.2±plus-or-minus±12.7 Adadelta Scratch - - 49.5±plus-or-minus±0.8 59.7±plus-or-minus±0.5 60.8±plus-or-minus±0.7 63.6±plus-or-minus±1.0 64.4±plus-or-minus±0.4 Original 99.6 9.3 80.1±plus-or-minus±0.6 86.6±plus-or-minus±0.4 87.1±plus-or-minus±0.2 87.3±plus-or-minus±0.3 87.3±plus-or-minus±0.3 Sophon 96.2 10.0 9.8±plus-or-minus±0.5 12.3±plus-or-minus±2.5 17.6±plus-or-minus±1.0 31.4±plus-or-minus±4.1 56.4±plus-or-minus±2.1 Adam Scratch - - 53.7±plus-or-minus±0.2 62.7±plus-or-minus±0.4 63.4±plus-or-minus±0.6 63.8±plus-or-minus±0.2 63.5±plus-or-minus±0.5 Original 99.6 9.3 87.4±plus-or-minus±0.9 89.8±plus-or-minus±0.3 90.1±plus-or-minus±0.3 90.0±plus-or-minus±0.4 89.9±plus-or-minus±0.3 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 9.8±plus-or-minus±0.4 10.0±plus-or-minus±0.0 23.2±plus-or-minus±10.8 50.2±plus-or-minus±4.0 † Scratch: training from scratch (B1). Original: fine-tuning the original model (B2). Sophon: fine-tuning the Sophon model. TABLE V: Effectiveness of Sophon against different optimizers. [t] Learning Rate Method Original ACC ACC in the Restricted Domain Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 5×10−65superscript1065× 10^-65 × 10- 6 Scratch - - 20.3±plus-or-minus±1.2 31.7±plus-or-minus±0.2 35.9±plus-or-minus±0.0 38.0±plus-or-minus±0.1 39.7±plus-or-minus±0.1 Original 99.6 9.3 11.1±plus-or-minus±0.3 20.9±plus-or-minus±0.4 37.8±plus-or-minus±0.4 55.3±plus-or-minus±0.2 63.0±plus-or-minus±0.1 Sophon 96.2 10.0 1.4±plus-or-minus±0.0 1.3±plus-or-minus±0.0 1.2±plus-or-minus±0.0 1.1±plus-or-minus±0.0 1.0±plus-or-minus±0.0 1×10−51superscript1051× 10^-51 × 10- 5 Scratch - - 25.0±plus-or-minus±0.9 35.8±plus-or-minus±0.1 39.6±plus-or-minus±0.1 41.9±plus-or-minus±0.3 43.5±plus-or-minus±0.2 Original 99.6 9.3 13.1±plus-or-minus±0.3 37.1±plus-or-minus±0.6 62.6±plus-or-minus±0.4 70.3±plus-or-minus±0.4 74.0±plus-or-minus±0.3 Sophon 96.2 10.0 1.4±plus-or-minus±0.0 1.2±plus-or-minus±0.0 1.0±plus-or-minus±0.0 0.9±plus-or-minus±0.0 0.9±plus-or-minus±0.0 1×10−41superscript1041× 10^-41 × 10- 4 Scratch - - 39.2±plus-or-minus±0.4 48.2±plus-or-minus±0.2 52.3±plus-or-minus±0.2 54.0±plus-or-minus±0.3 54.9±plus-or-minus±0.3 Original 99.6 9.3 53.0±plus-or-minus±1.0 79.8±plus-or-minus±0.2 83.7±plus-or-minus±0.1 84.6±plus-or-minus±0.1 84.8±plus-or-minus±0.2 Sophon 96.2 10.0 1.0±plus-or-minus±0.0 1.4±plus-or-minus±0.0 8.5±plus-or-minus±0.4 9.8±plus-or-minus±0.1 8.9±plus-or-minus±1.1 1×10−31superscript1031× 10^-31 × 10- 3 Scratch - - 48.8±plus-or-minus±0.3 56.5±plus-or-minus±0.3 58.4±plus-or-minus±0.1 58.8±plus-or-minus±0.0 58.7±plus-or-minus±0.0 Original 99.6 9.3 74.1±plus-or-minus±0.6 86.0±plus-or-minus±0.7 86.8±plus-or-minus±0.7 87.2±plus-or-minus±0.5 87.2±plus-or-minus±1.0 Sophon 96.2 10.0 8.0±plus-or-minus±2.5 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 1×10−21superscript1021× 10^-21 × 10- 2 Scratch - - 50.2±plus-or-minus±1.2 59.7±plus-or-minus±0.6 60.5±plus-or-minus±0.4 62.1±plus-or-minus±0.6 62.8±plus-or-minus±0.4 Original 99.6 9.3 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 23.5±plus-or-minus±13.5 41.9±plus-or-minus±31.9 Sophon 96.2 10.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 10.0±plus-or-minus±0.0 39.2±plus-or-minus±34.8 † Scratch: training from scratch (B1). Original: fine-tuning the original model (B2). Sophon: fine-tuning the Sophon model. TABLE VI: Effectiveness of Sophon against different learning rates. [t] Batch Size Method Original ACC ACC in the Restricted Domain Epoch 0 Epoch 0 Epoch 1 Epoch 5 Epoch 10 Epoch 15 Epoch 20 50 Scratch - - 46.2±plus-or-minus±0.3 54.6±plus-or-minus±0.4 55.9±plus-or-minus±0.2 56.4±plus-or-minus±0.2 57.0±plus-or-minus±0.3 Original 99.6 9.3 72.9±plus-or-minus±2.9 84.7±plus-or-minus±0.2 86.1±plus-or-minus±0.5 86.5±plus-or-minus±0.1 86.6±plus-or-minus±0.5 Sophon 96.2 10.0 1.1±plus-or-minus±0.1 8.2±plus-or-minus±1.2 10.0±plus-or-minus±0.0 9.2±plus-or-minus±1.4 9.9±plus-or-minus±0.0 100 Scratch - - 42.8±plus-or-minus±0.4 51.8±plus-or-minus±0.3 54.9±plus-or-minus±0.1 55.2±plus-or-minus±0.1 55.3±plus-or-minus±0.4 Original 99.6 9.3 63.0±plus-or-minus±3.9 83.1±plus-or-minus±0.3 85.4±plus-or-minus±0.1 85.7±plus-or-minus±0.2 85.7±plus-or-minus±0.2 Sophon 96.2 10.0 0.9±plus-or-minus±0.0 8.2±plus-or-minus±1.0 9.4±plus-or-minus±0.8 7.7±plus-or-minus±2.1 9.2±plus-or-minus±1.2 150 Scratch - - 40.5±plus-or-minus±0.3 49.7±plus-or-minus±0.1 53.6±plus-or-minus±0.2 54.9±plus-or-minus±0.1 55.2±plus-or-minus±0.5 Original 99.6 9.3 59.4±plus-or-minus±1.9 81.3±plus-or-minus±0.1 84.6±plus-or-minus±0.2 85.2±plus-or-minus±0.2 85.2±plus-or-minus±0.2 Sophon 96.2 10.0 0.9±plus-or-minus±0.0 1.6±plus-or-minus±0.1 9.8±plus-or-minus±0.0 8.4±plus-or-minus±1.1 8.2±plus-or-minus±1.6 200 Scratch - - 39.2±plus-or-minus±0.4 48.2±plus-or-minus±0.2 52.3±plus-or-minus±0.2 54.0±plus-or-minus±0.3 54.9±plus-or-minus±0.3 Original 99.6 9.3 53.0±plus-or-minus±1.0 79.8±plus-or-minus±0.2 83.7±plus-or-minus±0.1 84.6±plus-or-minus±0.1 84.8±plus-or-minus±0.2 Sophon 96.2 10.0 1.0±plus-or-minus±0.0 1.4±plus-or-minus±0.0 8.3±plus-or-minus±0.4 9.8±plus-or-minus±0.1 8.1±plus-or-minus±1.3 250 Scratch - - 38.0±plus-or-minus±0.3 46.9±plus-or-minus±0.2 51.1±plus-or-minus±0.1 53.1±plus-or-minus±0.3 54.0±plus-or-minus±0.2 Original 99.6 9.3 41.2±plus-or-minus±2.7 78.4±plus-or-minus±0.3 82.9±plus-or-minus±0.3 84.1±plus-or-minus±0.1 84.8±plus-or-minus±0.1 Sophon 96.2 10.0 1.0±plus-or-minus±0.0 1.2±plus-or-minus±0.0 3.6±plus-or-minus±0.2 9.8±plus-or-minus±0.0 9.9±plus-or-minus±0.0 † Scratch: training from scratch (B1). Original: fine-tuning the original model (B2). Sophon: fine-tuning the Sophon model. TABLE VII: Effectiveness of Sophon against different batch sizes. Appendix B Meta-Review The following meta-review was prepared by the program committee for the 2024 IEEE Symposium on Security and Privacy (S&P) as part of the review process as detailed in the call for papers. B.1 Summary This paper proposes a framework (named SOPHON) for model pre-training. The goal is to prevent pre-trained models from being fine-tuned to “unintended” use cases while preserving the performance of the original use case. The intuition is to trap the pre-trained model within a hard-to-escape local optimum of the restricted domains. At the core, SOPHON combines two loss terms designed to i) preserve the model’s general capabilities and i) reduce its capabilities for particular domains via fine-tuning simulation, which was inspired by model-agnostic meta-learning paradigms. Experimental evaluation showed SOPHON’s success against various models and datasets in vision domains. B.2 Scientific Contributions Provides a Valuable Step Forward in an Established Field. B.3 Reasons for Acceptance 1. Investigates an important problem – “non-fine-tunability”, a generalized case of “non-transferability”. 2. The system supports both classification and generative models. 3. The proposed approach nicely utilizes the insights from relevant literature to realize the goal. B.4 Noteworthy Concerns 1. In all settings, the worst penalty the attacker can pay is training from scratch. So, an increase in accuracy compared to training from scratch is a partial success for the attacker. 2. The paper considers a partial set of training regimes/domain-adaptation techniques. How the results would generalize for the untested/unseen domains is unclear.

Instant research discovery