Paper deep dive
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text
Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/3/2026, 2:15:33 AM
Summary
The paper introduces TextCloak, a reinforcement learning-driven framework designed to protect textual data from unauthorized Large Language Model (LLM) exploitation. It generates 'unlearnable examples' (UEs) by transforming clean text into semantically faithful but linguistically perturbed variants that degrade the performance of models fine-tuned on them. The method utilizes a novel optimization algorithm, GRPO-UE, which employs group-relative policy optimization to balance the trade-off between inducing downstream model degradation and preserving text utility for legitimate users.
Entities (6)
Relation Signals (6)
TextCloak → protectsagainst → Unauthorized LLM Exploitation
confidence 98% · TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM exploitation.
TextCloak → uses → GRPO-UE
confidence 95% · To optimize the policy, we introduce GRPO-UE... This bi-level optimization enables the generator to discover generalizable protective patterns
TextCloak → generates → Unlearnable Examples
confidence 92% · TextCloak employs a generative policy that transforms batches of clean text into unlearnable examples
GRPO-UE → optimizes → Generative Policy
confidence 90% · To optimize the policy, we introduce GRPO-UE... updates the generator parameters via group-relative policy optimization.
TextCloak → usesmetric → SBERT
confidence 85% · We measure the semantic fidelity of the protected batch and its clean counterpart using Sentence-BERT (SBERT)
TextCloak → usesmetric → GPT-2
confidence 85% · We quantify linguistic naturalness using perplexity under pre-trained GPT-2
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The rapid development of Large Language Models (LLMs) has led to significant advances across a wide range of language tasks, while simultaneously raising growing concerns about unauthorized data exploitation and privacy leakage. Unlearnable examples (UEs) offer a promising defense by introducing carefully designed perturbations into data such that models trained on them exhibit degraded utility. However, existing methods for text protection are primarily designed for classification tasks (e.g., sentiment analysis) in discriminative language models and often rely on injecting class-specific linguistic cues, which limits their effectiveness in the open-ended generation settings of LLMs. In this work, we propose TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM exploitation. TextCloak employs a generative policy that transforms batches of clean text into unlearnable examples while preserving semantic fidelity and linguistic naturalness. To optimize the policy, we introduce GRPO-UE, which rewards generated unlearnable text based on the downstream degradation they induce in fine-tuned surrogate LLMs and updates the generator parameters via group-relative policy optimization. This bi-level optimization enables the generator to discover generalizable protective patterns beyond class-specific cues. Comprehensive experiments on six publicly available datasets and nine state-of-the-art LLMs demonstrate that TextCloak consistently impairs unauthorized fine-tuning while maintaining text utility for legitimate use. Further analyses establish its transferability and robustness across model architectures, training configurations, and adaptive attacks, highlighting its broad applicability as a practical defense against unauthorized LLM exploitation.
Tags
Links
- Source: https://arxiv.org/abs/2607.28862v1
- Canonical: https://arxiv.org/abs/2607.28862v1
Trouble viewing inline? Open PDF directly →
Full Text
70,261 characters extracted from source content.
Expand or collapse full text
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples Chengshuai Zhao Pingchuan Ma Dawei Li Bohan Jiang Zhiyuan Yu Zhen Tan Huan Liu Abstract The rapid development of Large Language Models (LLMs) has led to significant advances across a wide range of language tasks, while simultaneously raising growing concerns about unauthorized data exploitation and privacy leakage. Unlearnable examples (UEs) offer a promising defense by introducing carefully designed perturbations into data such that models trained on them exhibit degraded utility. However, existing methods for text protection are primarily designed for classification tasks (e.g., sentiment analysis) in discriminative language models and often rely on injecting class-specific linguistic cues, which limits their effectiveness in the open-ended generation settings of LLMs. In this work, we propose TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM exploitation. TextCloak employs a generative policy that transforms batches of clean text into unlearnable examples while preserving semantic fidelity and linguistic naturalness. To optimize the policy, we introduce GRPO-UE, which rewards generated unlearnable text based on the downstream degradation they induce in fine-tuned surrogate LLMs and updates the generator parameters via group-relative policy optimization. This bi-level optimization enables the generator to discover generalizable protective patterns beyond class-specific cues. Comprehensive experiments on six publicly available datasets and nine state-of-the-art LLMs demonstrate the effectiveness, transferability, robustness, and broad applicability of TextCloak. Code: https://github.com/ympc08/TextCloak 1 Introduction Figure 1: Illustration of UEs for LLMs. The rapid proliferation of Large Language Models (LLMs) has transformed natural language processing, delivering unprecedented performance across tasks such as open-ended text generation, complex reasoning, and instruction following (Brown et al., 2020; Ouyang et al., 2022; Li et al., 2025; Zhao et al., 2026b). These advances are largely fueled by pre-training and fine-tuning on vast quantities of web-scraped textual data, which has sparked severe concerns regarding unauthorized data exploitation and privacy violations (Carlini et al., 2021; Zhao et al., 2026a). Unscrupulous entities frequently scrape proprietary, sensitive, or user-owned text without consent to fine-tune commercial LLMs, potentially leaking personal information or infringing upon intellectual property rights (Kandpal et al., 2022). Consequently, empowering data creators with proactive defense mechanisms to safeguard their textual assets against unauthorized LLM fine-tuning has become an urgent imperative. To counteract unauthorized data exploitation, unlearnable examples (UEs) have emerged as a promising defense strategy (Huang et al., 2021). By injecting carefully crafted small perturbations into data prior to publication, UEs induce a shortcut learning effect, rendering models trained on those datasets suffer severe utility degradation, while the underlying text remains functionally intact for legitimate human readers. While UEs have been widely explored in computer vision (Li et al., 2026), extending unlearnable examples to the LLM regime presents distinct challenges. Existing textual UEs methods are predominantly tailored for closed-set classification tasks (e.g., sentiment analysis or topic categorization) in pre-trained language models and rely heavily on injecting static class-specific linguistic cues or surface-level triggers (Li et al., 2023; Wallace et al., 2019). However, modern LLM fine-tuning primarily focuses on instruction following and reasoning tasks, where explicit class labels do not exist. As a result, existing classification-bound text protection techniques fail to generalize, leaving textual data vulnerable to unauthorized LLM exploitation. Moreover, textual data is semantically rich and coherent. Discrete textual edits via character substitutions or trigger insertion may alter meaning or produce conspicuous artifacts, which potentially undermine the utility of the text for legitimate users. A practical defense must therefore preserve semantic fidelity and linguistic naturalness. To bridge this gap, we propose TextCloak, a RL-driven framework designed to protect textual data against unauthorized LLM fine-tuning. Unlike existing heuristic approaches, TextCloak formulates unlearnable text generation as a constrained bi-level optimization problem. Specifically, we design a generative policy to transform batches of clean text into unlearnable variants. The policy operates directly in natural language space, allowing it to preserve the linguistic quality. To optimize this generative policy, we further introduce GRPO-UE, which explicitly considers the model fine-tuning process and dynamically measures the downstream performance degradation induced by the generated unlearnable text. Utilizing this degradation as a reward signal, GRPO-UE updates the generator parameters via group-relative policy optimization, enabling it to discover effective unlearnable patterns that impair LLM generalization ability. Our main contributions are summarized as follows: • Problem formulation. We identify the need to protect textual data in the era of LLMs and formalize it as a constrained bi-level optimization problem. • Novel framework. We introduce TextCloak, a RL-driven framework that crafts unlearnable text while preserving semantic fidelity and linguistic naturalness for legitimate use. To the best of our knowledge, TextCloak is the first framework to leverage unlearnable examples to defend against unauthorized LLM fine-tuning. • Optimization mechanism. We propose GRPO-UE, which dynamically measures protection through the downstream degradation and directly guides the generative policy to discover effective unlearnable patterns. • Comprehensive evaluation. We provide extensive evaluations on six public datasets and nine state-of-the-art LLMs, demonstrating the effectiveness, transferability, robustness, and broad applicability of TextCloak. 2 Related Work 2.1 Unlearnable Examples Unlearnable examples (Huang et al., 2021), which inject small perturbations into training data such that models trained on the protected samples exhibit degraded utility, were first introduced in computer vision. Subsequent studies have improved the robustness and practicality of UEs. For example, robust error-minimizing perturbations were developed to resist adversarial training (Fu et al., 2022), while transferable UEs aim to maintain their protective effect across model architectures, optimization procedures, and datasets (Ren et al., 2023). Other extensions relax the requirement that the defender and unauthorized trainer use identical class labels (Zhang et al., 2023). Recently, efforts have attempted to extend UEs to textual data. For instance, Li et al. (2023) crafts unlearnable text through gradient-guided token search and subsequently extracts reusable surface patterns from optimized examples. RegText (Java et al., 2024) similarly introduces spurious correlations through task-representative, low-frequency words to reduce the generalization of pre-trained language models. In the light of these pioneering works, TextCloak firstly explores unlearnable examples in the context of LLMs, focusing on natural language understanding and reasoning tasks. 2.2 Data Poisoning in LLMs Data poisoning attacks manipulate a model’s training corpus to alter its learned behavior (Steinhardt et al., 2017). Earlier poisoning studies commonly focused on classification models, where attackers modify labels or insert trigger features to induce targeted prediction errors (Fan et al., 2022). The increasing use of web-scale pre-training and instruction tuning has expanded this threat to LLMs. Poisoning instruction-tuning data can associate particular concepts or phrases with attacker-selected behaviors that transfer across multiple downstream tasks (Wan et al., 2023). Instructions themselves can also act as backdoor triggers, enabling adversaries to manipulate models without directly modifying individual input instances or their labels (Xu et al., 2024). Soft prompt injection further demonstrates that a small number of poisoned instruction–response pairs can implant persistent, context-dependent behavior while largely preserving performance on benign inputs (Yan et al., 2024). Collectively, these findings illustrate the sensitivity of LLM fine-tuning to carefully constructed training examples. Instead of exploiting this sensitivity for malicious purposes, TextCloak empowers data owners to proactively protect the text before release. 2.3 Defenses Against Unauthorized LLM Exploitation Existing safeguards against unauthorized LLM exploitation include usage control, post-hoc attribution, model-side remediation, and proactive data protection. Access restrictions, licenses, and crawler exclusion policies express content-owner preferences but depend on voluntary compliance (Jayaraman et al., 2026). Watermarking (Liu et al., 2024a; Zhang et al., 2024; Lau et al., 2024) instead embeds identifiable signals into protected content, enabling owners to test whether their data influenced a potentially unauthorized model. Machine unlearning seeks to remove the influence of sensitive or copyrighted data from trained LLMs (Cao and Yang, 2015; Yao and Xu, 2024), but requires cooperation and model access from the developer and may degrade retained knowledge. Proactive protection instead intervenes before data release, giving content owners direct control over the learnability of their text. Recent effort (Liu et al., 2024b) introduces minimally perceptible perturbations to reduce memorization and instance-level membership inference attack. TextCloak complements this line of work by developing a data-centric solution to thwart unauthorized LLM exploitation. Figure 2: Overview of the proposed TextCloak. 3 Preliminaries 3.1 Problem Formulation Let =(xi,yi)i=1ND=\(x_i,y_i)\_i=1^N denote a clean text corpus, where xix_i is an input or instruction and yiy_i is its target response sequence. A data owner applies a protection mechanism to obtain ~=(x~i,yi)i=1N D=\( x_i,y_i)\_i=1^N before releasing the corpus. The protected pair should convey the same information as the original one, as well as remain fluent and natural, so that the released text retains its utility for legitimate readers. We consider an unauthorized trainer who collects ~ D and fine-tunes an LLM fθf_θ for a specific task by minimizing the standard autoregressive training loss: θ⋆=argminθℒft(fθ;~).θ = _θL_ft(f_θ; D). (1) The data owner seeks protection that degrades the generalization of trained models. Let evalD_eval be held-out evaluation data and let ℒevalL_eval measure downstream error, with larger values indicating worse utility. The protection objective is ~⋆=argmax~∈()ℒeval(fθ⋆;eval), D = _ D (D)L_eval (f_θ ;D_eval ), (2) where ()C(D) is the set of admissible corpora satisfying constraints (e.g., semantic fidelity and linguistic naturalness). This formulation captures the central trade-off: protected text should impair models trained on it without explicitly changing the content presented to legitimate users. 3.2 Unlearnable Examples for Text Classific textual UEs (Li et al., 2023) modify discrete token sequences, injecting task-specific shortcuts (e.g., lexical patterns correlated with class labels) to degrade the generalization of models trained on them. Formally, let xi=(wi,1,…,wi,Ti)x_i=(w_i,1,…,w_i,T_i) be a token sequence and yiy_i be its class label. A textual modification ηi=(pi,si) _i=(p_i,s_i) replaces the token at position pip_i with a candidate token sis_i from the vocabulary and acquiesces the modified sequence xi⊕ηix_i _i. The admissible modifications (xi)A(x_i) are often constrained by an edit budget. Textual UEs can be constructed by solving the following bi-level min-min optimization problem: minθ1N∑i=1Nminηi∈(xi)ℓ(fθ(xi⊕ηi),yi). _θ\ 1N _i=1^N _ _i (x_i) (f_θ(x_i _i),y_i ). (3) The model parameters and textual modifications are optimized alternately. Because token replacement is non-differentiable, first-order gradients from a surrogate model are used to guide the search for effective modifications: si⋆=argmins∈(e(s)−e(wi,pi))⊤∇e(wi,pi)ℓ(fθ(xi),yi),s_i = _s (e(s)-e(w_i,p_i) ) _e(w_i,p_i) (f_θ(x_i),y_i ), (4) where V is the vocabulary and e(⋅)e(·) denotes the token embedding in pre-trained language models. 4 The Proposed TextCloak 4.1 Overview We propose TextCloak, an RL-driven framework that protects text from unauthorized LLM exploitation and comprises three key components. First, TextCloak leverages a generative policy that rewrites a clean corpus into semantics-preserving unlearnable text. Second, GRPO-UE measures the modified candidates by fine-tuning surrogate LLMs and evaluating the resulting degradation on held-out clean data. The candidates generated from the same batch form a comparison group, allowing their degradation scores to be converted into relative advantages without a learned value model. Third, a constrained bi-level loop alternates between inner surrogate fine-tuning and outer policy to update the generative policy parameters. The overall framework is illustrated in Figure 2. 4.2 UE Generative Policy 4.2.1 Batch-level generation. Let ℬ=(xi,yi)i=1bB=\(x_i,y_i)\_i=1^b denote a mini-batch of clean instruction-response pairs. We define a generative policy πϕ _φ that transforms ℬB into a protected batch ℬ~=(x~i,yi)i=1b B=\( x_i,y_i)\_i=1^b, where each (x~i,yi)( x_i,y_i) is a semantics-preserving rewrite of (xi,yi)(x_i,y_i). The policy is parameterized by ϕφ and is implemented as an autoregressive large language model to operate directly in natural language space. The generation process is conditioned on both the clean batch and an editing instruction p that specifies the tasks and requirements as detailed in Appendix C.1. Formally, we have ℬ~∼πϕ(⋅∣p,ℬ). B _φ(· p,B). (5) We note that the policy transforms the entire batch in one shot. Conditioning on the batch allows the policy to introduce more generalizable patterns across instances. Let a=(a1,…,aT)a=(a_1,…,a_T) denote the token sequence in a candidate batch and let s=(p,ℬ)s=(p,B). Its probability factorizes as πϕ(a∣s)=∏t=1Tπϕ(at∣s,a<t). _φ(a s)= _t=1^T _φ(a_t s,a_<t). (6) 4.2.2 Semantic fidelity. We measure the semantic fidelity of the protected batch and its clean counterpart using Sentence-BERT (SBERT) (Reimers and Gurevych, 2019) embeddings. For each instruction-response pair (xi,yi)(x_i,y_i), let ui=xi∥yiu_i=x_i \|y_i and u~i=x~i∥yi u_i= x_i \|y_i, where ∥ \| denotes sequence concatenation. We compute the batch-level semantic fidelity as Ssem(ℬ,ℬ~)=1b∑i=1bcos(ϵ(ui),ϵ(u~i)).S_sem(B, B)= 1b _i=1^bcos (ε(u_i),ε( u_i) ). (7) where ϵ(⋅)ε(·) denotes the SBERT encoder and cos(⋅,⋅)cos(·,·) denotes cosine similarity. To preserve the semantic content of the original text, we require Ssem(ℬ,ℬ~)≥τsemS_sem(B, B)≥ _sem. 4.2.3 Linguistic naturalness. We quantify linguistic naturalness using perplexity under pre-trained GPT-2 (Radford et al., 2019) pωp_ω. For a tokenized sequence z=(w1,…,wT)z=(w_1,…,w_T), the batch-level perplexity is computed as PPL(z)=exp−1T∑t=1Tlogpω(wt∣w<t),Sppl(ℬ,ℬ~)=1b∑i=1bmin1,PPL(ui)PPL(u~i). splitPPL(z)&= \- 1T _t=1^T p_ω(w_t w_<t) \,\\ S_ppl(B, B)&= 1b _i=1^b \1, PPL(u_i)PPL( u_i) \. split (8) We constrain the perplexity distance above a threshold to maintain the linguistic naturalness: Sppl(ℬ,ℬ~)≥τpplS_ppl(B, B)≥ _ppl. 4.3 Group-Relative Policy Optimization for UEs To optimize the generative policy, we introduce GRPO-UE. Classic unlearnable examples are optimized by minimizing the training loss of a surrogate model on the protected data. However, this doesn’t directly reflect the downstream degradation, which is the ultimate goal of data protection. Therefore, we simulate the unauthorized LLM fine-tuning process and measure the resulting degradation on held-out clean data, which provides an effective signal for policy optimization. 4.3.1 Group sampling. For each clean batch ℬB, the policy πϕroll _ _roll rolls out K times and formulate a group of candidate batches ℬ~(k)k=1K\ B^(k)\_k=1^K. All candidates share the same clean context and are generated independently, so they explore alternative protective patterns while remaining directly comparable. Let gψ0∼g_ _0 denote a surrogate drawn from the defender’s surrogate distribution. For each candidate k, we initialize an independent surrogate copy at ψ0 _0 and apply the prescribed inner-loop fine-tuning procedure: ψk⋆=argminψℒft(gψ;ℬ~(k)). _k = _ψL_ft (g_ψ; B^(k) ). (9) Similarly, we can also obtain a clean baseline gψcl⋆g_ _cl by applying the same fine-tuning procedure to ℬB. Sharing the clean batch, surrogate initialization, and inner optimization schedule isolates the effect of each candidate rewrite. 4.3.2 Reward design. We evaluate every fine-tuned surrogate on a clean held-out set valD_val. The degradation induced by candidate k can be quantified as: dk=ℒeval(gψk⋆;val)−ℒeval(gψcl⋆;val).d_k=L_eval(g_ _k ;D_val)-L_eval(g_ _cl ;D_val). (10) Thus, dk>0d_k>0 indicates that training on the protected candidate causes more held-out error than training on the corresponding clean batch. We observe that the ℒeval(gψcl⋆;val)L_eval(g_ _cl ;D_val) is a constant for all candidates in the same group, so it can be omitted from the reward computation in practice. To prevent the policy from increasing degradation by changing the meaning or producing unnatural text, we combine dkd_k with the two utility measurements. Let Ssem(k)=Ssem(ℬ,ℬ~(k))S_sem^(k)=S_sem(B, B^(k)) and Sppl(k)=Sppl(ℬ,ℬ~(k))S_ppl^(k)=S_ppl(B, B^(k)). The reward is rk=dk−λsem[τsem−Ssem(k)]+−λppl[τppl−Sppl(k)]+,r_k=d_k- _sem[ _sem-S_sem^(k)]_+- _ppl[ _ppl-S_ppl^(k)]_+, (11) where [u]+=max(u,0)[u]_+= (u,0), and λsem _sem and λppl _ppl control the semantic and naturalness penalties, respectively. 4.3.3 Policy update. GRPO-UE adapts group-relative policy optimization (GRPO) (Shao et al., 2024) as base RL framework for UEs. For the K candidate batches, we compute A^k=rk−r¯K−1∑j=1K(rj−r¯)2+ϵ,r¯=K−1∑j=1Krj. A_k= r_k- r K^-1 _j=1^K(r_j- r)^2+ε, r=K^-1 _j=1^Kr_j. (12) Given the rollout policy, the token-level importance ratio is ρk,t(ϕ)=πϕ(ak,t∣s,ak,<t)πϕroll(ak,t∣s,ak,<t). _k,t(φ)= _φ(a_k,t s,a_k,<t) _ _roll(a_k,t s,a_k,<t). (13) The policy maximizes the clipped objective JGRPO(ϕ)= J_GRPO(φ)= 1K∑k=1K1Tk∑t=1Tkℓk,t−βDKL(πϕ∥πref), 1K _k=1^K 1T_k _t=1^T_k _k,t-β D_KL( _φ\| _ref), (14) ℓk,t= _k,t= minρk,tA^k,ρ¯k,tA^k, \ _k,t A_k, ρ_k,t A_k\, ρ¯k,t= ρ_k,t= clip(ρk,t,1−ε,1+ε), ( _k,t,1- ,1+ ), where πref _ref is the initial reference policy. Clipping limits abrupt policy changes, and length normalization prevents long rewrites from dominating the update. The KL term measures the average token-level divergence across the sampled sequences, thereby discouraging the policy from drifting away from fluent natural-language generation. 4.4 Constrained Bi-level Optimization Overall, TextCloak solves the constrained bi-level optimization problem to balance the trade-off between unlearnability and utility. The outer problem maximizes the expected degradation on held-out clean data, while the inner problem simulates the unauthorized fine-tuning process. The semantic fidelity and linguistic naturalness constraints ensure that the protected text remains usable for legitimate readers: maxϕ[ℒeval(gψ⋆;val)]s.t.ψ⋆=argminψℒft(gψ;ℬ~),Ssem(ℬ,ℬ~)≥τsem,Sppl(ℬ,ℬ~)≥τppl. split _φ &E [L_eval(g_ψ ;D_val) ]\\ s.t. &ψ = _ψL_ft(g_ψ; B),\\ &S_sem(B, B)≥ _sem,\\ &S_ppl(B, B)≥ _ppl. split (15) We optimize Equation 15 by alternating three steps described in Algorithm 1. First, the current policy generates K candidate rewrites for each clean batch. Second, SBERT similarity and the normalized perplexity score measure their utility, while fresh surrogate copies are fine-tuned on the candidates and evaluated on clean held-out data. Third, Equations 11-14 convert the resulting degradation and utility scores into group-relative advantages and update the policy. Once training is complete, protection requires only one forward pass. Algorithm 1 Constrained bi-level optimization of TextCloak Input: Clean corpus D, validation set valD_val, initial policy πϕ _φ, reference policy πref _ref, and surrogate distribution S Parameters: Group size K, thresholds τsem _sem and τppl _ppl, and penalty weights λsem _sem and λppl _ppl Output: Protected corpus ~ D 1: repeat 2: Sample ℬ∼B and gψ0∼g_ _0 . 3: Set πϕroll←πϕ _ _roll← _φ. 4: Sample ℬ~(k)k=1K\ B^(k)\_k=1^K from πϕroll(⋅∣p,ℬ) _ _roll(· p,B). 5: Fine-tune a clean baseline gψcl⋆g_ _cl on ℬB. 6: for k=1,…,Kk=1,…,K do 7: Compute Ssem(k)S_sem^(k) and Sppl(k)S_ppl^(k). 8: Fine-tune gψk⋆g_ _k on ℬ~(k) B^(k). 9: Compute dkd_k and rkr_k using Equations 10 and 11. 10: end for 11: Compute A^kk=1K\ A_k\_k=1^K using Equation 12. 12: Update ϕφ by maximizing JGRPOJ_GRPO in Equation 14. 13: until the policy satisfies the stopping criterion 14: Generate ~ D with πϕ _φ and retain outputs satisfying both utility thresholds. 15: return ~ D 5 Experiments 5.1 Experimental Setup 5.1.1 Datasets. We consider six representative public-available datasets including ARC-Challenge (Clark et al., 2018), MATH (Hendrycks et al., 2021), MMLU-Pro (Wang et al., 2024), RACE (Lai et al., 2017), HumanEval (Chen et al., 2021), and MedQA (Jin et al., 2021), which covers diverse domains and tasks as summarized in Table 1. Detailed information about the datasets is provided in Appendix A.1. Table 1: Statistics of the evaluation datasets. Dataset Task Size ARC-Challenge Commonsense reasoning 2,590 MATH Mathematical reasoning 12,500 MMLU-Pro Multitask language understanding 12,032 RACE Reading comprehension 97,687 HumanEval Code generation 164 MedQA-USMLE Medical question answering 12,723 5.1.2 LLM Backbones. We evaluate nine state-of-the-art LLMs spanning multiple families and sizes: Qwen3-4B and Qwen3-14B (Yang et al., 2025), Gemma-3-12B-IT (Team, 2025), Mistral-7B-Instruct-v0.3 (Jiang et al., 2023), GPT-OSS-20B (Agarwal et al., 2025), Llama-3.2-3B and Llama-3.1-8B-Instruct (Grattafiori et al., 2024), Phi-4 (Abdin et al., 2024), and GLM-4-9B-Chat (Glm et al., 2024). 5.1.3 Baselines. Due to the lack of existing methods for defending against unauthorized LLM fine-tuning, we extend related works on UEs to our setting. Specifically, we formulate the following baselines: (i) Zero-Shot evaluates the target LLMs on clean test data without any fine-tuning. (i) Clean fine-tunes each target LLM on the original examples and establishes the utility attainable without protection. (i) Random-Prepend and Random-Append prepend or append five tokens sampled uniformly from vocabulary of Llama-3-8B. (iv) Textual UE (Li et al., 2023) leverages error-minimizing perturbations to construct unlearnable examples for text classification in pre-trained LLMs through token replacement. (v) MEM-3 and MEM-5 (Liu et al., 2024c) insert optimized text triggers of three and five tokens, respectively, to prevent generalization in multimodal contrastive learning. 5.1.4 Evaluation Metrics. For downstream performance, we use accuracy for ARC-Challenge, MMLU-Pro, RACE, and MedQA-USMLE; exact match for MATH; and pass@1 for HumanEval. Our primary metric is the absolute performance drop Δ after fine-tuning on the protected corpus. We repeat experiments with three random seeds and report the average. Δ=ℳclean−ℳprotected, =M_clean-M_protected, (16) Table 2: Protection performance and data quality across six datasets. Parentheses report the performance drop Δ from clean fine-tuning; larger Δ indicates stronger protection. SBERT and PPL are averaged across datasets. Bold and underlined task-performance values denote the lowest and second-lowest results, excluding Zero-Shot and Clean. Task Performance (%) Data Quality Method ARC MATH MMLU RACE HEval MedQA Avg. SBERT↑ PPL↓ Zero-Shot 78.7 (+13.2) 20.7 (+17.8) 36.3 (+12.8) 76.6 (+12.7) 56.3 (+18.8) 63.6 (+3.7) 55.4 (+13.2) 1.00 18.7 Clean 91.9 (0.0) 38.5 (0.0) 49.1 (0.0) 89.3 (0.0) 75.0 (0.0) 67.3 (0.0) 68.5 (0.0) 1.00 18.7 Random-Prepend 91.3 (+0.6) 38.1 (+0.4) 49.6 (-0.5) 88.0 (+1.3) 65.6 (+9.4) 63.9 (+3.4) 66.1 (+2.4) 0.91 29.7 Random-Append 91.2 (+0.7) 36.6 (+1.9) 47.9 (+1.2) 87.6 (+1.7) 65.6 (+9.4) 62.8 (+4.5) 65.3 (+3.2) 0.91 28.7 Textual UE 91.4 (+0.5) 38.3 (+0.2) 47.4 (+1.7) 87.1 (+2.2) 56.3 (+18.8) 63.4 (+3.9) 64.0 (+4.5) 0.92 23.1 MEM-3 90.3 (+1.6) 38.9 (-0.4) 50.5 (-1.4) 87.2 (+2.1) 56.3 (+18.8) 64.7 (+2.6) 64.6 (+3.9) 0.91 27.1 MEM-5 90.8 (+1.1) 36.5 (+2.0) 49.9 (-0.8) 88.3 (+1.0) 62.5 (+12.5) 63.2 (+4.1) 65.2 (+3.3) 0.87 34.0 TextCloak 86.6 (+5.3) 34.5 (+4.0) 17.7 (+31.4) 83.8 (+5.5) 53.1 (+21.9) 61.1 (+6.2) 56.1 (+12.4) 0.95 20.1 Table 3: Transferability of protection across unseen LLM architectures. Parentheses report the performance drop Δ . Bold and underlined values denote the lowest and second-lowest results. ARC-Challenge MATH MMLU-Pro Method GPT-20B Llama-8B Phi-4 Qwen-14B GPT-20B Llama-8B Qwen-4B Qwen-14B GLM-9B GPT-20B Phi-4 Qwen-14B Zero-Shot 21.6 (+64.6) 72.7 (+1.5) 18.6 (+46.3) 78.8 (+13.1) 14.9 (+25.9) 21.5 (+8.1) 19.0 (+7.1) 13.5 (+32.9) 28.1 (+4.8) 20.7 (+27.9) 8.1 (+18.2) 44.5 (+11.1) Clean 86.2 (0.0) 74.2 (0.0) 64.9 (0.0) 92.0 (0.0) 40.8 (0.0) 29.6 (0.0) 26.1 (0.0) 46.5 (0.0) 32.9 (0.0) 48.6 (0.0) 26.3 (0.0) 55.6 (0.0) Random-Prepend 79.7 (+6.5) 54.1 (+20.1) 48.0 (+16.9) 88.5 (+3.5) 42.3 (-1.5) 28.9 (+0.7) 28.3 (-2.2) 46.4 (+0.1) 32.5 (+0.3) 49.8 (-1.2) 29.9 (-3.5) 56.1 (-0.5) Random-Append 87.8 (-1.6) 51.2 (+23.0) 44.4 (+20.5) 83.1 (+8.9) 41.9 (-1.1) 26.7 (+2.9) 23.1 (+2.9) 43.5 (+2.9) 30.9 (+2.0) 44.7 (+3.9) 28.3 (-2.0) 12.3 (+43.3) Textual UE 84.5 (+1.7) 71.7 (+2.6) 62.6 (+2.3) 92.4 (-0.4) 40.9 (-0.1) 28.7 (+0.9) 27.1 (-1.0) 46.5 (-0.1) 32.1 (+0.7) 48.5 (+0.1) 46.6 (-20.3) 53.5 (+2.1) MEM-3 82.3 (+3.8) 52.4 (+21.8) 55.8 (+9.1) 87.2 (+4.8) 41.3 (-0.5) 28.5 (+1.1) 24.9 (+1.2) 44.9 (+1.5) 33.3 (-0.4) 49.3 (-0.7) 36.8 (-10.5) 56.3 (-0.7) MEM-5 87.6 (-1.5) 51.7 (+22.5) 53.8 (+11.0) 87.7 (+4.3) 41.5 (-0.7) 28.9 (+0.7) 25.1 (+1.0) 46.7 (-0.3) 33.2 (-0.3) 48.7 (-0.1) 35.5 (-9.1) 55.5 (+0.1) TextCloak 80.1 (+6.1) 59.2 (+15.0) 19.0 (+45.8) 84.8 (+7.2) 33.9 (+6.9) 26.1 (+3.5) 22.3 (+3.8) 42.9 (+3.6) 27.9 (+5.0) 23.1 (+25.5) 20.1 (+6.3) 16.3 (+39.3) RACE HumanEval MedQA-USMLE Method GPT-20B Mistral-7B Phi-4 Qwen-4B Gemma-12B Llama-8B Phi-4 Qwen-4B GLM-9B Phi-4 Qwen-4B Qwen-14B Zero-Shot 22.3 (+63.1) 76.8 (+0.8) 11.2 (+55.7) 81.5 (+2.6) 87.5 (+9.4) 59.4 (+6.2) 71.9 (+9.4) 62.5 (+15.6) 45.6 (+7.9) 17.5 (+44.2) 47.8 (+9.2) 64.3 (+3.1) Clean 85.3 (0.0) 77.6 (0.0) 66.9 (0.0) 84.1 (0.0) 96.9 (0.0) 65.6 (0.0) 81.3 (0.0) 78.1 (0.0) 53.6 (0.0) 61.7 (0.0) 57.0 (0.0) 67.5 (0.0) Random-Prepend 85.2 (+0.2) 77.8 (-0.2) 78.7 (-11.8) 83.7 (+0.4) 81.3 (+15.6) 56.3 (+9.4) 78.1 (+3.1) 37.5 (+40.6) 53.2 (+0.4) 53.2 (+8.5) 53.6 (+3.4) 68.4 (-0.9) Random-Append 82.5 (+2.9) 78.3 (-0.7) 73.9 (-7.0) 80.9 (+3.2) 87.5 (+9.4) 43.8 (+21.9) 78.1 (+3.1) 62.5 (+15.6) 51.0 (+2.6) 52.8 (+8.9) 50.2 (+6.8) 65.4 (+2.1) Textual UE 85.0 (+0.4) 75.4 (+2.2) 83.1 (-16.2) 84.4 (-0.3) 84.4 (+12.5) 56.3 (+9.4) 78.1 (+3.1) 37.5 (+40.6) 53.3 (+0.2) 63.2 (-1.5) 46.7 (+10.2) 64.8 (+2.7) MEM-3 85.1 (+0.2) 77.0 (+0.6) 73.2 (-6.2) 83.9 (+0.2) 87.5 (+9.4) 59.4 (+6.2) 75.0 (+6.3) 37.5 (+40.6) 53.2 (+0.4) 52.9 (+8.8) 53.8 (+3.1) 67.3 (+0.2) MEM-5 86.0 (-0.7) 76.8 (+0.8) 80.2 (-13.2) 84.0 (+0.1) 84.4 (+12.5) 62.5 (+3.1) 78.1 (+3.1) 40.6 (+37.5) 53.5 (+0.1) 55.9 (+5.7) 53.6 (+3.4) 67.2 (+0.2) TextCloak 70.8 (+14.5) 76.3 (+1.3) 48.0 (+18.9) 72.0 (+12.1) 78.1 (+18.8) 40.6 (+25.0) 75.0 (+6.3) 31.3 (+46.9) 47.3 (+6.3) 53.0 (+8.6) 46.5 (+10.5) 63.2 (+4.3) 5.1.5 Implementation Details. The generative policy is initialized from Llama-3-8B and is optimized for 2 epochs. We set the group size to 4. The inner-loop surrogate utilizes Qwen3-8, fine-tuned with LoRA-8. The constraints are τsem _sem and τppl _ppl as 0.9, with penalty weights λsem _sem and λppl _ppl as 1.0. To accelerate the experiments, we run on 8 NVIDIA A100 GPUs with a batch size of 8 per device, which costs ∼ 40s per batch. More implementation details are provided in Appendix A.3. 5.2 Protection Effectiveness RQ1: Can TextCloak reduce the performance of unauthorized LLMs while preserving utility? We compare TextCloak with baselines on six datasets, which are summarized in Table 2. Overall, TextCloak consistently outperforms the baselines across all datasets, achieving the largest average performance drop Δ while maintaining high semantic fidelity and a normalized perplexity score close to one. Notably, TextCloak achieves an average performance drop of 12.4%, which is significantly higher than the best baseline. Interestingly, TextCloak impairs the performance of unauthorized LLMs below the zero-shot baseline on some datasets (e.g., HEval), indicating that the protected text not only prevents learning from the data but also actively misleads the model. 5.3 Transfer Experiments RQ2: Does the protection transfer across unseen LLM architectures and fine-tuning configurations? 5.3.1 Transfer Across LLM Architectures. Table 3 evaluates the transferability of TextCloak across different LLM architectures. We can observe that TextCloak consistently achieves satisfactory transferability across unseen LLMs. Notably, TextCloak largely degrades the performance of Phi-4 and Qwen3 models on most datasets, indicating the good applicability of TextCloak for SOTA LLM families. However, the transferability of TextCloak is limited since the error-minimized text perturbations are specific to the surrogate. Figure 3: Transferability across fine-tuning configurations. 5.3.2 Transfer Across Fine-tuning Configurations. We also evaluate the transferability across different fine-tuning configurations, including LoRA-64, QLoRA, DoRA, soft-prompt, full fine-tuning (FFT), as shown in Figure 3. Overall, TextCloak demonstrates strong transferability across various fine-tuning methods. For instance, TextCloak achieves the largest performance degradation on soft prompt while remaining relatively less effective on FFT, which is because fine-tuning with larger parameter updates unlocks more capacity and leads to better adaptation to the protected text. 5.4 Ablation Study RQ3: How do the components of TextCloak contribute to its performance? We isolate the contribution of the degradation reward, utility constraints, and optimization algorithms in Table 4. We can find that removing the degradation reward and GRPO-UE significantly reduces the performance drop, while abating the semantic and naturalness constraints leads to a higher performance drop but lower language quality. This indicates that each component of TextCloak is essential for achieving a balance between protection and utility. Table 4: Ablation study of TextCloak. Values in parentheses denote absolute differences from the full method. Variant Δ ↑ SBERT↑ PPL↓ TextCloak 54.7 (-0.0) 0.91 (-0.00) 7.0 (+0.0) w/o degradation reward 9.0 (-45.7) 0.90 (-0.02) 8.5 (+1.5) w/o semantic constraint 34.3 (-20.4) 0.77 (-0.15) 7.2 (+0.3) w/o naturalness constraint 17.7 (-37.0) 0.86 (-0.05) 14.0 (+7.0) w/o GRPO-UE 7.2 (-47.5) 0.85 (-0.06) 7.9 (+0.9) 5.5 Robustness Analysis RQ4: How robust is TextCloak against various defense strategies? Unauthorized trainers may attempt to invalidate the protection by transforming the released corpus (e.g., lowercase normalization, punctuation removal, whitespace stripping, and paraphrasing) or adapting adversarial training (AT) techniques, as shown in Figure 4. Generally, the protected text maintains its effectiveness against various defense approaches. Among them, AT is the most effective while punctuation is the least effective, suggesting that TextCloak provides strong resistance beyond surface-level modifications. Figure 4: Task performance under various defense strategies. 5.6 Human Evaluation Table 5: Human evaluations under three-scaled rating. Scores are averaged across human annotators and LLM judges. Method Naturalness↑ Fluency↑ Human Utility↑ Clean 2.92±0.072.92± 0.07 2.93±0.052.93± 0.05 2.99±0.012.99± 0.01 Textual UE 2.25±0.122.25± 0.12 2.03±0.272.03± 0.27 1.65±1.191.65± 1.19 MEM-5 1.35±0.561.35± 0.56 1.57±0.311.57± 0.31 1.94±0.281.94± 0.28 TextCloak 2.52 ± 0.08 2.84 ± 0.11 2.68 ± 0.28 RQ5: Do the protected examples preserve language quality and overall utility for legitimate users? We sample 20 samples from each dataset and then employ three human annotators and three LLM judges to evaluate the naturalness, fluency, and utility of the protected text. The rating rubrics are provided in Appendix B.1. As showcased in Table 5, TextCloak achieves comparable scores to the clean examples and significantly outperforms the SOTA UEs across all three metrics. Significantly, the protected text generated by TextCloak remains similar fluency and human utility to the original text, indicating that the protected text is practically usable while less compromising the user experience. 5.7 Case Study RQ6: What are the qualitative characteristics of the protected examples and their impact on downstream model behavior? Figure 5 presents representative examples of protected text generated by TextCloak. TextCloak converts the original text into UE by introducing instance-specific thinking or reasoning shortcuts (e.g., semantic hints) such that disregarding the generalization ability of the downstream model. At the same time, the protected text remains semantically faithful and linguistically natural, demonstrating the practical effectiveness of TextCloak in preventing unauthorized LLM fine-tuning. Figure 5: Case study. 6 Conclusion We presented TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM fine-tuning. Its generative policy produces semantically faithful and natural unlearnable text, while GRPO-UE directly optimizes the policy using downstream degradation measured on fine-tuned surrogate models. Experiments across six datasets and nine LLMs indicate that TextCloak consistently impairs unauthorized LLM fine-tuning, transfers across model architectures and training configurations, and remains robust under adaptive defense strategies. These results demonstrate the potential of RL-guided unlearnable text as a practical, model-transferable approach for proactive data protection. Future work will develop sophisticated UEs in high-stakes domains. References M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, et al. (2024) Phi-4 technical report. arXiv preprint arXiv:2412.08905. Cited by: §5.1.2. S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Bao, et al. (2025) Gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925. Cited by: §5.1.2. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020) Language models are few-shot learners. Advances in neural information processing systems 33, p. 1877–1901. Cited by: §1. Y. Cao and J. Yang (2015) Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, p. 463–480. Cited by: §2.3. N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, et al. (2021) Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), p. 2633–2650. Cited by: §1. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: §A.1, §5.1.1. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord (2018) Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: §A.1, §5.1.1. J. Fan, Q. Yan, M. Li, G. Qu, and Y. Xiao (2022) A survey on data poisoning attacks and defenses. In 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), p. 48–55. Cited by: §2.2. S. Fu, F. He, Y. Liu, L. Shen, and D. Tao (2022) Robust unlearnable examples: protecting data against adversarial learning. arXiv preprint arXiv:2203.14533. Cited by: §2.1. T. Glm, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Zhang, D. Rojas, G. Feng, H. Zhao, et al. (2024) Chatglm: a family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793. Cited by: §5.1.2. A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. (2024) The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §5.1.2. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), External Links: Link Cited by: §A.1, §5.1.1. H. Huang, X. Ma, S. M. Erfani, J. Bailey, and Y. Wang (2021) Unlearnable examples: making personal data unexploitable. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2.1. A. Java, S. Shahid, and C. Agarwal (2024) Towards operationalizing right to data protection. arXiv preprint arXiv:2411.08506. Cited by: §2.1. B. Jayaraman, V. Marathe, H. Mozaffari, W. Shen, and K. Kenthapadi (2026) Permissioned llms: enforcing access control in large language models. Advances in Neural Information Processing Systems 38, p. 81743–81773. Cited by: §2.3. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. (2023) Mistral 7b. arXiv preprint arXiv:2310.06825. Cited by: §5.1.2. D. Jin, E. Pan, N. Oufattole, W. Weng, H. Fang, and P. Szolovits (2021) What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences 11 (14), p. 6421. Cited by: §A.1, §5.1.1. N. Kandpal, E. Wallace, and C. Raffel (2022) Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, p. 10697–10707. Cited by: §1. G. Lai, Q. Xie, H. Liu, Y. Yang, and E. Hovy (2017) Race: large-scale reading comprehension dataset from examinations. In Proceedings of the 2017 conference on empirical methods in natural language processing, p. 785–794. Cited by: §A.1, §5.1.1. G. K. R. Lau, X. Niu, H. Dao, J. Chen, C. Foo, and B. K. H. Low (2024) Waterfall: scalable framework for robust text watermarking and provenance for llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 20432–20466. Cited by: §2.3. D. Li, B. Jiang, L. Huang, A. Beigi, C. Zhao, Z. Tan, A. Bhattacharjee, Y. Jiang, C. Chen, T. Wu, et al. (2025) From generation to judgment: opportunities and challenges of llm-as-a-judge. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 2757–2791. Cited by: §1. X. Li, M. Liu, and S. Gao (2023) Make text unlearnable: exploiting effective patterns to protect personal data. In The Third Workshop on Trustworthy Natural Language Processing, p. 249. Cited by: §A.2, §C.1, §1, §2.1, §3.2, §5.1.3. Z. Li, J. Cai, G. Xu, H. Zheng, Q. Li, F. Zhou, S. Yang, C. Ling, and B. Wang (2026) Versatile transferable unlearnable example generator. Advances in Neural Information Processing Systems 38, p. 17495–17522. Cited by: §1. A. Liu, L. Pan, Y. Lu, J. Li, X. Hu, X. Zhang, L. Wen, I. King, H. Xiong, and P. Yu (2024a) A survey of text watermarking in the era of large language models. ACM Computing Surveys 57 (2), p. 1–36. Cited by: §2.3. R. Liu, T. Tran, T. Wang, H. Hu, S. Wang, and L. Xiong (2024b) ExpShield: safeguarding web text from unauthorized crawling and llm exploitation. arXiv preprint arXiv:2412.21123. Cited by: §2.3. X. Liu, X. Jia, Y. Xun, S. Liang, and X. Cao (2024c) Multimodal unlearnable examples: protecting data against multimodal contrastive learning. In Proceedings of the 32nd ACM International Conference on Multimedia, p. 8024–8033. Cited by: §A.2, §5.1.3. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, et al. (2022) Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, Cited by: §1. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. (2019) Language models are unsupervised multitask learners. OpenAI blog 1 (8), p. 9. Cited by: §4.2.3. N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 3982–3992. Cited by: §4.2.2. J. Ren, H. Xu, Y. Wan, X. Ma, L. Sun, and J. Tang (2023) Transferable unlearnable examples. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §2.1. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. (2024) Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §4.3.3. J. Steinhardt, P. W. W. Koh, and P. S. Liang (2017) Certified defenses for data poisoning attacks. Advances in neural information processing systems 30. Cited by: §2.2. G. Team (2025) Gemma 3 technical report. CoRR abs/2503.19786. Cited by: §5.1.2. E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh (2019) Universal adversarial triggers for attacking and analyzing nlp. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 2153–2162. Cited by: §1. A. Wan, E. Wallace, S. Shen, and D. Klein (2023) Poisoning language models during instruction tuning. In International Conference on Machine Learning, p. 35413–35425. Cited by: §2.2. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, et al. (2024) Mmlu-pro: a more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems 37, p. 95266–95290. Cited by: §A.1, §5.1.1. J. Xu, M. Ma, F. Wang, C. Xiao, and M. Chen (2024) Instructions as backdoors: backdoor vulnerabilities of instruction tuning for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 3111–3126. Cited by: §2.2. J. Yan, V. Yadav, S. Li, L. Chen, Z. Tang, H. Wang, V. Srinivasan, X. Ren, and H. Jin (2024) Backdooring instruction-tuned large language models with virtual prompt injection. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 6065–6086. Cited by: §2.2. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §5.1.2. Y. Yao and X. Xu (2024) Large language model unlearning. Advances in Neural Information Processing Systems 37, p. 105425–105475. Cited by: §2.3. J. Zhang, X. Ma, Q. Yi, J. Sang, Y. Jiang, Y. Wang, and C. Xu (2023) Unlearnable clusters: towards label-agnostic unlearnable examples. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3984–3993. Cited by: §2.1. R. Zhang, S. S. Hussain, P. Neekhara, and F. Koushanfar (2024) \remark-LLM\: a robust and efficient watermarking framework for generative large language models. In 33rd USENIX Security Symposium (USENIX Security 24), p. 1813–1830. Cited by: §2.3. C. Zhao, Z. Tan, D. Li, Z. Yu, and H. Liu (2026a) To see is not to learn: protecting multimodal data from unauthorized fine-tuning of large vision-language model. arXiv preprint arXiv:2605.14291. Cited by: §1. C. Zhao, Z. Tan, P. Ma, D. Li, B. Jiang, Y. Wang, Y. Yang, and H. Liu (2026b) Is chain-of-thought reasoning of llms a mirage? A data distribution lens. In ACL (Findings), p. 15231–15261. Cited by: §1. Appendix A Detailed Experiment Setting This appendix provides additional details for reproducing the experiments in the main paper. We focus on the experimental protocol used to evaluate whether protected text degrades unauthorized LLM fine-tuning while preserving the utility of the released corpus for legitimate users. Unless otherwise stated, the same data splits, prompts, metrics, and decoding configurations are used across TextCloak and all baselines. A.1 Datasets We evaluate TextCloak on six public datasets that cover factual reasoning, mathematical problem solving, multitask knowledge, reading comprehension, code generation, and medical question answering. The datasets are selected to stress different aspects of LLM fine-tuning. This diversity allows us to test whether the learned unlearnable patterns are tied to one narrow output format or remain effective across heterogeneous language tasks. ARC-Challenge (Clark et al., 2018) contains grade-school science questions that are difficult for retrieval-only or shallow pattern-matching systems. Each instance consists of a natural-language question and multiple answer options. We cast each example into an instruction-following format where the model is asked to choose the correct option. Accuracy is used as the downstream metric. MATH (Hendrycks et al., 2021) contains competition-style mathematical problems spanning algebra, geometry, probability, number theory, and related topics. Because answers are often short expressions or numbers, we evaluate MATH with exact match after standard answer normalization. This dataset tests whether protection remains effective for examples whose useful content depends on precise symbolic reasoning. MMLU-Pro (Wang et al., 2024) extends multitask language understanding evaluation with more challenging questions and a larger answer-option space than the original MMLU benchmark. We use it to evaluate broad-domain knowledge and reasoning. Each example is formatted as a multiple-choice instruction, and accuracy is reported. RACE (Lai et al., 2017) is a reading-comprehension benchmark collected from English examinations. Each example contains a passage, a question, and multiple answer candidates. We keep the passage and question together in the instruction field and evaluate the selected answer with accuracy. RACE is included because the protected text must preserve relatively long contextual passages HumanEval (Chen et al., 2021) evaluates functional code generation from natural-language programming prompts. Following the main paper, we report pass@1. This dataset tests whether unlearnable text can affect code-oriented fine-tuning where downstream success is determined by executable behavior. MedQA-USMLE (Jin et al., 2021) contains medical exam questions derived from the United States Medical Licensing Examination style. We formulate each item as a medical multiple-choice question and evaluate accuracy. Since this is a high-stakes domain, we use the dataset only as a benchmark for measuring unauthorized fine-tuning degradation. For all datasets, the protected corpus is generated only from training examples. Held-out validation data are used during GRPO-UE to compute the surrogate degradation reward, and final test data are reserved for evaluation of target LLMs. The target response yiy_i is kept unchanged by all protection methods; TextCloak rewrites the input or instruction text x~i x_i so that the released pair remains interpretable to human readers while impairing unauthorized LLM fine-tuning. A.2 Baselines We compare against clean fine-tuning, no fine-tuning, and several adapted textual unlearnable-example baselines. Because prior textual UE methods are primarily designed for classification or settings specific to other domains (e.g., segmentation, verification), we adapt them to the instruction-tuning in LLMs used in this paper while preserving their core mechanisms. Zero-Shot evaluates each target LLM directly on the clean test set without any task-specific fine-tuning. This baseline measures the capability already present in the model before it sees the released corpus. Clean fine-tunes each target LLM on the original unprotected training examples. Its score represents the performance an unauthorized trainer can obtain when the data owner releases the clean corpus. We use Clean as the reference for the performance drop Δ reported in the main paper. Random-Prepend and Random-Append add five randomly sampled tokens to each training input. The tokens are sampled from the Llama-3-8B tokenizer vocabulary and are inserted before or after the original input, respectively. These two baselines test whether arbitrary surface noise is sufficient to impair fine-tuning. Textual UE (Li et al., 2023) applies gradient-guided token replacement to construct unlearnable text. Since the original method is designed mainly for classification-style supervision in pre-trained language models, we adapt the loss to the autoregressive instruction-tuning objective and restrict replacements to the input side. The target response is kept fixed so that changes in downstream performance are caused by protected inputs corrupted labels. Further, we replace the vocabulary set with the byte-pair encoding (BPE) encoding of the target LLM to ensure that the perturbations are valid tokens. MEM-3 and MEM-5 (Liu et al., 2024c) insert optimized textual triggers of length three and five, respectively, which are originally designed for multimodal contrastive learning tasks in CLIP. Similar to Textual UE, we adapt the loss to the autoregressive instruction-tuning objective and restrict the trigger insertion to the input side and modify the vocabulary set to the target LLM’s BPE encoding. The triggers are optimized on the training set and then inserted into each input example before fine-tuning. All baselines are evaluated under the same target-model fine-tuning and test-time prompting protocol as TextCloak. A.3 Implementation Details Policy and surrogate models. We initialize the generative policy from meta-llama/Meta-Llama-3-8B and use Qwen/Qwen3-8B as the inner-loop surrogate. Both models run in bfloat16 with LoRA applied to all linear layers. We use rank 8, scaling factor 32, and dropout 0.05 for both adapters, while keeping the backbone parameters frozen. Policy optimization. We train the policy for two epochs with group size K=4K=4. The policy adapter is optimized with AdamW using a learning rate of 1×10−51× 10^-5, weight decay 1×10−41× 10^-4, and maximum gradient norm 1.0. The GRPO clipping radius is ε=0.2 =0.2. Rollout temperature decreases from 1.0 in the first epoch to 0.5 in the second epoch, with nucleus probability fixed at 1.0. The maximum rollout length ranges from 460 to 1,280 tokens across datasets. For each candidate, we restore the same surrogate and optimizer state, apply one LoRA fine-tuning step to the protected batch, and evaluate the updated surrogate on a clean validation mini-batch. The surrogate uses AdamW with learning rate 2×10−52× 10^-5, weight decay 10−310^-3, cosine decay to 2×10−72× 10^-7, and maximum gradient norm 1.0. The clean-baseline loss is constant within a candidate group and therefore does not affect the group-normalized advantages. We standardize rewards using a numerical constant of 10−810^-8. Quality control and corpus generation. Semantic fidelity is computed with the SBERT checkpoint sentence-transformers/all-MiniLM-L6-v2. Linguistic naturalness is computed with the GPT-2 checkpoint openai-community/gpt2. We set the degradation, semantic, and naturalness weights to 1.0, with τsem=τppl=0.9 _sem= _ppl=0.9. The scorers are frozen and applied to the full formatted clean and protected examples. Appendix B Additional Experiment Results Table 6: Human evaluation rubric for protected textual examples. Dimension Definition Criteria Text Naturalness The degree to which the protected text remains plausible as ordinary task text, without suspicious insertions, random strings, code-like fragments, irrelevant phrases, or stylistically abnormal content that would make the example appear manipulated to a human reader. 1: The text contains clearly random, gibberish, code-like, irrelevant, or strongly suspicious content that noticeably disrupts natural reading. 2: The text remains understandable but includes mild awkwardness, unusual wording, isolated substitutions, or phrase-like fragments that are stylistically abnormal. 3: The text appears naturally written, contextually appropriate, and free from noticeable suspicious or irrelevant content. Text Fluency The degree to which the protected text remains grammatical, coherent, readable, and locally well formed after rewriting. This dimension focuses on language quality. 1: The text is grammatically flawed, fragmented, or difficult to read. 2: The text is generally readable but contains minor grammatical errors, awkward phrasing, or local coherence issues. 3: The text is fluent, coherent, and easy to read. Human Utility The degree to which a legitimate human user can understand and respond to the intended task from the protected text without being hindered by ambiguity, missing information, or meaning changes. 1: The task cannot be answered reliably because essential information is missing, changed, misleading, or ambiguous. 2: The task remains answerable, but with noticeable uncertainty caused by mild ambiguity, reduced clarity, or incomplete evidence. 3: The task is clearly answerable from the protected text, with sufficient information for a confident human response. B.1 Human and LLM Evaluation Protocol We conduct a combined human and LLM evaluation to verify that protected examples remain useful to legitimate readers. For each dataset, we sample 20 examples and compare the clean text, Textual UE, MEM-5, and TextCloak. Each example is rated by three human annotators and three LLM judges: gemini-3.5-flash, claude-opus-4-8, and gpt-5.5. Raters are shown the clean and protected versions and are asked to assess naturalness, fluency, and human utility independently. Ratings use a three-point Likert scale, where higher scores indicate better quality. The full rubric is shown in Table 6. Human annotators and LLM judges are instructed to assess based on the legitimate user experience. Final scores are averaged across annotators, judges, datasets, and examples. The prompt for LLM judges is described in Appendix C.2. Appendix C Illustration of Prompt C.1 Prompt for Generative Policy The generative policy πϕ _φ is prompted to rewrite an entire mini-batch of training examples in a single rollout, conditioning on the shared editing instruction p used throughout policy generation. The prompt is deliberately structured around the two utility constraints of semantic fidelity and linguistic naturalness: it explicitly forbids modifying the target response yiy_i, since only the input field x~i x_i is protected, and it forbids surface-level artifacts (random strings, repeated tokens, broken grammar) that would depress the SBERT and perplexity scores used for filtering. At the same time, the prompt licenses the policy to introduce instance-specific reasoning or procedural framing rather than a single fixed lexical trigger, which is what allows GRPO-UE to discover generalizable shortcut patterns instead of the static, class-correlated cues used by prior textual UE methods (Li et al., 2023). The following template is used for every policy rollout, with dataset-specific field names substituted into the batch; the same template is reused verbatim across the K group members, with stochasticity coming only from sampling temperature. Prompt Template for the UE Generative Policy πϕ _φ [SYSTEM] You are a data-protection assistant. You rewrite text so that it resists unauthorized model fine-tuning while remaining fully usable and readable for legitimate human readers. [TASK] You will receive a batch of batch_size training examples. Each example has an input field and a fixed target field. Rewrite only the input field of every example to produce a protected version that will be released in place of the original. Your rewrites should introduce subtle, instance-specific reasoning cues, procedural hints, or framing shifts that a model fine-tuned on this batch may latch onto as shortcuts, rather than learning features that generalize to unseen data. Do not solve the task, and do not reveal, hint at, or alter the target answer. [REQUIREMENTS] 1. Preserve the original meaning, task, factual content, and answer options exactly; do not add, remove, or contradict any information needed to solve the task. 2. Never modify, leak, or otherwise change the target field; the rewritten input must remain answerable with the same target as the original. 3. Keep every rewritten input fluent, grammatical, and natural, as if written by a careful human author. 4. Do not introduce random strings, repeated tokens, code-like fragments, non-sequitur insertions, or any artifact that a human reader would find suspicious or out of place. 5. You may vary sentence order, phrasing, or add brief instance-specific reasoning or procedural framing, provided the correct answer and task semantics are unchanged. 6. Rewrite every example in the batch; return exactly batch_size examples, in the same order, matched by id. 7. Output only valid JSON, with no text before or after it: a list of objects with fields id and protected_input. [OUTPUT FORMAT] ["id": <id>, "protected_input": "<rewritten input>", ...] [BATCH] batch_examples [OUTPUT] During GRPO-UE training, K candidate batches are sampled independently from the same prompt at rollout temperature (Appendix A.3) to form a comparison group. Candidate batches that are malformed, omit examples, change the target answer, or fail the semantic or naturalness constraints receive the lowest reward within their group and are excluded from the final protected corpus. Dataset-specific prompts preserve the same system role and requirements but adapt the field names inside [BATCH]: multiple-choice tasks (ARC-Challenge, MMLU-Pro, MedQA-USMLE) expose the question and options, RACE additionally exposes the passage, HumanEval exposes the programming prompt and function signature, and MATH exposes the problem statement. C.2 Prompt for LLM Judges As described in Appendix B.1, each of the three LLM judges (gemini-3.5-flash, claude-opus-4-8, and gpt-5.5) independently rates the same 20 sampled examples per dataset that are shown to human annotators. Judges receive the clean input, the protected input produced by a given method, and the (unchanged) target response, and are asked to score only the protected input along the three dimensions of Table 6: text naturalness, text fluency, and human utility. The target response is included solely so that the judge can verify the protected input remains answerable, not to be evaluated itself. To avoid position and identity bias, the method identity is withheld, the clean/protected order is randomized across queries, and each dimension is scored independently before any overall judgment is formed. The exact rubric text embedded in the prompt is reproduced from Table 6 to ensure human and LLM raters are held to an identical standard. Prompt Template for LLM Judges [SYSTEM] You are an expert annotator evaluating text released for legitimate downstream use. You will compare a clean example with a protected (rewritten) version of the same example and judge only the protected version’s quality from the perspective of a legitimate human reader who must complete the underlying task. [TASK] Rate the protected input on the three dimensions below, each on a 3-point scale. The target response is provided only for reference; it is unchanged and is not itself being rated. [DIMENSIONS AND CRITERIA] Text Naturalness -- the degree to which the protected text remains plausible as ordinary task text, without suspicious insertions, random strings, code-like fragments, irrelevant phrases, or stylistically abnormal content. 1: Clearly random, gibberish, code-like, irrelevant, or strongly suspicious content that noticeably disrupts natural reading. 2: Understandable but with mild awkwardness, unusual wording, isolated substitutions, or stylistically abnormal fragments. 3: Naturally written, contextually appropriate, and free from suspicious or irrelevant content. Text Fluency -- the degree to which the protected text remains grammatical, coherent, readable, and locally well formed after rewriting. 1: Grammatically flawed, fragmented, or difficult to read. 2: Generally readable but with minor grammatical errors, awkward phrasing, or local coherence issues. 3: Fluent, coherent, and easy to read. Human Utility -- the degree to which a legitimate human user can understand and respond to the intended task from the protected text alone, without being hindered by ambiguity, missing information, or meaning changes. 1: Cannot be answered reliably because essential information is missing, changed, misleading, or ambiguous. 2: Answerable, but with noticeable uncertainty from mild ambiguity, reduced clarity, or incomplete evidence. 3: Clearly answerable, with sufficient information for a confident response. [INPUT] Task domain: task_domain Clean input: clean_input Protected input: protected_input Target response (reference only, not rated): target_response [OUTPUT FORMAT] Output only valid JSON, with no text before or after it: "naturalness": <1-3>, "fluency": <1-3>, "human_utility": <1-3>, "justification": "<one or two sentence rationale>" [OUTPUT] Scores are parsed from the JSON output and averaged across the three LLM judges; these are then combined with the three human-annotator scores as described in Appendix B.1 to produce the aggregate human-evaluation results reported in the main paper. Appendix D Notation Table Table 7: Summary of notation used in the main paper. Notation Description Notation Description D Clean training corpus. ~ D Protected corpus released by the data owner. (xi,yi)(x_i,y_i) Clean input or instruction and its target response. (x~i,yi)( x_i,y_i) Protected input paired with the unchanged target response. N Number of examples in the corpus. b Mini-batch size. fθf_θ Unauthorized target LLM with parameters θ. θ⋆θ Parameters after unauthorized fine-tuning. ℒftL_ft Autoregressive fine-tuning loss. ℒevalL_eval Held-out downstream evaluation loss or error. evalD_eval Held-out test/evaluation set. valD_val Validation set used to compute surrogate rewards. ()C(D) Set of admissible protected corpora satisfying utility constraints. ηi _i Discrete textual modification in classical textual UE baselines. (xi)A(x_i) Allowed edit set for input xix_i. V Token vocabulary. e(⋅)e(·) Token embedding function. ℬB Clean mini-batch sampled from D. ℬ~ B Protected mini-batch generated from ℬB. p Editing instruction used to condition the generative policy. πϕ _φ Generative policy with parameters ϕφ. πref _ref Reference policy used for KL regularization. πϕroll _ _roll Rollout policy used to sample candidate batches. a=(a1,…,aT)a=(a_1,…,a_T) Token sequence of a generated candidate batch. s=(p,ℬ)s=(p,B) Policy state consisting of prompt and clean batch. K Number of candidate protected batches in each group. uiu_i Concatenated clean input-response sequence xi∥yix_i \|y_i. u~i u_i Concatenated protected input-response sequence x~i∥yi x_i \|y_i. ϵ(⋅)ε(·) SBERT encoder for semantic similarity. SsemS_sem Batch-level semantic fidelity score. τsem _sem Semantic fidelity threshold. PPL(⋅)PPL(·) Perplexity under the reference language model. SpplS_ppl Normalized linguistic naturalness score. τppl _ppl Naturalness threshold. gψg_ψ Surrogate LLM with parameters ψ. S Distribution over surrogate models. ψk⋆ _k Surrogate parameters after fine-tuning on candidate k. ψcl⋆ _cl Surrogate parameters after fine-tuning on the clean batch. dkd_k Degradation induced by candidate k. rkr_k Reward for candidate k after utility penalties. λsem _sem Penalty weight for violating semantic fidelity. λppl _ppl Penalty weight for violating naturalness. A^k A_k Group-normalized advantage for candidate k. ρk,t _k,t Token-level importance ratio for candidate k at step t. ε Clipping radius in the GRPO objective. β KL regularization coefficient. JGRPOJ_GRPO Policy optimization objective used by GRPO-UE. Δ Performance drop relative to clean fine-tuning. ℳcleanM_clean Target-model performance after clean fine-tuning. ℳprotectedM_protected Target-model performance after protected fine-tuning. Table 7 summarizes the main notation used throughout the paper. Appendix E Use of Generative AI To enhance clarity and readability, we utilized the GPT-5.2 model exclusively as a language polishing tool. Its role was confined to proofreading, grammatical correction, and stylistic refinement—functions analogous to those provided by traditional grammar checkers and dictionaries. This tool did not contribute to the generation of new scientific content or ideas, and its usage is consistent with standard practices for manuscript preparation.