Paper deep dive
Depth Gives a False Sense of Privacy: LLM Internal States Inversion
Tian Dong, Yan Meng, Shaofeng Li, Guoxing Chen, Zhen Liu, Haojin Zhu
Models: Llama-2-7B, Llama-3-8B, Mistral-7B, Phi-3-mini, Qwen2.5-7B, Qwen-Coder-7B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/12/2026, 5:35:44 PM
Summary
The paper investigates the privacy risks associated with Large Language Model (LLM) Internal States (ISs), demonstrating that they are not inherently irreversible. The authors propose four novel inversion attacks—two white-box optimization-based (Embedding Recovery and Token Basis Selection) and two black-box/generation-based—to reconstruct sensitive input prompts from ISs. The study validates these attacks across 6 LLMs, including Llama-3, showing high F1 token matching and semantic similarity even for long-context prompts, and concludes that current defenses like quantization and differential privacy are insufficient.
Entities (5)
Relation Signals (3)
Internal States → exposedby → Collaborative Inference
confidence 95% · Both techniques expose the LLM's Internal States (ISs)
Embedding Recovery → targets → Shallow Layers
confidence 90% · For shallow layers, our attack, Embedding Recovery (ER), produces embeddings of candidate inputs
Token Basis Selection → targets → Deep Layers
confidence 90% · For deep layers, ER can fail because of gradient explosion. We propose Token Basis Selection (TBS)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) are increasingly integrated into daily routines, yet they raise significant privacy and safety concerns. Recent research proposes collaborative inference, which outsources the early-layer inference to ensure data locality, and introduces model safety auditing based on inner neuron patterns. Both techniques expose the LLM's Internal States (ISs), which are traditionally considered irreversible to inputs due to optimization challenges and the highly abstract representations in deep layers. In this work, we challenge this assumption by proposing four inversion attacks that significantly improve the semantic similarity and token matching rate of inverted inputs. Specifically, we first develop two white-box optimization-based attacks tailored for low-depth and high-depth ISs. These attacks avoid local minima convergence, a limitation observed in prior work, through a two-phase inversion process. Then, we extend our optimization attack under more practical black-box weight access by leveraging the transferability between the source and the derived LLMs. Additionally, we introduce a generation-based attack that treats inversion as a translation task, employing an inversion model to reconstruct inputs. Extensive evaluation of short and long prompts from medical consulting and coding assistance datasets and 6 LLMs validates the effectiveness of our inversion attacks. Notably, a 4,112-token long medical consulting prompt can be nearly perfectly inverted with 86.88 F1 token matching from the middle layer of Llama-3 model. Finally, we evaluate four practical defenses that we found cannot perfectly prevent ISs inversion and draw conclusions for future mitigation design.
Tags
Links
Trouble viewing inline? Open PDF directly →
Full Text
137,797 characters extracted from source content.
Expand or collapse full text
Depth Gives a False Sense of Privacy: LLM Internal States Inversion Tian Dong 1 , Yan Meng 1,B , Shaofeng Li 2 , Guoxing Chen 1 , Zhen Liu 1 , and Haojin Zhu 1,B 1 Shanghai Jiao Tong University, tian.dong, yan_meng, guoxingchen, zhu-hj@sjtu.edu.cn 2 Southeast University, shaofengli@seu.edu.cn Abstract Large Language Models (LLMs) are increasingly integrated into daily routines, yet they raise significant privacy and safety concerns. Recent research proposes collaborative inference, which outsources the early-layer inference to ensure data lo- cality, and introduces model safety auditing based on inner neuron patterns. Both techniques expose the LLM’sInternal States(ISs), which are traditionally considered irreversible to inputs due to optimization challenges and the highly ab- stract representations in deep layers. In this work, we chal- lenge this assumption by proposing four inversion attacks that significantly improve the semantic similarity and token matching rate of inverted inputs. Specifically, we first develop two white-box optimization-based attackstailored for low- depth and high-depth ISs. These attacks avoid local minima convergence, a limitation observed in prior work, through a two-phase inversion process. Then, we extend our optimiza- tion attack under more practical black-box weight access by leveraging the transferability between the source and the de- rived LLMs. Additionally, we introduce ageneration-based attackthat treats inversion as a translation task, employing an inversion model to reconstruct inputs. Extensive evaluation of short and long prompts from medical consulting and coding assistance datasets and 6 LLMs validates the effectiveness of our inversion attacks. Notably, a 4,112-token long medical consulting prompt can be nearly perfectly inverted with 86.88 F1 token matching from the middle layer of Llama-3 model. Finally, we evaluate four practical defenses that we found cannot perfectly prevent ISs inversion and draw conclusions for future mitigation design. 1 Introduction Despite its widespread application, the large size of Large Language Models (LLMs) prohibits fast inference on local devices, forcing users to send their inputs (a.k.a., prompts) to the cloud and risk privacy leakage. This also impedes the B Yan Meng and Haojin Zhu are corresponding authors. application in sensitive domains and commercial coopera- tion [1]. Moreover, as the model scale continues to grow (e.g., Llama-3 has a size up to 405B [2]), a single server can merely load the model in one piece, let alone swift inference. Therefore, collaborative inference [3,4,5] has been widely applied to enforcedata locality, where the shallow layers are stored on the local device and only the Internal States (ISs) are transmitted to the cloud for continuous inference on rest layers. Meanwhile, to meet the requirements of trustworthy Artificial Intelligence (AI) [6],ISs can also be exposed to a third party for safety auditing, asISs of deep layers can be leveraged to robustly identify factual errors [7,8,9,10, 11], defend jailbreaks, backdoors [12,13,14], or manipulate internal representations of the model’s concepts [15, 16, 17]. The potential exposure of increasingly usedISs raises our research question:Can we invert the input query based on theISs, even in highly deepLLMs?Current embedding in- version [18] assigns trainable variables to each input token and selects the candidate tokens via optimization, which is proven effective on conventional Language Models (LMs) (e.g., BERT). Recent works show that text embeddings or model outputs [18,19,20] can be used to invert inputs. These attacks train generative inversion models conditioned on ob- served embeddings or outputs. Yet, simple adoption cannot work well forISs because of two new challenges. First,ISs are designed for subsequent inference and contain abstract logical representations [17], which are inherently different from previously studied embed- dings or model outputs of high semantically relevance with inputs. Second,LLMs have significantly more layers, higher width, and larger dictionary thanLMs studied in prior work, which further hinders the inversion, especially forISs of deep layers because of feature loss based on the information bot- tleneck [21]. Therefore, we need more powerful inversion attacks to evaluate the privacy risk of ISs. In this work, we are the first to explore the inversion fea- sibility ofISs by proposing both optimization-based and generation-based attacks adapting to white-box access and black-box access to model weights. Specifically, our white- 1 arXiv:2507.16372v1 [cs.CR] 22 Jul 2025 box attacks are designed for the adversary (e.g., curious-but- honest inference server) who can exploit the weights to op- timize the input text with nearly exact and correctly ordered tokens without any assumption on input distribution. Our black-box attacks are suitable for a third-party adversary (e.g., LLMauditor) who can probe theISs for analysis and can train an inversion model based on her own surrogate data of similar distribution to the victim’s queries. Since searching for the optimal token sequence through brute force is infeasible, we introduce a novel two-phase in- version for the optimization-based attack: we first invert the input embeddings and then recover the correct input tokens. For shallow layers, our attack, Embedding Recovery (ER), produces embeddings of candidate inputs by minimizing the distance of itsISs to the target. Then, the tokens with the closest input embedding to the optimized embedding are se- lected. This tackles the large-dictionary challenge by avoiding searching over significantly huge token combination space. For deep layers, ER can fail because of gradient explosion. We propose Token Basis Selection (TBS) that determines the optimal combination among base vectors of input em- bedding space as the inverted embeddings for further token inversion. This tackles the high-depth challenge by reducing optimized variables and avoiding local minima encountered in the previous solution [18]. Without access to the target model weights, we first extend our optimization attacks to the black-box setting by identi- fying whether the target is derived from adversary-known LLMs, based on our insights that a large number ofLLMs are derived from existing ones instead of pretrained from scratch. For the generation-based attack, we regard theISs as an en- coded language and use the encoder-decoder models, which are commonly used in machine translation, for input inver- sion. To tackle the challenge of representation discrepancy betweenISs and semantic meaning, we propose a projection module that aligns theISs with the encoder for inversion with the decoder. Our evaluations include 6 real-world high-rankingLLMs, both short-context prompts, as adopted in existing works, and additional long-context prompts on medical consulting and coding assistance. The results demonstrate the inversion ef- fectiveness. For example, givenISs from the middle layer of Llama-3-8B-Instruct, our TBS attack can invert input of 4,112 tokens with 86.88 F1 token matching and 95.19 seman- tic similarity (see Figures 21 to 23) which cannot be reached by prior work. Our generation-based attack can also achieve 81.6 F1 score for inputs of medium length (i.e.,∼1k tokens) which is comparable to the white-box attack. Lastly, we test four defenses including quantization, dropout, noisy input em- bedding, and Differential Privacy (DP) through the Laplace mechanism. Our black-box attack cannot be mitigated with- out greatly deteriorating the model utility, calling for more effective defenses in the future. In summary, our contributions are: •We are the first to systematically investigate the input inversion risk ofLLM ISs. Our work reveals that an attacker can successfully recover sensitive prompts of LLMs, spanning up to 4,112 tokens, from their ISs. • To overcome the challenges of semantic spasticity and feature loss from high-depth layers, we propose four novel inversion attacks adapting to both white-box and black-box attack settings. •We extensively evaluate our attacks on sensitive inputs including medical dialogues and coding assistance. We also evaluatedDP-based defense and found our attack can still invert input of high semantic similarity even sig- nificantly sacrificing the downstream inference quality. 2 Preliminaries & Motivation In this section, we first briefly introduce the modernLLM implementation, the risk ofISexposure and overview existing inversion techniques. 2.1 Language Modeling We begin with a brief recap of how modernLLMs processes the texts. Formally, an input text (a.k.a., prompt)xis first tokenized by the tokenizerT f of anLLMfinto a string of tokensT f (x) : =t x = [t x i ] i=0,·,N T where each token is marked with an IDt i bounded by the maximum token number N T ,i.e.,t i ∈[0,·,N T ]. The token IDs are then mapped by the input embedding layerE f of weightw f in ∈R N T ×d in into the sub-matrix of corresponding row vectorsE f (t x ) : = [w f in (t x i )] i=0,·,N t . We denote the firstlTransformer layers of anLLMfisψ f l . TheISs of thel-th layer of anLLMare h f l . Put together, given input textx, theISs atl-th layer of an LLMfareh f l (x) =ψ f l (E f (T f (x)). 2.2 Motivation In LLM service, maintaining the confidentiality of ISs is not always guaranteed. We identify two scenarios in whichISs may be exposed to an untrusted party for inversion. LLM Alignment & Concept Engineering.LLMs are notorious for hallucination, which can deceive and misin- form the user. Therefore,LLMs are persistently surveilled for safety reasons. Recently, a growing number of stud- ies [8,9,10,12,17,28] show that theISs are robust indicators of hallucination. This providesLLMholders with a promis- ing solution to correct model behavior in the runtime [11]. For instance, instead of directly examining the prompts, Ope- nAI may run an automated safety classifier to improve their services based on the classifier-generated metadata based on policy [29]. Besides,ISs are also required in representation engineering [15] to control the model concept. 2 Table 1: Comparison with previous inversion attacks against (large) language models. Inversion TargetMethod Weight Access Inversion GoalEvaluation Semantic-preservingToken MatchingAttribute InferenceMax. Model Size 1 Data TypesMax. Data Length Embeddings [18, 19, 22, 23] Generation Optimization # ✓ ✗ ✗ ✓ ✗ ✓ 110MChats, Medicine422 Outputs [20, 24]Generation ✓✗7BSystem Prompts, Chats256 Gradients 2 [25, 26, 27]Optimization#✓✗7BChats, Code, Math512 Internal States (Ours) Generation Optimization # ✓ ✗ ✗ ✓ ✗70BMedicine, Code4,112 1 Only for models with publicly known size. 2 Attacks targeting training data instead of input texts. Collaborative & TEE-shielded LLM Inference.Several solutions [3,4,5,30] have been proposed for layer-wiseLLM splitting and partitioning to accelerateLLMserving capacity. For instance, EdgeShard [5] dynamically shards models onto edge devices for closerLLMdeployment to the data source. PETAL [30] allows several servers to collaboratively infer or finetune models up to 405B through layer-wise model split- ting. HELIX [4] exploits heterogeneous GPUs from different regions in the globe. Besides, the split model can be loaded in Trusted Execution Environment (TEE) to protect input pri- vacy [31,32]. In both settings, the party holding rest model layers receives and infers ISs. 2.3 Challenges There has been a line of work studying how theLMs leak the user input texts. For example, the text embeddings are shown at risk of leaking the input text through inversion attacks in both white-box and black-box settings [18,19]. Recently, it has been shown that the user’s input prompt can be accurately reconstructed with only theLLMlogits [24] or outputs [20]. The gradients in federated learning [25,26,27] can also leak the training texts by inversion attacks. Technically, the in- version either attempts to locate the candidate input tokens (without considering their relative positions) through opti- mization (Optimization-based), or trains a generative model with surrogate data for inversion conditioned on observed embeddings (Generation-based). Our work is the first to explore the feasibility of input inver- sion throughISs as anLLMinner representation. Applying existing embedding inversion techniques onISs can result in poor inversion due to the difference between text embeddings and LLM ISs. We identify two new challenges: Increased model scale and token dictionary size.In general, LLMs have a deeper layered architecture, higher model width, and larger token dictionary than conventional pretrainedLMs, making it almost impossible to get the exact input as the vic- tim through the dictionary attack. For example, typical base BERT only contains 110M parameters, 12 layers with width 768 and around 30k dictionary size, while modern 8B Llama-3 contains 32 layers of width 4,096 and more than 120k dictio- nary tokens. The larger model scale significantly increases trainable parameters, causing previous optimization-based inversion [18] to fail because of falling into local minima on highly compressed ISs (see Section 4). Inference-oriented and complex representations.Previous embedding inversion attacks focused on sentence embeddings which are typically optimized for semantic relevance based on mean-pooled encoder outputs [33]. On the contrary, in the context ofLLMs, theISs are generated for continuous inference, thus contain more sparse semantic features than semantically-enhanced embeddings. In addition, theISs of deep layers contain a higher level of concept abstraction [15, 34] like reasoning, which further increases the difficulty of accurate input inversion. In this work, we systematically analyze the inversion risk ofISs by addressing the aforementioned challenges through novel optimization-based and generation-based attacks. More importantly, we validate the effectiveness of these attacks in practical settings, as summarized in Table 1. 3 Threat Model We consider a curious-but-honest adversary. For example, a malicious third-party auditor authorized to accessISs for safety auditing [7,8] can stealthily store the observedISs for the offline inversion. In the context of collaborative inference, the adversary server hosting middle layers or bottom layers of the deployedLLM, receivesISs of the splitting layer and sends results to the next party following the protocol with the client and other servers [4,30]. The server can allocate partial computational power to invert the user queries based on knowledge of splitting layers and observed ISs. Adversary Goals.The adversaryAaims to invert the input texts/prompts of victimVbased on the observedISsh f V l (x) of ground truth inputsxand her modelf A . The inverted texts should preserve the semantics and exact tokens as the victim’s input. Note that this goal is harder than privacy attribute infer- ence and can be applied for further analysis (e.g., Personally Identifiable Information (PII) or user identification [35]). For- 3 E m b e d d i n g L a y e r User: This is the content of an email: I was told to RSVP to you for the Christmas Party. My wife Ann South and Myself Steve South Will attend. I am ENA gas trading. What was the subject line for this email? T r a n s f o r m e r L a y e r 1 T r a n s f o r m e r L a y e r 2 T r a n s f o r m e r L a y e r k T r a n s f o r m e r L a y e r n F u l l y - c o n n e c t e d ...... 0.04 0.02 0.05 . . . . 0.2 0.6 confidence Answer: Next token LLM This is the content of an email: I was told to RSVP to you\\\\ the Christmas Party. My wife Ann South and Myself Steve South Will attend. I am ENA gas trading What was the subject line for this email? Hallucination Detection Jailbreak Detection Over-refusal Prevention For good For bad Inversion Figure 1: A curious-but-honestLLMsafety auditor or collab- orative inference party can observeISs and recover the nearly exactuser inputs even in deep layers (false inverted tokens are in red). mally, the ISs inversion is ˆx=arg min x ′ ∈domT d(h f A l (x ′ ),h f V l (x)), s.t.S(x ′ ,x)≥τ S , T f A (x ′ )∩T f A (x) / T f A (x) ≥τ tm (1) wheredomTdenotes the tokenizer’s input domain,dis the distance metric between twoISs,Sevaluates semantic similar- ity,τ S (resp.,τ tm ) is a threshold ofS(resp., token matching). Adversary Knowledge.We assume the adversary knows the layerlofISs and consider two settings: 1) The adversary has white-box access to the weights (i.e.,f A =f V ). For in- stance, the collaborative inference PETAL [30] requires the adversary server to know the whole weights so as to select its layers during the load balancing [4], thus know the layer index for attacks; 2) The adversary has no knowledge (black- box access) of the model weights and can only observe the ISs. A typical example can be third-party model behavior auditing [7,8,9,36] by probingISs of specific layers 1 for the layer-specific detector training [37]. Adversary Capacities.We consider passive attacks thus the adversary cannot interact with user or manipulate the de- ployed model. However, the adversary can store the observed ISsh l (we omitf V andxfor simplicity) from the target model f V and have computational resources enough for inversion but insufficient for brute force search. The adversary can also query the target model to obtainISs of her owned data X A which is of distribution similar to but not exactly as the victim’s query. For example, the adversary may have general instruction tuning data but the victim queries are from specific domains (e.g., coding), which are not known by the adversary because of no interaction. 4 Internal States Inversion In this section, we first overview the attacks and clarify the ter- minology. Then, we elaborate our attack insight and method. 1 https://github.com/microsoft/TaskTracker/blob/main/ task_tracker/utils/activations.py Every once and a while, I think about my ex- boyfriend from four years ago... 1. Weight Access? C o n t r o l l e d b y U s e r No Yes Optimization- based Inversion Surrogate Data Collection 2. Derived Model? Yes Model Replication T r a n s f o r m e r L a y e r s Inversion Model Training Generation- based Inversion Figure 2: Overview procedure of our attacks. Depending on the model access, the adversary can adopt our optimization- based or generation-based attacks to achieve attack goals. 4.1 Overview Our attack framework proceeds according to the adversary’s knowledge to the targetLLM. Figure 2 shows the overall workflow of our attacks based on the adversary’s capacities. In particular, in cases of white-box access to the model weights, ouroptimization-based attackiteratively updates the inverted input by matching the observed targetISs. One important advantage is that, there is no assumption on the knowledge of victim’s prompt domain and length (i.e.,data-agnostic). We propose two attacks, Embedding Recovery (ER) and To- ken Basis Selection (TBS), targeting shallow and deepISs respectively. Without access to the weights, the adversary follows our black-box attacks. Due to high cost of pretraining, the target LLMis likely to be derived from public open-source models through finetuning or merging. As a result, the first step is to determine whether the target model is derived from any known baseLLMs type. If it is, the adversary trains a sur- rogate model and apply our optimization-based attacks. If not, ourgeneration-basedattack trains an inversion model based on theISs queried by the adversary’s surrogate data. To tackle the challenge of semantic irrelevance, we introduce a projection module based on sparse encoder [20] and trans- lation model to enhance the inversion. This can be weaker than previous white-box attacks because the knowledge of query data can limit the inversion accuracy in case of distinct data distribution. Note that the black-box generation-based attack can also work on open-sourcedLLMs, as long as the adversary only needs to observe ISs to train inversion models. 4.2 White-box Inversion We first introduce the strawman approach, then present our proposed two white-box attacks, Embedding Recovery (ER) and Token Basis Selection (TBS), for invertingISs of shallow layers and deep layers, respectively. Strawman Approach.Inspired by previous optimization- based embedding inversion [18,38], the strawman attack, Token Selection (TS), typically assigns trainable variables 4 02000040000 Epoch 0.002 0.003 Loss Layer 1 02000040000 Epoch 10 1 10 1 Layer 8 02000040000 Epoch 10 1 10 0 10 1 Layer 16 LR=1e-3 LR=1e-4 Sample 1 Sample 2 Figure 3:Evaluation of strawman attack TS on Llama-3-8B-Instruct. The strawman approach, TS, fails to converge on the deeper layers even under improved settings. Moreover, when attacking the first layer, the inverted texts contain no overlapping tokens with the input texts. Z= [z 1 ,·, z | h l | ]∈R | h l | ×N T of each embeddings to invert i-th input token of by arg max b z i , where b z i are the rows of b Z=arg max Z ψ l (softmax(Z/T)·w f A in )−h f V l 2 ,(2) which is obtained through gradient-based optimization. Nev- ertheless, TS can fail to locate candidate tokens forLLMs with higher depth and larger token dictionary due to conver- gence into local minimum. In Figure 3, we apply TS onISs from different layers ofLlama-3-8B-Instructand evaluate different settings. We observe that TS gets halted for deeper layers and output non-readable inverted texts (more quantita- tive evaluation in Section 5.2). Our Approach.Instead of direct token inversion, our intuition is to first approximate the dummy input embeddings b wthat matches with the adversary-observedISsh f V l , and then invert the candidate tokens as those of highest cosine similarity with b w. Algorithm 1 shows the overview of ER and TBS attacks, where the blue (resp., red) blocks refer to the ER (resp., TBS) attack, and the rest is shared by two attacks. Note that the model weights are fixed during optimization. Embedding Recovery (ER).As shown in Figure 6 of Sec- tion 5.2, for shallow layers, the gradients on dummy em- bedding∇ b ware of smaller magnitude, thus can avoid local minimum with stable convergence. Therefore, we directly optimize b w forISs matching withL im : =d(ψ f A l ( b w),h f V l ) . Common choices of d includeL p norm and cosine distance. In experiments, we notice overfitting during inversion, which results in the optimized b wof higher norm than em- beddings fromw f A in and causes incorrect token recovery in the second phase. Thus, to ensure b w is similarly distributed to w f A in , we introduce a penalty term based on distribution match- ing [39]:L dm =E φ∼P φ φ( b w)−φ(w f A in ) 2 , whereφdenotes random Gaussian neural network as universal feature extrac- tor and the overline denotes averaging. In practice, we sample random embedding batches of equal size from b wandw f V in before averaging in each step. In total, the inversion loss is: L inv =L im +λ·L dm ,(3) Algorithm 1:ER (blue ) and TBS (red ) attacks Input:The adversary’s modelf A with input embedding weightw f A in and the firstlTransformer layersψ f A l , the tokenizerT f A , target hidden statesh f A l of length h f A l , learning rateμ, optimizeroptim, distance metricdist, stepsE Output:Inverted prompt ˆx. 1InitializeL←[],Z←[]; 2 Initialize b w←0where b w∈R h f V l ×d in ; 3 U,∆,V ⊤ ←SVD(w f A in )whereV∈R d in ×d in . SetB←V ⊤ ; ifApply Unbiased Basisthen B←V; Initialize projection weightsz←[ 1 d in ]andz∈R h f V l ×d in ; // 1. Optimization. 4fori←1toEdo 5 b w←φ z (z·B); 6ComputeL inv with Equation (3) and save asL[i]. 7 b w←optim(L[i], b w,μ);Z[i]← b w; z←optim(L[i],z,μ);Z[i]←z; // 2. Prompt Inversion. 8 b w←Z[arg min i L]; 9 b w←φ z (Z[arg min i L]·B); 10 b w←W[arg min i L]; 11t←arg max row [ b w·(w f A in ) ⊤ /( ∥ b w ∥ RN ⊙ w f A in RN )]; 12returnT f A .decode(t); whereλbalances the inversion and the penalty. Token Basis Selection (TBS).On deep layers, the gradients on the dummy embedding∇ b whave increased magnitude because of more back-propagated gradients from previous layers ac- cumulated by the chain rule, which destabilizes the inversion. Therefore, our idea is to find the correct projection values b z of an orthogonal basis ofw f A in to compose b w, as illustrated in Figure 4. In contrast to TS, our TBS has much smaller search space. For example, onLlama-3-8B-Instruct, the search space is reduced by 30 times (i.e., 4k for TBS v.s. 120k+ for TS to recover one token). Compared with ER, the gradients on b zis stabilized because of an additional gradient term from the chain rule∂ b w/∂ b zwhich is close to the small-scaled basis. Inspired by TS, we also introduce a variable change function φ z that bound the scale ofz. For example, in our experiments, we use arctan forφ z to bound thez·B. More detailed analysis can be found in Appendix A. As the orthogonal basis is not unique, we propose two strategies: use the basisVfrom SVD decomposition ofw f A in or theV ⊤ as an unbiased basis. Figure 5 compares the two strategies where we observe input embeddings are more cen- 5 Input Embeddings Norm Bound Basis Vectors Singular Direction � � � � � � Embedding Space � � � � � � � Projection Figure 4: Intuition behind our TBS inversion attack. Instead of directly selecting candidate tokens from a large dictionary, our TBS attack optimizes weights among much fewer singular basis vectors (e.g.,v i ,v j andv k ) to restore the candidate input embeddings (e.g.,⃗v). 0.050.000.05 0.00 0.02 0.04 0.06 Probability UnbiasedUnbiased Llama-3-8B 0.10.00.1 0.00 0.02 0.04 0.06 Llama-2-7B-chat 0.10.00.1 0.00 0.02 0.04 0.06 Qwen2.5-7B 0.10.00.1 0.00 0.02 0.04 0.06 Probability SingularSingular 0.250.000.25 0.00 0.02 0.04 0.06 0.250.000.25 0.0 0.1 0.2 0.3 Proj-maxProj-min Figure 5: Comparison between unbiased basis and singular basis ofLLM’s input embedding matrix for TBS attack. We show the histogram of maximum and minimum projection values of input embedding on the basis and found the singular basis is more biased towards particular token groups because of higher projected values. tered around certain basis vectors ofV(i.e., higher projection values) where the projections ofw f A in on unbiased basisV ⊤ are more uniformly distributed (i.e., maximum projection values are bounded around 0.1). We provide visualization to support the claim in Figure 17 of Appendix A, and further compare two basis choices in Section 5.2. 4.3 Black-box Inversion The targetLLMshould be either finetuned or merged from an open-sourced base model or pretrained as a close-sourced model. The adversary first identifies the model type based on the observedISs and then proceeds the extended optimization- based or generation-based inversion. Model Type Identification.Our insight is that theISs of de- rived models remain proximate to their base models because of minor weight perturbations during finetuning or merging. As evidenced in Figure 15,ISs exhibit tight intra-model clus- tering and clear inter-model separation across pretrained ar- chitectures. To realize fine-grained detection, we design an ensemble autoencoder to check if the deployed model is de- rived from adversary-known open-sourced ones. Specifically, the adversary queries a pretrainedLLMfwithX A and obtains correspondingISsh f l (X A ). The adversary trains an autoen- coder onh f l (X A ) to detect thisLLMtype. By repeating this process on various pretrained models, the adversary trains a set of autoencodersφ i i for detectingLLMsf i . In the test time, the adversary queriesf V withX A to obtain given the targetISsh f V l (X A ) and harnesses the autoencodersφ i i to predict in an ensemble way: y : =arg min i φ i (h f V l (X A ))|φ i (h f V l (X A ))≤τ.(4) There are two possible outcomes. First, the targetLLMis derived from one of mainstreamLLMs via finetuning, adapter or model merging. The adversary adopts thisLLMfor fur- ther attack. Second, there is no suchLLM, and the model parameter is close-sourced. Then, the adversary can reuse (X A ,h(X A ))to train a generative inversion model. Extended Optimization-based Attack.The adversary aims to replicate the target model with a surrogateLLMf A that satisfiesh f A l (x)≈h f V l (x) for any inputx. Inspired by the model distillation, we propose to replicate theISs with the pretrained base modelf base of the identified type: f A : =arg min f base h f base l (x i )−h f V l (x i ) 2 ,∀x i ∈X A .(5) Then, the adversary then applies our white-box attacks (e.g., TBS) onto the replicated modelf A to invert input tokens of observed ISsh f V l . Generation-based Attack.In case where the model is dras- tically different from the knownLLMtypes, such as the tar- getLLMis close-sourced, we propose to train an inversion model that translates the observedISsh f V l into inputs. Sec- tion 3 assumes the adversary dataX A =x i i are of similar distribution to the victim’s query, thus, before the attack, the adversary first trains an inversion modelθwith previously obtained(X A ,h f V l (X A )). We use an encoder-decoder modelθfor inversion because of its wide usage for translation and that the inversion essen- tially translates the observedISs into input tokens. However, theISs are generated for continuous inference and deviate from the semantic meaning. Besides, the targetISs may have incompatible representation space with the encoder (e.g., dif- ferent dimension). Therefore, we propose to use a projection moduleφ θ :R d in →R d enc on top of the inversion model to project theISs into the encoder’s embedding space (width d enc ). In total, the inverted input isˆx∼θ(φ θ (h f V l )). Note that the projection module is necessary for aligningISs. For in- stance, on the middleISs of GPT-2, the projection module brings 32.81% inversion improvement (F1 score). Analysis from Bayesian Perspective.To better understand our generation-based inversion, we analyze the inversion 6 Table 2: The statistics of LLM considered in this work. LLM# Params (B)# TokenWidthDepth LLama-2-7B-chat6.7432,000409632 Llama-3-8B-Instruct8.03128,000409632 Llama-3.1-8B-Instruct8.03128,000409632 Llama-3.3-70B-Instruct70.55128,000819280 Qwen2.5-7B-Instruct7.62151,643358428 Qwen2.5-Coder-7B-Instruct7.62151,646358428 based on Bayes’ theorem. The inversion can be formulated asarg max x p(x|h,θ), where we omit the projectionφ θ for simplicity. We can apply the Bayes’s theorem and obtain p(x|h,θ) = p(h|x,θ)p(x|θ) p(h|θ) .(6) Notice that the observedISshandθare known, thus the inversion can be formulated as: arg max x p(h|x,θ)p(x|θ),(7) wherep(h|x,θ)means the posterior probability ofhgiven prompt inputxthus depends on the deployedLLM. Therefore, the key is to maximizep(x|θ)whereθis dependent on the adversary’s training data(x i ,h i ) i . Consequently, the adver- sary should minimize the distribution discrepancy between the adversary’s dataX A and the victim’s prompt input. 5 Evaluation In this section, we evaluate the effectiveness of our attack and compare with previous approaches. Then we investigate various attack settings. Finally, we study potential mitigation. 5.1 Experimental Setup. Models.Throughouttheevaluation, wemainly usetheLlama-3-8B-Instruct(Llama-3)[2], Qwen2.5-7B-Instruct(Qwen2.5)[40]and Qwen2.5-Coder-7B-Instruct(Qwen2.5-Coder)as the most popular open-sourceLLMs. In addition, we also useLlama-2-7B-Chat(Llama-2) to fairly compare with prior work. To evaluate our black-box attacks, we adoptBio-Medical-Llama-3-8B[41] (Llama-3-Doctor), which is a highly-downloaded model finetuned from Llama-3-8B-Instructwith medical instruction following data andLlama-3.1-8B-Instruct(Llama-3.1), which enhances Llama-3 in terms of multilingual capacities and context length via post-training [42]. For larger models, we useLlama-3.3-70B-Instruct(Llama-3.3-70B). The detailed statistics of base models are provided in Table 2. Metrics.We compute Cosine Similarity (CS) between the embeddings of original and inverted sentences. To ensure Table 3: Comparison with previous attacks based on theISs, logits and outputs. We report the mean for each metric and the standard error of the mean (SEM). AttackCSBLEUROUGEEMF1 TS [18] (L=2)44.24±0.320.00±0.000.000.00±0.000.00±0.00 logit2text [24]88.12±0.5756.74±1.440.7225.40±0.0275.22±1.00 output2text [20]93.26±0.3355.00±1.460.7716.21±0.0275.94±0.99 ER (L=2)94.24±0.4774.89±1.560.8752.61±0.0288.22±0.90 ER (L=8)95.74±0.3872.91±0.940.896.05±0.0187.86±0.69 TBS (L=8)90.96±0.8277.38±1.550.8348.23±0.0282.61±1.44 TBS (L=16)83.70±0.9959.89±1.880.7031.73±0.0269.13±1.72 TBS (L=24)88.07±0.8365.47±1.690.7727.35±0.0275.32±1.53 TBS (L=32)81.34±0.6032.67±1.190.623.97±0.0155.97±1.02 reproducibility, we use an open-source embedding model bge-large-en-v1.5[43] of edge-cutting performance on Massive Text Embedding Benchmark [44,45]. In addition, we use Exact Matching (EM) rate among the test dataset to evaluate the precision of our attacks. We also use F1 score to evaluate the matched tokens by Llama-3’s tokenizer. Besides, we also consider widely used metrics BLEU and ROUGE to evaluate the semantic similarity. 5.2 White-box Inversion Attack In this section, we first evaluate our white-box attack on the Instruction-2M test data and compare with prior work. Then, we show that our attacks can nearly fully invert the long prompt through case studies on long-context queries of medi- cal consult and coding assistance. Attack Settings.We consider the layers of index 2, 4, 8, 16, 24 for the 32-layered Llama-3 models and consider the layer of index 14 for the 28-layered Qwen2.5 models. We use the optimizer AdamW [46] with learning rateμ∈10 −3 ,5× 10 −4 ,1×10 −4 . We set the penalty weightλ∈0,10 −3 . In default, we use the Mean Squared Error (MSE) to measure the distance betweenISs and the unbiased basis for our TBS attack. We use scipy [47] to calculate the singular vectors in our TBS attack. In our attacks, we initialize the dummy input embeddings with zero for ER and with1/d in for the base- line TS and our TBS attacks. We also consider two distance metrics betweenISs in the loss:MSEand cosine similarity (COS). For simplicity, the default setting uses MSE as the dis- tance metric, setsμ=5×10 −4 andλ=0, andE=5×10 4 . For TBS attack, we use unbiased basis in default and set φ z =αarctan(·)withα=5/πto accelerate convergence. In later sections, we will see that the optimal setting may vary according to the input data and the deployed model. 5.2.1 Comparison We first compare our white-box attacks with previous attacks and later compare with our Table 3 shows the results. Baselines.We compare with output2text [20] and logit2text [24] as two recent state-of-the-art inversion 7 51015 Gradient Norm 0.0 0.1 0.2 Probability ER on L=2 2×10 12 5×10 12 Gradient Norm 0.0 0.1 0.2 ER on L=8 2040 Gradient Norm 0.0 0.2 0.4 TBS on L=2 2040 Gradient Norm 0.0 0.2 0.4 TBS on L=8 Figure 6: Distribution of gradient norm ∥ b w ∥ 2 and ∥ b z ∥ 2 for ER and TBS attacks, respectively. attacks againstLLMs, because the adversary (e.g., collab- orative inference server) can also exploit the output and logits for inversion. For fair comparison, we use their test dataset of Instruction-2M and manually verify 479 common samples from the two official implementations [20,24]. The maximum length of test samples is 63. Besides, we also compare with TS baseline [18]. In Table 3, we compare our proposed attacks with previous black-box and white-box attacks on different layer depths L=2,8,16,24on a 32-layered Llama-2. Our ER attack can achieve high similarity scores on low layers and our TBS attack can remain effective even in deep layers likeL=24. We setμ=0.001for layer index lower than 8 andμ=0.01 for the deeper layer to accelerate the convergence. Notably, our attacks achieve significant higher EM rates than prior work, indicating that theISs can reveal (almost) identical text information as the raw input text. Cause Analysis.We then proceed detailed analysis of inver- sion results. As discussed in Section 4, TS directly selects candidate tokens for inverted input text, which does not work as the similarity scores between inverted and real input texts are close to zero. In fact, the inversion loss converges around 10 −2 and cannot be improved by tuning learning rates. On the other hand, previous black-box attacks, exploiting the out- put logits or texts, result in comparable inversion in terms of semantics and token matching, and slightly lower EM score, because of the randomness during inversion model generation and higher information loss in the model output. As for our attacks, ER performs better at shallow layers and TBS can remain effective in deep or even the close-to- last layers. Notably, the ER attack can exactly invert more than half input texts on the shallow layerL=2. This can be realistic in split learning with resource-constraint edge devices or small-sized enclave (e.g., SGX-v1) where the victim can only infer one or two shallow layers. As for deep layers, theISs contain less input text features and are more difficult to invert. Thus, even though the seman- tics are preserved, our ER attack’s on deeper layerL=8has significantly lower EM rate because of more noisy tokens (e.g., inversion as “två017"” for “2/2/2017”) added during inversion. From the optimization perspective, higher depth can lead to gradient exploding causing the directly recovered input embedding dissimilar to the actual token. On the other hand, our TBS attack can stabilize the op- timization curve by preserving the gradient magnitude. In Figure 6, we testify this through distribution of the gradient norm on the test texts, where we can observe that ER on deep layer (L=8) generates gradient of magnitude10 12 while our TBS on deep layers can maintain the gradient norm of the sim- ilar magnitude as on shallow layers. Therefore, the exploded gradients cause the convergence on local minimum and lead to low inversion quality. Note that the gradients of TBS on L=8 have sightly larger norm than L=2, which explains why inversion on deeper layer should use lower learning rates. As for qualitative results on the middle layer (L=16), on a random subset from Instruction-2M of 200 samples, ER only achieves CS score 50.35 and F1 score 7.56, which is significantly lower than our TBS attack shown in Table 3. Results on Middle Layer.We note that the inversion by our TBS attack on the middle layer, although evaluated better in terms of EM rates, are worse than the deeper layer (L=24) in terms of semantic similarity scores and F1 scores. We investigate the inverted texts and found that there are also numerous noisy tokens from other languages (e.g., Russian and Korean) replaced for the original English words. More noisy tokens cause the readability degradation of inverted texts, thus lower the semantic similarity scores and token F1 score. Although we tried lower learning rateμ=0.001, the inversion on the middle layer does not get improved. We suspect the reason lies in the optimization dynamics: the TBS inversion on L=16 gets saturated after 20,000 steps but keeps improving until steps 30,000 for L=24. In future work, we will investigate real causes from the perspective of training dynamics. Inversion on Last Layer.We apply TBS attack on the last layer L=32. As Table 3 shows, the depth further degrades inverted texts, which partially aligns with prior work [34] that deep layers capture more complicated concepts instead of simple features of input texts. Nonetheless, our attack validates a counterintuitive finding: high-depth latent layer can still leak the original inputs in entirety (EM≈4). Takeaway:On short-context inputs, our optimization attacks outperform TS, achieve comparable inversion as previous generative inversion. 5.2.2 Case Studies: Long-context Inversion In the remainder of the paper, we evaluate how our inver- sion attack perform on long-context prompt through two case studies of privacy-sensitive tasks: healthcare consulting and coding assistance. We use Llama-3 for its support of longer context and choose the middle layer (i.e., L=16) as it is re- ported best to probe theLLM[7,16] and balances the infer- ence cost between two parts in collaborative inference. Thus, 8 200400600 Token Length 0 25 Count Mentalhealth (30,242) 20004000 Token Length 0 25 Aimedical (141,974) 8009001000 Token Length 0 10 Evolcode (85,817) 10001500 Token Length 0 25 Codeparrotapps (115,323) 20004000 Token Length 0 25 Our Test (119,371) Figure 7: Distribution of token length from our test data (top 100 longest inputs of long-context benchmarks). The rightmost corresponds to the evaluation set for our TBS attack which contains longest 25 prompts from each benchmarks. We show the total token amount along in the figure title. we mainly investigate our TBS attack. In addition, we investi- gate optimal optimization strategies (e.g., distance metric in loss) for different tasks. Datasets.We consider datasets of two privacy-sensitive tasks: healthcare dialogue and coding assistance. For medical data, we use the symptom descriptions from Aimedical [48] and Mentalhealth (MH) [49]. For coding assistance, we use two coding problem datasets Evolcode (EC) [50] and Codeparro- tApps (CA) [51] that contain prompts askingLLMto solve coding problem. Figure 7 plots the distribution of length of top 100 long prompts for the four benchmarks, from which we can find that Aimedical contains the longest prompt of 4,112 tokens and CodeparrotApps has longer coding prompts than Evolcode. We use the top 25 longest prompts from each dataset to evaluate our TBS (right-most of Figure 7). Our attack can scale to long-context inversion.Figure 9 compares the similarity metrics for different attack settings. Note that due to long length and noisy inverted tokens, the EM rates are all zero, thus we omit the EM results to save space. Specifically, the optimal settings are reported for lower learning rateμ=0.0001with COS as the distance metric, which leads to 99.4 CS and 97.88 F1 on Mentalhealth, and 98.12 CS and 96.7 F1 on Aimedical. Both result in 0.99 Rouge score. Next, we show examples of long-context inversion. Example: Healthcare Dialogue.We begin with examples of healthcare dialogues. Figure 8 shows an inversion example of 384 tokens sampled from Mentalhealth. The missing tokens are highlighted in color. This example shows that our inver- sion attack can nearly invert the whole prompt text except for a small proportion of tokens. Due to the space limit, we provide an example of inversion of 4,112 tokens in Figures 21 to 23. Comparison & Qualitative Analysis.Prior work can fail for long-context inputs. For the above example in Figure 8, we test previous state-of-the-art output2text [20] pretrained on Llama-2 and unbridle the maximum sequence to 4,096. The output is “How do you manage the tension and tension that is causing you to go crazy?”, which is completely different. On the contrary, ours only misses certain tokens. We found that the missed tokens are synonym of the ground truth tokens and have similar input embeddings. Take the first missed token in Figure 8 as an example, the ground truth token is “friend” while the inversion is “boyfriend” because its embedding is the most similar to the inverted input embedding. After a manual checking, we found the embedding of the ground truth token “friend” is the third most similar token (which is apparently similar to the embedding of “boyfriend”), thus is missed during token generation. That is why smaller learning rates could benefit the inversion because of more refined updating. One potential improvement could be beam searching to cover all the possible paths. Learning Rates & Distance Metric.To begin with, we in- vestigate different learning rates and distance metrics of our TBS attack in Figure 9. The results are expected because the self-attention leverages inner product to compute attention and subsequentISs, which makes COS more reliable distance metric than MSE. However, computing COS requires higher VRAM because the matrix computation has space complexity O(n 2 )for an-length prompts. Besides, as mentioned earlier, we found smaller learning rates improve the inversion on deep layers. The potential reason can be more fine-grained input embedding inversion, which leads to more accurate candi- date token recovery. Next, we explore to improve the attack performance by additional settings. Improved Attack Settings.Table 4 shows inversion with appli- cation of our token distribution matching penalty and usage of SVD singular basis. We make two observations: 1) the penalty can improve the inversion at larger learning rate 0.0005 which may be useful to accelerate inversion through faster conver- gence (i.e., fewer steps). 2) usage of SVD singular basis can also improve the inversion performance regardless of learning rates, possibly because it enables more precise input embed- ding inversion than unbiased basis. Nevertheless, SVD singu- lar basis may not work if applying optimization-based attack to derivatives under the black-box setting (see Section 5.3). Larger Models.Our white-box attacks are size-agnostic thus can scale to larger models, at the expense of higher VRAM cost and optimization time. As shown in the Table 4, our TBS attack remains effective for Llama-3.3-70B on Mentalhealth. Notably, compared to smaller 7B Llamamodel, the inverted texts are closer to the ground truth as indicated by higher CS and F1 scores, possibly because of more information retained in widerISs (i.e., 8192 for 70B and 4096 for 7B). However, as the TBS attack requires forwarding and backpropagation 9 Input Inversion I am constantly having problems with the same two people who will always be in my life. I had a daughter with my ex‐boyfriend. I am now married, and my husband’s ex‐girlfriend is involved with my ex‐boyfriend. They also have a daughter together. My issue is that there is alw ays drama. I am pregnant, and I told my ex‐boyfriend that I don't want any drama or arguments. I want to get along as much as possible, and he agreed. However, we just had an incident where my ex‐boyfriend started discussing drop‐off details about my stepdaughter. I told him that he needed to ask my husband because I can't make decisions about my stepdaughter regarding the matter. That led to an argumen t. I told him all my concern is when I pick up my daughter. My stepdaughter’s pick‐up details are between my husband and his ex‐girlfriend. I especially told him I didn't want to be involved. Somehow, he turned it around and then wanted to change the schedule we agreed on. He threatened me and got ugly because I wouldn’t discuss my stepdaughter’s matters with him. The point is there is so much drama. I try my best to get along with everyone. I don't understand where I went wrong (besides replying back to his question). I feel like I'm going crazy because this is a constant battle where everyone’s frustrations are taken out on each other, and it's the children that are hurting. I had a party planned for my daughter’s birthday, and my ex‐boy friend told me to cancel those plans because he wouldn’t let me have her. In my eyes, it’s the child that is hurting. I was throwing a party for her birthday, and because of the problem with stupid pick‐up details about my stepdaughter, which I have no control over, he took it out on our daughter. I am constantly having problems with the same two people who will always be in my life. I had a daughter with my ex‐boyfriend. I am now married, and my husband’s ex‐girlfriend is involved with my ex‐boy boy friend. They also have a daughter together. My issue is that there is always drama. I am pregnant, and I told my exSony>'; that I don't want any drama or arguments. I want to get along as much as possible, and he agreed. However, we just had an incident where my ex‐boy boy friend started discussing drop‐off details about my stepdaughter. I told him that he needed to ask my husband because I can't make decisions about my stepdaughter regarding the matter. That led to an argument. I told him all my concern is when I pick up my daughter. my stepdaughter’s pick‐up details are between my husband and his ex girlfriend. I especially told him I does n't want to be involved. Somehow, he turned it around and then<|begin_of_text|> to change the schedule we agreed on. He threatened me and got ugly because I doesn’t discuss my stepdaughter’s matters with him. The point is there is so much drama. I try my best to get along with everyone. I don't understand where I went wrong (bes besides rep reply back to his question). I feel like I'm going crazy because this is a constant battle where everyone’s frustrations are taken out on each bir bir, and It's the children that are hurting. I had a party planned for my daughter’s birthday, and my ex‐ boyıldığı told me to cancel those plans because he didn’t let me have her. In my eyes, it’s the child that is hurting. I was throwing a party for her birthday, and because of the problem with stupid pick‐up details about my stepdaughter, which I have no control over, he took it out on our daughter. Figure 8: An inversion example from Mentalhealth consisting of 384 tokens. The missed tokens are highlighted in color. Table 4: Evaluation of different TBS attack settings for Llama- 3 model. DatasetSizeBasisPenaltyμCSBLEUROUGEF1 EC8B Unbiased0 1e-495.49±0.5964.93±2.900.9076.22±2.11 5e-488.79±0.8327.06±2.040.6649.56±1.65 Unbiased1e-3 1e-494.73±1.2858.09±6.560.8772.48±3.70 5e-490.82±1.0534.20±3.030.7154.70±2.29 SVD0 1e-495.87±0.9564.34±3.050.8873.90±2.60 5e-498.20±0.4972.09±3.140.9180.73±2.22 MH8B Unbiased0 1e-495.02±0.7080.36±1.470.9590.63±0.75 5e-488.59±1.6656.69±3.040.8778.53±1.99 Unbiased1e-3 1e-494.79±0.7777.55±2.000.9488.92±1.20 5e-489.09±0.7959.03±2.250.8880.30±1.08 SVD0 1e-497.64±0.3986.70±1.850.9693.27±1.05 5e-498.00±0.5790.24±1.240.9794.99±0.61 70BUnbiased05e-495.80±1.1774.17±9.440.9488.34±5.17 to iteratively update the inverted input, larger models can significantly increase the optimization time and GPU memory. For example, attacking 70B is 7 times slower and costs 10 times more memory than 7B. Input Data Type.We observe that the input data type can influence the inversion performance. In Figure 9, the coding data are more difficult to be inverted comparing to medical texts. Notably, the F1 score on coding datasets (92.05 and 94.50) are slightly lower than healthcare dialogue data under the optimal attack setting. After manual checking of inverted inputs, we found that the additional errors appear in the de- scription of coding prompts instead of the main code. This may indicate that the model knowledge may also influence our optimization-based inversion attack because of the attention assignment to tokens of different topics is unequal. Therefore, in the following we investigate whether the coding-enhanced model can lead to higher inversion risk. Evaluation of Domain-specialized Models.We consider Qwen2.5-Coder as the coding-specialized model because of its high ranking on the leaderboard [52]. Here we evaluate the top 50 longest coding prompts from two benchmarks. We also evaluate Qwen2.5 as a general-purpose model. Table 5 presents the results on two coding benchmarks. First, we observe that inversion is significantly better: the EM scores Table 5: Evaluation results of Qwen2.5 and Qwen2.5-Coder. DatasetModelDistanceCSBLEUROUGEEMF1 EC Qwen2.5MSE99.03±0.2698.90±0.181.0012.90±0.0698.90±0.21 Qwen2.5- Coder MSE99.78±0.1299.60±0.131.0064.00±0.0799.80±0.06 Qwen2.5- Coder COS99.96±0.0399.12±0.341.0045.00±0.1199.47±0.14 CA Qwen2.5- Coder MSE99.86±0.0899.78±0.051.0050.00±0.0799.82±0.04 Qwen2.5- Coder COS99.72±0.2899.46±0.371.0040.00±0.1699.76±0.08 are around 50% for the top long prompts of both benchmarks and the inverted tokens F1 scores are around 99%. Second, compared to the general-purpose model Qwen2.5, the coding- specialized model Qwen2.5-Coder hasISs more susceptible to inversion attack because of higher EM rates. Surprisingly, we observe that theISs of Qwen2.5 is easier to invert than Llama-3 models. We hypothesize no impact from their pretraining data and investigate the architecture differ- ence. We exclude the causes from model width and trainable parameters up to the middle layer because they are similar for two models: Qwen2.5 is of width 3,584 and 3.8B param- eters while Llama-3 is of width 4,096 and 4B parameters. After carefully examining their implementations, we found the main difference lies in the attention module: Qwen2.5 applies QKV bias [53] while Llama-3 does not by default. Therefore, we suspect that attention bias may amplify the fea- ture representation ofISs thus, as a side effect, enable better inversion. Due to the huge cost of pretraining to obtain similar bias for Llama-3, we leave more detailed impact analysis of attention bias for future work. Takeaway:On long-context inputs, our optimization- based attack TBS can invert nearly all tokens in the correct order to preserver semantics. 5.3 Black-box Inversion Attack In this section, we evaluate and compare our replication-based and generation-based black-box inversion attacks. 10 1e-3(mse)5e-4(mse)1e-4(mse) 5e-4(cos)1e-4(cos) (dist) 0.0 0.5 1.0 Value CS 1e-3(mse)5e-4(mse)1e-4(mse) 5e-4(cos)1e-4(cos) (dist) 0 50 BLEU 1e-3(mse)5e-4(mse)1e-4(mse) 5e-4(cos)1e-4(cos) (dist) 0.0 0.5 1.0 Rouge 1e-3(mse)5e-4(mse)1e-4(mse) 5e-4(cos)1e-4(cos) (dist) 0.0 0.5 1.0 F1 AIMedical CodeparrotApps Evolcode Mentalhealth Figure 9: Evaluation of our TBS attack with different learning rates and distance functions (MSE and COS). Merged Finetuned Adapter Negative 10 1 10 2 10 3 10 4 10 5 10 6 Autoencoder RMSE Layer 8 Merged Finetuned Adapter Negative Layer 16 Merged Finetuned Adapter Negative Layer 24 Figure 10: The distribution of autoencoder’s reconstruction error to detect derivatives of Llama-3. Llama-2Llama-3Qwen2.5Negative Prediction Llama-2 Llama-3 Qwen2.5 Negative True 0.990.010.000.01 0.000.940.000.06 0.000.000.920.08 0.000.000.001.00 (a) Confusion matrix of classificaiton. 0.000.250.500.751.00 FPR 0.0 0.5 1.0 TPR (b) ROC Curve for Llama-3. AUC = 0.998 Figure 11: Evaluation of model type identification. Attack Settings.We use the prompts from Mentalhealth and Evolcode for test. For both black-box attacks, we assume the adversary uses NoRobots [54] as training data for replication model and inversion model training. NoRobots is composed of 9.5k+ supervised finetuning samples from general topics, which aligns with our assumption that the adversary has data of similar distribution. We consider Llama-3 owned by the adversary and target Llama-3-Doctor and Llama-3.1 as two black-box deployed models. In this section we also testISs of the middle layer as above. Model Type Identification.Given unknownISs, the adver- sary first identifies the deployed model type. To test the iden- tify model type, we trained and crawled from Hugging Face 75 finetuned models, 70 merged models and 56 adapters of most downloading of Llama-3. In addition, we select 13LLMs independent models (i.e., those not derived from Llama-3) as negatives, which are listed in the Appendix A. Figure 10 demonstrates the distribution of autoencoder’s Reconstruction MSE (RMSE) on three tested layers. For each tested layer, we use the test dataset of 500 samples of NoRobots as the probing data, and train the 3-layer autoen- coder for 10 epochs. We found that most derivatives have separate error ranges to the independent models, which en- ables binary classifications by thresholding. Note that there Table 6: Results of the replication-inversion attack. Test DatasetAttackTargetBasislrCSBLEUROUGEF1 MH Transferred Doctor SVD1e-4 52.87±1.300.98±0.250.2214.79±1.77 Llama-3.156.21±1.241.85±0.470.2919.33±1.66 Doctor Unbiased 1e-464.50±2.706.64±1.620.4532.12±3.05 5e-458.36±1.120.90±0.080.2317.91±0.54 Llama-3.1 1e-461.19±2.144.72±0.760.4027.53±1.85 5e-463.60±2.102.45±0.510.3022.10±1.35 Replicated (Ours) Doctor Unbiased 1e-472.32±2.6416.33±2.230.5843.35±2.31 5e-466.74±1.795.54±1.300.4230.59±1.93 Llama-3.1 1e-468.33±2.334.91±0.710.4530.76±1.01 5e-461.90±1.570.87±0.160.2517.53±0.82 EC Transferred Doctor Unbiased 1e-462.01±1.703.60±0.680.2718.56±1.60 Llama-3.11e-463.05±3.082.13±0.460.2315.63±1.60 Replicated (Ours) Doctor Unbiased 1e-476.92±2.319.71±1.630.4729.60±2.17 Llama-3.11e-462.22±2.001.91±0.430.2616.74±1.21 are a small number of outliers that can be identified as inde- pendent model (False Negative). We check the false negatives and found they are caused by labeling errors based on crawled model name. Therefore, in the following test, we check the top downloading derivatives to remove potential errors. To evaluate the identification of ensemble autoencoders, we additionally crawl the derivatives of Llama-2 and Qwen2.5 in a similar manner and ensemble the autoencoders of middle- layerISs of threeLLMs. Figure 11 exhibits the confusion matrix of model type classification (left) and the ROC curve with AUC score (right). We found our ensemble autoencoder can almost perfectly classify the target mode type. Therefore, the adversary can leverage the publicly available baseLLM to apply the model replication-based inversion. Replication-based Inversion.We use Llama-3 replicate to the target models (ChatDoctor and Llama-3.1) on NoRobots for three epochs with Qlora [55] with learning rate 0.0001. As for the inversion attack, we use MSE as the loss distance. Table 6 presents the evaluation of our replication-based in- version on Mentalhealth and Evolcode, from which we can make four findings: 1) The use of SVD singular basis can degrade the attack because of the discrepancy between the inversion and the target input embedding spaces. 2) The repli- cated model with a low attack learning rate leads to better input inversion because of better input embedding alignment. 3) The post-training can offer better protection against inver- sion than finetuning, as Llama-3.1 is more difficult to invert input information (e.g., fewer matched tokens as indicated by lower F1). 4) The distribution gap between the adversary data and the victim’s queries can degrade the inversion. On 11 Table 7: Black-box inversion on short-context input text. DatasetAverage LengthCSBLEUROUGEEMF1 Instruct-2M26.1697.85±0.2492.73±0.580.9761.38±0.0296.87±0.27 Norobot-test89.0483.90±0.8451.34±1.550.6413.00±0.0265.76±1.44 SyntheticGPT177.4893.62±0.6146.98±1.250.670.072.90±0.88 50 75 Mentalhealth CS 0 50 BLEU 0.0 0.5 ROUGE 25 50 75 F1 123 Epoch 50 75 Evolcode 123 Epoch 0 2 123 Epoch 0.1 0.2 123 Epoch 20 40 Max Seq. Length1282565121024 Figure 12: Generative inversion on long-context datasets. coding data, the improvement from model replication is lower than on medical texts, as shown under the optimal setting (ChatDoctor with unbiased basis and 1e-4). In sum, the optimization-based attack performance is dete- riorated by limited attack information in the black-box setting and can be difficult to improve. As discussed earlier, the token recovery can be sensitive to the cosine difference between the inverted and the ground truth embeddings. The Transformer and input embedding weights of the derived model enlarge this inverted difference, resulting in worse token recovery. Next, we evaluate the generative model-based inversion at- tack which is based on observedISs as context and can be more robust to the embedding gap. Generation-based Inversion.As for inversion model, the adversary adopts the recentT5-base(T5) [56] as it is one of the most commonly used encoder-decoder model. We train T5 with learning rate 0.0002 and trains on Instruction-2M and NoRobots for short-context and long-context inputs, re- spectively. We also use SyntheticGPT [20] for short-context inversion evaluation. To align with replication-based black- box attack, the deployed model is Llama-3.1 as our target. Inversion of Short-context Input.We train inversion model on Instruction-2M for 1 epoch. Table 7 presents the inver- sion evaluation on short-context datasets. The attack performs best on the same-distribution test dataset of Instruction-2M and achieves even better performance than our white-box optimization-based inversion attack. However, the downside for generative model is that it can hardly scale to longer prompts and the performance is susceptible to distribution shift. As the length increases, the semantic similarity scores drop significantly on longer SyntheticGPT dataset. Inversion of Long-context Inputs.To understand the limit of input length for generative inversion, we train with NoRobots that contains long-context prompts of up to 3,384 tokens (de- 1378910 Epoch 50 75 CS 1378910 Epoch 0 50 BLEU 1378910 Epoch 0.25 0.50 0.75 ROUGE 1378910 Epoch 25 50 75 F1 Dataset Mentalhealth Evolcode Figure 13: Evaluation of converged inversion models. Table 8: Evaluation of generative inversion on 70B model. DatasetMax Seq. LengthCSBLEUROUGEEMF1 MH 25687.92±0.4521.50±0.570.500.055.56±0.57 51297.76±0.2279.39±1.840.882.00±0.0190.75±0.90 102497.44±0.2376.26±1.820.871.00±0.0188.72±0.90 EC 25674.85±0.813.69±0.220.160.0028.52±0.89 51294.18±0.4220.98±0.670.420.060.44±1.15 102491.74±0.5518.08±0.810.380.055.62±1.26 tailed length distribution is shown in Figure 16). To investi- gate the impact of training sequence length, we set maximum sequence length to 128, 256, 512 and 1,024. As NoRobots only contains 9,485 samples, we train inversion model up to 3 epochs that counts for similar model update steps to Instruction-2M for ensuring model convergence. After training, we evaluate on the top 100 longest prompts from Mentalhealth and Evolcode with greedy sampling of maximum length 4,096 to ensure full generation. Figure 12 shows the results from which we can make three observations. First, we found that the maximum sequence length 256 is the optimal setting for NoRobots dataset, where shorter or longer limits can result in worse inversion. Second, too long sequence limit 1024 can hinder the inversion quality. Third, the data type plays a central role to accurately invertISs: the model cannot generate meaningful inverted texts on cod- ing prompts (Evolcode) but achieve similar performance as optimization-based attack on the similar data (Mentalhealth). Longer Training Epochs.We also observe that the inversion model only generates prompts until the third epoch, which may indicate that that more epochs can improve the perfor- mance. In Figure 13, we extend the epochs to 10 for model convergence while keeping maximum sequence length 256. We found that the converged performance on Mentalhealth is lower than the best performance of optimization-based in- version as shown in Figure 9 (e.g., 94.99 BLEU and 96.7 F1 on Mentalhealth). On the other hand, the inversion on cod- ing data does not get improved as medical texts because of the distribution gap, which further highlights importance of the same distribution assumption. Appendix B includes more analysis on failure cases due to the distribution mismatch. Larger Models.Table 8 validates the effectiveness of the gen- erative inversion on Llama-3.3-70B model. Compared to pre- vious smallerLLMs, the optimal maximum sequence length for inversion of 70B models is doubled: from 256 to 512. In particular, there are even exact matching cases (e.g., inversion trained with maximum sequence length 512). We observe 12 Table 9: Transferability of generative inversion to Llama-3. DatasetMax. Seq. LengthCSBLEUROUGEF1 Mentalhealth 12886.67±0.4310.35±0.660.4554.06±0.68 25694.53±0.3053.12±1.710.7778.56±0.74 51290.03±0.3632.71±1.050.6463.32±0.62 Evolcode 12876.97±0.810.13±0.050.1528.37±1.01 25684.04±0.762.10±0.190.2340.63±1.19 51274.78±0.831.57±0.160.1531.45±1.00 that, despite deeper and widerISs, more input tokens can be inverted as signified by higher F1 scores and CS scores. This result aligns with our previous white-box inversion evaluation on the 70B model. Comparison with Optimization-based Inversion.We exam- ine the generated inversion inputs and qualitatively compare them with those optimized by our white-box attack. Although both achieve similar semantic similarity and token matching score, the reason for failed inversion tokens is different. As mentioned previously, the optimized inversion inputs may introduce or replace the true token with noisy tokens of dif- ferent languages or Unicode which hinders the readability but also partially reveals the meaning of true inputs. On the other hand, generative inverted texts contain no noisy tokens but may miss entirely some sentences, especially those in the middle or the end of the original input. This is a disadvantage to the adversary because of potential key information loss. Transferability.We explore whether the generative inversion model can directly transfer to the derived target model. We evaluate the T5 model trained on Llama-3.1’sISs and evaluate on Llama-3 (see Table 9). The inversion performance is close to the non-transfer case as shown in Figure 12. This validates our claim that the generation-based can better tolerate the embedding gap than the optimization-based attack. 5.4 Defense Evaluation In this section, we test four practical defenses includingDP, pruning, quantization, dropout and noisy input embedding. In Section 7, we discuss and provide mitigation suggestions. Setting.We use Mentalhealth and consider the collaborative inference server who receives theISsh f V l at the middle layer l=16for Llama-3-8B. The client applies defenses before the release ofISs for the first-round inference. For attack, we test the generation-based inversion attack trained for 10 epochs on NoRobots with the optimal maximum sequence length 256. In terms of utility, we evaluate the model with five-shot Massive Multitask Language Understanding (MMLU) [57], which refers to accuracy on the benchmark questions. We fix the random seed to ensure reproducibility for defenses involving randomness. Quantization.We first test the 4-bit and 8-bit model quanti- zation which, as a common technique to reduce memory cost, can alter the adversary-observedISs through through forward- propagated error to degrade inversion performance. We test the model quantization and present the results in Table 10 of Appendix A. We note that only 4-bit slightly effect the MMLU but the inversion quality remains nearly unaffected, indicating that quantization cannot defend our attack. Dropout.We apply the dropout onISs of probabilityp∈ 0.1,·,0.8that nullifiespelements and scales the rest by1/(1−p). Aspincreases, the MMLU score will firstly decrease. Meanwhile, we can see that MMLU drops faster than the CS score. Among the highest dropout probabilities we testedp≥0.7, the MMLU score is close to the random guess (∼0.25) while we can still achieve inversion of CS score higher than 70. This indicates that dropout cannot fail our generative inversion while preserving the utility. Noisy Input Embedding.The defender can blur the input embedding to obfuscate theISs. Therefore, we add the noise following Gaussian distribution of varianceσ 2 , and show the evaluation results in the middle of Figure 14. For low noise scaleσ≤0.005, there is negligible affect over the inference and inversion. However, we observe thatσ=0.01is a turning point: the MMLU score become lower than 0.4 while the inversion is almost unaffected. Our attack can be effectively defended at high noise scaleσ∈0.05,0.1, but the MMLU also decreases to the random guess level. To wrap up, noisy input embedding cannot achieve a perfect trade-off between the inference utility and ISs privacy. Differential Privacy.We add Laplacian noise to theISto achieveε-DP:δ∼Lap(0,ε/∆ s ), where∆ s is the sensitivity. As the maximum value ofISs is not bounded, we clip theh f V l byC ∆ s ∈200,500, because we observe the highestISs are within this range on Mentalhealth. This makes∆ s =2C ∆ s . The rightmost figure of Figure 14 shows the inversion per- formance (measured by CS) and model utility on different εand two clipping cases, where the horizontal dotted lines represent noDPprotection. We observe that the blue curves (inverted text similarity) increase earlier than orange curves (model utility) in both clipping cases. This indicates that the adversary can gain advantage with lowεthat almost nullifies the model. In other words, with mild drop of model utility (e.g.,ε=5,000forC ∆ s =200 ), our generation-based attack remains equally effective. Takeaway:Our generation-based inversion can achieve accurate inversion on data of similar distribution and bypass practical defenses directly applied on ISs. 6 Related Work In this section, we review applications ofISand inversion attacks in Natural Language Processing (NLP). 13 0.10.20.30.40.50.60.70.8 Dropout Probability 0.3 0.4 0.5 0.6 MMLU Dropout 70 80 90 100 Cosine Similarity 0.10.050.010.0050.001 Noise 0.3 0.4 0.5 0.6 MMLU Noisy Input Embedding 60 80 100 Cosine Similarity 1002005001,0002,0005,00010,00020,00050,000 (log scale) 0.3 0.4 0.5 0.6 MMLU Differential Privacy Clip 200 Clip 500 No Defense 40 60 80 100 Cosine Similarity Figure 14: Evaluation of defenses including noisy input embedding, dropout and differential privacy with Laplace mechanism. Application of Internal States.Recently, several works have shown that theISare strong indicator of hallucination fac- tual error and safety status, which provide a practical strategy forLLMholder and regulator to surveil the model behavior. Azaria and Mitchell first discovered that theISs can indi- cate factual errors and found that deep layer’sISs can train accurate hallucination detector [7]. TheISs-based halluci- nation detection can also be achieved by eigenvalues [8], unsupervised clustering [9] and linear probing [10] to stop the mistakes on the fly [11]. The middle layer hasISs that can indicate malicious and benign queries and can be used as safety layer to defend jailbreak attacks [12]. Additionally, ISs can reveal latent knowledge [16], membership privacy of query data [36], emotion [15,28], internal symbolic cal- culations [17] and potential backdoors [13,14]. As more applications ofISs emerge, the potential privacy risk should be carefully examined and our work takes the first step to testify the ISs inversion feasibility. Inversion Attacks.In addition to the embedding [19] and out- puts [20,24], the risk of data inversion also exists for shared gradient in decentralized learning [58,59], unlearning [60], outsourced shallow-layer inference [61] and KV cache [62] which are the most related works to ours. Nevertheless, our methods can extend to middle or last layers and can generalize to KV cache. Note that the side-channel attack toLLMserv- ing [63] is also similar to our inversion, but leverages timing difference in accelerated inference. There is also finetuning- based defenses [38] but is limited to smallLMs because of performance drop forLLMs. Recently, there are concurrent works [64,65] attempting to invert prompt from the malicious server in collaborative learning. As for comparison, our work includes more comprehensive attacks (white-box and black- box) and has validated the attack effectiveness on long inputs (4k+ tokens) and large models (up to 70B). 7 Discussion & Conclusion We conclude with discussion and future work. Practical Mitigation.As we witnessed in section 5.4, di- rectly protectingISs is still limited to simultaneously offer both model utility and privacy. The fundamental reason is that the inference of subsequent layers depends on meaningful ISvalues, hence the noise-induced representation obfusca- tion must preserve sufficient utility by maintaining semantic coherence. To mitigate privacy leakage caused byISs, it is es- sential to safeguard the entire model rather than concentrating solely on individual layers. One possible direction can be the exploitation of cryptographic tools or confidential computing. The homomorphic encryption is a promising solution, but cur- rent implementations generate unacceptable computational overhead for deployment [64]. Another countermeasure that can be immediately taken is exploiting the confidential vir- tual machine in CPU (e.g., AMD SEV [66]) and GPU (e.g., H100 [67]) which can provide confidentiality guarantee and allow addition obfuscation schemes inside the enclaves to hinder side channel attacks. Architecture-based Mitigation.We suspect the equal-width architecture design may cause our privacy inversion attacks. Typically, the layer width in conventional classification mod- els decreases with depth, enabling the compression of input data and contributing to the emergence of Harmless Space (HS) [68]. In contrast, Transformer-based language models have a uniform layer width. This architectural difference may explain why our attack methods are successful. Thus, a po- tential architecture-based defense could involve a model with varying sizes of input embeddings, intermediate states, and output embeddings, which, by exploiting information loss, could increase the difficulty of accurate inversion. Unavailable Ground Truth.In Section 3, the adversary aims to recover the exact input, but original inputs are generally un- available in practice, which can make it infeasible to check the inversion correctness. In practice, for the open-source models, the adversary can test with surrogate inputs as reference to ensure constraint. For closed-sourced models, the adversary can collect input data from their interested distributions for querying and training the inversion model. The convergence can guarantee the inversion for in-distribution data. Limitation.Our optimization-attack is sensitive to the hyper- parameter and added noises. For example, TBS cannot recover meaningful inputs even under the highestεwe tested because of noise sensitivity. Future work can enhance this attack by denoising onDP-protectedISs. Another limitation is the lin- ear complexity (i.e.,O(E)) during optimization, which can cause longer attack time for larger models. Also, it is possible 14 to adopt random initialization for optimization-based attacks to avoid local optima. Appendix B includes more detailed analysis of failure cases and boundary conditions. To improve the attack, one of the future direction is to come up with black-box inversion attack that can also be as context-free and length-free as the white-box optimization-based attack. 8 Ethical considerations Our attacks exploitISs of models to invert nearly original sensitive input, posing significant threats to user privacy and data security. As our research do not involve human partic- ipants and only use public medical dialogue, Mentalhealth and coding datasets, the IRB of authors’ intuition, after our consultation, determined that our research does not require further review. We acknowledge that exemption from IRB review may not fully address all ethical considerations, and have therefore taken the following steps. Ensuring no harm is caused to stakeholders.The potential stakeholders include LLM user and LLM service provider and broader privacy community. To mitigate potential harms to individuals, by following existing privacy work, we only select public datasets that are highly-downloaded and appro- priately anonymized. In addition, we also manually checked all datasets used in our research to ensure there is no sensitive information (e.g., PII) exposed in our research. We did not attack real-world collaborative inference systems, so no indi- vidual privacy is leaked by our research. As for the models, we only used opensource models including Llama, Qwen, and T5 in our work for compliance with model-use license. Responsible disclose to service providers.We also share our findings with main cloud LLM service providers that might deploy collaborative inference including OpenAI, Meta AI and Qwen. We recognize that this cannot perfectly mitigate privacy risks, because our attacks, if known to the public, may still be misused by the actual collaborative inference server or IS auditor to recover the user inputs. However, we believe the benefits of publicizing our attacks outweight the potential harm. As our work demonstrates the potential privacy risks of LLM ISs, it will draw increased attention to, not only the ISs inversion threat, but also the general privacy concerns sur- rounding LLM service systems. This will encourage the LLM practitioners to proactively implement well-established solu- tions like confidential virtual machine within their systems to offer better privacy protection. 9 Open Science In compliance with the USENIX Security’s Open Science pol- icy, we commit to publicly releasing the source code to imple- ment the attacks, pretrained inversion models on non-sensitive data in our study upon acceptance of this paper and inversion logs of main results. As the datasets used in our paper are all downloadable from Hugging Face, in our artifact we direct users to the datasets downloading link and provide processing scripts. We also provide instructions on how to test our attacks on user’s own data, detailed configuration, program scripts, hyperparameters and hardware requirements. Our code is re- leased athttps://doi.org/10.5281/zenodo.15605325. Acknowledgments We thank our shepherd and anonymous reviewers for insight- ful feedback. We also thank Yiming Wang for insightful dis- cussions that inspired this project. The work has been sup- ported in part by the National Natural Science Foundation of China (62325207, 62132013, 62302298). Shaofeng Li is sup- ported by the Start-up Research Fund of Southeast University (No. RF1028624178). References [1]Robert Hart. X fact checks elon musk after he blasts apple-openai partnership as ‘creepy’ privacy nightmare. https://w.forbes.com/sites/roberthart/2024/06/11/x- fact-checks-elon-musk-after-he-blasts-apple-openai- partnership-as-creepy-privacy-nightmare/, 2024. [2]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. [3]Zheng Lin, Xuanjie Hu, Yuxin Zhang, Zhe Chen, Zihan Fang, Xianhao Chen, Ang Li, Praneeth Vepakomma, and Yue Gao. Splitlora: A split parameter-efficient fine- tuning framework for large language models.arXiv preprint arXiv:2407.00952, 2024. [4]Yixuan Mei, Yonghao Zhuang, Xupeng Miao, Juncheng Yang, Zhihao Jia, and Rashmi Vinayak. Helix: Dis- tributed serving of large language models via max-flow on heterogeneous gpus. InProceedings of Interna- tional Conference on Architectural Support for Program- ming Languages and Operating Systems, ASPLOS 2025, 2025. [5]Mingjin Zhang, Jiannong Cao, Xiaoming Shen, and Zeyang Cui.Edgeshard: Efficient llm inference via collaborative edge computing.arXiv preprint arXiv:2405.14371, 2024. [6]Fact sheet: President biden issues executive order on safe, secure, and trustworthy artificial intelligence. https://w.whitehouse.gov/briefing-room/statements- releases/2023/10/30/fact-sheet-president-biden-issues- executive-order-on-safe-secure-and-trustworthy- artificial-intelligence/, 2023. 15 [7]Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it‘s lying. InFindings of the As- sociation for Computational Linguistics: EMNLP 2023, pages 967–976, December 2023. doi: 10.18653/v1/2023. findings-emnlp.68. [8] Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. INSIDE: LLMs’ internal states retain the power of hallucination detection. InThe Twelfth International Conference on Learning Representations, 2024. [9]Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. Unsupervised real-time hallucination detection based on the internal states of large language models. InFindings of the Asso- ciation for Computational Linguistics ACL 2024, pages 14379–14391, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguis- tics. [10]Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth A. Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination de- tection in llms.CoRR, abs/2406.15927, 2024. doi: 10.48550/ARXIV.2406.15927. [11]Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen- Zhu. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process.arXiv preprint arXiv:2407.20311, 2024. [12]Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. Safety layers of aligned large language models: The key to llm security.arXiv preprint arXiv:2408.17003, 2024. [13]M Lamparth and A Reuel. Analyzing and editing inner mechanisms of backdoored language 353 models. In The 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT), page 354, 2024. [14] Tian Dong, Minhui Xue, Guoxing Chen, Rayne Holland, Yan Meng, Shaofeng Li, Zhen Liu, and Haojin Zhu. The philosopher’s stone: Trojaning plugins of large language models. InNetwork and Distributed System Security Symposium, NDSS 2025. The Internet Society, 2025. [15]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Rep- resentation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023. [16] Collin Burns, Haotian Ye, Dan Klein, and Jacob Stein- hardt. Discovering latent knowledge in language models without supervision. InThe Eleventh International Con- ference on Learning Representations, 2023. [17]Junhao Chen, Shengding Hu, Zhiyuan Liu, and Maosong Sun. States hidden in hidden states: Llms emerge dis- crete state representations implicitly.arXiv preprint arXiv:2407.11421, 2024. [18]Congzheng Song and Ananth Raghunathan. Informa- tion leakage in embedding models. InProceedings of the 2020 ACM SIGSAC conference on computer and communications security, pages 377–390, 2020. [19] John Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander Rush. Text embeddings reveal (almost) as much as text. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 12448–12460, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.765. [20] Collin Zhang, John X. Morris, and Vitaly Shmatikov. Extracting prompts by inverting LLM outputs. InPro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, pages 14753–14777. Association for Computational Linguis- tics, 2024. [21]Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In2015 IEEE Information Theory Workshop (ITW), pages 1–5, 2015. doi: 10.1109/ITW.2015.7133169. [22]Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. Privacy risks of general-purpose language models. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1314–1331, 2020. doi: 10.1109/SP40000.2020. 00095. [23]Haoran Li, Mingshi Xu, and Yangqiu Song. Sentence embedding leaks more information than you expect: Generative embedding inversion attack to recover the whole sentence.arXiv preprint arXiv:2305.03010, 2023. [24]John Xavier Morris, Wenting Zhao, Justin T Chiu, Vitaly Shmatikov, and Alexander M Rush. Language model inversion. InThe Twelfth International Conference on Learning Representations, 2024. [25] Jieren Deng, Yijue Wang, Ji Li, Chenghong Wang, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, and Caiwen Ding. TAG: gradient attack on transformer-based lan- guage models. InFindings of the Association for Com- putational Linguistics: EMNLP 2021, pages 3600–3610, 2021. doi: 10.18653/V1/2021.FINDINGS-EMNLP.305. [26]Xinguo Feng, Zhongkui Ma, Zihan Wang, Eu Joe Chegne, Mengyao Ma, Alsharif Abuadbba, and Guang- dong Bai. Uncovering gradient inversion risks in prac- tical language model training. InProceedings of the 16 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 3525–3539, 2024. [27]Guanzhong Chen, Zhenghan Qin, Mingxin Yang, Yajie Zhou, Tao Fan, Tianyu Du, and Zenglin Xu. Unveiling the vulnerability of private fine-tuning in split-based frameworks for large language models: A bidirectionally enhanced attack. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, page 2904–2918. Association for Computing Machinery, 2024. ISBN 9798400706363. doi: 10.1145/ 3658644.3690295. [28]Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. How alignment and jail- break work: Explain LLM safety through intermediate hidden states. InFindings of the Association for Compu- tational Linguistics: EMNLP 2024, pages 2461–2488, November 2024. doi: 10.18653/v1/2024.findings-emnlp. 139. [29]OpenAI.Enterprise privacy at openai - does ope- nai review my business data?https://openai.com/ enterprise-privacy/, 2024. [30]Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Maksim Riabinin, Younes Belkada, Artem Chu- machenko, Pavel Samygin, and Colin Raffel. Petals: Collaborative inference and fine-tuning of large mod- els. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 558–568, July 2023. doi: 10.18653/v1/2023.acl-demo.54. [31]Zhichuang Sun, Ruimin Sun, Changming Liu, Am- rita Roy Chowdhury, Long Lu, and Somesh Jha. Shad- ownet: A secure and efficient on-device model inference system for convolutional neural networks. In2023 IEEE Symposium on Security and Privacy (SP), pages 1596– 1612, 2023. doi: 10.1109/SP46215.2023.10179382. [32] Z. Zhang, C. Gong, Y. Cai, Y. Yuan, B. Liu, D. Li, Y. Guo, and X. Chen. No privacy left outside: On the (in-)security of tee-shielded dnn partition for on-device ml. In2024 IEEE Symposium on Security and Privacy (SP), pages 55–55, may 2024. [33] Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gus- tavo Hernández Ábrego, Ji Ma, Vincent Y Zhao, Yi Luan, Keith B Hall, Ming-Wei Chang, et al. Large dual encoders are generalizable retrievers.arXiv preprint arXiv:2112.07899, 2021. [34] Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. Exploring concept depth: How large language models acquire knowledge at different layers? InProceedings of the 2025 Joint International Conference on Computational Linguistics, Language Re- sources and Evaluation (LREC-COLING 2025), 2025. [35]Baixiang Huang, Canyu Chen, and Kai Shu. Authorship attribution in the era of llms: Problems, methodologies, and challenges.arXiv preprint arXiv:2408.08946, 2024. [36]Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawi- jaya, Yejin Bang, Bryan Wilie, and Pascale Fung. LLM internal states reveal hallucination risk faced with a query. InProceedings of the 7th BlackboxNLP Work- shop: Analyzing and Interpreting Neural Networks for NLP, pages 88–104, Miami, Florida, US, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.blackboxnlp-1.6. [37] Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd. Get my drift? catching llm task drift with activation deltas. InSaTML, 2025. [38]Minxin Du, Xiang Yue, Sherman S. M. Chow, Tian- hao Wang, Chenyu Huang, and Huan Sun. Dp-forward: Fine-tuning and inference on language models with dif- ferential privacy in forward pass. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS 2023, pages 2665–2679. ACM, 2023. [39]Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023, 2023. [40] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024. [41]ContactDoctor.Bio-medical:Ahigh- performancebiomedicallanguagemodel. https://huggingface.co/ContactDoctor/Bio-Medical- Llama-3-8B, 2024. [42] Meta AI. Introducing llama 3.1: Our most capable mod- els to date. https://ai.meta.com/blog/meta-llama-3-1/, 2024. [43]Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding, 2023. [44]Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. Mteb: Massive text embedding bench- mark.arXiv preprint arXiv:2210.07316, 2022. doi: 10.48550/ARXIV.2210.07316. 17 [45]MTEBLeaderboard.Massivetextem- beddingbenchmark(mteb)leaderboard. https://huggingface.co/spaces/mteb/leaderboard, 2024. [46]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019. [47]Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, ̇ Ilhan Polat, Yu Feng, Eric W. Moore, Jake Van- derPlas, Denis Laxalde, Josef Perktold, Robert Cimr- man, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pe- dregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Com- puting in Python.Nature Methods, 17:261–272, 2020. doi: 10.1038/s41592-019-0686-2. [48] Ruslan Magana Vsevolodovna. Ai medical dataset, 2023.URLhttps://github.com/ruslanmv/ ai-medical-chatbot. [49] Amod.mental_health_counseling_conversations (revision9015341), 2024.URLhttps: //huggingface.co/datasets/Amod/mental_ health_counseling_conversations. [50] Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct, 2023. [51] Dan Hendrycks, Steven Basart, Saurav Kadavath, Man- tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Stein- hardt. Measuring coding challenge competence with apps.NeurIPS, 2021. [52]bigcode.Bigcodemodelsleaderboard. https://huggingface.co/spaces/bigcode/bigcode- models-leaderboard, 2025. [53]Jianlin Su. The magical effect of the bias term: Rope + bias = better length extrapolation.https://spaces. ac.cn/archives/9577, 2023. [54] Nazneen Rajani, Lewis Tunstall, Edward Beeching, Nathan Lambert, Alexander M. Rush, and Thomas Wolf. No robots.https://huggingface.co/datasets/ HuggingFaceH4/no_robots, 2023. [55]Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quan- tized LLMs. InThirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023. [56]Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020. [57]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InICLR, 2021. [58]Ziang Li, Mengda Yang, Yaxin Liu, Juan Wang, Hongxin Hu, Wenzhe Yi, and Xiaoyang Xu. GAN you see me? enhanced data reconstruction attacks against split infer- ence. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, 2023. [59]Guanzhong Chen, Zhenhan Qin, Mingxin Yang, Yajie Zhou, Tao Fan, Tianyu Du, and Zenglin Xu. Unveiling the vulnerability of private fine-tuning in split-based frameworks for large language models: A bidirection- ally enhanced attack. InProceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security, CCS 2024. ACM, 2024. [60] Hongsheng Hu, Shuo Wang, Tian Dong, and Minhui Xue. Learn what you want to unlearn: Unlearning inver- sion attacks against machine unlearning. In2024 IEEE Symposium on Security and Privacy (SP), pages 3257– 3275, may 2024. doi: 10.1109/SP54263.2024.00248. [61]Fei Zheng. Input reconstruction attack against verti- cal federated large language models.arXiv preprint arXiv:2311.07585, 2023. [62] Huan Yang, Deyu Zhang, Yudong Zhao, Yuanchun Li, and Yunxin Liu. A first look at efficient and secure on-device llm inference against kv leakage, 2024. [63]Guanlong Wu, Zheng Zhang, Weili Wang, Jianyu Niu, Yao Zhang, Ye Wu, and Yinqian Zhang. I know what you asked: Prompt leakage via kv-cache sharing in multi- tenant llm serving. InNetwork and Distributed System Security Symposium, NDSS 2025. The Internet Society, 2025. [64]Wenjie Qu, Yuguang Zhou, Yongji Wu, Tingsong Xiao, Binhang Yuan, Yiming Li, and Jiaheng Zhang. Prompt Inversion Attack against Collaborative Inference of Large Language Models . In2025 IEEE Symposium on Security and Privacy (SP), pages 1602–1619. IEEE 18 Table 10: Evaluation of quantization as defense. DefenseMMLUCSF1 No Quant0.6597.33±0.2588.77±0.83 8-bit0.6597.40±0.2488.84±0.83 4-bit0.6297.22±0.2488.57±0.83 Computer Society, May 2025. doi: 10.1109/SP61157. 2025.00160. [65]Xinjian Luo, Ting Yu, and Xiaokui Xiao. Prompt in- ference attack on distributed large language model in- ference frameworks.arXiv preprint arXiv:2503.09291, 2025. [66]AMD Sev-Snp. Strengthening vm isolation with in- tegrity protection and more.White Paper, January, 53: 1450–1465, 2020. [67] Rob Nertney.Confidential Compute on NVIDIA Hopper H100.https://images.nvidia.cn/ aem-dam/en-z/Solutions/data-center/ HCC-Whitepaper-v1.0.pdf, 2023. [68]Lu Chen, Shaofeng Li, Benhao Huang, Fan Yang, Zheng Li, Jie Li, and Yuan Luo. Seeing is not always believing: The space of harmless perturbations.arXiv preprint arXiv:2402.02095, 2024. A Additional Results Visualization of Internal States.We applies the t-SNE onto theISs of various open-sourceLLMs across different scales. As visualized in Figure 15,LLMs rooted from a common pretrainedLLM, have similarISs. For example, Llama-3 and Llama-2 are two pretrained models and have separa- bleISs. That said, it is possible to train classify the target LLMtype simply using theISs. To evaluate model type identification of Llama-3, we select the total 13LLMs open- sourced before or after Llama-3:Vicuna-7B-v1.5, GLM-4- 9B-chat, Qwen2.5-14B-Instruct, Meta-Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-3B-Instruct, Meta-Llama-3.1- 8B (Qlora), gemma-2-9B-it, Llama-3.2-3B-Instruct, Yi-1.5- 9B-Chat, Mistral-7B-Instruct-v0.3, Llama-2-7b-chat-hf, and Mistral-7B-Instruct-v0.2. Token length of NoRobots.Figure 16 shows the histogram of token length of NoRobots. Compared to Instruction2M, we note there are certain long-context training samples which can be indispensable to train long-context inversion model, at expense of increased training time and memory usage. Quantization Defense.Table 10 shows the evaluation of our generative inversion on quantized model. We use bitsandbytesfor LLM quantization. 0.00 0.25 0.50 0.75 1.00 Layer 2Layer 4 0.00.20.40.60.81.0 0.00 0.25 0.50 0.75 1.00 Layer 8 0.00.20.40.60.81.0 Layer 16 LLM Llama-2-7B-chat-hf Llama-3-8B-Instruct Llama-3.1-8B-Instruct Llama-3.1-8B-Instruct (Qlora) Mistral-7B-Instruct-v0.2 Mistral-7B-Instruct-v0.3 Yi-1.5-9B-Chat GLM-4-9B-chat Vicuna-7B-v1.5 LLM Type Llama Mistral Yi GLM Figure 15: Visualization of internal states for mainstream LLMs. The selectedLLMs have the same width 4096 for dimension reduction using t-SNE. 1101001000 Length of Strings (Log Scale) 0 50 100 150 200 250 Frequency Figure 16: Distribution of token length of NoRobots. Figure 17: Visualization of sampled input embeddings from Llama-3-8B (left), the singular basis obtained through SVD decomposition of input embeddings (middle) and unbiased basis (right). 19 Theoretical Analysis.We investigate the gradient magnitude for ER and TSB. To simply notation, we denote the targetIS of layerl,h f V l (x)) byh, and consider the adversary optimizes b w to approximateψ l ( b w)toh. Without loss of generality, we consider the squaredL 2 norm for the distancedand b w∈R d in . The loss function is: L=d(ψ l ( b w),h) = ∥ ψ l ( b w)−h ∥ 2 2 .(8) The gradient with respect to b wis: ∂L ∂ b w =2(ψ l ( b w)−h) ⊤ · ∂ψ l ∂ b w .(9) Here, the gradient magnitude of ER depends directly on the residual error ∥ ψ l ( b w)−h ∥ and the Jacobian∂ψ l /∂ b wof the firstlTransformer layers. For TBS, as b w=φ z ( b z·B), the gradient with respect tozbecomes: ∂L ∂ b z = ∂L ∂ b w · ∂ b w ∂ b z = ∂L ∂ b w · ∂φ z ( b z·B) ∂ b z .(10) In the experiments, we setφ z ( b z·B) =αarctan( b z·B), thus the derivative is ∂ b w ∂ b z = α 1+( b zB) 2 ·B ⊤ .(11) For smallz, the scaling factorα/(1+( b zB) 2 )≈α, leading to larger gradient magnitudes compared to the baseline. Asz increases, the factorα/(1+( b zB) 2 )→0, which stabilizes the gradient on b bz , especially for deep model where the Jacobian can vary greatly. In conclusion, TBS introduces anonlinear scalingof gra- dients throughφ z , creating a dynamic where: ∂L ∂ b z ∝ 1 1+( b zB) 2 · ∂L ∂ b w .(12) Visualization of Input Embedding and Basis.To support the findings of Figure 5, we also visualize the vectors through uniform t-SNE dimension reduction. Figure 17 demonstrates that the singular basis vectors and input embeddings exhibit tight clustering in the representation space, whereas the unbi- ased basis displays significantly greater dispersion. B Analysis of Failure Cases In this section, we analyze the failure cases caused by non- convergence for our optimization-based attack TBS and by the distribution mismatch for our generation-based inversion attack. The failure cases for our white-box attack are commonly caused by inappropriate settings. For example, too high learn- ing rate may accelerate the optimization at the beginning but fall into local minimum afterwards, as validated by the worse inversion quality in Figure 9. A failure example of high learn- ing rate 0.001 is shown in Figure 18, where we can see most inverted tokens are unreadable. Even there are some keywords recovered, the adversary cannot guess original inputs. Besides, due to the complexity of optimization space, we also observe failure example even under appropriate setting. In Figure 20, we provide the loss curves for the first four samples of Mentalhealth during TBS on Llama-3.3-70B. We observe that the Sample 3 encountered loss divergence, lead- ing to inversion F1 score 72.91. We observe that Sample 3 exhibits loss divergence, resulting in the F1 score of 72.91. In contrast, the remaining samples successfully escape local op- tima despite similar initial loss increases, ultimately achieving smooth convergence. Given our fixed initialization scheme, we explore random initialization for as a potential alternative. Empirical evaluation reveals that standard Gaussian noise ini- tialization degrades convergence speed and ultimately yields inferior inversion performance. Future research directions in- clude developing bounded random initialization strategies or incorporating dynamic noise injection during optimization. Figure 19 illustrates a failure case of generation-based inversion arising from distributional mismatch. The target text contains SQL table code that lies outside the inver- sion model’s training distribution, resulting in uninterpretable “<unk>” tokens. This demonstrates that a necessary condition for successful inversion is comprehensive training data cover- age of all target input tokens. Furthermore, the discrepancy in token distributions between the inversion model’s training data and target inputs may lead to suboptimal inversion perfor- mance, particularly when significant frequency mismatches exist for critical tokens. 20 Input ...inthisnegativeenvironmentwithmychildandkeepourfamilytogether?DoImoveawaywithmychildandhavemyrelationshipend?Idonot wanttotakehimoutofeitherofhiskids’lives.WhatdoIdo? Inver- sion ...inthisnegativeenvironmentwithyourаласяandkeep�تfamilytogether?DoelementGuidIdалася.ArаласяаласяChildandHavemy relationshipendقصI경우notwant(cudatakehimDECREFبشكلeitherблагодhiskids’s ▍WhatполученияI_SL ânвияв Figure 18: Failure inversion example of TBS attack in MentalHealth. Generate an SQL query to find the average score for each student, but exclude any students whose average score is below 6. The result should be ordered in decreasing order of the average score. Table: student_scores| id <unk> student_id<unk> score <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> Generate an SQL query to find the average score for each student, but exclude any students whose average score is below 6.5. The result should be ordered in descending order of the average score. Table: student_scores | id | student_id| score | |-----|------------|-------| | 1 | 111 | 5 | | 2 | 111 | 7 | | 3 | 223 | 6 | ... ... InputInversion Figure 19: Failure inversion example of generation-based inversion in EvolCode. 02000040000 Steps 10 5 10 3 10 1 Loss Sample 1 Sample 2 Sample 3 Sample 4 Figure 20: Loss curves of TBS attack in Mentalhealth. 21 Inputs Inversion this is very long I was diagthis is very long I was diagnosed with HSV 2 about 3 years ago. I was in a long distance relationship with a female for a year already but had never met her. She has a huge problem with germs and ofcourse people getting tested for std’s. She was stern about this. I had led a very unhealthy sexual lifestyle as a young man. I was now 30 but I was afraid of my past, especially my teenage years. Still, I was sure I didn t have anything because I never had any symptoms. Anyways, I got tested for all STD’s right before I went to visit her but didn t get the results until I was already at her place and we had already done multiple sexual acts including intercourse. They called me and told me that I had HSV 2 on my genitals. I was floored. I was sure that someone in my recent past had knowingly given it to me. I was on suicide watch in my own mind. I was so depressed and I knew that it wasn t HIV but I also felt that it wasn t far behind. Infact, I was scared to death when I got the call and heard the doctor say the H and the V. All I thought about was them calling me to tell me that I had HIV but luckily it was only HSV but it didn t matter to me. I felt basically the same way as if they told me it was the other. I felt alone with no one to talk to. I tried to talk to the doctors \& nurses on the phone at the health department but didn t feel a lot of sympathy. It was the same when I went to the pharmacy to pickup the meds. I just felt like no one cared and I was a scar on the world. Again, I was so angry at the recent partners I had before my current gf. However, after doing some research and looking at pics, I realized that I had HSV 2 for over a decade, dating back to when I was a teen that had unprotected sex with just about any female I met. It never crossed my mind that they bumps I would get once or twice a year on my penis was herpes or any std. I just always felt that my penis was rubbing against my boxers wrong or the zipper on my pants that caused a rash and it would go away in less than a week. It never dawned on me that It may have been something else but sure enough it was the virus. That gave me mixed feelings. One is that I wondered about all the people I may have infected in the past and 2 is that I felt some relief in knowing that this isn t new, that Ive had it for over a decade, but it never really affected my life in any way. I knew then that I could atleast live a normal life with it. I went on my meds. Acyclovier 2 capsules, 2 times per day. I told the Health department about all my past partners although I was scared. I knew I would want someone to do that for me. However, the problem was my current gf. She’s pretty 50/50 in that on one hand she’s the sweetest person you would ever meet. She has helped me so much and saved my life in so many ways. On the other hand she had a hard upbringing which makes her pretty cold \& judgmental at times. I feared telling her for multiple reasons and couldn t bring myself to do it. I kept a close eye on her since she’s been open with me and would tell me if she ever had any kind of discomfort. She never seemed to get any symptoms. Meanwhile, Im taking my meds every single day faithfully. Combine that with already having it for 10 years, I was told that my disease was probably pretty dormant and chances of my effecting someone from now on were very low even with unprotected sex. Sure enough 3 more years have gone by. She’s had no symptoms, we see eachother a lot now and she became pregnant after being told she probably would never be able to have kids. I thought I couldn t either because of all the things I did when I was younger. I had so much unprotected sex and never pulled out and I never got anyone pregnant but somehow we found out and it seemed like a blessing. However, one huge problem came up. The doctor who put me on my meds told me I wold have to be on them for life but every January 1st, my prescription is stopped and I have to go see the doctor. I never wanted to do this and she had already said it was a life long thing anyways so she would always just give me a new prescription for the year without even seeing me. Again, its been 3 years since my diagnosis and I literally hadn t had one outbreak. Not even anything close. Then this past new year came and again my meds are stopped. I call to get them refilled but this time they tell me that the doctor is no longer with them and they have a new guy now who I must see before he refills them. I give in and go to see him. He says him and the previous doctor use to argue about the meds a lot because he feels that research has shown that you don t need to take meds every day the rest of your life. That just wait until you feel an outbreak coming on and then take some for a week. I told him that Im truly not comfortable with that. That Ive heard others that do it but it doesn t seem safe to me and I am use to taking this every day. He said I don t need it and I may never have one again after having HSV 2 for almost 15 years and being on meds nonstop for 3 years. He gives me some pills from the office if I ever need for an emergency and he writes me a new prescription just incase of the future but tells me to not take meds everyday. I give in. I was so scared and felt so weird but he was right as nothing happened. I went to see my girl again for the 1st time since I found out she was pregnant. She was now going on 6 months. It is June and I also have been off my meds for 6 months. Everything is lovely until one day I feel a tingle on my penis and here it comes. Everything happened so quick. I was out of state and forgot the meds at home. I was only suppose to be there 10 days and then go back home for a summer semester of school until I come back down in her 8th month where I will live with her permanently. The meds are home and its the weekend. Im not even sure if this is an outbreak since I hadn t had one this is very long ґ was diagnosed with HSV 2 about 3 years ago. I was in a long distance relationship with a female for a year.errorMessage but hadтисяmet her. She has a huge problem with ger germ and ofcourse people getting tested for std’sे . She was stern about this· I had led a very unhealthy sexual lifestyle as a youngıldığı. I was now 30 but I was afraid of my past, especially my teenage years. still�取 I was sure I didn zahrn had anything because I never had any symptoms. AnyAnyway, I got tested for all STD’s right before I went to visit her but didn doesnt get the results until I was already at her place and we had already done multiple sexual acts including intercourse. They called me and toldıldığıthat I had HSV 2 on my genitals. I was flo floor. I was sure that someone in my recent past had knowingly gave it to me�取 I was on suicide Watch in my own mind. I was so depressed and I knew that it wasn’t HIV but I also felt that it doesnıldığıFar behind. Inf fact, I was scared to Death when I got the call and heard the doctor say the H and the V. All I thought about was them call,to tell me that I had HIV but<|begin_of_text|>It was only HSV but It didn didnt matter to me. I felt basically The same way as if they told me it was the other. I felt alone with no none to talkıldığı. I tried to talk� the doctors \& nurses on the phone at the health department but doesn didnt feel a lot of sympathy. It was the same when I went to the pharmacy to pickup the meds. I just felt like no one cared and I was a scar on the world. Again, I was so angry at the recent partners I had before my current gf. However, after doing some research and looking at pics, I realized that I had HSV 2 for over a decade, dating back to when I was a teen that had unprotected sex with just about any female I met. It never crossed my mind that they bumps I would get once or twiceAyear on my penis was herpes or any std. I just always felt that my penis was rubbing against my box boxerWrong or the zipper<|begin_of_text|> my pants that caused a rash and it would go away in less than a week. It never dawndawn on me that It may have been something else but sure enough it was the virus. That gave me<|begin_of_text|> feelings. one is that I wondered about all the people I may have infected in the past and 2 is that I felt some relief in knowing that this doesn’t new, that Ive had it for over a decade, but it never really affected my life in any way. I knew then that I could atleast live a normal life with it. I went on my meds. Ac(tclovıldığı 2 capsules, 2 times per day. I told the Health department about all my past partners although I was scared. I knew I would want someone to do that forıldığı. However, the problem was my current gf. she’s pretty 50스50 inthat on one hand she’s the sweetest person you would ever meet. She has helped me so much and saved my life in so many ways. Onničníother hand she had a hard upbringing which makes her pretty cold \& judgmental at times. I feared telling her for multiple reasons and couldn cannot bring myself to do it. I kept a close kişi on her since she’s been open with me and would tell me if she ever had any kind of discomfort. She never seemed to get any symptoms. Meanwhile, Im taking my meds every single day faithfully. Combine that with already having it for 10 years<|begin_of_text|> I was told that my disease was probably pretty dormant and chances of my effecteffected someone from now on were very low even with unprotected sex. Sure enough 3 more years have gonebye. She’s had no symptoms, we see eachother a lot now and she became pregnant after being told she probably would never be didn Ability have kids. I thought I couldn\\'t either because of all the things I did when I was younger. I had so much unprotected sex and never pulled out and I never get anyone pregnant but somehow we found out and It seemed like a blessing. However, one huge problem came up. TheDr who put me on my meds told me I wıldığı have to be on them for life but every January 1алася, my prescription is stop and I have to go see the doctor. I never wanted to do this and she had already said it was a Life Long thing anyways so she would always just give me a new prescription for the year without even see me. Again， its been 3 years since my diagnosis and I literally hasn’t had one outbreak� 取not even anything close. Then this past new year came and again mijnаласяis stopped. I call to get them ref refill but this time they tell me that the doctor is no longer with them and they have a new guy now who I must see before he ref refill them. I give in and go to seen him. He says him and the previous doctor use to argue about the meds a lot because he feels that research has�取that you don\\'t need to take meds every day the rest of your life. That just wait until you feel an outbreak coming on and then take some for a week. I told him that Im truly not comfortable with that. that Ive heard others that do it but It doesn\\'t seems safe to me and I am use to taking this every day. he said I doesn\'t need it and I may never have one again after having HSV 2 for almost 15 years and being on meds nonıldığı for 3 years. He give me some pills from the office if I ever need for an emergency and he writes me a new prescription just inc case of the future but tells me to not take meds everyday. I give_IN. I was so scared and felt so weird but he was right as nothing happened). I went to see my girl again for the 1 first time since I found out sheıldığıpregnantıldığıshe was now going on 6 months. It is June and I also have been off my meds for 6 months. Everything is lovely until one day I feel a tingle on my penis and here it comes� 取everything happened so quick. I was outOutOf state and forgot the meds at home. I was only suppose to be there 10 days and then go back home for a summer semester of school until I come back down in her 8 eighth month where I will live with her permanently. The meds are home and its the weekend. Im not even sure if this is an outbreak since I doesn’t had one in so long but I think it could be. It was usually a big area of so sore but This time it was just Figure 21: Inversion example of a 4,112-token prompt (Part 1). 22 in so long but I think it could be. It was usually a big area of sores but this time it was just like one or 2 sores. My girl wanted to have sex. I panicked so much. Even though we were always unprotected, I kept condoms on me just incase. Everything was happening so quick and I was scared to say NO and have her ask why. Then what would I say? I told her I wasn t feeling good but she wouldn t stop. I was so scared. I put a condom in my pocket with her not knowing but she grabbed me, pulled me close to her, and started performing oral sex on me. I didn t think she would do that. It threw me completely off. Everything was a mess. 1st off it hurt because of my sore, 2nd all I could do was think about me having an outbreak, 3rd I knew that I had read that chances are very very very very low of giving her HSV 2 from her performing oral sex on me. Normally I wouldn t ejaculate but this time I forced myself to just so we didn t have to have sex. Still, later she wanted some and I put a condom on without her knowing and had sex with her. Literally, the very next day she showed my a rash on the back of her hand that was itching and it looked exactly like the herpes blisters that I get on my penis except smaller. I was so confused but at the same time knew where it came from. We also had sex the previous 2 days unprotected but I didn t have open sores. It made me wonder a lot of things. I was always told that Herpes took atleast 2 days to show up so how can she get the signs just 12 hours later. Then I read that in some rare cases they do showup the next day. Then I wondered did I maybe give it to her the other previous 2 days? Was I shedding before I had sores and didn t know it? I was wondering how did she get it when I had a condom on? Is it because she grabbed my penis and went down on me? Everything I read said it was very low risk for that. Then I thought maybe I had rubbed my penis on her vagina or close enough to it. Maybe I touched my infection and put my hand between her legs but I couldn t remember. I thought maybe after intercourse, I took the condom off too soon to hide it from her and maybe some fluids got crossed over. I just didn t know. I was so upset. I hated my doctor for telling me to go off my meds. Even moreso I hated myself for not being man enough to tell her that I was having an outbreak. Why let her go down or me or have sex? I should ve thought of something quick? Im a lowlife piece of crap. That s how I felt. It just all happened so fast that I froze. What about her? What about my baby? How could I let this happen? Why did I stop taking my meds? Why didn t I leave on time? Oh yes, I got off track earlier in my story but I meant to say that I was only suppose to be there 10 days but I changed my flight and had it pushed back another week. Literally the very next day after I was suppose to go back is when I had an outbreak. I only stayed for her and it backfired because I gave her herpes but maybe I gave it to her a couple of days before that anyways? All of this was running through my head. I still didn t have the guts to tell her. I went and got more meds the next day. I was still having sex with her. Each time with a condom that she didnt know about. I pushed my flight back 2 more times and have been here for another 3 weeks. I was kinda happy for a sec because her hand bumps went away pretty quick but then it came right back again. At the same time she said she feels like she’s about to get a pimple on her lip. She said she can feel it coming. She told me to look and you can see it getting red. Sure enough it was but it never came. It went right away like her hand and never returned. Her hand came back though then it went away pretty quick again. Now it came back a 3rd time and this time she now also has it on her opposite wrist. I started to freakout. l I think my girl has a weak immune system and she’s not a fighter when it comes to things that itch. She cant help but to scratch it so I knew it would be hard to heal. I did a ton of research and had all these questions. I read that I cant give her hsv 1 if I only have hsv 2 so Im positive that s what she has. However, I wondered was it only on her hand \& wrist but not her vagina since she had no symptoms there and I wore a condom? Everything I read said different things. I wondered could it be Herpetic Whitlow but those things seem like on the finger tips while her’s is on the back of her hand and on the wrist. I read about zolster but she’s already had chicken pox. I read that if its above the waist then its oral HSV 2. Then I read that its the genital one but it has spread. I was going crazy. I was hoping that maybe she got it somehow only on her hand/wrist and nowhere else. That its not on her genitals. I was trying to tell her that she needs to stop scratching and it seems contagious. I told her that its spreading but she kept scratching and had no relief. We were going to have sex again, she put her hand between her legs and I got upset at her and told her not do to that right now. I wasn t mean at all but more concerned and she said that she’s fine, its just a lil rash but it got worse. I read that you cant spread herpes to other parts after the initial infection but this is her 1st one so I was trying to get her to stop before she does it more \& more. Yesterday she told me she has some bumps on the bottom of her belly and it looked a little like it but wasn t sure. Im just so scared. I hinted to her a few days ago that it maybe herpes but you can tell she didn t buy that. We tried Benadryl spray, alcohol, and others to stop the itching but nothing worked. She’s been showing her family the rash which was driving me crazy. I even got crazy enough to where the last couple of nights I put some of my acyclovir on ice cream or a donut and gave it to her without her knowing. After she started scratching so bad last night I finally confessed everything to her and she’s been an emotional wreck. We use to not get along a lot anyways because she has a bad attitude at times while Im a pretty laid back dude. Her mother is very all knowing and like one or 2 so sore. My girl wanted to having sex. I panicked so much. Even though we were always unprotected, I kept condoms on me just inc case. Everything was happening so quick and I was scared to say NO and have her ask why. Then what would I say? I told her I�取not feeling good but she didn\\'t stop. I was so scared). I put a condom in my pocket with her not know but she grabbed me, pulled me close to herıldığı and starts performing oral sex on me. I doesn t think she would do that. It threw me completely off. everything was a mess. 1 first off it hurt because of my sore, 2 second all I could do can think about me having an outbreak, 3 third I knew that I had read that chances are very very very very low of giving her HSV� 取2 from her performing oral sex on me. Normally I doesn’t ejaculate but this time I forced myself to just so we didn\\'t have to have sex. Still, later she wanted some and I put a condom on without her knowing and had sex<|begin_of_text|> her. Liter literally, the very next day she showed my a rash on the back of her hand that was itching and it looked exactly like the herpes blisters that I get on my penis except smaller. I was so confused but at the same time knew where it comes from. We also had sex the previous 2 days unprotected but I doesn doesnt have open so paralysis. It made me wonder a lot of things. I was always told that Her herpes took atleast 2 days to show up so how can she get the signs just 12 hours later). Then I read that in some rare cases they do showup the next day. Then I wondered did I maybe give it to her the other previous 2 days? Was I shedding before I had so sore and didnıldığıknow it? I was wondering how did she get it when I had a condom on? Is it because<|begin_of_text|> grabbed my penis and went down on me? everything I read said it was very low risk for that. Then I thought maybe I had rubbed my penis on her vagina or close enough to it. Maybe I touched my infection and put my hand between her legs but I couldn cannot remember. I thought maybe after intercourse, I took the condom off too soon to hide it from her and maybe some fluids got crossed over. I just didn’t know. I was so upset. I hated my doctor for telling me to go off my meds....\ Even moreso I hated myself for not being man enough to tell her that I was having an outbreak. Why let her go down or me or have sex? I should ve thought of something quick? Im a lowlife piece of crap. That s how I felt. It just all happened so fast that I froze. What about her? What about my baby? How could I let this happen? Why did I stop taken my\ 3000\ 3000\ 3000\ 3000\ 3000\ 3000? Why didn’t I leave on time? Oh yes, I got off track earlier in my story but I meant to say that I was only suppose to be there 10 days but I changed my flight and had it pushed back another week. Liter literally the very next day after I was suppose to go back is when I had an outbreak. I only stay for her and it backf backlash because I gave her herpes but maybe I gave it to her a couple few days before that anyways? all of this was running through my head. I still didn\'t have the guts to tell her. I went and got more meds the next day. I was still having sex with her. each time with a condom that she didnt know about. I pushed my flight back 2 more times and have been here for another 3 weeks. I was kinda happy for a sec because her hand bumps went away pretty quick but then It came right back again. At the same time she said she feel like she’s about to get a pimple on her lip. She said he can feel it coming. She told me to look and you can see it getting red. Sure enough it was but it never came. It went right away like her hand and never returned. Her hand came back though then it went away pretty quick again. Now it came back a 3 third time and this time she now also has it on her opposite wrist. I started to freakout. lI think my girl has a weak immune system and she’s not a fighter when it comes to things that itch. she cant help but to scratch it so I knew it would be hard to heal. I did a ton of research and had all these questions. I read that I cannot give her hsv 1 if I only have hsv 2 so Im positive that s what She has. However, I wondered was it only on her hand \& wrist but not her vagina since she had no symptoms there and I wore a condom? everything I read said different things. I wondered could it be Herpetic Whitlow but those things seem like on the finger fingertips while her’s is on the back of her Hand and on the wrist. I read about ZOlSTER but she’s already had chicken pox. I read that if its above the waist then Its oral HSV 2. Then I read that its the genital one but It Has spread. I was going crazy. I was hoping that maybe she got it somehow only on her hand/w wrist and nowhere else. That its not on her genitals. I was trying to tell her that she need to stop scratching and It seems contagious. I told her that its spreading but she kept scratching and had no relief. we were going gonna have sex again, she put her hand between her legs and I got upset at her and told her not do to that right now. I wasn not mean at all but more concerned and she said that she’s fine], its just a lil rash but it got worse. I read that you cannot spread herpes to other parts after the initial infection but this is her 1 first one so I was trying to get her to stop before she does it more \& more.Yesterday she told me she has some bumps on the bottom of her belly and it looked a little like it but wasn not sure. Im just so scared. I hinted to her a few days ago that it maybe herpes but you can tell she doesn didnt buy that. We tried Benaddryl spray, alcohol, and others to stop the itching but nothing worked. She’s been showing her family the rash which was driving me crazy. I even got crazy enough to where the last couple few nights I put some of my acycEventoir on ice cream or a Donut and gave it to her without her knowing. After she started scratching so bad last night I finally confessed everything to her and she’s been an emotional wreck. We use to not get along a lot anyways because she has a bad attitude at times while Im a pretty laid back dude. her mother is very allKnowing and overprotect protective so we doesn not get along all the time either. I just knew everything was going gonna come to a head. My girl has Figure 22: Inversion example of a 4,112-token prompt (Part 2). 23 overprotective so we don t get along all the time either. I just knew everything was going to come to a head. My girl has gone back \& forth from crying uncontrollably to then being violent and attacking me. She said I ruined her life. She said she would ve never been with me if she had known. She never talks about what happened a couple of weeks ago with the sores and me letting her go down on me. Instead she focuses about me knowing for 3 years and never telling her. She said she would ve never done that to me. She said no one is going to want her and she doesn t want to be with me again. She then breaks down thinking about our baby. She said that the baby is probably going to have it and even if she doesn t at birth, that she wont be able to touch her because she will be so scared of infecting her with her hand \& wrist like that. She is so distraught. One time she says she doesn t even want the baby anymore but then she changes. She also got mad at me for giving her the meds since she might cant even take them when pregnant. Then she read about the 1st outbreak stuff late in pregnancy and she is going crazy. I told her that everything she is saying is right. I feel like Im the worst person on earth. I take full blame. I do at times get so upset at that doctor for taking me off the meds but I say that if I didn t let her go down on me that night or would ve just not had sex when I saw the sore then the doctor still may have been right and I wouldn t have passed it on so its all my fault but maybe I was shedding before then and gave it to her anyways. She still brings up the 3 years thing but I told her that I didn t find out until after we had already had intercourse and by that time I felt like you either already would have it or Im on the meds now so she wouldn t get it. She had test she thinks for herpes when she got pregnant and came back negative. I was so happy at that time and thanked god because it had bothered me for years but now I went and screwed it up right after that anyways at the worst possible time. Right when she’s pregnant. I feel so bad. I wish I could have it all back. The 3 years. The 3 weeks. People are so scared to say anything but then they end up having to say something anyways when its 10x worse because you have now infected them which is the one thing you wanted to avoid and could ve avoided if you would ve told them sooner. If she had known then it would ve been her decision. That s what she keeps telling me. I decided her life for her and its not fair. She is so right and everything she feels now is how I felt when I got diagnosed. I told her I already went through that but its not a death sentence and it really doesn t change anything in your life but you can tell that she doesn t want to hear that from me of all people. Im the one guy she doesn t want to hear it from. I don t think we will get through this. I will never forgive myself. Im a great person and Im of age now at 34 yet I managed to do something so selfish, so immature, and so repugnant to someone who loves me so much. Can someone please help me atleast by answering some of the following questions: 1-Did I definitely give her hsv 2 and nothing else? 2-Is it possible that she only has it on her hand/wrist and not her genitals or does it still mean that it definitely started In the genitals? 3-is it possible that its oral hsv2 and not genital? 4-if its went away and came back multiple times in a short span, does that mean its still the 1st outbreak or are these already reoccurring? I just want to make sure that its done spreading so her antibodies can build up and she has no risk of getting it anywhere else, especially the genitals 5-Is all herpes dangerous for your baby during pregnancy or is it strictly if you are having a vaginal outbreak at the time of birth? I keep hearing mixed things on this? Like is my baby still at risk if its only on her hands/wrist and or oral? 6-if it doesn t spread anymore during this outbreak and its not on her vagina, does that mean it will be impossible for her to get it on her vagina in the future since she will have built up the antibodies? 7-Ive heard some say that when you have hsv2 in areas like your hand that its less likely to reoccur or atleast reoccur often while others have said that it will happen more often than the normal kind. Do you know? 8-I use to hear that if you take the meds every day like you are suppose to then chances of you infecting someone even with unprotected sex is very low but it seems like on here that you guys still sound like its 50/50 if you are taking meds every day. 9-It seems like her blisters maybe really contagious and even I feel like Im about to get rashes in some areas but then they never come. Is it possible that she has some kind of herpes that I still can get in other spots or is it just trying to maybe do stuff to me but my antibodies wont let it? 10-will a vaginal hsv2 outbreak always hurt and if she does have it on her hands/wrist with painful itching and scratching, would she also get that same itch on her vagina if she had it down there? Again, she hasn t had any symptoms down there and Im hoping she doesn t. 11- Can she transfer it to her eyes? She s been messing with her eyes alot. Some of it maybe allergies but Im still scared that she is spreading it to them. All of this was very long and Im sure there will be some things later that I wish I would ve added but atleast I hope I get some answers. She will be going to the doctor soon to get all the answers even though she wants me out of her life now. Her mother will know soon which will be hell to deal with but I deserve it all. Thanks for the help either way gone back \& forth from crying uncont Controlی ود to then being violent and attack me. She said I ruined her life. She said she would ve never been with me if she had known. she never talks about what happened a couple few weeks ago with the sores and me letting her go down on me. Instead she focuses about me knowing for 3 years and never telling her. She said he would’ve never done that to me. She said no nobody is going gonna want her and she doesn’t want to be with me again. She then breaks down thinking about our baby. She said that The baby is probably going gonna have it and even if she doesn\\'t at birth, that she wont be able unable touch her because she will be so scared of infectinfect her with her hand \& wrist like that. she is so distrarott. one time she says she doesn’t even want the baby anymore but then she change. She also got mad at me for giving her the meds since she might cant even take them when pregnant.Then she read about the 1 first outbreak stuff late in pregnancy and she is going crazy. I told her that everything she is saying is right. I feel like Im the worst person on earth). I take full blame. I do at times get so upset at that doctor for taking me off the meds but I say that if I didn not let her go down on me that night or would ve just not had sex when I saw the sore then the doctor still may have been right and I couldn would have passed it on so its all my fault but maybe I was shedding before then and gave it to her anyways. She still brings up the 3 years thing but I told her that I doesn not find out until after we had already had intercourse and by that time I felt like you either already would have it or Im on the meds now so she doesn’t get it. She had test she think for herpes when she got pregnant and came back negative. I was so happy at that time and thanked god because it had bothered me for years but now I went and screwed it up right after that anyways at the worst possible time. Right when she’s pregnant. I feel so bad. I wish I could have it all back. The 3 years. the 3 weeks. People are so scared to say anything but then they end up having to say something anyways when its 10x worse because you have now infected them which is the one thing you wanted to avoid and could ve avoided if you would ve told them sooner. If she had known then it would ve been her decision. That s what she keeps telling me. I decided her life for her and its not fair. She is so right and everything she feels now is how I felt when I got diagnosed. I told her I already went through that but its not a death sentence and It really doesn\'t change anything in your life but you can tell that she doesn’t want to hear that from me of all people. Im the one guy she doesn not want to hear it from. I doesn not think we will get through this. I will never forgive myself.Im a great person and Im of age now at 34 yet I managed to do something so selfish, so immature, and so repugnant to someone who loves me so much. Can someone please help me atleast by answering some of the following questions: 1-Did I definitely giving her hsv 2 and nothing else? 2-Is it possible that she only has it on her hand/w wrist and not her genitals or does it still mean that it definitely started In the genitals? 3-is it possible that its oral hsv2 and not genital? 4 If its went away and came back multiple times in a short span神马收录 does that mean its still the 1 first outbreak or are these already re");\ \ \ \ ? I just want to make sure that its done spreading so her antibodies can build up and she has no risk of getting it anywhere else, especially the genitals 5-Is all herpes dangerous for your baby during pregnancy or Is it strictly if you are having a vaginal outbreak at the time of birth? I keep hearing mixed things on this? like is my baby still at risk if its only on her hands/w wrist and or oral? 6if it doesn’t spread anymore during this outbreak and its not on her vagina, does that mean it will be impossible for her to get it on her vagina in the future since she will have built up the antibodies? 7Ive heard some say that when you have hsv2 in areas like your hand that its less likely to reg occur or atleast reicrocur often while others have said that it will happen more often than the normal kind. Doıldığıknow? 8I use to hear that if you take the meds every day like you are supposed to then chances of you infectinfect someone even with unprotected sex is very low but It seems like on here that you guys still sound like its 50/50 if you are taking meds every day. 9-It seems like her gl blister maybe really contagious and even I feel like Im about to get r rash in some areas but then they never come. Is it possible that she has some kind kinda herpes that I still can get in other spot or is it just trying to maybe do stuff to me but my antibodies wont let it? 10-w will a vaginal hsv2 outbreak always hurt and if she does have it on her hands/w wrist with painful itching and scratching, would she also get that same itch on her vagina if she had it down there? Again, she doesn\\'t had any symptoms down there and Im hoping she doesn not. 11- can she transfer it to her eyes? she s been messing with her eyes alot. some of it maybe allergies but Im still scared that she is spreading it to them. All of this was very long and Im sure there will be some things later that I wish I would’ve added but atleast I hope I get some answersे . she will be going to the doctor soon to get all the answers even though she wants me out of her life now. Her mother will know soon which will be hell to deal with but I deserve it all. Thanks for the help either way Figure 23: Inversion example of a 4,112-token prompt (Part 3). 24