Paper deep dive
Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings
Yubo Wang, Shujie Cui, James Bailey, Hongzhi Yin, Wenyu Liang, Min Tang, Shiyue Qin, Weiqing Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/20/2026, 4:50:16 AM
Summary
This paper introduces DAEI (Denoising-Aware Embedding Inversion), a novel pipeline designed to recover original text from noise-protected text embeddings. The authors identify a 'Double Noise Trap' where standard generative inversion methods fail because additive Gaussian noise creates misleading correction signals. DAEI addresses this by employing a residual denoising autoencoder (DAE) trained unsupervised via Stein's Unbiased Risk Estimate (SURE) to remove noise before inversion. Experiments demonstrate that DAEI significantly outperforms existing baselines like Vec2Text and ZSInvert, challenging the assumption that simple Gaussian perturbation ensures embedding privacy.
Entities (8)
Relation Signals (7)
DAEI → usescomponent → Denoising Autoencoder
confidence 95% · DAEI... combines a residual denoising autoencoder with generative text inversion
DAEI → usestechnique → Stein's Unbiased Risk Estimate
confidence 93% · the denoiser is trained in an unsupervised manner using Stein's unbiased risk estimate
Gaussian Noise → causes → Double Noise Trap
confidence 92% · Additive Gaussian noise pushes the target embedding off the clean text manifold... exposes the standard inversion process to what we term the 'Double Noise Trap'.
DAEI → outperforms → Vec2Text
confidence 90% · DAEI achieves approximately 154% relative improvement in BLEU over the existing generative inversion baseline
Double Noise Trap → prevents → Vec2Text
confidence 90% · Double Noise Trap... fundamentally prevents standard generative inversion models from achieving high-quality reconstruction.
DAEI → outperforms → ZSInvert
confidence 88% · substantially outperforming ZSInvert
DAEI → mitigates → Privacy Leakage
confidence 85% · The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a "Double Noise Trap", which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein's unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154\% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32--60\%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations.
Tags
Links
- Source: https://arxiv.org/abs/2608.18610v1
- Canonical: https://arxiv.org/abs/2608.18610v1
Trouble viewing inline? Open PDF directly →
Full Text
60,668 characters extracted from source content.
Expand or collapse full text
Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings Yubo Wang1, Shujie Cui1, James Bailey1, Hongzhi Yin2, Wenyu Liang1, Min Tang1, Shiyue Qin3, Weiqing Wang1 Affiliation: 1Monash University, Melbourne, Australia 2The University of Queensland, Brisbane, Australia 3Northeastern University, Shenyang, China yubo.wang2, shujie.cui, james.a.bailey, Teresa.Wang@monash.edu wlia0047@student.monash.edu, bupttm@163.com, h.yin1@uq.edu.au, qinsy@mail.neu.edu.cn Abstract Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a “Double Noise Trap”, which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein’s unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32–60%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations. The code for reproduction is provided below: https://github.com/ChrisWang233/DAEI Index Terms: Machine Learning, Dense Text Embeddings, Embedding Inversion Attack, Privacy Leakage, Denoising Autoencoder I Introduction Dense text embeddings have become a standard data representation in modern data mining and machine learning systems [28, 12, 21]. Across applications such as semantic search, user modeling, vector databases, and retrieval-augmented analytics, raw textual records are increasingly transformed into compact continuous vectors that are stored, searched, or used for various tasks by downstream systems [15, 39, 9]. This representation is attractive because it transforms raw data into low-dimensional and semantically rich vector representations, making subsequent tasks more efficient[28, 39]. Yet these benefits create a potential privacy issue: high-quality embeddings are explicitly trained to preserve semantic and lexical information, and may therefore also preserve sensitive entities and attributes[31, 16]. Consequently, any leakage allows adversaries to apply sophisticated extraction techniques to recover the underlying sensitive data, resulting in a privacy vulnerability. Recent works have made this privacy risk concrete. Generative embedding inversion attacks reconstruct full sentences from sentence embeddings, rather than simply recovering unordered keywords [16]. Vec2Text further frames inversion as controlled generation by introducing an iterative correction framework, substantially improving reconstruction accuracy for short inputs [20]. In practice, embeddings may be exposed through compromised vector databases, honest-but-curious storage or retrieval providers, and distributed systems that transmit embeddings online across components where the adversaries can attack the embedding during transmission [19, 35]. Once an adversary can observe these embeddings, the embeddings themselves may become a text disclosure channel. One natural defense strategy is to perturb embeddings before release. Differential privacy formalizes the broader principle of adding calibrated noise to sensitive computations [7], and prior work on privacy-preserving textual analysis has shown that perturbations in embedding spaces can maintain downstream utility while reducing disclosure risks in data mining workflows [8, 36, 6]. The most prevalent defense mechanism is additive Gaussian noise injection [20, 40, 29]. In practice, adding zero-mean Gaussian noise with standard deviation σ=0.01σ=0.01 to the final representations of the target embedder can severely degrade generative inversion performance while barely affecting downstream task utility [20]. However, as far as we know, there are no existing embedding inversion methods studying whether embedding-to-text inversion remains possible under this noise-protected setting. The closest related work is ZSInvert [38], which focuses on zero-shot inversion instead of focusing on noise-protected setting. We consider it the closest work because it includes an experimental case study to show that the proposed approach is noise robust [38]. The possible reason is that its search-based optimization offers some robustness to noisy embeddings, but the recovered texts are often less readable and weakly token-aligned, which is also validated in our experiments. Therefore, we include it as a noise-robust baseline, while noting that conventional generative inversion methods remain largely ineffective under the noise-protected embedding setting. This difficulty goes beyond a harder inversion task: noise creates a fundamentally different and underexplored inversion paradigm. In this setting, adversaries are deprived of clean embeddings and can observe only noisy vectors. This makes inversion substantially more challenging for two reasons: first, the absence of clean targets invalidates standard supervised denoising objectives that rely on reconstruction error against clean references; second, the inverse mapping must disentangle the underlying semantic signal from high-dimensional noise before text can be reliably recovered. Consequently, these constraints motivate the central question of our work: can adversaries recover sufficient textual information from black-box noisy embeddings without relying on clean embeddings? We investigate this question from a security-evaluation perspective. Specifically, we study the no-clean-embedding regime and propose DAEI: Denoising-Aware Embedding Inversion, an integrated pipeline for measuring the privacy risk of noise-protected text embeddings. Building upon Vec2Text as our generative backbone for text recovery, we prepend a residual denoising autoencoder (DAE) [33] to the inversion decoder to separate semantic signals from high-dimensional noise. A conventional DAE typically relies on a Mean Squared Error (MSE) objective against clean reference embeddings, which are unavailable in our noisy-only setting [33, 34]. Under the Gaussian noise model, we therefore use Stein’s unbiased risk estimate (SURE) [32, 27] to remove this reliance on clean embeddings, yielding an unbiased risk estimate computable from noisy observations alone. We further use Monte-Carlo Hutchinson estimation [27] to approximate the required divergence term with an explicit variance-based error bound. To align denoising with text generation, we jointly fine-tune both modules. This coupled optimization guides the DAE to preserve critical semantic directions, while dynamically adapting the inverter to the refined embedding space. Furthermore, unlike methods such as Noise2Noise [14] that require multiple noisy observations of each sample which are usually unavailable in real world applications, DAEI operates with only one noisy observation per sample. In summary, our main contributions are as follows: • We challenge the prevailing assumption that noise guarantees text embedding privacy. We are the first to formulate and systematically evaluate an adaptive “denoising-inversion” approach. By demonstrating that underlying textual information can still be reliably recovered from noise-protected embeddings, we provide a critical re-evaluation of current representation security standards. • We systematically analyze the failure of existing generative inversion approach in reconstructing text from noisy embeddings and introduce “Double Noise Trap” (Section I-B). • To the best of our knowledge, we are the first to study denoising autoencoders for noisy text embedding inversion under a strict no-clean-target setting. • We propose DAEI, a novel denoising-aware inversion pipeline that couples DAE pre-training, DAE-shielded inversion, components joint fine-tuning, and DAE-shielded correction, enabling robust text recovery from noisy observations. • We validate DAEI across multiple benchmarks and embedding backbones. Compared with noisy Vec2Text, DAEI achieves approximately 154% relative improvement in BLEU and 32–60% gains in F1 and ROUGE-L, while also substantially outperforming ZSInvert. I Problem Formulation and Motivation I-A Problem Formulation Fig. 1: Geometric intuition of the “Double Noise Trap”. Additive Gaussian noise pushes the target embedding off the clean text manifold, causing vanilla inversion to chase off-manifold directions. DAEI instead projects the noisy representation back toward the clean manifold before inversion. Let x∈x denote an input text and let fclean:→ℝdf_clean:X ^d be a frozen black-box text encoder. In the standard setting, the exact latent representation is given by z=fclean(x),z=f_clean(x), (1) We study a privacy-preserving regime where the system releases only a perturbed version of the final embedding, denoted as y: y=f(x)=z+ϵ,ϵ∼(0,σ2Id).y=f(x)=z+ε, ε (0,σ^2I_d). (2) where f denotes a noisy embedder, and the standard deviation of noise scale σ is assumed to be known or estimable, but the clean embedding z remains strictly inaccessible to the attacker. Adversary assumption: In our threat model, we assume that the adversary has no knowledge of the embedder such as architecture and parameters. 1) Adversary can only query the released embedding interface as a black box and obtain an output embedding for a given attacker-chosen input text, thereby constructing noisy embedding–text training pairs (y,x)(y,x); 2) The adversary is assumed to know that the perturbation is additive Gaussian noise, since it is a standard and widely adopted mechanism for protecting text embedding; 3) The adversary either knows the noise scale σ, as prior work commonly adopts σ=0.01σ=0.01 to achieve the best tradeoff between effectiveness and privacy [20]; or can estimate σ through repeated calibration queries on any adversary-chosen input text by observing the variation of the returned embeddings. Specifically, for n repeated observations of the same calibration text, y(j)=z+ϵ(j)j=1n\y^(j)=z+ε^(j)\_j=1^n, an unbiased estimator of the noise variance is σ^2=1d(n−1)∑j=1n‖y(j)−y¯‖22,y¯=1n∑j=1ny(j). σ^2= 1d(n-1) _j=1^n\|y^(j)- y\|_2^2, y= 1n _j=1^ny^(j). (3) Under Gaussian noise, d(n−1)σ^2/σ2d(n-1) σ^2/σ^2 follows a chi-square distribution with d(n−1)d(n-1) degrees of freedom [3], so the relative standard error of σ^2 σ^2 is 2/(d(n−1)) 2/(d(n-1)). For a 768-dimensional embedder, even n=10n=10 calibration queries give a relative standard error of about 1.7%1.7\% for σ^2 σ^2 and about 0.85%0.85\% for σ σ. Please note that, even though an attacker can probably estimate the clean embedding for the text they input to the encoder multiple times and these clean embeddings might be helpful in reconstructing the original text that the adversaries input. But the text we want to protect is NOT the text that the adversaries input but the hidden text of the target systems. For these protected texts, the adversaries only observe the released noisy embeddings and they cannot estimate the clean embeddings for them. Given the observed noisy embedding y and black-box embedder f, the task is to reconstruct the original text x. A natural initial attempt is to directly train an inversion model on the noisy pairs (y,x)(y,x). However, as observed in prior experiments by Zhuang et al. [40], this strategy remains largely insufficient. Below, we analyze theoretically why this naive approach fails. I-B Vec2Text Fails due to “Double Noise Trap” Generative inversion methods such as Vec2Text [20] are designed to map a target embedding z back to text x by modeling pθ(x∣z)p_θ(x z) via an inverter, and refine a hypothesis x x via a corrector by comparing the target embedding with the embedding of the hypothesis f(x^)f( x). Under a noisy embedding interface, both quantities are observed with perturbations: ytar=z+ϵ1,yhyp=f(x^)=fclean(x^)+ϵ2,y_tar=z+ _1, y_hyp=f( x)=f_clean( x)+ _2, (4) where ϵ1,ϵ2∼(0,σ2Id) _1, _2 (0,σ^2I_d) are independent. The observed correction residual is therefore Δobs _obs =ytar−yhyp =y_tar-y_hyp (5) =z−fclean(x^)⏟semantic residual+(ϵ1−ϵ2)⏟noise residual. = z-f_clean( x)_semantic residual+ ( _1- _2)_noise residual. Thus, the correction signal contains not only the desired semantic residual, but also a residual noise term formed by subtracting two independent perturbations. Since ϵ1−ϵ2∼(0,2σ2Id) _1- _2 (0,2σ^2I_d), the residual noise has doubled variance, and its typical magnitude is ‖ϵ1−ϵ2‖2≈2σd.\| _1- _2\|_2≈ 2σ d. (6) For a 768-dimensional embedding with σ=0.01σ=0.01, this residual noise magnitude is approximately 0.390.39, which is larger than the single-noise displacement σd≈0.28σ d≈ 0.28. As shown in Fig. 1, clean embeddings reside on a text-induced manifold, and additive noise pushes y off this manifold. Consequently, the corrector attempting to match the hypothetical embedding f(x^)f( x) against the noisy target y is fundamentally misguided. This geometric mismatch exposes the standard inversion process to what we term the “Double Noise Trap”. The optimization objective expands as follows: ‖ytar−yhyp‖22 \|y_tar-y_hyp\|_2^2 =‖z−fclean(x^)‖22 =\|z-f_clean( x)\|_2^2 (7) +2⟨z−fclean(x^),ϵ1−ϵ2⟩+‖ϵ1−ϵ2‖22. +2 z-f_clean( x), _1- _2 +\| _1- _2\|_2^2. The first term is the desired semantic matching objective, while the remaining terms are caused by noise. As refinement improves and the semantic residual ‖z−fclean(x^)‖22\|z-f_clean( x)\|_2^2 becomes smaller, the stochastic cross term and the doubled noise baseline can dominate the correction signal. The corrector may therefore reward hypotheses whose embeddings align with residual noise directions rather than true semantic directions. This motivates our decoupled design, which first estimates a denoised clean-manifold representation and then performs generation and correction through denoised embeddings. I-C Infeasible Supervised Denoising The strict no-clean-embedding constraint fundamentally changes the nature of the denoising process. The standard supervised denoising objective, minϕ[‖hϕ(y)−z‖22], _φ\;E [\|h_φ(y)-z\|_2^2 ], (8) where hϕh_φ denotes a denoising neural network parameterized by ϕφ, is inherently unavailable because the clean target z is unavailable. Consequently, these analyzes underscore the critical need for a novel inversion pipeline: one capable of reconstructing the original text relying strictly on the noisy observations. I method Fig. 2: The overall architecture and data flow of our proposed DAEI pipeline. The system consists of four functional components (grey vertical panels) optimized across four stages. (1) Upon obtaining a noisy embedding from the hidden text via the Noisy Embedder, we first apply the DAE (indicated by the shield icons) to reconstruct a denoised embedding z~ z. (2) It is then processed by the Inverter to generate an initial hypothetical text x^(k) x^(k). (3) During training, the token-level cross-entropy loss (ℒCEL_CE) backpropagates directly through the active DAE to achieve joint fine-tuning (grey dashed lines). (4) Finally, the generated text enters an iterative correction loop: it is re-encoded by the embedder and passed through the DAE to yield a denoised hypothesis embedding e~(k) e^(k). Both e~(k) e^(k) and the original denoised target z~ z are then fed into the Corrector to predict the next hypothetical text, repeating until the loop terminates to produce the final output. We propose DAEI, a novel pipeline for robust noisy text embedding inversion, with its overall architecture and data flow illustrated in Fig. 2. Our system is built upon four components: a black-box noisy embedder f(⋅)f(·), a denoising autoencoder (DAE) hϕh_φ, a generative inverter θ, and an iterative corrector. Rather than a naive end-to-end approach, the DAEI pipeline comprises four distinct stages: (1) SURE-based unsupervised DAE pre-training, (2) DAE-shielded inverter training, (3) joint SURE-CE fine-tuning, and (4) DAE-shielded corrector training. Notably, the architectures for Stages 2 and 4 remain virtually identical to vanilla Vec2Text, with the distinction that the input embeddings are denoised by our DAE in advance. I-A SURE-based Unsupervised DAE Stage 1 aims to learn a denoising module hϕh_φ that constructs a denoised embedding z~ z from the noisy observation y: z~=hϕ(y,σ). z=h_φ(y,σ). (9) The oracle denoising risk Eq. (8) cannot be directly optimized because the clean embedding z is unobserved. To check if the dependency on z can be removed, we first expand Eq. (8): ‖hϕ(y,σ)−z‖22=‖hϕ(y,σ)‖22−2z⊤hϕ(y,σ)+‖z‖22.\|h_φ(y,σ)-z\|_2^2=\|h_φ(y,σ)\|_2^2-2z h_φ(y,σ)+\|z\|_2^2. (10) The last term ‖z‖22\|z\|_2^2 is independent of ϕφ, while the cross term z⊤hϕ(y,σ)z h_φ(y,σ) is inaccessible because z is hidden. However, under the Gaussian noise model y=z+ϵy=z+ε, where ϵ∼(0,σ2Id)ε (0,σ^2I_d), Stein’s lemma gives [32] ϵ[ϵ⊤hϕ(y,σ)]=σ2ϵ[divyhϕ(y,σ)].E_ε [ε h_φ(y,σ) ]=σ^2E_ε [div_yh_φ(y,σ) ]. (11) Since z=y−ϵz=y-ε, the inaccessible cross term can be replaced in expectation by quantities depending only on y, the noise variance σ2σ^2, and the divergence of the denoiser. This leads to SURE [32]: ℒSURE(y,ϕ)=‖hϕ(y,σ)−y‖22+2σ2divyhϕ(y,σ)−dσ2.L_SURE(y;φ)=\|h_φ(y,σ)-y\|_2^2+2σ^2div_yh_φ(y,σ)-dσ^2. (12) Here, divyhϕ(y,σ)=Tr(∂hϕ/∂y)div_yh_φ(y,σ)=Tr(∂ h_φ/∂ y) measures the local sensitivity of the denoiser with respect to its noisy input. Therefore, although z is never observed, minimizing Eq. (12) is equivalent in expectation to minimizing the oracle denoising risk in Eq. (8). We further parameterize the denoising module as a residual correction around the observed noisy embedding, rather than directly predicting z~ z from scratch: hϕ(y,σ)=y+rϕ(y,σ),h_φ(y,σ)=y+r_φ(y,σ), (13) where rϕ(y,σ)r_φ(y,σ) estimates the displacement needed to suppress the noise component and move the observation back toward the clean embedding manifold. Substituting this parameterization into the divergence term gives divyhϕ(y,σ)=d+divyrϕ(y,σ)div_yh_φ(y,σ)=d+div_yr_φ(y,σ). After removing constants independent of ϕφ, the training objective becomes ℒSURE(y,ϕ)≡‖rϕ(y,σ)‖22+2σ2divyrϕ(y,σ).L_SURE(y;φ)≡\|r_φ(y,σ)\|_2^2+2σ^2div_yr_φ(y,σ). (14) We approximate the divergence term with Monte-Carlo Hutchinson estimation [27]: divyrϕ(y,σ)≈1K∑k=1Kvk⊤Jrϕ(y,σ)vk,vk∈−1,+1d,div_yr_φ(y,σ)≈ 1K _k=1^Kv_k J_r_φ(y,σ)v_k, v_k∈\-1,+1\^d, (15) where JrϕJ_r_φ is the Jacobian of the residual network with respect to the noisy embedding. Let div^K div_K denote the Monte-Carlo estimator in Eq. (15). For Rademacher probes satisfying [vkvk⊤]=IE[v_kv_k ]=I, this estimator is unbiased and its variance is bounded by v[div^K] _v [ div_K ] =divyrϕ(y,σ), =div_yr_φ(y,σ), (16) Var[div^K] [ div_K ] ≤2K‖Jrϕ(y,σ)‖F2. ≤ 2K\|J_r_φ(y,σ)\|_F^2. Thus, the approximation is theoretically well grounded: it is unbiased, and its variance decays at rate O(1/K)O(1/K). In practice, we use Rademacher probes and spectral normalization in the residual MLP to stabilize the trace estimate. I-B DAE-Shielded Inverter After we obtain denoised representation z~ z, the inverter θ is subsequently trained on the pairs (z~,x)( z,x) rather than the raw noisy embeddings. It prevents the generative model from overburdening its capacity with high-dimensional noise-filtering and avoids overfitting to non-semantic perturbations. The training objective is the standard token-level cross-entropy loss: ℒCE(θ;ϕ)=−∑t=1Tlogpθ(xt∣x<t,z~)L_CE(θ;φ)=- _t=1^T p_θ (x_t x_<t, z ) (17) where T denotes the sequence length, xtx_t is the target token at step t, and x<tx_<t represents the preceding autoregressive context. I-C Joint SURE-CE Fine-tuning Following the independent training of the DAE and the inverter, stage 3 unifies the architecture. To explicitly align the unsupervised denoising process with the downstream generative task, we jointly fine-tune the DAE and the inverter: ℒSURE-CE(ϕ,θ)=ℒSURE(y,ϕ)+λCE(t)ℒCE(θ,ϕ).L_SURE -CE(φ,θ)=L_SURE(y;φ)+ _CE(t)\,L_CE (θ;φ ). (18) Here, the weighting coefficient λCE(t) _CE(t) is warmed up during the early training steps. The key difference from Stage 2 is that the cross-entropy gradient is allowed to pass through hϕ(y,σ)h_φ(y,σ). Consequently, the DAE receives two complementary supervisory signals: the SURE objective ensures that the output remains strictly aligned with the denoising manifold, while the CE (cross-entropy) loss provides semantic feedback, preserves the latent directions for decoding. Concurrently, the inverter is continuously trained on the embeddings denoised dynamically by the fine-tuning DAE to adapt to the optimized embedding distribution. However, the optimization directions of these two objectives may conflict. We therefore employ distinct learning rates for the DAE and the inverter, and apply gradient surgery PCGrad [37] exclusively to the DAE parameters ϕφ. Let gSURE≜∇ϕℒSUREg_SURE _φL_SURE and gCE≜∇ϕℒCEg_CE _φL_CE denote the gradients with respect to the DAE parameters. If these two gradients exhibit a negative inner product (i.e., ⟨gCE,gSURE⟩<0 g_CE,g_SURE <0), they are conflicting. In such cases, we project the CE gradient onto the normal plane of the SURE gradient before merging the updates: gCE′ g_CE =gCE−⟨gCE,gSURE⟩‖gSURE‖22gSURE, =g_CE- g_CE,g_SURE \|g_SURE\|_2^2g_SURE, (19) ϕ φ ←ϕ−ηDAE(gSURE+gCE′). ←φ- _DAE(g_SURE+g_CE ). This orthogonal projection guarantees that the text feedback (gCE′g_CE ) shapes the final representation without unlearning the fundamental denoising capabilities. I-D DAE-Shielded Corrector Conventionally, once an inverter is trained to generate an initial text hypothesis, a corrector iteratively refines this hypothesis by comparing its embedding against the target embedding. However, when the available target is noisy, directly performing this comparison inevitably triggers the “Double Noise Trap” described in Section I-B. To mitigate this, we introduce a DAE Shield to both sides of the correction interface. Specifically, instead of comparing raw representations, we pass both the noisy target y and the embedding of the current hypothesis x^(k) x^(k) through our Stage 3 denoiser hϕ⋆h_φ : z~=hϕ⋆(y,σ),e~(k)=hϕ⋆(f(x^(k)),σ), z=h_φ (y,σ), e^(k)=h_φ (f( x^(k)),σ), (20) where x^(k) x^(k) is the generated text at correction step k, and ϕ⋆φ denotes the fully fine-tuned DAE parameters. The corrector then conditions exclusively on the denoised target z~ z, the denoised hypothesis e~(k) e^(k), and their residual difference. By completely shielding the correction interface, we constrain the iterative refinement strictly within the learned clean manifold, ensuring the model aligns with semantic text directions rather than chasing irreducible noise residuals. IV Experimental Setup Our study aims to determine the extent to which noise-protected embeddings from black-box encoders remain vulnerable to advanced inversion attack paradigms. Specifically, we investigate the following research questions: RQ1: How does the proposed DAEI pipeline compare against vanilla Vec2Text and existing noise-robust baselines in recovering target text from noisy embeddings? RQ2: Can the proposed DAEI pipeline be utilized in different embedding model backbones? RQ3: Can DAEI maintain strong inversion performance on out-of-domain test sets? RQ4: Can the DAE component effectively reduce noise? RQ5: Does the joint fine-tuning module improve reconstruction quality? RQ6: To what extent can DAEI maintain its inversion performance when the noise σ varies? IV-A Dataset IV-A1 Training set All main experiments use the same training mixture of: • Natural Questions (NQ) is a question answering benchmark built from real Google search queries [13]. • MS MARCO is a large-scale machine reading retrieval dataset constructed from real Bing queries, web passages, and human-generated answers [1]. • Yahoo Answers is a community question-answering domain covering broad everyday topics. We capped the combined mixture at 7M text examples as raw text sources to construct the inversion datasets. While the underlying text corpora remain constant, the exact input-target configurations vary across our four training stages to accommodate the specific objectives of each module: • Stage 1 (DAE Pre-training): To train the residual DAE, the system requires noisy embeddings only. • Stage 2 (Base Inverter Training): To train the initial text decoding module, we utilize (denoised embedding, text) pairs. These denoised embeddings are computed by the Stage 1 DAE. • Stage 3 (Joint Fine-tuning): For end-to-end joint optimization, the dataset consists of (noisy embedding, text) pairs. Here, the noisy embeddings are dynamically denoised on-the-fly by the actively tuning DAE during the forward pass. • Stage 4 (Corrector Training): To train the iterative corrector, we rely on expanded input tuples consisting of (denoised embedding, hypothetical text, denoised hypothetical embedding), paired with the original text as label. Crucially, these denoised embeddings are computed by the fully tuned Stage 3 DAE, ensuring the corrector operates strictly within the optimal denoised distribution. IV-A2 Evaluation sets For evaluation, we consider both in-domain and out-of-domain settings. The in-domain test set is the held-out splits from the same NQ, MS MARCO, and Yahoo Answers mixture, while the out-of-domain test sets are drawn from five unseen corpora: AG News, Anthropic Toxic Prompts, Python Code Alpaca, Climate-FEVER [5], and MedMCQA [23]. These out-of-domain datasets allow us to evaluate whether the inversion methods generalize beyond the training distribution. The data sources for all training and evaluation sets are provided in our repository: https://anonymous.4open.science/r/DAEI-1F1C/ IV-B Target Embedding Models To rigorously evaluate the generalizability of our DAEI pipeline across different latent space distributions, we select two widely adopted, state-of-the-art dense text embedding models as our black-box targets: • GTR-base [22]: The GTR (Generalizable T5-based dense Retrievers) model leverages a bi-encoder architecture initialized from the T5 [25] checkpoints. It is specifically optimized for robust zero-shot retrieval tasks, producing highly structured semantic representations. • GTE-base [17]: The General Text Embeddings (GTE) model trained on large-scale contrastive data represents a highly optimized encoder-only architecture. We deliberately select GTR and GTE since they represent two fundamentally distinct backbone architectures (T5-based vs. BERT-based) and training paradigms, thereby demonstrate the generalizability of our method. IV-C Baselines and Our Models Given the scarcity of existing work on noisy text embedding inversion, we use ZSInvert [38] (an adversarial decoding approach) as our primary external baseline even though it is not based on the mainstream encoder-decoder architecture because it is the only existing work that we know has shown its effectiveness in the noisy text embedding inversion scenario. Our core focus remains on a rigorous comparison between various configurations of the encoder-decoder models and our proposed approach. To comprehensively assess inversion performance, we categorize the baselines and our models into three distinct groups: IV-C1 Clean Upper Bound Clean-to-Clean Baseline evaluates the vanilla Vec2Text [20] model on clean embeddings. It represents the state-of-the-art text recovery performance under ideal, noiseless conditions. This serves as the theoretical ceiling that our noisy inversion pipelines aim to approach. IV-C2 Generative Inversion under Noise This group compares the standard autoregressive inversion models against our proposed pipelines when facing perturbed embeddings: • Clean-to-Noisy Baseline: Tests the severe distribution shift faced by a vanilla Vec2Text model that is trained solely on clean embeddings but evaluated on noisy ones. • Noisy Inverter Baseline: Trains a vanilla Vec2Text directly on noisy embeddings. This provides a much stronger baseline that adapts to the corrupted embedding distribution but lacks denoising mechanism. • DAE-only inverter (Ours): Our proposed pipeline without joint fine-tuning. A DAE first denoises the embedding, then an inverter is trained on the denoised cache. • DAEI (Ours): Our complete pipeline that jointly fine-tunes both the DAE and the inverter. IV-C3 Adversarial-Decoding-Based Baseline To step outside the generative paradigm, we introduce ZSInvert as an external baseline. This method frames inversion as adversarial decoding, iteratively optimizing discrete tokens to minimize embedding distance. Table I summarizes the above systems evaluated in our main experiments. TABLE I: Main experimental systems. ZSInvert base is training-free; its listed training embedding type refers only to corrector training. System Train embedding Test embedding Clean-to-Clean Baseline Clean Clean ZSInvert Noisy Noisy Clean-to-Noisy Baseline Clean Noisy Noisy Inverter Baseline Noisy Noisy DAE-only (ours) Denoised by stage1 DAE Denoised by stage1 DAE DAEI (ours) Denoised by stage3 DAE Denoised by stage3 DAE IV-D Metrics. For evaluating the inverters, we report sentence-level BLEU [24], token-level F1 [26], ROUGE-L [18], and the cosine similarity between the embeddings of the reconstructed and reference texts. Specifically, BLEU assesses generation precision and local fluency by measuring the n-gram overlap between the reconstructed and reference texts; token-level F1 assesses keyword recovery by measuring unigram overlap independently of sequence order; ROUGE-L evaluates structural similarity and information recall based on the longest common subsequence; and embedding cosine similarity measures semantic closeness in the target embedding space. For evaluating the DAE, we also report the MSE and embedding cosine similarity between the DAE’s denoised outputs and the clean embeddings. We emphasize that these clean embeddings are strictly isolated for offline evaluation purposes only; they are never observed by the model during any stage of training, nor are they used to compute any optimization loss. IV-E Implementation Setting. Our method assumes the noise scale σ is prior knowledge (see Section I-A). Unless otherwise specified in the ablation studies, we apply zero-mean Gaussian noise with a standard deviation of σ=0.01σ=0.01 [20, 40]. The DAE module is pre-trained with a learning rate of 1×10−31× 10^-3. We configure a hidden dimension of 1024 and a depth of 3, spectral normalization, and five Monte-Carlo probes. All inverter variants use the Vec2Text architecture with a T5 [25] backbone and are optimized with a learning rate of 1×10−31× 10^-3. Following the original Vec2Text setting [20], both input and target texts are truncated to a maximum length of 128 tokens during training, while evaluation is conducted with a maximum length of 32 tokens. During the joint fine-tuning stage, the DAE learning rate is reduced to 2×10−42× 10^-4, and the cross-entropy loss weight is linearly warmed up over 10,000 steps. All corrector variants use the Vec2Text architecture with a learning rate of 5×10−45× 10^-4. We enforce an early-stop threshold of 1×10−31× 10^-3: refinement terminates once the cosine similarity gain between consecutive steps falls below this value, preventing the corrector from overfitting to residual noise. V Results TABLE I: Main results for in-domain evaluation with GTR-base. Gray rows indicate reference settings that are not available under the current threat model. Improvements in parentheses are absolute gains over the Noisy Inverter Baseline. Best feasible results excluding the clean upper bound are in bold. Category Model BLEU F1 ROUGE-L Cosine Similarity Clean Upper Bound Clean-to-Clean Baseline 0.6252 0.8656 0.8704 0.9500 Adversarial Decoding ZSInvert 0.013 0.1503 0.1117 0.8568 Generative Inversion Clean-to-Noisy Baseline 0.1348 0.5005 0.5012 0.7785 Noisy Inverter Baseline 0.1916 0.5814 0.5660 0.8349 DAE-only (Ours) 0.3616 0.7206 0.7200 0.9055 DAEI (Ours) 0.4875 (+0.2959) 0.7717 (+0.1903) 0.7788 (+0.2128) 0.9254 (+0.0905) TABLE I: Main results for in-domain evaluation with GTE-base. Formatting follows Table I. Category Model BLEU F1 ROUGE-L Cosine Similarity Clean Upper Bound Clean-to-Clean Baseline 0.4391 0.7622 0.7570 0.9211 Adversarial Decoding ZSInvert 0.0104 0.1189 0.0894 0.9641 Generative Inversion Clean-to-Noisy Baseline 0.0846 0.3222 0.3271 0.7081 Noisy Inverter Baseline 0.1249 0.3960 0.4044 0.7520 DAE-only (Ours) 0.2688 0.5854 0.5722 0.8141 DAEI (Ours) 0.3152 (+0.1903) 0.6344 (+0.2384) 0.6324 (+0.2280) 0.8556 (+0.1036) V-A Overall Performance (RQ1 & RQ2) DAEI achieves the best noisy inversion performance. Tables I and I report the main inversion results on GTR-base and GTE-base, respectively. On GTR-base, DAEI substantially improves over the Noisy Inverter baselines across all metrics. In particular, DAEI increases BLEU from 0.1916 to 0.4875, yielding an absolute gain of nearly 30 percentage points and a 154.4% relative improvement over the Noisy Inverter baseline. It also improves token-level F1 from 0.5814 to 0.7717 and ROUGE-L from 0.5660 to 0.7788, indicating that DAEI can recover richer textual information from noisy embeddings alone. Moreover, the embedding cosine similarity increases from 0.8349 to 0.9254, suggesting that the DAE-shielded correction process more effectively aligns the hypothesis embedding with the target embedding after removing a substantial portion of the perturbation noise. Noisy training alone provides limited gains. The comparison between the Clean-to-Noisy baseline and the Noisy Inverter baseline further illustrates the limitation of directly training Vec2Text on noisy embedding. On GTR-base, noisy training improves text reconstruction metrics by only 5–8 points over the Clean-to-Noisy baseline, indicating limited adaptation to noisy inputs. However, due to the Double Noise Trap, the correction stage compares a noisy hypothesis embedding with a separately perturbed target, causing the residual signal to be contaminated by doubled noise. As a result, this gain remains far lower than that achieved by DAEI. DAEI approaches the clean upper bound and outperforms ZSInvert in text reconstruction. Relative to the clean upper bound, DAEI recovers 78.0%/89.2% of BLEU/F1 on GTR-base and 71.7%/83.2% on GTE-base. These results suggest that noise-protected embeddings still retain considerable recoverable textual information when the adversary adopts a denoising-aware inversion pipeline. However, ZSInvert performs poorly on text-overlap metrics for both embedders, achieving only around 0.01 BLEU and 0.08–0.15 on F1 and ROUGE-L, despite obtaining a relatively high embedding cosine similarity of 0.85–0.96. This gap indicates that adversarial decoding matches the embedding space but fails to reliably recover token-aligned text, whereas DAEI better balances semantic matching and fluent reconstruction. DAEI generalizes across embedding backbones. DAEI shows similar gains on GTE-base, answering RQ2. Although the overall scores on GTE-base are lower than GTR-base, the clean upper bound is also lower, suggesting that the underlying embedding space is intrinsically more difficult for Vec2Text framework to invert. One possible reason is that our inverter uses a T5-base backbone, which is more closely aligned with the T5-based GTR encoder than with the BERT-style GTE encoder [17, 22]; meanwhile, GTE is trained with multi-stage contrastive learning for general-purpose semantic retrieval [17], which may emphasize semantic invariance while preserving less information needed for token-level reconstruction. Even under this harder setting, DAEI improves over the Noisy Inverter baseline by approximately 19–23 points on text reconstruction metrics, while also increasing cosine similarity by 10.36 points. These consistent improvements over two architecturally different encoders demonstrate that DAEI is generalized across different noisy embedding interfaces. TABLE IV: Out-of-domain evaluation results for DAEI. Upper Bound denotes Clean-to-Clean Baseline, and Baseline denotes the Noisy Inverter Baseline. OOD Dataset Method BLEU F1 ROUGE-L AG News Upper Bound 0.1023 0.4296 0.4887 ZSInvert 0.0130 0.1629 0.1142 Baseline 0.0531 0.2749 0.3109 DAEI 0.0791 0.3346 0.3915 Anthropic Toxic Prompts Upper Bound 0.8114 0.8912 0.9118 ZSInvert 0.0137 0.1451 0.1149 Baseline 0.2128 0.5581 0.5590 DAEI 0.7217 0.8881 0.8885 Python Code Upper Bound 0.6732 0.8588 0.8783 ZSInvert 0.0174 0.1928 0.1434 Baseline 0.1933 0.6085 0.5961 DAEI 0.5080 0.8213 0.8316 Climate fever Upper Bound 0.5350 0.7843 0.7824 ZSInvert 0.0132 0.1518 0.1105 Baseline 0.1377 0.4853 0.4980 DAEI 0.3489 0.6651 0.6961 Medmcqa Upper Bound 0.6689 0.8476 0.8800 ZSInvert 0.0128 0.1286 0.1070 Baseline 0.1739 0.5252 0.5576 DAEI 0.4871 0.7730 0.8037 V-B Out-of-Domain Generalization (RQ3) Table IV evaluates whether DAEI can generalize to Out-Of-Domain (OOD) test sets. Across all OOD datasets, DAEI consistently outperforms the Noisy Inverter and ZSInvert on BLEU, F1, and ROUGE-L, showing that DAEI remains effective when the target texts come from unseen domains. The performance varies across datasets. DAEI performs especially well on Anthropic Toxic Prompts, Python Code, and MedMCQA, while AG News obtains lower scores for both the clean upper bound and DAEI. One likely reason is that AG News examples are longer on average, with about 61 tokens compared with 22–30 tokens for the other datasets. Although evaluation outputs are truncated to 32 tokens, longer source texts still contain more information to recover and are more likely to suffer from truncation and partial reconstruction. TABLE V: DAE component performance across training stages. MSE and cosine similarity are computed against clean embeddings before and after DAE denoising, with Δ reporting the denoising improvement. Embedder Stage MSE ↓ Cosine ↑ Noisy DAE Δ Noisy DAE Δ GTE-base Stage 1 0.0765 0.0583 0.0182 0.9662 0.9727 0.0065 Stage 3 0.0765 0.0602 0.0163 0.9662 0.9713 0.0051 GTR-base Stage 1 0.0766 0.0542 0.0224 0.9179 0.9322 0.0142 Stage 3 0.0766 0.0575 0.0191 0.9179 0.9280 0.0101 (a) Mean LID with SEM. (b) Median LID. Fig. 3: LID-based geometric evaluation of DAE projection using clean embeddings as the reference set. The two subfigures report mean LID with SEM (Standard Error of the Mean) error bars and median LID, respectively, showing DAE reduces the LID of noisy embeddings back to the clean level. V-C Components Effectiveness (RQ4 & RQ5) V-C1 DAE Effectiveness DAE reduces noise-induced embedding distortion. Table V demonstrates DAE can effectively reduce the perturbation noise without using clean embeddings during training. The Stage 1 DAE reduces MSE to the clean embeddings by 29.2% on GTR-base and 23.8% on GTE-base, while improving cosine similarity from 0.9179 to 0.9322 and from 0.9662 to 0.9727, respectively. Since the injected noise is relatively small, the raw noisy embeddings are already close to the clean embeddings in cosine similarity, making the cosine gains modest. Nevertheless, the consistent MSE reduction and cosine improvement show that the DAE shifts noisy embeddings closer to their clean counterparts, even without using clean embeddings as training targets. Different roles of Stage 1 and Stage 3 DAEs. Interestingly, Table V shows that the Stage 3 DAE has slightly weaker overall denoising metrics than the Stage 1 DAE in terms of MSE and cosine similarity to the clean embedding. This is expected because the two stages optimize for different objectives: Stage 1 focuses purely on denoising, while Stage 3 is additionally guided by the cross-entropy objective toward semantic directions for text reconstruction, resulting in a representation that is better aligned with the generative decoder. DAE projects noisy embeddings toward the clean manifold. We further evaluate the DAE from a geometric perspective using Local Intrinsic Dimensionality (LID) [10]. For each test embedding, we estimate its LID with respect to a held-out clean embedding reference set using the MLE (Maximum Likelihood Estimation) estimator with 100 nearest neighbors. As shown in Fig. 3, Gaussian noise increases the mean LID from 47.65 to 51.15, indicating that noisy embeddings deviate from the clean embedding manifold. After DAE projection, the LID decreases back to the clean level: Stage 1 obtains a slightly lower LID (47.19) than clean embedding, it may because a purely denoising DAE pulls embeddings toward more averaged regions of the clean distribution, result in lower estimated local dimensionality. In contrast, Stage 3 reaches 47.65, almost exactly matching the clean embeddings, because joint fine-tuning introduces semantic alignment with the clean manifold. The median results show the same trend, confirming that the reduction is not driven by outliers. DAE improves downstream inversion. The effectiveness of the DAE is also reflected in the downstream inversion results through the comparison between the Noisy Inverter baseline and DAE-only. Across the two embedders, DAE-only improves the text reconstruction metrics by approximately 14–19 percentage points, while increasing embedding cosine similarity by about 6–7 percentage points. This demonstrates the fundamental effectiveness of the denoising-aware strategy: removing part of the perturbation noise before inversion makes the input representation more semantically stable and easier for the generative decoder to interpret. V-C2 Role of Joint Fine-tuning While DAE-only already provides strong gains, the full DAEI model further improves performance by jointly fine-tuning the DAE and the inverter. Across the two embedders, DAEI brings additional improvements of approximately 5–13 percentage points on text reconstruction metrics over DAE-only, while also further increasing embedding cosine similarity by 2–4 points. These consistent gains highlight the effectiveness of joint fine-tuning, showing that a small sacrifice in denoising accuracy can be worthwhile when it better aligns the representation with the downstream text decoding objective. TABLE VI: In-domain evaluation of DAEI on GTR-base under different Gaussian noise scales. σ BLEU F1 ROUGE-L Cosine 0.005 0.4927 0.7823 0.7850 0.9293 0.008 0.4932 0.7807 0.7836 0.9288 0.010 0.4875 0.7717 0.7788 0.9254 0.012 0.4813 0.7729 0.7793 0.9242 0.015 0.4628 0.7578 0.7634 0.9218 0.018 0.4377 0.7421 0.7480 0.9240 0.020 0.3966 0.7200 0.7269 0.9150 V-D Generalization Across Varying Noise Levels (RQ6) Table VI evaluates the in-domain robustness of DAEI on GTR-base under different Gaussian noise scales. The model is trained with σ=0.01σ=0.01 and tested on other noise levels. DAEI remains highly stable when σ varies from 0.005 to 0.018: BLEU stays above 0.4377, F1 remains above 0.7421, and cosine similarity remains around 0.924–0.929. Even at σ=0.02σ=0.02, DAEI still achieves 0.3966 BLEU and 0.7200 F1, indicating that DAEI can tolerate moderate mismatch between the training noise scale and the test-time perturbation. This result is especially relevant when the defender does not use a fixed noise scale, but instead samples σ from a small range, such as [0.005,0.015][0.005,0.015], for each released embedding: although the attacker may only estimate an average σ, DAEI can still work effectively. Larger noise scales may further reduce inversion performance, but prior work has shown that overly large perturbations also damage the downstream utility of the protected embedder [20]. Thus, within the practical noise regime, DAEI is robust to realistic noise-scale fluctuations. VI Related Work VI-A Text Embedding Inversion Attacks Dense text embeddings can leak substantial information about their original inputs. Early work demonstrated this risk by showing that sentence embeddings can reveal sensitive attributes and keywords [31]. Later, Li et al. proposed a generative embedding inversion attack that reconstructs complete sentences [16]. Morris et al. introduced Vec2Text, which achieves high-fidelity reconstruction of short texts [20], while later methods reduce attacker requirements via few-shot, transfer-based, or zero-shot inversion [4, 11, 38]. These works demonstrate the practical privacy risk of text embeddings, but they mainly focus on general embeddings settings rather than embeddings explicitly protected by noise. VI-B Privacy Protection for Text Embeddings Noise injection is a standard mechanism for privacy protection, with differential privacy providing a formal foundation for calibrated random perturbations [7]. In NLP, Feyisetan et al. applied calibrated multivariate noise in word embedding space to achieve privacy-utility tradeoffs for textual analysis [8], and Xu et al. further refined this idea with a Mahalanobis metric [36]. More recent work extends perturbation-based privacy to sentence-level representations [6, 2]. Recent evaluations also show that Gaussian noise can degrade the performance of standard embedding inversion attacks [29, 20, 38]. Such a lightweight but effective defense poses a substantial challenge to embedding inversion attacks. VI-C Denoising Techniques Denoising autoencoders learn robust representations by reconstructing clean inputs from corrupted observations[33]. Standard denoising training typically relies on clean reconstruction targets. However, Noise2Noise relaxes this requirement by learning from pairs of independently corrupted observations of the same underlying signal [14]. Stein’s unbiased risk estimate provides a way to estimate Gaussian denoising risk from noisy observations alone [32], and Monte Carlo SURE extends this idea to general denoisers through randomized divergence estimation [27, 30]. However, these techniques have not been studied in the context of text embedding inversion. As a result, the community still lacks a systematic understanding of whether high-quality text can be reconstructed from noise-protected embeddings. This gap motivates our study of denoising-aware embedding inversion in the noisy-only setting. VII Conclusion In this paper, we studied the privacy risk of noise-protected text embeddings under a threat model where attackers observe only perturbed embeddings. We propose DAEI, a denoising-aware embedding inversion pipeline that combines unsupervised denoising autoencoder with generative inversion. Extensive experiments across embedding backbones, domains, and ablation settings show that DAEI outperforms existing baselines, demonstrating that Gaussian noise alone is insufficient to prevent textual information leakage. These findings highlight the need to evaluate privacy-preserving embedding systems against adaptive denoising-aware attacks and to develop stronger defenses for embedding-based applications. VIII Acknowledgment This work was supported by Monash eResearch capabilities, including M3 HPC, the Australian Research Council through the Discovery Early Career Researcher Award (DE250100032), the Discovery Project (Grant No. DP260100326) and the Linkage Projects (Grant Nos. LP230200892, LP240200546 and LP250200778). References [1] P. Bajaj, D. Campos, N. Craswell, L. Deng, J. Gao, X. Liu, R. Majumder, A. McNamara, B. Mitra, T. Nguyen, et al. (2016) MS marco: a human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268. Cited by: 2nd item. [2] D. Bollegala, S. Otake, T. Machide, and K. Kawarabayashi (2025) A metric differential privacy mechanism for sentence embeddings. ACM Transactions on Privacy and Security 28 (2), p. 1–34. Cited by: §VI-B. [3] G. Casella and R. Berger (2024) Statistical inference. Chapman and Hall/CRC. Cited by: §I-A. [4] Y. Chen, Q. Xu, and J. Bjerva (2025) Algen: few-shot inversion attacks on textual embeddings via cross-model alignment and generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 24330–24348. Cited by: §VI-A. [5] T. Diggelmann, J. Boyd-Graber, J. Bulian, M. Ciaramita, and M. Leippold (2020) Climate-fever: a dataset for verification of real-world climate claims. arXiv preprint arXiv:2012.00614. Cited by: §IV-A2. [6] M. Du, X. Yue, S. S. Chow, and H. Sun (2023) Sanitizing sentence embeddings (and labels) for local differential privacy. In Proceedings of the ACM Web Conference 2023, p. 2349–2359. Cited by: §I, §VI-B. [7] C. Dwork, F. McSherry, K. Nissim, and A. Smith (2006) Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, p. 265–284. Cited by: §I, §VI-B. [8] O. Feyisetan, B. Balle, T. Drake, and T. Diethe (2020) Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations. In Proceedings of the 13th international conference on web search and data mining, p. 178–186. Cited by: §I, §VI-B. [9] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, H. Wang, H. Wang, et al. (2023) Retrieval-augmented generation for large language models: a survey. arXiv preprint arXiv:2312.10997 2 (1), p. 32. Cited by: §I. [10] M. E. Houle (2017) Local intrinsic dimensionality i: an extreme-value-theoretic foundation for similarity applications. In International Conference on Similarity Search and Applications, p. 64–79. Cited by: §V-C1. [11] Y. Huang, Y. Tsai, H. Hsiao, H. Lin, and S. Lin (2024) Transferable embedding inversion attack: uncovering privacy risks in text embeddings without model queries. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 4193–4205. Cited by: §VI-A. [12] V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih (2020) Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), p. 6769–6781. Cited by: §I. [13] T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, et al. (2019) Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7, p. 453–466. Cited by: 1st item. [14] J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, and T. Aila (2018) Noise2Noise: learning image restoration without clean data. In International Conference on Machine Learning, p. 2965–2974. Cited by: §I, §VI-C. [15] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, p. 9459–9474. Cited by: §I. [16] H. Li, M. Xu, and Y. Song (2023) Sentence embedding leaks more information than you expect: generative embedding inversion attack to recover the whole sentence. In Findings of the Association for Computational Linguistics: ACL 2023, p. 14022–14040. Cited by: §I, §I, §VI-A. [17] Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang (2023) Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281. Cited by: 2nd item, §V-A. [18] C. Lin (2004) Rouge: a package for automatic evaluation of summaries. In Text summarization branches out, p. 74–81. Cited by: §IV-D. [19] X. Luo, T. Yu, and X. Xiao (2025) Prompt inference attack on distributed large language model inference frameworks. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, p. 1739–1753. Cited by: §I. [20] J. Morris, V. Kuleshov, V. Shmatikov, and A. M. Rush (2023) Text embeddings reveal (almost) as much as text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 12448–12460. Cited by: §I, §I, §I-A, §I-B, §IV-C1, §IV-E, §IV-E, §V-D, §VI-A, §VI-B. [21] N. Muennighoff, N. Tazi, L. Magne, and N. Reimers (2023) Mteb: massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, p. 2014–2037. Cited by: §I. [22] J. Ni, C. Qu, J. Lu, Z. Dai, G. H. Abrego, J. Ma, V. Zhao, Y. Luan, K. Hall, M. Chang, et al. (2022) Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 9844–9855. Cited by: 1st item, §V-A. [23] A. Pal, L. K. Umapathi, and M. Sankarasubbu (2022) Medmcqa: a large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, p. 248–260. Cited by: §IV-A2. [24] K. Papineni, S. Roukos, T. Ward, and W. Zhu (2002) Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, p. 311–318. Cited by: §IV-D. [25] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21 (140), p. 1–67. Cited by: 1st item, §IV-E. [26] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang (2016) Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 conference on empirical methods in natural language processing, p. 2383–2392. Cited by: §IV-D. [27] S. Ramani, T. Blu, and M. Unser (2008) Monte-carlo sure: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on image processing 17 (9), p. 1540–1554. Cited by: §I, §I-A, §VI-C. [28] N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 3982–3992. Cited by: §I. [29] D. Seputis, Y. Li, K. Langerak, and S. Mihailov (2025) Rethinking the privacy of text embeddings: a reproducibility study of “text embeddings reveal (almost) as much as text”. In Proceedings of the Nineteenth ACM Conference on Recommender Systems, p. 822–831. Cited by: §I, §VI-B. [30] S. Soltanayev and S. Y. Chun (2018) Training deep learning based denoisers without ground truth data. Advances in neural information processing systems 31. Cited by: §VI-C. [31] C. Song and A. Raghunathan (2020) Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, p. 377–390. Cited by: §I, §VI-A. [32] C. M. Stein (1981) Estimation of the mean of a multivariate normal distribution. The annals of Statistics, p. 1135–1151. Cited by: §I, §I-A, §I-A, §VI-C. [33] P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol (2008) Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, p. 1096–1103. Cited by: §I, §VI-C. [34] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P. Manzagol, and L. Bottou (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion.. Journal of machine learning research 11 (12). Cited by: §I. [35] Y. Wang, M. Tang, N. Shen, S. Cui, and W. Wang (2025) Privacy risks of llm-empowered recommender systems: an inversion attack perspective. In Proceedings of the Nineteenth ACM Conference on Recommender Systems, p. 812–821. Cited by: §I. [36] Z. Xu, A. Aggarwal, O. Feyisetan, and N. Teissier (2020) A differentially private text perturbation method using regularized mahalanobis metric. In Proceedings of the Second Workshop on Privacy in NLP, p. 7–17. Cited by: §I, §VI-B. [37] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn (2020) Gradient surgery for multi-task learning. Advances in neural information processing systems 33, p. 5824–5836. Cited by: §I-C. [38] C. Zhang, J. X. Morris, and V. Shmatikov (2025) Universal zero-shot embedding inversion. arXiv preprint arXiv:2504.00147. Cited by: §I, §IV-C, §VI-A, §VI-B. [39] W. X. Zhao, J. Liu, R. Ren, and J. Wen (2024) Dense text retrieval based on pretrained language models: a survey. ACM Transactions on Information Systems 42 (4), p. 1–60. Cited by: §I. [40] S. Zhuang, B. Koopman, X. Chu, and G. Zuccon (2024) Understanding and mitigating the threat of vec2text to dense retrieval systems. In Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, p. 259–268. Cited by: §I, §I-A, §IV-E.