Paper deep dive
Membership Inference for Contrastive Pre-training Models with Text-only PII Queries
Ruoxi Cheng, Yizhong Ding, Hongyi Zhang, Yiyan Huang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/22/2026, 5:06:42 AM
Summary
The paper introduces the Unimodal Membership Inference Detector (UMID), a text-only auditing framework for contrastive pre-training models like CLIP and CLAP. UMID detects PII memorization by performing text-guided cross-modal latent inversion to extract similarity and variability signals, which are then analyzed via an ensemble of unsupervised anomaly detectors. This approach eliminates the need for computationally expensive shadow models and avoids the 'auditor's dilemma' by not requiring paired biometric inputs for queries.
Entities (5)
Relation Signals (3)
UMID → audits → CLIP
confidence 95% · Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness
UMID → audits → CLAP
confidence 95% · Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness
UMID → detects → PII
confidence 90% · UMID, a membership inference detector for contrastive pretraining models that operates solely on textual PII queries.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Contrastive pretraining models such as CLIP and CLAP underpin many vision-language and audio-language systems, yet their reliance on web-scale data raises growing concerns about memorizing Personally Identifiable Information (PII). Auditing such models via membership inference is challenging in practice: shadow-model MIAs are computationally prohibitive for large multimodal backbones, and existing multimodal attacks typically require querying the target with paired biometric inputs, thereby directly exposing sensitive biometric information to the target model. We propose Unimodal Membership Inference Detector (UMID), a text-only auditing framework that performs text-guided cross-modal latent inversion and extracts two complementary signals, similarity (alignment to the queried text) and variability (consistency across randomized inversions). UMID compares these statistics to a lightweight non-member reference constructed from synthetic gibberish and makes decisions via an ensemble of unsupervised anomaly detectors. Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness and efficiency over prior MIAs, delivering strong detection performance with sub-second auditing cost while complying with realistic privacy constraints.
Tags
Links
- Source: https://arxiv.org/abs/2603.14222v1
- Canonical: https://arxiv.org/abs/2603.14222v1
Trouble viewing inline? Open PDF directly →
Full Text
66,097 characters extracted from source content.
Expand or collapse full text
Membership Inference for Contrastive Pre-training Models with Text-only PII Queries Ruoxi Cheng∗ Yizhong Ding∗ Hongyi Zhang Yiyan Huang† Abstract Contrastive pretraining models such as CLIP and CLAP underpin many vision-language and audio-language systems, yet their reliance on web-scale data raises growing concerns about memorizing Personally Identifiable Information (PII). Auditing such models via membership inference is challenging in practice: shadow-model MIAs are computationally prohibitive for large multimodal backbones, and existing multimodal attacks typically require querying the target with paired biometric inputs, thereby directly exposing sensitive biometric information to the target model. We propose Unimodal Membership Inference Detector (UMID), a text-only auditing framework that performs text-guided cross-modal latent inversion and extracts two complementary signals, similarity (alignment to the queried text) and variability (consistency across randomized inversions). UMID compares these statistics to a lightweight non-member reference constructed from synthetic gibberish and makes decisions via an ensemble of unsupervised anomaly detectors. Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness and efficiency over prior MIAs, delivering strong detection performance with sub-second auditing cost while complying with realistic privacy constraints. keywords: Membership Inference, Contrastive Pre-training, Privacy Leakage, Anomaly Detection fn1fn1footnotetext: ∗Co-first authors. †Corresponding author. E-mail: huangyiyan@gbu.edu.cn [besti]organization=Beijing Electronic Science and Technology Institute, city=Beijing, postcode=100070, country=China [ali]organization=Alibaba Group, city=Beijing, postcode=100027, country=China [ntu]organization=Nanyang Technological University, city=Singapore, postcode=637371, country=Singapore [gbu]organization=School of Computing and Information Technology, Great Bay University, city=Dongguan, state=Guangdong, postcode=523000, country=China 1 Introduction Contrastive pretraining models [53], exemplified by Contrastive Language–Image Pretraining (CLIP) [35] and Contrastive Language–Audio Pretraining (CLAP) [10], have become foundational backbones for modern vision–language and audio–language systems. Their strong performance, however, is largely enabled by web-scale training data that can contain Personally Identifiable Information (PII) [38, 41], raising increasing concerns about PII leakage and downstream misuse [50, 14]. Recent studies further suggest that multimodal models may inadvertently memorize membership signals or identity attributes, exposing individuals to privacy risks when sensitive biometric data (e.g., faces or voices) appear in the training set [54]. Consequently, auditing these multimodal models for membership leakage has become a critical imperative, not only to meet regulatory requirements, but also to provide an accessible safeguard that enables the public to assess potential data exposure in widely deployed foundation models without technical barriers [24]. Current auditing protocols primarily rely on Membership Inference Attacks (MIAs) [40], which aim to determine whether a specific data sample was used for model training. However, when applied to contrastive pretraining models, existing MIAs face two critical challenges. (1) Computationally prohibitive costs: standard MIAs require training multiple shadow models to approximate the target model’s behavior, which is often computationally infeasible for large-scale multimodal architectures [17]. Even heuristic alternatives, such as cosine similarity-based attacks [22] or self-influence functions [9], either reduce detection accuracy or still demand substantial computational resources [32]. (2) The “auditor’s dilemma”: existing multimodal MIAs typically assume access to explicitly paired inputs, such as a face image with a name or a speaker’s voice with textual metadata, to query the target model [22], creating a fundamental paradox because third-party auditors (e.g., regulators) usually do not possess, and should not handle, users’ sensitive biometric data. Even worse, submitting such sensitive inputs to an untrusted model can introduce new exposure risks, undermining the very purpose of the privacy audit [15]. These challenges underscore an urgent need for auditing methods that avoid querying the target model with matched multimodal pairs, in line with the principle of multimodal data protection [27]. To this end, we study MIAs in a more realistic identity-level auditing setting. Specifically, we assume each training sample consists of a textual PII description tit_i (e.g., a name) and a corresponding non-textual modality mim_i (e.g., a face image for CLIP or a voice recording for CLAP). Given only a target text query t, the auditor seeks to determine whether there exists any training sample (ti,mi)(t_i,m_i) such that ti=t_i=t. Crucially, this must be done under a strict unimodal constraint: even if the auditor possesses real biometric samples, they are prohibited from submitting them to the target model. This restriction enables lightweight, text-only queries that mitigate privacy leakage risk and are practical for third-party deployment, as shown in Figure 2. Figure 1: Comparison of UMID and traditional MIA methods. Traditional approaches rely on shadow models or bimodal input, while UMID turns membership inference problem into anomaly detection using text-only queries. Figure 2: Visualization of geometric separation. The extracted similarity and variability features via latent inversion exhibit a clear distributional gap between samples within and outside the training dataset of the target model. In this paper, we propose Unimodal Membership Inference Detector (UMID), a text-only query framework that detects PII memorization in contrastive models without exposing sensitive biometric inputs or training costly shadow models. Given a target text description, UMID performs cross-modal latent inversion with multiple randomized initializations and extracts two key statistics: (1) similarity, quantified as the average cosine alignment between the optimized embedding and the queried text embedding; and (2) variability, measured by the mean squared dispersion of the optimized embeddings across runs. UMID then compares these statistics against a lightweight non-member baseline built from synthetic textual “gibberish” and makes a membership decision using a voting ensemble of anomaly detectors. The intuition behind this design is simple yet heuristic: texts that correspond to identities seen during training (members) tend to produce more accurate and more consistent modal reconstructions under inversion than unseen texts (non-members), creating a clear distributional separation that can be detected reliably, as illustrated in Figure 2. Contributions Our contributions are three-fold: 1. We introduce UMID, a membership inference detector for contrastive pretraining models that operates solely on textual PII queries. UMID eliminates costly shadow-model training and thus overcomes the computationally prohibitive costs faced by prior MIAs. Moreover, UMID avoids querying the target model with explicitly paired multimodal inputs, resolving the auditor’s dilemma and adhering to the principle of multimodal data protection. 2. We provide both theoretical and empirical evidence showing that member and non-member texts induce well-separated feature distributions under text-guided cross-modal inversion. This enables membership inference via unsupervised anomaly detection using randomly generated textual gibberish. 3. We conduct comprehensive experiments across diverse CLIP and CLAP architectures, demonstrating that UMID consistently outperforms existing MIA methods in both effectiveness and efficiency, even though it queries with text alone, rather than paired multimodal inputs. 2 Related Work Contrastive pretraining models Multimodal contrastive pretraining [35] has become a core framework for learning unified representations across heterogeneous modalities, including images, audio, and natural language [25]. Representative methods, such as CLIP [20] and CLAP [49], train dual encoders with contrastive objectives on web-scale paired data to construct a shared embedding space, where semantically matched pairs are pulled together while mismatched pairs are pushed apart. Such an alignment in a shared embedding space enables strong zero-shot transfer, cross-modal retrieval, and a broad range of downstream applications without fine-tuning via task-specific supervision [47]. Recent work further improves contrastive pretraining by strengthening cross-modal alignment and scalability [3, 16], introducing codebook-based representations, and leveraging larger and more curated audio–text resources [51]. Despite these advances, the scale and opacity of the underlying training corpora raise growing concerns [15]: sensitive information present in the data may be implicitly retained in learned embeddings, leading to privacy risks [11]. PII leakage in MLLMs The unprecedented capabilities of Multimodal Large Language Models (MLLMs) stem from extensive pretraining on massive and weakly supervised datasets [2]. Such model training process, however, inevitably introduces privacy risks as these web-scale datasets frequently contain personally identifiable information (PII), including faces, names, and voiceprints [30, 46]. A growing number of studies demonstrate that multimodal encoders and associated LLM components can memorize identity-linked features and sensitive attributes, rendering them susceptible to various privacy attacks such as model stealing, knowledge extraction, data reconstruction, and membership inference [22]. These risks are especially pronounced in contrastive frameworks such as CLIP, where strong cross-modal alignment allows adversaries to elicit sensitive identity associations through crafted queries [21]. Consequently, developing rigorous auditing protocols to detect such representation-level PII leakage remains a critical open problem [34]. Membership inference attacks Membership inference attacks (MIAs) serve as a fundamental and widely-used instrument for quantifying privacy leakage by ascertaining whether a specific data sample was included in a model’s training set [40]. Conventional MIA pipelines typically rely on the training of shadow models to approximate the target model’s decision boundaries [40] or the exploitation of loss-based confidence signals [48]. However, as architectures of modern large models usually scale to billions of parameters, these strategies have become computationally prohibitive and often impractical [52]. Recent efforts have adapted MIAs to contrastive models such as CLIP by leveraging cross-modal alignment behavior, including identity detection inference attacks (IDIA) [13], cosine similarity attacks (CSA) [22], and weak supervision attacks (WSA) [36]. While these approaches successfully reduce or eliminate the need for shadow training, they paradoxically necessitate querying the target model with biometric inputs (e.g., facial images or voice recordings) during inference [43]. This highlights an urgent need for novel membership detectors that can operate effectively without involving sensitive bimodal PII. 3 Method 3.1 Problem Formulation Auditing scenario Consider a multimodal contrastive pretraining model ℳM (e.g., CLIP [20] or CLAP [49]) pre-trained on a dataset DtrainD_train. Each training sample si=(ti,mi)s_i=(t_i,m_i) represents a paired identity (i.e., PII) of an individual, where tit_i denotes a textual description (e.g., a person name) and mim_i denotes a corresponding non-textual modality (e.g., a face image for CLIP or a voice recording for CLAP). The objective of the detector is to perform identity-level membership inference: given a target identity t, the detector aims to determine whether the PII sample (t,m)(t,m) was present in trainD_train. Notably, this is conducted under a unimodal privacy constraint: the auditor queries the model ℳM using only textual descriptions. Even if the auditor possesses real biometric samples of the target, they are prohibited from submitting them to ℳM to prevent secondary privacy leakage, which we term as the auditor’s dilemma in Section 1. Detector’s capability We operate under a gray-box model auditing setting, where the detector is assumed to have query and gradient access to the frozen encoders, ϕtext _text and ϕmod _mod to perform the latent inversion described in Section 3.2. Crucially, this assumption does not require access to the original training pipeline: the auditor remains completely oblivious to the private training dataset trainD_train, the specific training hyperparameters, or the optimizer states. This setting aligns with standard safety auditing protocols for open-weight foundation models, where model parameters are accessible but training data remains proprietary. Figure 3: Pipeline of UMID. We employ an optimizer guided by target model to align non-text embeddings with PII text embeddings, maximizing their cosine similarity. By analyzing similarity and variability features of these optimized samples relative to the synthetic gibberish baseline, an anomaly detection system identifies abnormal patterns to infer the membership of the input text. 3.2 UMID: Unimodal Membership Inference Detector We propose UMID, a framework that reformulates membership inference as an unsupervised anomaly detection task based on cross-modal latent inversion. As illustrated in Figure 3, UMID operates in three logical steps: (1) extracting membership signatures via randomized modality inversion; (2) grounding the separability of these signatures via geometric theory; and (3) performing membership inference via anomaly detection calibrated on a semantic-null baseline. Feature Extraction via Latent Inversion For a target textual identity t, UMID elicits its membership signature by optimizing a modality input x to align with the text embedding vt=ϕtext(t)v_t= _text(t). Specifically, we perform n independent optimization runs. In each run i∈1,…,ni∈\1,…,n\, we initialize x0(i)x_0^(i) from random noise and iteratively update it via gradient ascent to maximize the cosine similarity S(x,t)=ϕmod(x)⊤vt‖ϕmod(x)‖‖vt‖S(x,t)= _mod(x) v_t\| _mod(x)\|\|v_t\|. Let v(1)(t),…,v(n)(t)v^(1)(t),…,v^(n)(t) denote the final optimized embeddings. We extract two trajectory statistics as the feature representation f(t)f(t): 1. Similarity (SnS_n): Measured by the average final similarity across n runs, reflecting the model’s ability to reconstruct a modality counterpart for t: Sn(t):=vt⊤(1n∑i=1nv(i)(t)).S_n(t):=v_t ( 1n _i=1^nv^(i)(t) ). (1) 2. Variability (Dn2D_n^2): Measured by the mean squared dispersion of the optimized embeddings, capturing the consistency of the optimization landscape: Dn2(t):=1n∑i=1n‖v(i)(t)−v¯n(t)‖22,where v¯n(t)=1n∑i=1nv(i)(t).D_n^2(t):= 1n _i=1^n\|v^(i)(t)- v_n(t)\|_2^2, v_n(t)= 1n _i=1^nv^(i)(t). (2) Note that Sn(t)S_n(t) and Dn2(t)D_n^2(t) serve as proxies for alignment and consistency, respectively. Specifically, a higher Sn(t)S_n(t) indicates higher similarity, while a lower Dn2(t)D_n^2(t) signifies lower variability (higher stability). Intuitively, member identities tend to yield higher similarity and lower variability due to stronger training-induced alignment. We formalize this intuition in the following theoretical analysis. Geometric Separation with Theoretical Justification We provide a theoretical guarantee that the statistics (Sn,Dn2)(S_n,D_n^2) fundamentally separate members from non-members. We model the latent space using K unit-norm prototypes μkk=1K\ _k\_k=1^K. The insights are motivated by distinct geometric behaviors: For a non-member toutt_out, its embedding is isotropic relative to the prototypes, implying that the optimization explores the prototype space uniformly (pk(tout)≈1/Kp_k(t_out)≈ 1/K). In contrast, a member tint_in aligns with a specific prototype μy⋆ _y with a margin γ. This alignment forces the optimization to concentrate exponentially on y⋆y (py⋆(tin)≈1p_y (t_in)≈ 1). These properties are well-grounded in contrastive learning mechanics: the training objective (e.g., InfoNCE) explicitly optimizes for high cosine similarity (margin) for members, whereas unseen non-members naturally exhibit high-dimensional isotropy. Based on these properties, we define the population-level statistics S∞(t)S_∞(t) and D∞2(t)D_∞^2(t), which capture the ideal behavior as n→∞n→∞. Members exhibit high alignment (S∞≈γS_∞≈γ) and low dispersion (D∞2≈0D_∞^2≈ 0), whereas non-members exhibit low alignment (S∞≈0S_∞≈ 0) and high dispersion (D∞2≈1−1/KD_∞^2≈ 1-1/K). The following theorem provides a finite-sample guarantee for this separation rule. Theorem 3.1 (Finite-sample geometric separation). Given a member text tint_in and a non-member text toutt_out satisfying the geometric properties above. Let ΔS:=S∞(tin)−S∞(tout) _S:=S_∞(t_in)-S_∞(t_out) and ΔD:=D∞2(tout)−D∞2(tin) _D:=D_∞^2(t_out)-D_∞^2(t_in) be the population gaps, and define Γ:=minΔS,ΔD>0 := \ _S, _D\>0. For sufficiently large embedding dimension d=Ω(log(K/δ))d= ( (K/δ)) and number of runs n=Ω(Γ−2log(1/δ))n= ( ^-2 (1/δ)), there exist thresholds sthrs_thr and dthr2d_thr^2 such that with probability at least 1−δ1-δ, Sn(tin)≥sthr,Dn2(tin)≤dthr2,Sn(tout)≤sthr,Dn2(tout)≥dthr2. S_n(t_in)≥ s_thr,\ D_n^2(t_in)≤ d_thr^2, S_n(t_out)≤ s_thr,\ D_n^2(t_out)≥ d_thr^2. (3) Theorem 3.1 formally validates that our randomized reverse-optimization operationalizes the geometric distinction described above. It guarantees that the empirical statistics (Sn,Dn2)(S_n,D_n^2) inherit the population-level separation with high probability. Furthermore, it quantifies the sample complexity, highlighting that the computational cost scales inversely with the square of the separation margin Γ . Detailed proofs are deferred to B. Membership Inference via Anomaly Detection Translating the theoretical separation into a practical algorithm requires estimating the decision boundaries. As the theoretical thresholds are unknown, we reformulate the problem as unsupervised anomaly detection. Semantic-null baseline To approximate the non-member distribution, we generate a reference set of ℓ gibberish strings =g1,…,gℓG=\g_1,…,g_ \. Since G contains no semantic information and did not appear in training, it satisfies the isotropy condition stated in Section 3.2 and serves as a typical proxy for the null hypothesis. Inference procedure We extract features ℱF_G from G to train an ensemble of kdetk_det anomaly detectors (e.g., Isolation Forests). Each detector learns a boundary encompassing the non-member region. For a test identity t, UMID classifies it as a member if the majority of detectors flag its features f(t)f(t) as an anomaly, indicating a significant shift toward the high-similarity and low-variability characteristic of members. Algorithm of UMID The complete algorithm, including feature extraction, geometric separation, and membership inference, is stated in Algorithm 1. Optional Enhancement with Local Modality Samples In scenarios where the auditor possesses local real modality samples (which cannot be submitted to ℳM), UMID can include an auxiliary coherence feature. We use an external feature extractor F (e.g., DeepFace [42]) to compute R(t)R(t): the average pairwise ℓ2 _2 distance between the local samples and the optimized embeddings provided by UMID. Intuitively, members yield smaller R(t)R(t) as optimized samples converge to true identity features. We incorporate R(t)R(t) by performing K-means clustering (K=2K=2) on the augmented feature set (Sn,Dn2,R)\(S_n,D_n^2,R)\, adding the cluster assignment as an additional vote to the ensemble. Algorithm 1 The algorithm of UMID. Input: Model ℳM; Target text t; number of runs n; iterations per run m; learning rate η. Output: Membership decision (member/non-member). 1:Phase 1: Baseline construction (Offline) 2:Generate gibberish set =g1,…,gℓG=\g_1,…,g_ \ 3:for gk∈g_k do 4: Sn(gk),Dn2(gk)←LatentInversion(ℳ,gk,n,m,η)S_n(g_k),D_n^2(g_k)← LatentInversion(M,g_k,n,m,η) 5:end for 6:Train ensemble detectors ℰ=D1,…,DkE=\D_1,…,D_k\ on features (Sn(gk),Dn2(gk))\(S_n(g_k),D_n^2(g_k))\ 7:Phase 2: Membership inference (Online) 8:Sn(t),Dn2(t)←LatentInversion(ℳ,t,n,m,η)S_n(t),D_n^2(t)← LatentInversion(M,t,n,m,η) 9:Votes ←∑j=1k(Dj(Sn(t),Dn2(t))=Anomaly)← _j=1^kI(D_j(S_n(t),D_n^2(t))=Anomaly) 10:Return Member if Votes >N>N, else Non-member. 11: 12:function LatentInversion(ℳ,t,n,m,ηM,t,n,m,η) 13: vt←ϕtext(t)v_t← _text(t); ←∅V← ⊳ Set of optimized embeddings 14: for i=1i=1 to n do 15: x0∼(0,I)x_0 (0,I) ⊳ Random initialization 16: for j=0j=0 to m−1m-1 do 17: g←gj←∇xj(vt⊤ϕmod(xj)‖vt‖‖ϕmod(xj)‖)g← g_j← _x_j ( v_t _mod(x_j)\|v_t\|\| _mod(x_j)\| ) 18: xj+1←xj+η⋅gx_j+1← x_j+η· g 19: end for 20: ←∪ϕmod(xm)V ∪\ _mod(x_m)\ 21: end for 22: Compute Sn,Dn2S_n,D_n^2 via Eqn. (1) and Eqn. (2) 23: Return Sn,Dn2S_n,D_n^2 24:end function 4 Experiments We present a comprehensive evaluation of UMID across two distinct multimodal contrastive frameworks: CLIP (vision-text) and CLAP (audio-text). The experiments aim to validate UMID’s superiority in both detection effectiveness and computational efficiency compared to state-of-the-art baselines. 4.1 Experimental Setup Datasets Image-text (CLIP) We construct the CelebA dataset following previous work [13]. We integrate image-text pairs from FaceScrub [18] and LAION-5B [37] into the C3M [4] backbone to form the training set, ensuring a gender-balanced distribution across 200 selected identities. Specifically, we set a threshold to exclude individuals with excessively high frequencies in LAION-400M, maintaining 100 members and 100 non-members to ensure experimental reliability. The training set comprises 100 celebrities (members) and 100 hold-out identities (non-members). To simulate varying degrees of memorization, we curate two subsets: One-Shot (1 photo/person) and Many-Shot (75 photos/person). Audio-text (CLAP) We utilize LibriSpeech [33] and construct a richer speaker recognition dataset based on CommonVoice18.0 [1]. The latter involves 3,000 speakers (1,500 members/1,500 non-members) with detailed PII descriptions (ID, age, gender) augmented by GPT-4o to generate semantic background narratives. Similarly, we evaluate under One-Shot (1 audio/user) and Many-Shot (50 audios/user) settings. Models and Baselines Models For CLIP, we conduct membership inference on six target models spanning three visual backbones: ResNet-50, ResNet-50x4, and ViT-B/32. The ResNet-based models follow the standard ResNet architecture [12, 45], while ViT-B/32 adopts the Vision Transformer design, covering both CNN- and Transformer-based encoders. For CLAP, we adopt a dual-encoder architecture in which the audio encoder is HTSAT [6], a transformer composed of four groups of Swin Transformer blocks [29], and the text encoder is RoBERTa [28]. The penultimate-layer outputs of both encoders (768 dimensions) are projected to a shared 512-dimensional embedding space via a two-layer MLP with ReLU activation. All target models are trained from scratch on their respective member datasets to establish unambiguous ground-truth membership labels. When real facial images are available for optional enhancement, DeepFace [39] is used as an external feature extractor. Baselines We compare UMID against a spectrum of MIA methods: 1. Shadow-model-based: Standard MIA [40] is the base membership inference attack using shadow models; Audio Auditor [31] trains shadow models and extracts audio features for inference; and AuditMI [44] trains shadow model using input utterances and features from model outputs; SLMIA-SR [5] employs a shadow speaker recognition system to train attack model. These require training proxy models to mimic target behavior. 2. Metric-based: IDIA [13] detects training membership by verifying if a model correctly predicts a target’s identity from candidate prompts across multiple reference images; WSA [22] infers membership via image-text cosine similarity, enhanced by a weakly supervised MIA framework trained on post-release non-member data; C-WSA [36] treats low-variance samples as pseudo-members to train a robust image-feature classifier via confidence-based weak supervision. These rely on thresholding signals like loss or cosine similarity, often requiring real modality inputs (e.g., images/audio). Implementation and Evaluation All experiments are conducted on four NVIDIA GeForce RTX 3090 GPUs. Each setting is repeated five times, with the mean and standard deviation reported. UMID performs optimization in latent inversion for n=100n=100 epochs, each consisting of m=1000m=1000 gradient descent iterations with a learning rate of η=3×10−2η=3× 10^-2. The anomaly detection ensemble includes LocalOutlierFactor [8], IsolationForest [26], OneClassSVM [23, 19], and AutoEncoder [7]. These detectors are trained using ℓ=100 =100 synthetic gibberish strings generated by prompting GPT-3.5-turbo (see A for examples), with a default voting threshold of N=3N=3. When real non-textual samples are available, a K-means model (K=2K=2) is additionally incorporated, and the threshold is adjusted to N′=4N =4. Table 1: Comparison of CLIP (image-text) methods. Best and second-best results are bolded. Setting Method Real data Effectiveness Efficiency Precision Recall Accuracy Time ResNet-50 (one-shot) MIA [40] 1 img .7251 .6873 .7042 4.32h IDIA [13] 3 imgs .6922 .4031 .6836 0.845s WSA [22] 1 img .6653 .2925 .6675 2158.1s C-WSA [36] 1 img .7210 .4530 .7125 2141.4s UMID (Ours) Null .8634 .9821 .9172 0.628s UMID (Ours) 1 img .9145 .9912 .9528 0.745s ResNet-50 (many-shot) MIA [40] 1 img .7530 .7086 .7219 4.32h IDIA [13] 3 imgs .6901 .3998 .6907 0.851s WSA [22] 1 img .6625 .2867 .6710 2143.7s C-WSA [36] 1 img .7245 .4605 .7218 2149.5s UMID (Ours) Null .8642 .9835 .9031 0.634s UMID (Ours) 1 img .9168 .9925 .9554 0.752s ResNet-50x4 (one-shot) MIA [40] 1 img .7937 .6492 .7218 4.37h IDIA [13] 3 imgs .6625 .3980 .6957 0.851s WSA [22] 1 img .6712 .2912 .6808 2144.3s C-WSA [36] 1 img .7305 .4592 .7234 2148.6s UMID (Ours) Null .8613 .9747 .9355 0.633s UMID (Ours) 1 img .9128 .9896 .9535 0.685s ResNet-50x4 (many-shot) MIA [40] 1 img .7654 .6981 .7189 4.36h IDIA [13] 3 imgs .7085 .3904 .7167 0.844s WSA [22] 1 img .6724 .2935 .6685 2156.7s C-WSA [36] 1 img .7364 .4670 .7321 2150.3s UMID (Ours) Null .8712 .9916 .9462 0.637s UMID (Ours) 1 img .9156 .9908 .9562 0.691s ViT-B/32 (one-shot) MIA [40] 1 img .6923 .5987 .6694 4.35h IDIA [13] 3 imgs .6783 .3746 .6772 1.245s WSA [22] 1 img .6323 .2964 .6812 2143.1s C-WSA [36] 1 img .6945 .4230 .6912 2146.2s UMID (Ours) Null .7091 .6385 .6837 0.667s UMID (Ours) 1 img .7462 .6815 .7145 0.701s ViT-B/32 (many-shot) MIA [40] 1 img .6958 .6341 .6732 4.35h IDIA [13] 3 imgs .6890 .3811 .6927 1.251s WSA [22] 1 img .7045 .2806 .6895 2144.8s C-WSA [36] 1 img .7012 .4305 .7008 2147.9s UMID (Ours) Null .7182 .6372 .6947 0.652s UMID (Ours) 1 img .7498 .6844 .7186 0.712s Additionally, we evaluate effectiveness using Precision, Recall, and Accuracy. Efficiency is assessed via Time, measured as the wall-clock latency per query. 4.2 Comparison with State-of-the-Art Methods Performance on vision-language models Table 1 provides a comprehensive comparison of UMID against prior membership inference baselines across CLIP backbones, data exposure regimes, and query modalities. On ResNet-based CLIP models (ResNet-50 and ResNet-50x4), UMID exhibits a clear and consistent advantage, achieving over 90%90\% accuracy using text-only (without real data) queries while driving recall to 9797–99%99\%. This represents a substantial improvement over the strongest baseline C-WSA, with gains of up to +18+18 accuracy points and, more critically, over +50+50 recall point. This suggests that metric- and shadow-model-based attacks incur severe false negatives and fail to capture subtle memorization effects in contrastive models. Additionally, UMID still maintains a consistent superior effectiveness over all baselines, despite that Vision Transformers (ViT-B/32) exhibit weaker text-only signals due to stronger generalization. This indicates its ability to effectively probe the more resilient Transformer latent space. Importantly, the effectiveness gains across all settings come without increased computational cost: UMID completes inference in sub-second time (≈0.6≈ 0.6–0.80.8s), delivering orders-of-magnitude speedups over shadow-model approaches while eliminating reliance on real biometric samples. Performance on audio-language models Table 2 compares CLAP membership auditing methods across two datasets, revealing three consistent findings: (i) Effectiveness and robustness: UMID forms the clear top tier in all settings, uniquely maintaining high precision and high recall simultaneously. On LibriSpeech, UMID already performs strongly without any real target audio (UMID-Null: 91.27%/93.07%91.27\%/93.07\% accuracy in one-/many-shot), exceeding the strongest baseline AuditMI by a clear margin. Notably, with only one audio query, the gap further widens (up to +7.08+7.08 accuracy points) and recall approaches saturation. This advantage is amplified on the more challenging CommonVoice dataset, where all baselines degrade markedly (e.g., AuditMI ≈75%≈ 75\% accuracy) but UMID remains robust, widening both accuracy and recall gaps, indicating better generalization beyond clean speech. (i) Exposure sensitivity. UMID’s one-shot performance is already close to many-shot (accuracy gaps ≤2.2≤ 2.2 points), suggesting that membership leakage is detectable even under minimal exposure when cross-modal evidence is properly exploited. (i) Efficiency and deployability. In addition to its strong effectiveness, UMID runs in sub-second time (≈0.6≈ 0.6–0.80.8s), offering an order-of-magnitude speedup over other baselines. Table 2: Comparison of CLAP (audio-text) methods. Best and second-best results are bolded. Setting Method Real data Effectiveness Efficiency Precision Recall Accuracy Time LibriSpeech (one-shot) Audio Auditor [31] 1 audio .6338 .7324 .6519 1.273s SLMIA-SR [5] 1 audio .7521 .8864 .8342 1.528s AuditMI [44] 1 audio .8257 .9526 .8791 13.578s UMID (Ours) Null .8649 .9649 .9127 0.603s UMID (Ours) 1 audio .8921 .9868 .9354 0.647s LibriSpeech (many-shot) Audio Auditor [31] 1 audio .6559 .8013 .6659 1.245s SLMIA-SR [5] 1 audio .7619 .9007 .8433 1.547s AuditMI [44] 1 audio .8341 .9804 .8816 13.511s UMID (Ours) Null .8812 .9876 .9307 0.612s UMID (Ours) 1 audio 91.63 99.57 95.24 0.659s CommonVoice (one-shot) Audio Auditor [31] 1 audio .5485 .6822 .6052 1.347s SLMIA-SR [5] 1 audio .6539 .7691 .7048 1.597s AuditMI [44] 1 audio .7143 .8145 .7436 15.214s UMID (Ours) Null .7496 .8601 .8179 0.721s UMID (Ours) 1 audio .7602 .8955 .8356 0.753s CommonVoice (many-shot) Audio Auditor [31] 1 audio .5611 .7358 .6135 1.391s SLMIA-SR [5] 1 audio .6628 .7927 .7218 1.652s AuditMI [44] 1 audio .7352 .8481 .7564 15.931s UMID (Ours) Null .7647 .8946 .8233 0.815s UMID (Ours) 1 audio .7934 .9113 .8569 0.776s 4.3 Ablation Study We conduct ablation studies in the following four perspectives to quantify how key design choices affect detection performance on both CLIP and CLAP (Figure 4 and Figure 5). Figure 4: Detection accuracy for CLIP model (ResNet-50) under various parameters. Figure 5: Detection accuracy for CLAP model (LibriSpeech) under various parameters. Number of gibberish texts Figure 4(a) and Figure 5(a) show that accuracy improves as the number of randomly generated gibberish strings (ℓ ) increases, and stabilizes once ℓ≥50 ≥ 50 for both modalities. With fewer than 20 strings, training becomes noisy and less reliable. Beyond 50 strings, the gain is marginal (typically <0.5%<0.5\% in accuracy), so we use ℓ=100 =100 as a practical choice that balances performance and overhead. Real sample enhancement As shown in Figure 4(b) and Figure 5(b), adding even a single real sample consistently improves accuracy (by roughly 1.5−3.5%1.5-3.5\%, depending on the dataset and training set size). In contrast, using more than one real sample provides limited additional benefit (typically <1%<1\%), suggesting that real samples are sufficient but not necessary to capture the modality-specific structure needed by the clustering-based vote. Detection threshold analysis We study the sensitivity of the ensemble decision to the voting threshold. As shown in Figure 4(c) and Figure 5(c), text-only inputs achieve the best accuracy with a three-vote threshold, whereas incorporating real samples favors a four-vote threshold (Figure 4(d) and Figure 5(d)). A higher threshold tends to miss true anomalies (lower recall), while a lower threshold increases false positives. The selected thresholds provide a robust balance between precision and recall across both modalities. Optimization parameters We examine two hyperparameters used during feature extraction: the number of epochs (Figure 4(e) and Figure 5(e) and the number of iterations per epoch (Figure 4(f) and Figure 5(f)). Overall, performance peaks at 100 epochs with 1000 iterations per epoch for CLIP, and 50 epochs with 1000 iterations per epoch for CLAP. Increasing iterations beyond 100 per epoch yields negligible gains (typically <0.3%<0.3\% in accuracy) while substantially increasing computation. Likewise, using more than 100 epochs leads to diminishing returns. 5 Robustness against Potential Defenses To comprehensively assess the practical effectiveness of our approach, we evaluate UMID’s robustness against potential defenses that model owners might deploy to mitigate membership inference attacks. Table 3: UMID robustness evaluation under differential privacy defense (ϵ=1.0ε=1.0, δ=10−5δ=10^-5). Dataset Training samples Defense Effectiveness Efficiency Accuracy Time CelebA (CLIP) one-shot Without .9535 0.685s DP .8742 0.688s many-shot Without .9562 0.691s DP .8595 0.689s LibriSpeech (CLAP) one-shot Without .9354 0.647s DP .8788 0.651s many-shot Without .9524 0.659s DP .8340 0.663s CommonVoice (CLAP) one-shot Without .8356 0.753s DP .7590 0.749s many-shot Without .8569 0.776s DP .7915 0.772s Gaussian noise We apply differential privacy to embedding computations by adding calibrated Gaussian noise. Specifically, for each query, noise sampled from (0,σ2Id)N(0,σ^2I_d) is added to returned embeddings, where σ is calibrated to achieve (ϵ,δ)(ε,δ)-differential privacy with ϵ=1.0ε=1.0 and δ=10−5δ=10^-5. Table 3 reports the robustness of UMID against a Differential Privacy (DP) defense in both the image and audio modalities. After injecting calibrated Gaussian noise (ϵ=1.0ε=1.0), UMID exhibits only a moderate degradation: accuracy drops by roughly 6%−12%6\%-12\%, depending on the dataset and the number of samples. Concretely, for CLIP (ResNet-50x4) on CelebA, UMID still achieves 87.42%87.42\% accuracy with 1 sample and 85.95%85.95\% with 75 samples under the defense. Likewise, for CLAP on LibriSpeech, performance remains high at 87.88%87.88\% and 83.40%83.40\%, and on CommonVoice it remains competitive at 75.90%75.90\% and 79.15%79.15\%. Importantly, runtime is essentially unchanged across all settings (averaging 0.70.7s), indicating that DP introduces negligible additional computational overhead for the attacker. Overall, these results suggest that while DP noise reduces the fidelity of embedding-based leakage, UMID’s reliance on geometric vulnerabilities remains effective under a moderate privacy budget; in practice, stronger privacy budgets (ϵ<1.0ε<1.0) and/or complementary defenses may be necessary to fully secure these models. Covert gibberish generation In practice, a target model may incorporate input filters that flag overtly anomalous queries (e.g., nonsensical gibberish) and return misleading outputs, causing UMID to misclassify the presence of PII. To obtain more covert queries, we generate gibberish-like strings that remain visually and phonotactically plausible by replacing a small subset of characters with syllables drawn from a different language. For example, the detector can synthesize query texts by mixing English-style names with syllables from Arabic medical terminology. Concretely, we first prompt an LLM (e.g., GPT-3.5-turbo) to produce lists of common English initial and final syllables, then post-process the lists to remove duplicates and encourage coverage. We randomly concatenate syllables to form pseudo-English names (e.g., Karinix”, Zylogene”, “Renotyl”), and explicitly verify their novelty against a name lexicon to avoid collisions with real entities. Finally, we prompt the LLM to instantiate these syllable templates into full strings, yielding covert gibberish that closely resembles authentic names (see Table 6). Table 4 summarizes UMID’s robustness to the Covert Gibberish (CG) defense for both CLIP and CLAP. UMID remains highly resilient: CG reduces accuracy by only 1.23%1.23\%–1.54%1.54\% across all datasets and training sample sizes. For example, on CelebA with 75 training samples, UMID still achieves 93.85%93.85\% accuracy under CG. Running time is also essentially unchanged (about 0.650.65–0.780.78s), suggesting that CG neither introduces meaningful computational overhead nor effectively disrupts the identification procedure. Overall, UMID preserves both effectiveness and efficiency under covert, text-like perturbations. Table 4: UMID robustness evaluation under covert gibberish defense. Dataset Training samples Defense Effectiveness Efficiency Accuracy Time CelebA (CLIP) one-shot Without .9535 0.685s CG .9412 0.689s many-shot Without .9562 0.691s CG .9385 0.687s LibriSpeech (CLAP) one-shot Without .9354 0.647s CG .9210 0.652s many-shot Without .9524 0.659s CG .9395 0.655s CommonVoice (CLAP) one-shot Without .8356 0.753s CG .8220 0.756s many-shot Without .8569 0.776s CG .8415 0.781s 6 Conclusion This work revisits membership inference for contrastive pretraining models through the lens of identity-level auditing under a unimodal privacy constraint, motivated by the practical limitations of existing MIAs: prohibitive shadow-model training and the need to query targets with paired biometric inputs. In this paper, we introduced UMID, a text-only auditing framework that avoids both requirements by leveraging text-guided cross-modal latent inversion. The key insight is that member identities induce reconstructions that are not only better aligned with the queried text but also more consistent across randomized inversions. UMID operationalizes this intuition via two statistics (similarity and variability) and a lightweight non-member reference built from synthetic gibberish, enabling membership decisions through an ensemble of unsupervised anomaly detectors. Extensive experiments on CLIP and CLAP across architectures and exposure regimes validate that UMID consistently outperforms prior baselines while remaining highly efficient, making it suitable for third-party auditing where sensitive biometrics are unavailable or impermissible to submit. Overall, UMID provides a practical and privacy-compliant pathway to assess PII memorization in widely deployed multimodal foundation models. References [1] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber (2019) Common voice: A massively-multilingual speech corpus. CoRR abs/1912.06670. External Links: Link, 1912.06670 Cited by: §4.1. [2] H. I. Ashqar, A. Jaber, T. I. Alhadidi, and M. Elhenawy (2025) Advancing object detection in transportation with multimodal large language models (mllms): a comprehensive review and empirical testing. Computation 13 (6), p. 133. Cited by: §2. [3] Z. Cao, Z. Deng, Z. Yang, J. Ma, and L. Ma (2025) Supervised contrastive pre-training models for mammography screening. Journal of Big Data 12 (1), p. 24. Cited by: §2. [4] S. Changpinyo, P. Sharma, N. Ding, and R. Soricut (2021) Conceptual 12m: pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 3558–3568. Cited by: §4.1. [5] G. Chen, Y. Zhang, and F. Song (2024) SLMIA-sr: speaker-level membership inference attacks against speaker recognition systems. In Proceedings of the 31st Annual Network and Distributed System Security (NDSS) Symposium, Cited by: item 1, Table 2, Table 2, Table 2, Table 2. [6] K. Chen, X. Du, B. Zhu, Z. Ma, T. Berg-Kirkpatrick, and S. Dubnov (2022) Hts-at: a hierarchical token-semantic audio transformer for sound classification and detection. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 646–650. Cited by: §4.1. [7] Z. Chen, C. K. Yeo, B. S. Lee, and C. T. Lau (2018) Autoencoder-based network anomaly detection. In 2018 Wireless telecommunications symposium (WTS), p. 1–5. Cited by: §4.1. [8] Z. Cheng, C. Zou, and J. Dong (2019) Outlier detection using isolation forest and local outlier factor. In Proceedings of the conference on research in adaptive and convergent systems, p. 161–168. Cited by: §4.1. [9] G. Cohen and R. Giryes (2024) Membership inference attack using self influence functions. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 4892–4901. Cited by: §1. [10] B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang (2023) Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: §1. [11] A. Golatkar, A. Achille, Y. Wang, A. Roth, M. Kearns, and S. Soatto (2022) Mixed differential privacy in computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8376–8386. Cited by: §2. [12] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778. Cited by: §4.1. [13] D. Hintersdorf, L. Struppek, M. Brack, F. Friedrich, P. Schramowski, and K. Kersting (2024) Does clip know my face?. Journal of Artificial Intelligence Research 80, p. 1033–1062. Cited by: §2, item 2, §4.1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1. [14] L. Hu, A. Yan, H. Yan, J. Li, T. Huang, Y. Zhang, C. Dong, and C. Yang (2023) Defenses to membership inference attacks: a survey. ACM Computing Surveys 56 (4), p. 1–34. Cited by: §1. [15] P. Hu, Z. Wang, R. Sun, H. Wang, and M. Xue (2022) M4i: multi-modal models membership inference. Advances in Neural Information Processing Systems 35, p. 1867–1882. Cited by: §1, §2. [16] B. Hunt, E. Kwan, J. Bergquist, J. Brundage, B. Orkild, J. Dong, E. Paccione, K. Yazaki, R. S. MacLeod, D. J. Dosdall, et al. (2025) Contrastive pretraining improves deep learning classification of endocardial electrograms in a preclinical model. Heart Rhythm O2 6 (4), p. 473–480. Cited by: §2. [17] M. Jagielski, M. Nasr, K. Lee, C. A. Choquette-Choo, N. Carlini, and F. Tramer (2024) Students parrot their teachers: membership inference on model distillation. Advances in Neural Information Processing Systems 36. Cited by: §1. [18] I. Kemelmacher-Shlizerman, S. M. Seitz, D. Miller, and E. Brossard (2016) The megaface benchmark: 1 million faces for recognition at scale. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 4873–4882. Cited by: §4.1. [19] S. S. Khan and M. G. Madden (2014) One-class classification: taxonomy of study and review of techniques. The Knowledge Engineering Review 29 (3), p. 345–374. Cited by: §4.1. [20] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan (2020) Supervised contrastive learning. Advances in neural information processing systems 33, p. 18661–18673. Cited by: §2, §3.1. [21] S. Kim, S. Yun, H. Lee, et al. (2024) Propile: probing privacy leakage in large language models. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §2. [22] M. Ko, M. Jin, C. Wang, et al. (2023) Practical membership inference attacks against large-scale multi-modal models: a pilot study. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4871–4881. Cited by: §1, §2, §2, item 2, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1. [23] K. Li, H. Huang, S. Tian, and W. Xu (2003) Improving one-class svm for anomaly detection. In Proceedings of the 2003 international conference on machine learning and cybernetics (IEEE Cat. No. 03EX693), Vol. 5, p. 3077–3081. Cited by: §4.1. [24] S. Li, R. Cheng, and X. Jia (2025) Tuni: a textual unimodal detector for identity inference in clip models. In Proceedings of the Sixth Workshop on Privacy in Natural Language Processing, p. 1–13. Cited by: §1. [25] Y. Li, H. Wang, Y. Duan, J. Zhang, and X. Li (2025) A closer look at the explainability of contrastive language-image pre-training. Pattern Recognition 162, p. 111409. Cited by: §2. [26] F. T. Liu, K. M. Ting, and Z. Zhou (2008) Isolation forest. In 2008 eighth ieee international conference on data mining, p. 413–422. Cited by: §4.1. [27] X. Liu, X. Jia, Y. Xun, S. Liang, and X. Cao (2024) Multimodal unlearnable examples: protecting data against multimodal contrastive learning. In Proceedings of the 32nd ACM International Conference on Multimedia, p. 8024–8033. Cited by: §1. [28] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (1907) RoBERTa: a robustly optimized bert pretraining approach. corr 2019. arXiv preprint arXiv:1907.11692. Cited by: §4.1. [29] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo (2021) Swin transformer: hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, p. 10012–10022. Cited by: §4.1. [30] Z. Liu, G. Dou, M. Jia, Z. Tan, Q. Zeng, Y. Yuan, and M. Jiang (2025) Protecting privacy in multimodal large language models with mllmu-bench. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 4105–4135. Cited by: §2. [31] Y. Miao, M. Xue, C. Chen, L. Pan, J. Zhang, B. Z. H. Zhao, D. Kaafar, and Y. Xiang (2021) The audio auditor: user-level membership inference in internet of things voice services. Proceedings on Privacy Enhancing Technologies 1, p. 209–228. Cited by: item 1, Table 2, Table 2, Table 2, Table 2. [32] M. G. Oh, L. H. Park, J. Kim, J. Park, and T. Kwon (2023) Membership inference attacks with token-level deduplication on korean language models. IEEE Access 11, p. 10207–10217. Cited by: §1. [33] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur (2015) Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , p. 5206–5210. External Links: Document Cited by: §4.1. [34] T. Pham, Q. Vo, H. Dao, and K. Fukuda (2025) I never willingly consented to this! investigate pii leakage via sso logins. IEEE Transactions on Privacy. Cited by: §2. [35] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, p. 8748–8763. Cited by: §1, §2. [36] D. Samira, E. Habler, Y. Elovici, and A. Shabtai (2025) Variance-based membership inference attacks against large-scale image captioning models. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 9210–9219. Cited by: §2, item 2, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1. [37] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. (2022) Laion-5b: an open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 35, p. 25278–25294. Cited by: §4.1. [38] P. M. Schwartz and D. J. Solove (2011) The pii problem: privacy and a new concept of personally identifiable information. NYUL rev. 86, p. 1814. Cited by: §1. [39] S. I. Serengil and A. Ozpinar (2020) LightFace: a hybrid deep face recognition framework. In 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), p. 23–27. External Links: Document, Link Cited by: §4.1. [40] R. Shokri, M. Stronati, C. Song, and V. Shmatikov (2017) Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), p. 3–18. Cited by: §1, §2, item 1, Table 1, Table 1, Table 1, Table 1, Table 1, Table 1. [41] R. Sun, H. Hu, W. Luo, Z. Zhang, Y. Zhang, H. Yuan, and L. Y. Zhang (2025) When better features mean greater risks: the performance-privacy trade-off in contrastive learning. In Proceedings of the 20th ACM Asia Conference on Computer and Communications Security, p. 488–500. Cited by: §1. [42] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf (2014) Deepface: closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 1701–1708. Cited by: §3.2. [43] J. Tao and R. Shokri (2025) Range membership inference attacks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), p. 346–361. Cited by: §2. [44] F. Teixeira, K. Pizzi, R. Olivier, A. Abad, B. Raj, and I. Trancoso (2025) Exploring features for membership inference in asr model auditing. Computer Speech & Language, p. 101812. Cited by: item 1, Table 2, Table 2, Table 2, Table 2. [45] D. Theckedath and R. Sedamkar (2020) Detecting affect states using vgg16, resnet50 and se-resnet50 networks. SN Computer Science 1 (2), p. 79. Cited by: §4.1. [46] L. Tran, W. Sun, S. Patterson, and A. Milanova (2025) Privacy-preserving personalized federated prompt learning for multimodal large language models. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2. [47] W. Tsai, P. Le, W. Ho, N. Chi, J. J. Lin, S. Tang, and S. Hsieh (2025) Construction safety inspection with contrastive language-image pre-training (clip) image captioning and attention. Automation in Construction 169, p. 105863. Cited by: §2. [48] Z. Wang, C. Zhang, Y. Chen, N. Baracaldo, S. R. Kadhe, and L. Yu (2025) Membership inference attacks as privacy tools: reliability, disparity and ensemble. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, p. 1724–1738. Cited by: §2. [49] J. Wu, W. Li, Z. Novack, A. Namburi, C. Chen, and J. McAuley (2025) Collap: contrastive long-form language-audio pretraining with musical temporal structure augmentation. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: §2, §3.1. [50] Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, and T. Wang (2024) Defending pre-trained language models as few-shot learners against backdoor attacks. Advances in Neural Information Processing Systems 36. Cited by: §1. [51] Z. Xie, X. Xu, Z. Wu, and M. Wu (2025) Audiotime: a temporally-aligned audio-text benchmark dataset. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: §2. [52] M. Xue, C. Yuan, C. He, Y. Wu, Z. Wu, Y. Zhang, Z. Liu, and W. Liu (2023) Use the spear as a shield: an adversarial example based privacy-preserving technique against membership inference attacks. IEEE Transactions on Emerging Topics in Computing 11 (1), p. 153–169. External Links: Document Cited by: §2. [53] X. Yuan, Z. Lin, J. Kuen, J. Zhang, Y. Wang, M. Maire, A. Kale, and B. Faieta (2021) Multimodal contrastive training for visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6995–7004. Cited by: §1. [54] H. Zhao, H. Chen, Y. Xiao, and Z. Zhang (2023) Privacy-enhanced federated learning against attribute inference attack for speech emotion recognition. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , p. 1–5. External Links: Document Cited by: §1. Appendix A Gibberish Samples Table 5: Samples of randomly generated textual gibberish generated by GPT-3.5-turbo. b+dh43u!f9de545w 53e(s3erg24=pnI<S fe3_5fg;@f34f.e2w/ d3|%542ferf34G_ 4teh<E434fef234ter 5gt%b@5435-hgF #4c3rd4scd324g ’2_:g435thyngb Table 6: Covert gibberish that seem to be real names generated by GPT-3.5-turbo. Karinix Zylogene Glycogenyx Zylotrax Vexilith Dynatrix Exodynix Novylith Glycosyne Xenolynx Rynexis Delphylith Appendix B Theoretical Guarantees for Geometric Separation In this appendix, we provide the detailed proofs for the theoretical claims made in Section 3.2. We establish that under mild geometric assumptions about the contrastive latent space, the proposed statistics SnS_n and Dn2D_n^2 fundamentally separate member identities from non-members with high probability. B.1 Preliminaries and Assumptions We fix a text t with embedding vt=ϕ(t)∈ℝdv_t=φ(t) ^d. The randomized modality inversion produces n independent optimized embeddings v(1),…,v(n)v^(1),…,v^(n). Recall the definitions of our statistics: v¯n v_n :=1n∑i=1nv(i),Sn:=vt⊤v¯n,Dn2:=1n∑i=1n‖v(i)−v¯n‖22. := 1n _i=1^nv^(i), S_n:=v_t v_n, D_n^2:= 1n _i=1^n\|v^(i)- v_n\|_2^2. (4) We model the latent space geometry using M unit-norm prototypes μkk=1M⊂d−1\ _k\_k=1^M ^d-1. We restate the core assumptions formally for the proof: Assumption B.1 (Prototype orthogonality). The prototypes μ1,…,μM _1,…, _M are approximately orthogonal. Specifically, let ρd:=maxy≠z|μy⊤μz| _d:= _y≠ z| _y _z|. For δ1∈(0,1) _1∈(0,1), with probability at least 1−δ11- _1, ρd≤4log(2M2/δ1)d _d≤ 4 (2M^2/ _1)d. Assumption B.2 (Text-prototype geometry). Non-member: For a non-member toutt_out, its embedding voutv_out is isotropic relative to random directions. For any unit u, ℙ(|vout⊤u|≥ϵ)≤2exp(−cdϵ2)P(|v_out u|≥ε)≤ 2 (-cdε^2). Member: For a member tint_in, there exists a prototype y⋆y such that vin⊤μy⋆≥γin>0v_in _y ≥ _in>0. Assumption B.3 (Randomized Optimization). Each run i selects a prototype index C(i)∼p(t)C^(i) p(t) and lands near it: v(i)=μC(i)+Δ(i)v^(i)= _C^(i)+ ^(i), with ‖Δ(i)‖2≤εopt\| ^(i)\|_2≤ _opt with high probability. For members, p(tin)p(t_in) concentrates on y⋆y : py⋆≥1−δ⋆p_y ≥ 1-δ (where δ⋆δ is small). For non-members, p(tout)p(t_out) is nearly uniform: py≈1/Mp_y≈ 1/M. B.2 Population-Level Separation We first analyze the population statistics defined as n→∞n→∞ (assuming εopt→0 _opt→ 0): S∞(t):=vt⊤m(t)S_∞(t):=v_t m(t) and D∞2(t):=1−‖m(t)‖22D_∞^2(t):=1-\|m(t)\|_2^2, where m(t):=∑y=1Mpy(t)μym(t):= _y=1^Mp_y(t) _y is the mean prototype vector. Proposition B.4 (Population separation gaps). Under the assumptions above, with high probability over prototypes: 1. Member: S∞(tin)≥γin−2δ⋆S_∞(t_in)≥ _in-2δ and D∞2(tin)≤2δ⋆+3ρdδ⋆≈0D_∞^2(t_in)≤ 2δ +3 _dδ ≈ 0. 2. Non-member: |S∞(tout)|≤O(d−1/2)≈0|S_∞(t_out)|≤ O(d^-1/2)≈ 0 and D∞2(tout)≥1−1M−ρd≈1D_∞^2(t_out)≥ 1- 1M- _d≈ 1. Proof. For Member: S∞=∑pyvin⊤μy=py⋆vin⊤μy⋆+∑y≠y⋆pyvin⊤μyS_∞=Σ p_yv_in _y=p_y v_in _y + _y≠ y p_yv_in _y. Using py⋆≥1−δ⋆p_y ≥ 1-δ , vin⊤μy⋆≥γinv_in _y ≥ _in, and trivial bounds |v⊤μ|≤1|v μ|≤ 1, we get S∞≥(1−δ⋆)γin−δ⋆≈γinS_∞≥(1-δ ) _in-δ ≈ _in. For dispersion, D∞2=1−‖py⋆μy⋆+∑y≠y⋆pyμy‖22D_∞^2=1-\|p_y _y + _y≠ y p_y _y\|_2^2. The cross-terms are bounded by ρd _d. Dominant term is 1−py⋆2≈1−(1−δ⋆)2≈2δ⋆1-p_y ^2≈ 1-(1-δ )^2≈ 2δ . For Non-member: S∞=vout⊤m(tout)S_∞=v_out m(t_out). Since voutv_out is isotropic and independent of m(tout)m(t_out), S∞S_∞ concentrates around 0 with rate d−1/2d^-1/2 (Assumption B.2). For dispersion, ‖m(tout)‖22=‖∑pyμy‖22=∑py2+∑y≠zpypzμy⊤μz\|m(t_out)\|_2^2=\|Σ p_y _y\|_2^2=Σ p_y^2+ _y≠ zp_yp_z _y _z. With py≈1/Mp_y≈ 1/M, ∑py2≈1/MΣ p_y^2≈ 1/M. Cross terms are bounded by ρd(∑py)2=ρd _d(Σ p_y)^2= _d. Thus D∞2≥1−(1/M+ρd)D_∞^2≥ 1-(1/M+ _d). ∎ B.3 Finite-Sample Concentration We now show that empirical statistics (Sn,Dn2)(S_n,D_n^2) converge to population values as O(1/n)O(1/ n). Decompose the empirical mean v¯n=m~n+Δ¯n v_n= m_n+ _n, where m~n=1n∑μC(i) m_n= 1nΣ _C^(i) is the average of selected prototypes and Δ¯n _n is the average optimization residual. Lemma B.5 (Concentration Bounds). Conditioned on small residuals ‖Δ(i)‖≤εopt\| ^(i)\|≤ _opt, for any ϵ>0ε>0: ℙ(|Sn−S∞|≥ϵ+εopt) (|S_n-S_∞|≥ε+ _opt) ≤2exp(−2nϵ2), ≤ 2 (-2nε^2), (5) ℙ(|Dn2−D∞2|≥4ϵ+Cεopt) (|D_n^2-D_∞^2|≥ 4ε+C _opt) ≤2exp(−nϵ2). ≤ 2 (-nε^2). (6) Proof. For SnS_n: Sn=vt⊤m~n+vt⊤Δ¯nS_n=v_t m_n+v_t _n. The term |vt⊤Δ¯n|≤εopt|v_t _n|≤ _opt. The term vt⊤m~n=1n∑Ziv_t m_n= 1nΣ Z_i where Zi=vt⊤μC(i)Z_i=v_t _C^(i) are i.i.d. variables bounded in [−1,1][-1,1] with mean S∞S_∞. Hoeffding’s inequality gives the result. For Dn2D_n^2: Using the identity Dn2=1n∑‖v(i)‖2−‖v¯n‖2D_n^2= 1nΣ\|v^(i)\|^2-\| v_n\|^2, and decomposing v(i)v^(i), we can bound |Dn2−D∞2|≤2‖m~n−m(t)‖2+O(εopt)|D_n^2-D_∞^2|≤ 2\| m_n-m(t)\|_2+O( _opt). Since m~n m_n is the average of bounded i.i.d. vectors with mean m(t)m(t), Hoeffding inequality yields the exponential tail bound. ∎ B.4 Proof of Theorem 3.1 The goal is to show that with probability at least 1−δ1-δ, the empirical statistics (Sn,Dn2)(S_n,D_n^2) for members and non-members fall on opposite sides of the thresholds sthrs_thr and dthr2d_thr^2. We structure the proof in three logical steps: (1) ensuring valid population geometry, (2) bounding the optimization and sampling errors, and (3) deriving the separation condition. We allocate the total failure probability δ into three components: prototype geometry failure (δproto _proto), optimization localization failure (δopt′ _opt ), and sampling concentration failure (δsamp _samp), such that δtotal=δproto+δopt′+δsamp≤δ _total= _proto+ _opt + _samp≤δ. We set each component to δ/4δ/4. (1) Existence of population gaps. By Proposition B.4 and the prototype orthogonality assumption (Assumption B.1), there exists an event ℰprotoE_proto with ℙ(ℰproto)≥1−δ/4P(E_proto)≥ 1-δ/4 where the population gaps are strictly positive. We define the minimum margin Γ:=minS∞(tin)−S∞(tout),D∞2(tout)−D∞2(tin)>0 := \S_∞(t_in)-S_∞(t_out), D_∞^2(t_out)-D_∞^2(t_in) \>0. The decision thresholds are defined as the midpoints: sthr=12(S∞(tin)+S∞(tout))s_thr= 12(S_∞(t_in)+S_∞(t_out)) and dthr2=12(D∞2(tin)+D∞2(tout))d_thr^2= 12(D_∞^2(t_in)+D_∞^2(t_out)). (2) Concentration of empirical statistics. We require the empirical statistics to concentrate around their population means within a radius smaller than Γ/2 /2. First, consider the optimization localization. Let ℰopt(t)E_opt(t) be the event that all n runs satisfy ‖Δ(i)‖2≤εopt\| ^(i)\|_2≤ _opt. By the union bound over 2n2n runs (for both tint_in and toutt_out), we have ℙ(ℰoptc)≤2nδoptP(E_opt^c)≤ 2n _opt. We require 2nδopt≤δ/42n _opt≤δ/4. Next, consider the sampling noise. Conditioned on ℰoptE_opt, Lemma B.5 guarantees concentration. We choose an error tolerance ϵε and residual εopt _opt such that the total deviation is bounded by Γ/4 /4. Specifically, let the target deviation bound be Δdev:=4ϵ+CΔεopt _dev:=4ε+C_ _opt. We require Δdev≤Γ/2 _dev≤ /2. By Lemma B.5, the probability that any of the four statistics (Sn,Dn2S_n,D_n^2 for member/non-member) deviates by more than Δdev _dev is bounded by 8exp(−nϵ2)8 (-nε^2). Setting this to δ/4δ/4 yields the sample complexity requirement: n≥1ϵ2log(32δ)=Ω(Γ−2log(1/δ)).n≥ 1ε^2 ( 32δ )= ( ^-2 (1/δ) ). (3) Geometric separation condition. Let ℰgoodE_good be the intersection of valid geometry, optimization localization, and sampling concentration. On this event, for the similarity statistic Sn(tin)S_n(t_in): Sn(tin)−sthr S_n(t_in)-s_thr =Sn(tin)−12(S∞(tin)+S∞(tout)) =S_n(t_in)- 12 (S_∞(t_in)+S_∞(t_out) ) =12(S∞(tin)−S∞(tout))⏟≥Γ−|Sn(tin)−S∞(tin)|⏟≤Δdev = 12 (S_∞(t_in)-S_∞(t_out) )_≥ - |S_n(t_in)-S_∞(t_in) |_≤ _dev ≥Γ2−Γ2=0⟹Sn(tin)≥sthr. ≥ 2- 2=0 S_n(t_in)≥ s_thr. Similarly, for the non-member, sthr−Sn(tout)≥Γ2−Δdev≥0s_thr-S_n(t_out)≥ 2- _dev≥ 0, implying Sn(tout)≤sthrS_n(t_out)≤ s_thr. The exact same logic applies to the dispersion Dn2D_n^2: dthr2−Dn2(tin)≥Γ2−Δdev≥0⟹Dn2(tin)≤dthr2. d_thr^2-D_n^2(t_in)≥ 2- _dev≥ 0 D_n^2(t_in)≤ d_thr^2. Thus, on ℰgoodE_good, the separation conditions hold simultaneously. Together, the probability of failure is bounded by the sum of failure probabilities for each step: ℙ(ℰgoodc)≤δproto+δopt′+δsamp≤δP(E_good^c)≤ _proto+ _opt + _samp≤δ. This completes the proof. ∎