Paper deep dive
Cross-Service Threat Intelligence in LLM Services using Privacy-Preserving Fingerprints
Waris Gill, Natalie Isak, Matthew Dressman
Models: ModernBert, OpenAI text-embedding-3-large
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/11/2026, 12:58:42 AM
Summary
BinaryShield is a privacy-preserving threat intelligence system designed for LLM services. It enables the secure sharing of attack fingerprints across compliance boundaries by utilizing a pipeline of PII redaction, semantic embedding, binary quantization, and differential privacy (randomized response). This approach allows organizations to correlate prompt injection attacks across siloed services without violating privacy regulations, achieving high F1-scores and significant computational efficiency compared to traditional dense embedding methods.
Entities (5)
Relation Signals (3)
BinaryShield â outperforms â SimHash
confidence 98% · BinaryShield achieves an F1-score of 0.94, significantly outperforming SimHash (0.77)
BinaryShield â mitigates â Prompt Injection
confidence 95% · BinaryShield, the first privacy-preserving threat intelligence system that enables secure sharing of attack fingerprints across compliance boundaries.
BinaryShield â uses â PII Redaction
confidence 95% · BinaryShield transforms suspicious prompts through a unique pipeline combining PII redaction
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The widespread deployment of LLMs across enterprise services has created a critical security blind spot. Organizations operate multiple LLM services handling billions of queries daily, yet regulatory compliance boundaries prevent these services from sharing threat intelligence about prompt injection attacks, the top security risk for LLMs. When an attack is detected in one service, the same threat may persist undetected in others for months, as privacy regulations prohibit sharing user prompts across compliance boundaries. We present BinaryShield, \emph{the first privacy-preserving threat intelligence system that enables secure sharing of attack fingerprints across compliance boundaries.} BinaryShield transforms suspicious prompts through a unique pipeline combining PII redaction, semantic embedding, binary quantization, and randomized response mechanism to potentially generate privacy-preserving fingerprints that preserve attack patterns while providing privacy. Our evaluations demonstrate that BinaryShield achieves an F1-score of 0.94, significantly outperforming SimHash (0.77), the privacy-preserving baseline, while achieving storage reduction and 38x faster similarity search compared to dense embeddings.
Tags
Links
- Source: https://arxiv.org/abs/2509.05608
- Canonical: https://arxiv.org/abs/2509.05608
Trouble viewing inline? Open PDF directly â
Full Text
93,492 characters extracted from source content.
Expand or collapse full text
Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) BinaryShield: Cross-Service Threat Intelligence in LLM Services using Privacy-Preserving Fingerprints Waris Gill Microsoft Redmond, USA warisgill@microsoft.com Natalie Isak Microsoft New York, USA natalieisak@microsoft.com Matthew Dressman Microsoft Redmond, USA matthew.dressman@microsoft.com AbstractâThe widespread deployment of LLMs across en- terprise services has created a critical security blind spot. Organizations operate multiple LLM services handling billions of queries daily, yet regulatory compliance boundaries prevent these services from sharing threat intelligence about prompt injection attacks, the top security risk for LLMs. When an attack is detected in one service, the same threat may persist undetected in others for months, as privacy regulations prohibit sharing user prompts across compliance boundaries. We presentBinaryShield, the first privacy-preserving threat intelligence system that enables secure sharing of attack fingerprints across compliance boundaries.BinaryShieldtransforms suspi- cious prompts through a unique pipeline combining PII redaction, semantic embedding, binary quantization, and randomized response mechanism to potentially generate privacy-preserving fingerprints that preserve attack patterns while providing privacy. Our evaluations demonstrate thatBinaryShieldachieves an F1-score of 0.94, significantly outperforming SimHash (0.77), the privacy-preserving baseline, while achieving storage reduction and 38x faster similarity search compared to dense embeddings. I. INTRODUCTION The rapid adoption of Large Language Models (LLMs) in digital services is driving a paradigm shift in human-computer interaction. Companies now operate many LLM-based services across diverse domains, collectively handling billions of queries each day. Typically, organizations like Microsoft maintain multiple, logically isolated (siloed) LLM services, such as enterprise AI assistants, consumer chat applications, API-based LLM services, and developer-focused coding agents. These services are siloed because they are tailored to meet distinct business objectives and end-user requirements, necessitating separate operational boundaries. Each service has distinct model stacks, logging pipelines, and compliance boundaries with strong internal data governance policies to protect user data and privacy. In addition to internal privacy regulations, these services are also subject to governmental regulations such as GDPR and HIPAA, which mandate strict data handling prac- tices to protect user privacy and sensitive information. While this siloed architecture is crucial for maintaining operational independence, it has an unintended consequence: fragmented security telemetry and a weakened collective defense. With little correlation or data sharing between services, organizations are left with a disjointed security posture as each service operates independently, responding to threats in isolation. This fragmentation creates a significant challenge for incident response teams, as they lack a unified view of the threat landscape across their LLM services. Prompt Injection and Current Defenses. The new interaction model, based on natural language prompts, introduces a critical security risk: prompt injection attacks. Often described as the âSQL injectionâ of AI, prompt injection is recognized as the top threat in the OWASP 2025 Top 10 for LLMs. Prompt injection exploits LLMsâ inability to distinguish legitimate user instructions from malicious commands embedded in input. These attacks can manipulate system prompts to leak private data, execute malicious code via tool interfaces, or generate harmful content. With emerging technologies like Model Context Protocol (MCP) and autonomous agents, the impact of prompt injection extends beyond text manipulation to arbitrary code execution and potential system compromise [1]. Organizations and researchers have developed various defense mechanisms against these attacks [2], [3]. Furthermore, more than one defense is deployed to enhance security within a compliance boundary [3]. However, a critical limitation stems directly from privacy regulations that prevent the sharing of user prompts between compliance boundaries (i.e., services). Motivation and Problem Statement. The current defense paradigm is compartmentalized, with each service responding independently to detected attacks. When a new attack vector is identified (typically through media reports, user complaints, or post-breach analysis), security teams patch their systems (e.g., update prompts classifiers). However, this knowledge remains confined to the affected service. Furthermore, attacks are inevitable because existing defenses are probabilistic and can be bypassed [4], [5]. Thus, organizations need retrospective systems to pinpoint the perpetrator and timing of incidents. This reactive, siloed approach prevents organization-wide visibility into the threat landscape. An attack discovered in one service today may have existed undetected in another for months. Security analysts lack tools to search for similar historical attack patterns across service boundaries, creating a âcorrelation gapâ. As a result, they cannot fully assess the scale and sophistication of an attack, identify its variants, or proactively protect all services from emerging threats. Traditional malware defense addressed similar coordination challenges decades ago by exchanging signatures: antivirus engines share hash or pattern-based fingerprints of malicious binaries without revealing proprietary information [6]. To our 1 arXiv:2509.05608v2 [cs.CR] 28 Feb 2026 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) knowledge, no comparable, privacy-preserving, and practically deployable threat intelligence mechanism exists for natural- language prompts in LLM services. This paper addresses the following problem: How can an organization securely share threat intelligence about prompt injection attacks across its compliance boundaries without violating privacy regulations? Ideal Characteristics. A privacy-preserving fingerprinting mechanism for prompt injection attacks is essential for secure information sharing across compliance boundaries. Such a fingerprint must encode the semantic core of the malicious prompt, capturing its essential characteristics beyond surface tokens. It should be privacy preserving, so the original input (prompt) cannot be recovered. The technique must support secure approximate matching for âfind-similarâ searches across data stores and remain lightweight for real-time generation and distribution. When an attack is detected in one service, the fingerprint can be broadcast to peer services. These peers can then search historical logs for related incidents, flag live traffic, and train local defenses, enabling proactive, collaborative security. Cross-boundary threat intelligence will enable faster patching without compromising user privacy. Our Contributions. We propose the concept of threat intel- ligence across compliance boundaries in LLM services. We materialize this idea inBinaryShield, the first privacy- preserving fingerprinting technique for prompt injection attacks that enables secure information sharing between otherwise siloed services. When a serviceâs defense mechanism detects a potential prompt-injection threat,BinaryShieldgenerates a privacy-preserving, lightweight fingerprint that captures the malicious promptâs semantic essence, allowing actionable threat signals to be shared without compromising privacy. Note, we use prompt injection as a broader term, similar to its use in the OWASP Top 10 for LLMs, where jailbreaking is treated as a form of prompt injection [7]. Our approach balances the need for threat intelligence sharing with user privacy and regulatory compliance. BinaryShield achieves this through a multi-stage process that progressively removes identifying information while retaining the essential semantic characteristics for effective threat detection. The process begins with personally identifiable information (PII) redaction, whereBinaryShieldremoves sensitive data such as social security numbers, names, and other identifying markers from input prompts. This initial privacy layer is crucial for protecting users whose prompts may be incorrectly flagged as malicious by automated defenses. How- ever, PII redaction alone is insufficient, as the remaining text may still contain contextual information that could compromise user privacy when shared externally. To capture the semantic essence of potentially malicious prompts,BinaryShield generates high-dimensional embeddings from the redacted text. These embeddings, which represent text as dense floating- point vectors, achieve state-of-the-art performance in natural language processing tasks by encoding semantic relationships in vector space. However, embeddings are not secure for sharing across compliance boundaries. Recent research demonstrates that embeddings pose significant privacy risks, as they can Defense D1LLM Service-1 Defense D2LLM Service-2 Defense D3LLM Service-3 Services in Cloud Attack Log Take actions based on policy. Prompt Injection Attacks Fingerprint Generation within Service 2 Boundary Search Attack Fingerprint within Service 1 and Service 3 Compliance Boundaries Attack Detected BinaryShield Pipeline Raw Logs Public Features Privacy Preserving Features Service 2 Attack Fingerprint Service 2 Attack Fingerprint Service 1 Queries Fingerprints Service 3 Queries Fingerprints Search Service 3 Service 1 Attacker System Metadata (e.g., Region) Private Data (e.g., Prompt) 13 2 5 Similar Fingerprints 6 7 Similar Fingerprints 6 4 Fig. 1:BinaryShieldsystem design. Suspicious prompts are processed within the compliance boundary to generate privacy-preserving fingerprints, which are then shared across services for collaborative threat detection. allow adversaries to reconstruct original input text [8]â[11]. Moreover, traditional embeddings present practical challenges for large-scale threat intelligence system. Each dimension typically requires 32 bits of storage, and similarity computations using cosine similarity or dot products demand substantial computational resources, often necessitating GPU acceleration for efficient threat matching. BinaryShieldaddresses these fundamental challenges through two innovative non-reversible transformations by exploiting insights from quantization and differential privacy. First,BinaryShieldperforms binary quantization on the floating-point embeddings, converting each dimension to a single bit based on its sign, assigning 1 for positive values and 0 for negative values. This quantization achieves remark- able efficiency gains, reducing storage requirements while simultaneously enhancing privacy. The key insight underlying this approach is that by discarding magnitude information and retaining only directional information, we make prompt reconstruction exponentially more difficult while preserving sufficient semantic structure for threat sharing and detection. To render original prompt reconstruction practically impossible, BinaryShieldapplies differential privacy to the binary embeddings. Leveraging the insights from the principle of ran- domized response [12]â[14],BinaryShieldindependently flips each bit of the binary vector with a precisely calibrated probability. This noise addition fundamentally alters the binary vector representation while preserving its calibrated utility for threat correlation, ensuring that adversaries cannot reverse- engineer the original prompt from the shared fingerprint. The resulting binary vector is efficient in storage and computation, enabling rapid Hamming distance [15] similarity checks. More importantly, after these privacy-preserving transformations, the resulting binary vector becomes safe to share across compliance boundaries, enabling effective cross-service threat intelligence without compromising user privacy or regulatory compliance. Evaluations. We conduct comprehensive evaluation of BinaryShieldspanning threat correlation effectiveness, privacy calibration, scalability, and operational efficiency (Sec- tion I).BinaryShieldsignificantly outperforms privacy- 2 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) Algorithm 1: BinaryShield Input: Suspicious prompt y; system metadata m; privacy parameter α; set of peer services S Output: Privacy-preserving fingerprint f broadcast to S 1 y redacted â PII_Redact(y) ; // Remove all PII, replace with placeholders 2eâ Embed(y redacted ) ; // Compute semantic embedding vector 3bâ [] ; // Initialize empty binary vector 4 for iâ 1 to d do 5 b i â ( 1 if e i > 0 0 otherwise ; 6Append b i tob ; 7 p keep â e α e α +1 ; // Probability of keeping the bit 8 Ì bâ [] ; // Initialize empty privatized vector 9 for iâ 1 to d do 10Sample u i ⌠Uniform(0, 1) ; 11if u i < p keep then 12 Ì b i â b i ; // Keep the true bit 13else 14 Ì b i â 1â b i ; // Flip the bit 15Append Ì b i to Ì b ; 16 f â Concatenate( Ì b,m) ; // Combine privatized bits with non-private metadata 17 Broadcast(f , S) ; // Send fingerprint to peers preserving baseline across attack variants (17-point F1 advan- tage on sophisticated attacks) and exhibits smooth privacy- utility trade-offs with theoretical noise alignment. In real- world enterprise-scale scenarios,BinaryShieldachieves 79.2% threat correlation accuracy (93% of non-private baseline performance) alongside efficiency gains (38x search speedup). I. BINARYSHIELD DESIGN The proliferation of LLM-based services within organizations creates a fundamental tension between security effectiveness and privacy compliance. While traditional malware defense systems successfully share threat signatures across bound- aries [6], no comparable mechanism exists for LLM services. BinaryShieldaddresses this gap by enabling organizations to share actionable threat intelligence about prompt injection attacks without violating privacy regulations. The system is designed to achieve four primary objectives: (1) Semantic Preservation: capture the essential characteristics of malicious prompts that enable cross-service threat correlation; (2) Privacy Guarantee: ensure that original prompts cannot be reconstructed from shared fingerprints, hence upholding regulations; (3) Computational Efficiency: enable efficient fingerprint generation and search; and (4) Operational Compli- ance: maintain strict separation of user data across compliance boundaries while enabling collaborative defense. To this end, BinaryShieldtransforms potentially malicious prompts through a series of privacy-preserving operations, ultimately producing compact binary fingerprints that can be safely shared between services. The key insight underlying our approach is that prompt injection attacks, despite surface-level variations, share semantic patterns that can be captured and compared without revealing the original content. High-Level Architecture. As a motivating example, consider a company that offers both Enterprise AI and Consumer AI services operating under separate compliance boundaries. When Enterprise AI detects a prompt injection attack, it cannot warn Consumer AI due to regulations prohibiting raw prompt sharing, leaving Consumer AI vulnerable to the same attack. Figure 1 illustrates howBinaryShieldaddresses this challenge. An attacker issues semantically related prompts (x,y,z) to the services of an enterprise organization that reside in separate compliance boundaries (Figure 1-â). Each service applies their own defense mechanism (D1,D2, andD3) to incoming prompts. Service-2 flags a prompt-injection attempt (Figure 1-â). After this step,BinaryShieldâs fingerprinting pipeline is invoked in the serviceâs compliance boundary (Figure 1-â). At this stage, the attack log is separated into two components. The first and most critical component is private information: this contains the sensitive content of the prompt itself, holding the essential details of the attack. Although critical for accurate event correlation, this data must remain within the serviceâs compliance boundary to safeguard privacy. The suspicious prompt then undergoes a series of privacy- preserving transformations to generate a fingerprint within the attacked serviceâs compliance boundary (Figure 1-â). The fingerprint generation process is detailed in SectionII-A. The second part consists of non-private system metadata, such as the tools used during response generation and the geographical region. Since this metadata contains no sensitive data, it can be safely shared among peer services as part of complete signature. Sharing these features enhances attack correlation by providing complementary evidence to surface relevant patterns pointing to attack source and channel, which helps reduce false positives and prioritize alerts. For instance, matching system metadata across services, such as both accessing an email tool, significantly increases the confidence in the correlation of potential attacks. Concatenating both parts forms a composite attack fingerprint that discloses no readable text (Figure 1-â) This fingerprint is then securely broadcast to peer services for threat correlation (Figure 1-â). Note that peer services (i.e., Services 1 and 3) use the exact same technique to fingerprint their local private queries and system metadata as used in Step âof Figure 1. For instance, after searching their logs, Service 1 and Service 3 independently find two and one similar attacks, respectively (Figure 1-â). Policy-driven response actions are then executed for each service based on the correlation results (Figure 1-â). Note that there is currently no public dataset available for system metadata. Therefore, our evaluation of BinaryShieldfocuses exclusively on private information (prompt) fingerprinting. Nonetheless, including system metadata features in the overall design demonstrates the full architectural intent ofBinaryShieldand highlights additional layers of security and context. A. BinaryShield Fingerprint Generation Design BinaryShield design is guided by several key require- ments. A primary goal is to ensure data privacy by trans- forming original prompts into fingerprints through a one-way 3 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) Potential Prompt Injection PII Redaction Module Semantic Embedding Generation Binary Quantization Randomized Response (Bit Flipping) Privacy-Preserving Binary Fingerprint Raw prompt Redacted text Float vector eâ R d Binary vector bâ0, 1 d Noisy binary Ì bâ0, 1 d Transformation: b i = ( 1 if e i > 0 0 otherwise Privacy mechanism: Pr[ Ì b i = b i ] = e α e α +1 Fig. 2:BinaryShieldapproach for privacy-preserving fingerprint generation for cross-service threat intelligence sharing. The pipeline transforms potential prompt injections through PII redaction, semantic embedding, binary quantization, and differential privacy to produce shareable fingerprints that preserve privacy while enabling threat correlation across compliance boundaries. 10 0 10 1 10 2 10 3 10 4 Count PERSON URL LOCATION US_DRIVER_LICENSE DATE_TIME NRP IP_ADDRESS PHONE_NUMBER EMAIL_ADDRESS US_BANK_NUMBER US_SSN US_PASSPORT Entity Type 7894 3771 957 855 854 521 50 14 12 8 1 1 Fig. 3: Distribution of PII entities in prompts flagged by BinaryShield. Most common: person names. process, making the reconstruction of the original prompt computationally difficult. Simultaneously, these fingerprints must preserve enough semantic information to be useful for identifying related attack patterns via approximate matching. Operationally, the system is built for computational efficiency to handle millions of daily queries with low latency and is designed with cross-boundary compatibility in mind, allowing fingerprints to be shared across services with varying internal regulatory requirements. Algorithm 1 and Figure 2 depicts the BinaryShieldâs pipeline for generating privacy-preserving fingerprints of prompt injection attacks. Below, we detail each component of the pipeline. 1) PII Redaction:BinaryShieldâs pipeline initial stage directly targets privacy risks by identifying and removing personally identifiable information (PII) that may be present in flagged prompts. While automated defenses aim to identify malicious content, they may inadvertently flag benign prompts containing sensitive user data. The main challenge is removing PII while preserving the semantic structure of potential attacks. Our approach addresses this by detecting and redacting structured data (using [16]) such as social security numbers, credit card numbers, phone numbers, email addresses, person names, organizations, locations, and other identifying entities. Identified PII is replaced with generic placeholders that maintain command structure, entity relationships, and linguistic patterns indicative of injection attempts. For example, âTransfer $5000 from John Smithâs account 123456789â becomes âTransfer [AMOUNT] from [PERSON]âs account [ACCOUNT]â. This balance ensures the redacted text retains sufficient information for downstream processing while protecting user privacy from the earliest stage of the pipeline. This one-way redaction minimizes PII exposure before any downstream analysis. Figure 3 reports the PII entities and their respective counts thatBinaryShieldflags in the prompt injection dataset from [17], used inBinaryShieldâs evaluation (Section I). However, because redaction alone is insufficient to protect all sensitive information, the pipelineâs next stage generates a privacy-preserving semantic fingerprint. 2) Semantic Feature Extraction: Following PII redaction, BinaryShieldmust capture the semantic essence of the potentially malicious prompt in a format suitable for com- parison and analysis. To this end,BinaryShieldemploys state-of-the-art embedding models [18], [19] to generate high- dimensional embeddings that encode the semantic relationships within the text. This transformation serves as the foundation for our semantic preservation requirement. This captures the meaning and intent of the potentially malicious prompt. The embedding process transforms the redacted text into a dense vectore â R d , wheredis the embedding dimension. The embedding space exhibits several properties that are crucial for threat detection. First, similar attack patterns tend to cluster together in the embedding space, regardless of surface-level variations. Second, the embeddings are robust to paraphrasing, so minor rewording or synonym substitution results in nearby embeddings, enabling the detection of attack variants. Binary Quantization. While semantic embeddings effectively capture prompt meaning, they pose significant challenges for privacy-preserving threat intelligence. Dense embeddings can leak information about the original text [8]â[11]. They also require substantial computational resources for large-scale similarity searches. Our binary quantization addresses both concerns. We transform the continuous embeddingeâ R d into a binary vector bâ0, 1 d using sign-based quantization: b i = ( 1 if e i > 0 0 otherwise , wheree i is thei-th dimension of the embedding vector andb i is the corresponding bit in the binary vector. This transformation provides several critical benefits by design. First, it acts as a lossy compression mechanism. By mapping each continuous value inRto a binary state in0, 1based on its sign, the process discards all magnitude information. This many-to-one mapping makes it computationally difficult to infer the original floating-point vectors, thus enhancing data protection. Second, despite this information reduction, the high dimensionality (d) of the binary embedding preserves essential geometric properties needed for similarity detection. Finally, this method is highly storage-efficient, reducing the memory footprint for each dimension from 32 bits (for a float32) to just 1 bit, a 32x reduction. 4 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 202 1 0 1 2 P1 P1' P2 P2' P3 P3' P4 P4' P5 P5' P6 P6' P7 P7' P8 P8' P9 P9' P10 P10' =0.25 Avg Displacement: 3.28 202 P1 P1' P2 P2' P3 P3' P4 P4' P5 P5' P6 P6' P7 P7' P8 P8' P9 P9' P10 P10' =1 Avg Displacement: 2.09 202 P1 P1' P2 P2' P3 P3' P4 P4' P5 P5' P6 P6' P7 P7' P8 P8' P9 P9' P10 P10' =2 Avg Displacement: 0.97 202 P1 P1' P2 P2' P3 P3' P4 P4' P5 P5' P6 P6' P7 P7' P8 P8' P9 P9' P10 P10' =3 Avg Displacement: 0.54 PC1 PC2 Original Binary VectorBinaryShield Randomized Response () Noisy Vector Fig. 4: Impact ofBinaryShieldâs privacy parameter (α). Asαincreases, privatized vector (red triangles) move closer to original vector, showing improved utility while maintaining differential privacy through controlled bit-flipping (i.e.,1âp) noise. 3)BinaryShieldâs Randomized-Response Layer: To build upon the baseline protections of data anonymization and binary quantization, our pipeline incorporates Local Differential Privacy (LDP) for a more formal privacy guarantee. By applying a randomized response mechanism [12]â[14] to each bit before a fingerprint crosses the compliance boundary, we gain several critical advantages. This approach provides a strong, mathematically-defined upper bound on privacy loss for each fingerprint, ensuring plausible deniability with a formal guarantee of privacy regardless of any auxiliary information a person might possess. The guarantee is enforced locally before data leaves its origin, which is ideal for cross-boundary threat intelligence as it removes the need for a trusted central aggregator. Furthermore, unlike in the global DP, the magnitude of the required noise is independent of the dataset size, ensuring our privacy protection remains constant and does not degrade as the system scales to millions of daily fingerprints. Consider ad-bit fingerprintb = (b 1 ,...,b d ) â 0, 1 d and letα > 0denote the per-bit privacy budget. Classical randomized response publishes a perturbed vector Ì bobtained by flipping each bit independently with probability1âp, where p = e α e α + 1 . The mechanism releases the randomized bit. Ì b i = ïŁ± ïŁČ ïŁł b i with probability p, 1â b i with probability 1â p. This DP mechanism inevitably perturbs the fingerprint. The expected self-distance introduced by it is E H(b, Ì b) = (1â p)d, whereHdenotes Hamming distance anddis the fingerprint dimensionality. The Hamming distance between two binary vectorsb, Ì b â 0, 1 d is defined as the number of positions at which the corresponding bits differ: H(b, Ì b) = d X i=1 I[b i Ìž= Ì b i ], whereI[·]is the indicator function, which equals 1 if its argument is true and 0 otherwise. Becausepremains close to one even for moderateα, the added noise is sparse and, as we show experimentally in Section I, preserves enough structure for accurate approximate matching while providing the privacy guarantees demanded by cross-service threat intelligence. The randomization budgetαserves as a control parameter governing the privacy-utility tradeoff inBinaryShield. Whenαapproaches zero, the mechanism maximizes privacy protection (i.e., infinite privacy) but renders fingerprints un- usable for correlation. Asαincreases, utility improves while maintaining privacy guarantees in a controlled manner. Figure 4 illustrates this relationship through PCA visualization of binary vectors before and after applying the Randomized Response mechanism [12]. Atα = 0.25, privatized vectors (red triangles) appear completely random. Asαincreases (reducing noise), these vectors progressively align closer to their original non- private counterparts (blue circles). Notably, even atα = 3, a non-zero average displacement between original and privatized vectors persists, making reconstruction of the original prompt challenging. This visualization confirms that largerαvalues preserve more of the vectorâs structure while maintaining formal differential privacy guarantees through controlled bit-flipping. In practice, higher privacy budgets may be appropriate for intra- service sharing where additional security measures already exist within organizational boundaries. 4)BinaryShieldâs Cross-Service Threat Correlation: When a service detects a potential prompt injection, it generates a fingerprint using theBinaryShieldâs pipeline. For clarity, we denote the locally randomized-response fingerprint Ì b â 0, 1 d asfwhen broadcasting and correlating across services (i.e.,f â Ì b). This fingerprint is broadcast asynchronously to participating services. Upon receiving a fingerprint, services scan their historical logs for similar instances. If matches are found, alerts are triggered and defenses can be updated proactively. To maintain privacy and compliance, only aggregate match statistics are shared, and no specific prompt content is revealed. This workflow enables rapid propagation of threat intelligence while respecting organizational boundaries. Formally, letS = S 1 ,S 2 ,...,S N denote the set ofN services, each with its own compliance boundary. Each serviceS i maintains a log of previously observed fingerprints F i = f (1) i ,f (2) i ,...,f (M i ) i , wheref (m) i â 0, 1 d is ad- 5 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) dimensional binary fingerprint. Suppose serviceS q detects a suspicious prompt and generates a fingerprintf q â0, 1 d us- ing theBinaryShieldpipeline. This fingerprint is securely broadcast to all other servicesS j (j Ìž= q). Each recipient service S j performs the following search: M j (f q ) = n f (m) j âF j H(f q ,f (m) j )â€ Ï o , whereM j (f q )is the set of fingerprints belonging to service S j (i.e., fingerprints inF j ) that matchf q within a Hamming distance thresholdÏ, andH(·,·)is the Hamming distance as previously defined. This enables rapid, privacy-preserving propagation of threat intelligence across allNservices. This protocol generalizes to any number of services. For example, if Service 2 (S 2 ) generates a fingerprintf 2 , it broadcastsf 2 to Services 1 and 3 (S 1 ,S 3 ), which independently search their logs for similar fingerprints. IfS 1 finds two matches andS 3 finds one, these services take action based on their policy and only the counts are reported back, enabling rapid, privacy-preserving propagation of threat intelligence across organizational boundaries. Summary.BinaryShieldoffers a scalable and privacy- preserving solution for sharing prompt injection threat in- telligence across organizational compliance boundaries. By combining PII redaction, semantic embedding, binary quan- tization, and differential privacy, the system achieves strong privacy protections without sacrificing detection performance or scalability. The modular architecture allows for privacy parameters to be tuned to specific requirements, while efficient processing supports production-scale deployments. As LLM adoption expands,BinaryShieldestablishes a foundation for collaborative defenses against evolving attacks. I. EVALUATION We present a comprehensive evaluation ofBinaryShield, structured around the following research questions: âąRQ1 (Detection Effectiveness). How accurately does BinaryShieldmatch prompt injection attacks across systematically generated adversarial variants, from simple word substitutions to comprehensive semantic paraphras- ing, compared to the SimHash [15] baseline? âą RQ2 (Privacy-Utility Trade-off). What is the quantita- tive relationship betweenBinaryShieldâs randomized response mechanism (α) and threat detection utility? âąRQ3 (Noise Calibration & Predictability). Does the empirical self-Hamming distortion introduced by random- ized response match the theoretical(1âp)dcurve across α, letting us confidently choose a privacy setting? âąRQ4 (Scalability). DoesBinaryShieldmaintain con- sistent detection accuracy as corpus size scales from thousands to hundreds of thousands of entries, reflecting realistic enterprise deployment scenarios where malicious prompts are sparse within benign prompts? âąRQ5 (Computational Efficiency). What are the computa- tional gains ofBinaryShieldâs fingerprints compared to dense embeddings, and do these improvements enable practical real-time threat correlation at enterprise scale? A. Evaluation Settings 1) Datasets: We evaluateBinaryShieldusing the recent prompt injection dataset from [17], which contains real-world attack and benign prompts. From this, we construct a compre- hensive evaluation set simulating adversarial scenarios, enabling controlled and realistic assessment of threat intelligence. Given the inherent challenge of obtaining large-scale labeled datasets of semantically similar prompt injection variants, we develop a systematic methodology for generating controlled attack modifications to validate threat intelligence. Our syn- thetic data generation process creates two primary types of variants: Word-Flipping Variants: For each adversarial prompt, we randomly selectxwords (wherex â 1, 3, 5, 10, 20) longer than four characters and replace them with semantically equivalent synonyms using GPT-4o. This simulates subtle ad- versarial modifications that maintain attack intent while altering surface-level tokens. The language model is guided using the structured prompt listed in Appendix Listing 1. Paraphrase Variants: We generate comprehensive paraphrased versions of prompts using GPT-4o while preserving the underlying attack intent. Appendix Listing 2 contains the prompt. Benign Pairs: Benign prompts are systematically paired such that each pair consists of two semantically unrelated prompts, ensuring they serve as appropriate negative examples for similarity detec- tion algorithms (e.g., SimHash andBinaryShield). The resulting dataset undergoes rigorous filtering to ensure quality and consistency (e.g., remove duplicates, empty prompts). The final dataset is balanced with equal numbers of attack and benign pairs, with attack pairs labeled as 1 and benign pairs labeled as 0. This balanced approach ensures unbiased evaluation across both positive and negative cases. Our synthetic generation methodology enables systematic evaluation across varying levels of attack sophistication, from minimal single- word modifications to comprehensive paraphrasing. 2) Baseline: Our evaluation comparesBinaryShield against SimHash [15], a widely-adopted locality-sensitive hashing baseline.BinaryShieldoperates by first extracting semantic features using an embedding model. To demonstrate its flexibility with both open-source and proprietary options, we evaluateBinaryShieldusing ModernBert [19] and an OpenAI model. 3) Evaluation Metrics: We employ a comprehensive set of metrics to assess both detection performance and privacy-utility tradeoffs. For detection performance, we use precision, recall, accuracy, and F1-score across varying similarity thresholds to evaluate attack detection accuracy. Threshold analysis is performed using precision-recall curves to identify the optimal threshold for Hamming distance and to understand how performance is affected by threshold selection, as discussed in [15]. We also include confusion matrices for detailed analysis of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) at optimal thresholds. 6 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 0.0 0.5 1.0 V-1V-3V-5 200300400 0.0 0.5 1.0 V-10 200300400 V-20 200300400 V-Paraphrase Hamming Distance Threshold (Ï ) Precision and Recall Score PrecisionRecall (a) Precision-recall curves across attack variants. Regular PI 9511 4948 V-1 (Ï =256) 9525 0957 V-3 (Ï =286) 9461 9938 V-5 (Ï =264) RegularPI Regular PI 9214 9916 V-10 (Ï =276) RegularPI 8716 9868 V-20 (Ï =278) RegularPI 88476 50910 V-Paraphrase (Ï =306) Predicted Label True Label (b) Confusion matrices at optimal thresholds. Fig. 5:BinaryShieldperformance analysis across attack variants withα = 2.0. (a) Precision-recall curves demonstrate consistent performance across variants. (b) Confusion matrices at optimal Hamming distance thresholds show TP, TN, FP, and FN counts, indicating strong detection capabilities even under complex paraphrasing attacks. Note that PI in the confusion matrices stands for Prompt Injection and is used to distinguish between attack and benign prompts. B. BinaryShield Comparison with Baseline To provide a rigorous performance analysis, we systemat- ically compareBinaryShieldagainst SimHash, a widely used locality-sensitive semantic hashing method [15]. Both algorithms generate binary fingerprints and use Hamming distance for similarity measurement, making them directly comparable. This baseline is chosen to evaluate not only detection performance but also the privacy-preserving capabili- ties essential for cross-boundary threat intelligence sharing as SimHash also generates privacy-preserving binary fingerprints. For this evaluation, we configureBinaryShieldwith a differential privacy parameter ofα = 2.0, which provides meaningful privacy protection while maintaining strong de- tection performance. Our comparative analysis encompasses six distinct attack variants: minimal single-word modifications (V-1), progressive word changes (V-5, V-20) and extensive modifications in paraphrasing scenarios (V-Paraphrase) that fundamentally restructure prompts while preserving malicious 0.0 0.5 1.0 V-1V-3V-5 020 0.0 0.5 1.0 V-10 020 V-20 020 V-Paraphrase Hamming Distance Threshold (Ï ) Precision and Recall Score PrecisionRecall (a) Precision-recall curves across attack variants. Regular PI 9520 7945 V-1 (Ï =15) 9543 20937 V-3 (Ï =18) 9398 17930 V-5 (Ï =19) RegularPI Regular PI 9178 40885 V-10 (Ï =19) RegularPI 86413 63814 V-20 (Ï =20) RegularPI 703257 196764 V-Paraphrase (Ï =27) Predicted Label True Label (b) Confusion matrices at optimal thresholds. Fig. 6:SimHashbaseline performance analysis across attack variants. Confusion matrices reveal growing FP and FN, particularly for paraphrasing scenarios. 1 Words3 Words5 Words 10 Words20 Words Paraphrase Attack Variant 0.80 0.85 0.90 0.95 1.00 F1 Score (Higher is Better) SimHashBinaryShield Fig. 7: F1-score comparison betweenBinaryShieldand SimHash across attack variants.BinaryShieldoutperforms SimHash, particularly in complex paraphrasing scenarios, demonstrating superior robustness to semantic modifications. intent. This progression allows us to assess how each methodâs performance degrades as attack sophistication increases during threat correlation. Figure 5 and Figure 6 present comprehensive performance analysis through precision-recall curves and confusion matrices at optimal operating points. For minimal attack modifications (V-1 to V-10), both algorithms demonstrate exceptional detection capabilities with near-perfect performance (Figure 7). At optimal thresholds, both methods achieve optimal F1-scores (Figure 7), establishing a strong baseline for more complex evaluations. For instance, in 7 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) the V-10 scenario, SimHash produces 8 false positives and 40 false negatives at Hamming distance threshold (Ï) 19 (Figure 6), whileBinaryShieldgenerates only 4 false positives at threshold 276 (Figure 5). As attack complexity increases to extensive word modi- fications (V-20), performance differences begin to emerge. BinaryShieldmaintains robust performance with an F1- score of 0.99 (precision: 0.99, recall: 0.98), while SimHash experiences measurable degradation to an F1-score of 0.96 (precision: 0.98, recall: 0.93). The 3.3% gap starts to illustrate BinaryShieldâs superior stability as lexical modifications increase. The most challenging scenario involves comprehensive paraphrasing attacks (V-Paraphrase), where prompts undergo fundamental restructuring while preserving malicious intent. Here, the performance differential becomes most pronounced: BinaryShieldachieves an F1-score of 0.94 (precision: 0.92, recall: 0.95), while SimHash degrades significantly to an F1- score of 0.77 (precision: 0.75, recall: 0.80). This 17% gap in F1 score demonstratesBinaryShieldâs fundamental advantage in threat intelligence that transcends surface-level linguistic variations. Additionally, examining confusion matrices at optimal thresholds, provides additional insights into error patterns in Figures 5b and 6b.BinaryShielddemonstrates consis- tent performance as compared to SimHash. SimHash shows increasing error variability as attack complexity increases, struggling particularly to distinguish between semantically similar malicious content and unrelated benign prompts. The 17 percentage point F1-score gap in paraphrasing attacks, the most challenging and operationally relevant scenario, validates the effectiveness ofBinaryShield. These performance improvements are achieved while simultaneously providing privacy guarantees essential for cross-service threat intelligence. Summary.BinaryShieldconsistently outperforms SimHash in threat correlation, especially under complex para- phrasing attacks, achieving an F1-score of 0.94 compared to SimHashâs 0.77. These results highlightBinaryShieldâs superior robustness with enabling privacy, making it highly effective for cross-service threat intelligence. 1) Empirical Analysis of Randomized Response Impact: We quantify the impact of randomized response mechanism [12] noise on detection utility by sweeping the per-bit privacy budgetα â [0.25, 3.0](Algorithm 1, Step 4) on the most challenging paraphrase attack variant. For eachα, a bit in the 768-dimensional binary embedding is kept with probability p = e α e α +1 and flipped otherwise, yielding an expected self-Hamming distortion(1â p)d, wheredis the fingerprint dimension (here,d = 768). Atα = 0.25,p = 0.562and the mechanism flipsâ 0.438Ă 768 â 336bits (i.e., the whole binary vector is approximately completely randomized). As shown in the top-right confusion matrix of Figure 8, at this extreme privacy setting there are 821 false positives and 97 false negatives, resulting in an F1-score of only 0.65. Such 123 BinaryShield Privacy Parameter () [Lower = More Private] 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Score (Higher is Better) BinaryShield F1 SimHash F1 Dense Embeddings F1 123 BinaryShield Privacy Parameter () [Lower = More Private] 0.4 0.5 0.6 0.7 0.8 0.9 1.0 BinaryShield Precision BinaryShield Recall SimHash Precision SimHash Recall RegularPI Regular PI 139821 97863 = 0.25, F1 = 0.65 RegularPI Predicted Label Regular PI 92436 54906 = 3.0, F1 = 0.95 True LabelTrue Label Fig. 8: Privacy-utility trade-off for paraphrased prompt injection detection. (Left)BinaryShieldF1 rises smoothly with privacy budgetα, dominating SimHash (privacy-preserving baseline) across most of the spectrum and approaching non- private dense embeddings. (Center) Precision lags recall at high noise, then rapidly converges afterαâ1as FP collapse. Horizontal dashed lines: SimHash precision/recall. (Right) Confusion matrices at extreme settings: severe noise (α = 0.25) induces many FP; higher α (3) sharply improves precision. high noise levels provide maximal privacy but render the binary fingerprints nearly random (Figure 9), offering little practical utility for threat detection. Figure 8 shows that utility improves smoothly with larger αas noise shrinks. Atα = 1.0precision increases to 0.79 with high recall of 0.82. The inflection regionαâ[1.25, 1.75] marks the transition where precision catches up to recall: by α = 1.5 BinaryShieldattains F1=0.90, already exceeding SimHashâs paraphrase F1 (0.77) by +13 points while providing privacy that is absent in dense cosine embeddings. From α = 2.5onward the F1 curve is close to the dense embeddings while still providing privacy as bit flipping probability is not zero (Figure 9). The confusion matrices at the privacy extremes (Figure 8, right) illustrate this shift: moving from α = 0.25 to 3.0 reduces false positives 22.8Ă (821â36) and false negatives (97â54), yielding an approximately 30-point F1 gain (0.65â0.95). This predictable performance curve lets operators easily adjustαto meet privacy requirements, estimate detection accuracy, and securely share threat intelligence without revealing raw prompts or embeddings. Randomized Response and Hamming Distance. To empiri- cally validateBinaryShieldâs privacy model, we conducted a controlled experiment measuring the actual Hamming distance introduced by the BinaryShieldâs differential privacy. To this end, for 500 prompts from our evaluation dataset, BinaryShieldgenerated 768-dimensional binary finger- prints and applied its differential privacy transformation by varyingαvalues from 0.2 to 3.4. For each configuration, we computed the Hamming distance between original and noise- perturbed fingerprints, comparing observed distances against the theoretical expectation(1â p)d. As a baseline reference, we generated 1,000 pairs of independent random binary vectors to establish the average distance for completely uncorrelated binary vectors. The mean Hamming distance for these indepen- dent vectors is384.44± 13.65, which serves as a baseline for comparison. Figure 9 demonstrates that our theoretical model precisely predicts the observed self-Hamming distortion across all privacy levels confirmingBinaryShieldâs alignment 8 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 0.51.01.52.02.53.03.5 BinaryShield's Privacy Parameter () 50 100 150 200 250 300 350 400 Expected Hamming Distance Theoretical: (1-p)d Independent Random Binary Vectors Observed (Real Data) Fig. 9: Calibration of randomized-response noise. Empirical mean Hamming distances (blue points,n = 500each) between original and privatized 768-bit fingerprints tightly follow the theoretical curve(1â p)d(solid black) across privacy budgets α. The dashed red line shows the mean distance between independent random binary vectors (384), illustrating how increasingαmovesBinaryShieldsmoothly from near- randomization to low-distortion states. with theory. Atα = 0.2,BinaryShieldâs perturbed binary fingerprint reaches a Hamming distance of346± 13.41, nearly matching the random baseline. This results in lower F1 scores at small α (Figure 8). Asαincreases, the Hamming distance decreases to206.45± 13.08atα = 1.0. This decay directly correlates with the smooth utility recovery in Figure 8. The tight correspondence between theoretical predictions and observed measurements validates thatBinaryShieldâs privacy mechanism operates exactly as designed, providing operators with precise control over the privacy-utility trade-off through a single, well-characterized parameterα. Another noticeable observation is that even at higherαvalues, as shown in Figure 9, the Hamming distance remains nonzero. This indicates thatBinaryShield continues to provide privacy protection, since the nonzero Hamming distance ensures privacy is maintained, even as its utility approaches that of dense embeddings (Figure 8 and 9). In contrast, dense embeddings offer no privacy at all, so even a nonzero Hamming distance (i.e., some privacy) is fundamentally better than nothing, ensuring that sensitive information is not directly exposed. Summary.BinaryShieldexhibits the fundamental dif- ferential privacy trade-off: lowα(strong privacy) yields near-random fingerprints, whereas increasingα(meaningful privacy) produces a smooth, monotonic F1 rise that quickly approaches non-private baseline performance. C. Real-World Deployment Analysis While earlier sections validateBinaryShieldâs superior performance as compared to the privacy-preserving baseline (SimHash), we now turn to evaluatingBinaryShieldâs effectiveness in realistic cross-service threat intelligence sce- narios. Specifically, it must be shown that privacy-preserving correlation remains effective when (i) malicious prompts are sparse relative to benign traffic and (i) corpus size scales by an order of magnitude (i) privacy-utility trade-off is maintained, i.e., the accuracy gap betweenBinaryShieldand non- private dense embeddings remains bounded, and (iv) efficiency is maintained in terms of computational and storage overhead. 1) Large-Scale Threat Intelligence Scalability: To simu- late realistic enterprise conditions, we query attack finger- prints against corpora where benign prompts vastly out- number malicious ones, measuring Accuracy@k as suc- cessful detection when the true variant ranks within the top-k retrieved candidates. The goal is to quantify how BinaryShieldâs privacy-preserving fingerprint correlation accuracy behaves as the total corpus grows, and characterize the accuracy gap relative to a non-private dense embedding baseline. In this experiment,BinaryShielduses OpenAIâs text-embedding-3-large to generate embeddings. We study this setting using hybrid corpora that interleave a (fixed) malicious prompt injection (V-Paraphrase) with large volumes of real user interactions from WildChat [20], thereby emulating an enterprise log in which only a small fraction of entries are attacks. For each target corpus size C â10K, 20K,..., 100K, we construct a hybrid dataset by injecting a fixed malicious prompt set intoC â Mbenign WildChat prompts (whereMis the number of distinct attack prompts). Each malicious prompt is issued as a query; success is recorded if its exact counterpart (or semantically identical variant, depending onk) appears within the top-kcandidates when ranked by Hamming distance overBinaryShieldâs fingerprints, or cosine similarity for the dense baseline. We fix the local differential privacy parameter atα = 2(differentα values are also evaluated in Section I-C2). Figure 10 shows the results of threat intelligence scalability analysis, with the X-axis showing corpus sizes (10K to 100K) and the Y-axis representing retrieval accuracy. Accuracy@k represents the percentage of queries for which the correct malicious prompt is returned in the first k-results. Accuracy@1 for the dense baseline declines modestly by only 1.2% points across a 10Ăincrease in corpus size (86.5%â85.3%). BinaryShieldshows a similarly shallow decline (81.1% â79.2%; 1.9% points). This indicates that neither binary quantization nor randomized response mechanism introduces scale-sensitive degradation. Furthermore, Figure 13 (in the appendix) shows that the Accuracy@k gaps betweenBinaryShieldand the dense baseline with X-axis showing corpus sizes (10K to 100K) and Y-axis showing the gap at different top-k accuracy. The gap remains tightly bounded: it ranges from 5.31% to 6.15% (mean 5.66% points) atk = 1, indicating thatBinaryShield retains over 93% of the dense baselineâs accuracy even under tight local differential privacy constraints. Crucially, the gap does not significantly widen with corpus size. Furthermore, increasingkrapidly closes the gap, as shown in Figure 10 and Figure 13. For instance at 100K entries,BinaryShield achieves 79.2% Accuracy@1, 90.8% Accuracy@3, and 92.9% Accuracy@5. On the same 100K corpus, the dense embed- dings achieve 85.3% Accuracy@1, 94.7% Accuracy@3, and 96.5% Accuracy@5. The gap atk = 5is only 3.6% points, indicating thatBinaryShieldâs with privacy feature reaches 9 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 10k20k30k40k50k60k70k80k90k 100k 0 25 50 75 100 Accuracy (%) 86.5 86.4 86.186.1 86.0 85.8 85.7 85.6 85.5 85.3 81.1 80.9 80.6 80.380.380.3 80.1 80.0 79.7 79.2 Accuracy@1 10k20k30k40k50k60k70k80k90k 100k 93.1 93.0 92.992.992.992.9 92.8 92.692.6 92.4 89.4 89.1 88.8 88.6 88.488.4 88.3 88.2 88.1 88.0 Accuracy@2 10k20k30k40k50k60k70k80k90k 100k Corpus Size 95.1 95.0 94.994.994.9 94.894.894.8 94.794.7 92.0 91.9 91.791.791.7 91.4 91.191.191.1 90.8 Accuracy@3 10k20k30k40k50k60k70k80k90k 100k 96.4 96.1 96.096.096.0 95.9 95.895.895.895.8 92.7 92.6 92.592.592.592.5 92.4 92.3 92.2 92.1 Accuracy@4 10k20k30k40k50k60k70k80k90k 100k 96.9 96.7 96.8 96.7 96.696.696.6 96.5 96.6 96.5 93.693.6 93.4 93.393.393.3 93.293.2 93.0 92.9 Accuracy@5 Dense Embedding (No Privacy)BinaryShield Fig. 10: Scalability ofBinaryShield. Accuracy@k forBinaryShieldvs non-private dense embeddings baseline across hybrid corpora (10K-100K). BinaryShield retains 93% of baseline Accuracy@1 while providing privacy. close to the non-private baseline of dense embeddings. This confirms that the semantic signal preserved after (i) sign-only quantization and (i) per-bit randomized response still remains sufficiently rich for practical threat correlation. For every corpus size,BinaryShieldâs incremental gains fromk = 1to k = 2average roughly +8% (Figure 10). This reflects that most residual misses atk = 1are near-boundary semantic neighbors retrievable with minimal expansion. Subsequent marginal gains (k > 2) diminish smoothly. Figure 13 shows the gap curves monotonically compress askincreases. The gap is attributable to two irreversible transformations: (a) many- to-one sign mapping (loss of magnitude information) and (b) randomized response bit flips calibrated byα = 2. That the gap does not drift upward with scale empirically supports that these transformations inBinaryShieldbehave as scale- neutral perturbations rather than compounding sources of semantic erosion. In an operational setting where an analyst (or automated response engine) can examine the top 3â5 correlated candidates,BinaryShielddelivers privacy-preserving threat intelligence with accuracy comparable to a fully non-private dense embedding pipeline. This constitutes strong utility under privacy constraints and validatesBinaryShieldâs suitability for multi-service deployment. Summary.BinaryShieldretains over 93% accuracy of dense embeddings (non-private baseline) and remains scalable in practical deployment settings with randomized re- sponse mechanism, demonstrating robust privacy-preserving threat correlation at enterprise scale. 2)BinaryShieldâs Privacy Parameter Performance at Scale: The relationship between privacy protection and threat detection utility at scale represents a fundamental tension in the cross-boundary threat intelligence system. Similar to Sec- tionIII-B1, to characterize this trade-off precisely, we evaluate BinaryShieldâs performance across a spectrum of privacy budgets (α) on a fixed 50K-entry hybrid corpus containing both malicious prompts and benign WildChat interactions (SectionIII-C1). The privacy parameterαdirectly controls the randomized response mechanism through the bit preservation probabilityp = e α /(e α + 1), with the corresponding bit flip probability being1â p. Smallerαvalues provide stronger privacy guarantees by increasing the probability of bit flips 0.00.51.01.52.02.53.03.5 BinaryShield Privacy Budget () 0 20 40 60 80 100 Accuracy (%) (Higher is Better) BinaryShield (Privacy Preserving) @1 BinaryShield (Privacy Preserving) @5 Dense Embedding (No Privacy) @1 Dense Embedding (No Privacy) @5 Fig. 11: Privacy-utility trade-off forBinaryShieldacross αvalues on 50K hybrid corpus. Accuracy@1 (blue line) and Accuracy@5 (black line) show phase transition behavior as privacy budget increases. in the binary fingerprint, while larger values preserve more semantic information at the cost of reduced privacy protection. Our experimental results, visualized in Figure 11, reveal a sharp phase transition in detection accuracy as the privacy budget increases from highly restrictive to meaningful protec- tive settings. The X-axis represents the privacy parameterα, while the Y-axis shows correlation accuracy (Accuracy@1 and Accuracy@5) for bothBinaryShieldand the non-private dense embedding baseline. At extreme privacy levels (α = 0.1), the bit flip probability reaches 0.475, meaning nearly half of all bits are randomly inverted, effectively reducing fingerprints to near-random noise. Consequently, Accuracy@1 drops to 0%, rendering threat correlation impossible. This represents the theoretical limit where maximal privacy completely eliminates utility. As we relax the privacy constraint the bit flip probability decreases. Withα = 0.75(bit flip probability 0.32), Accuracy@1 rises to 14.17% and Accuracy@5 to 26.15%, marking the onset of effective threat detection. Atα = 1.0 BinaryShieldto reach 44.06% Accuracy@1 and 64.90% Accuracy@5. While this is roughly half the non-private baseline (86.04% and 96.56%), it shows that meaningful threat correlation is feasible even under strong privacy constraints. Betweenα = 1.0 andα = 2.0, accuracy improves smoothly with meaningful privacy and utility tradefoff. Atα = 1.25, Accuracy@1 reaches 62.40% and Accuracy@5 82.81%. Atα = 1.5, Accuracy@1 is 72.60% and Accuracy@5 is 89.90%, retaining 84.4% of 10 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) baseline Accuracy@1. The results show a smooth transition in the privacy-utility trade-off, especially betweenα = 0.75andα = 2.0. Since organizations have already strict security policies, for internal sharing, relaxing privacy toα = 2.0â2.5yields near-optimal utility with realistic meaningful privacy protection. Overall, BinaryShieldpotentially can operate in a wide range of privacy-utility trade-offs. Summary.BinaryShieldempowers organizations to balance privacy and utility as needed, consistently deliv- ering strong accuracy and realistic privacy protection. The organizations can flexibly tune the privacy-utility trade-off based on their operational and regulatory needs. 3)BinaryShieldComputational Efficiency Analysis: The operational viability of any threat intelligence system fundamentally depends on its ability to process vast quantities of data with minimal latency and storage overhead. Historical lessons from traditional cybersecurity infrastructure underscore this requirement. One of the major reasons that signature- based ecosystems (antivirus, IDS) scaled is because compact hashes (e.g., MD5, fuzzy-hash families). Similarly, modern LLM services face an even more daunting challenge, processing billions of queries daily across distributed infrastructure. For BinaryShieldto enable practical cross-boundary threat intelligence, it must achieve orders-of-magnitude improvements in both computational speed and storage efficiency compared to non-private dense embeddings. Every extra byte per fingerprint multiplies into material storage cost, and every millisecond of per-fingerprint scan latency stretches incident response time. Figure12revealscomputationaladvantagesof BinaryShieldâs fingerprints over dense embeddings. At a corpus size of 10K entries, dense embeddings require 0.87 seconds for similarity search of 968 prompt injections, whileBinaryShieldcompletes the same operation in just 0.032 seconds, a 26.7x speedup. This performance gap widens dramatically as corpus size increases. At 100K entries, dense embeddings demand 14.52 seconds, rendering real-time threat correlation infeasible for high-volume services. In contrast, BinaryShieldmaintains sub-second performance at 0.38 seconds, achieving a 38.1x speedup. The computational efficiency stems from the fundamental difference in similarity computation. Dense embeddings require floating-point dot products or cosine similarity calculations, operations that scale poorly even with optimized linear algebra libraries. BinaryShieldâs binary fingerprints enable Hamming distance computation through simple XOR operations. We discussBinaryShieldâs storage utilization in the appendix. The practical implications of these efficiency gains extend beyond raw performance metrics (SectionIII-BandIII-C1). Consider an organization processing 100 million queries daily across ten services. With dense embeddings, maintaining a rolling 30-day threat intelligence window would require approximately 7.1 TB of storage per boundary and dedicated GPU clusters for similarity search making it near to imprac- 10k20k30k40k50k60k70k80k90k100k Corpus Size 10 1 10 0 10 1 Search Time (seconds) (Lower is Better) 0.87 1.83 2.79 5.23 4.33 5.98 6.41 16.30 13.58 14.52 0.03 0.06 0.10 0.14 0.20 0.21 0.24 0.31 0.38 0.38 Dense Embedding (No Privacy) BinaryShield (Privacy Preserving) Fig. 12: Search time comparison between non-private dense embeddings andBinaryShieldacross corpus sizes. Note the logarithmic scale on the Y-axis.BinaryShieldmain- tains sub-second performance, demonstrating its suitability for real-time threat correlation at enterprise scale. tical choice.BinaryShieldreduces this to 111 GB per boundary, fitting comfortably in memory on commodity servers while enabling CPU-based similarity search. This efficiency democratizes threat intelligence capabilities, allowing even resource-constrained services to participate in collaborative defense without significant infrastructure investment. Furthermore, the computational efficiency enables new operational capabilities. Security teams can now perform retrospective threat hunting across months of historical data in minutes rather than hours. The reduced storage footprint also facilitates comprehensive threat intelligence archival, enabling long-term trend analysis and attribution of persistent attack campaigns that evolve over extended periods. Summary.BinaryShieldâs computational and storage ef- ficiency fundamentally transforms cross-boundary threat intel- ligence economics. By reducing search latency up to 38x and storage requirements while maintaining 93% of non-private baseline top-1 accuracy with privacy,BinaryShield enables enterprise-scale threat correlation. This positions BinaryShieldas a practical foundation for collaborative threat intelligence systems against prompt injection attacks. IV. RELATED WORK AND DISCUSSION Attacks. Several works have focused on systematically charac- terizing prompt injection attacks. For instance, [21] demonstrate how hidden instructions can be inserted into external content to trigger harmful model behaviors. Other works [17], [22]â[29] further contribute to our understanding by exploring a variety of prompt injection scenarios. In addition, [30] proposeSOS, a training-time âsoft promptâ attack that implants backdoors into open-source LLMs. The LLM behaves normally, until a trigger token activates malicious behaviors like jailbreaks, prompt stealing, or output manipulation. In contrast, [31] introduceCrescendo, an inference-time jailbreak attack that unfolds over multiple dialogue turns. Supporting this observation, [32] performs an in-depth analysis of such multi- turn jailbreak attacks and show why defenses designed for single-turn interactions, like circuit breakers [33], may be insufficient. By examining internal model representations, [32] shows that multi-turn prompts can gradually reframe harmful 11 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) outputs as safe, effectively evading safeguards designed for single-turn interactions.BIPIA[34] systematically assesses the risks posed by malicious instructions embedded in third- party content and introduces two defense paradigms: one based on prompt engineering and in-context strategies in a black-box setting, and another using adversarial training with special tokens in a white-box setting. Finally, [3] introduces a realistic testbed for indirect prompt injection attacks targeting an LLM-based email assistant. Collectively, these studies [3], [17], [21]â[32], [34] provide a comprehensive overview of the diverse strategies employed to exploit prompt injection attacks, underscoring the ongoing need for robust security measures in LLM-integrated applications and services. Attack Defenses. Recent work proposes a variety of defense strategies against prompt injection attacks. These approaches differ in their methods and operating settings, ranging from inference-time countermeasures to white-box techniques and architectural safeguards.RigorLLM[35] framework enhances content moderation. Hines et al. [36] build on the work of [34] to address indirect prompt injection attacks by proposing a defense strategy based on spotlighting.CaMeL[4] is a defense mechanism that explicitly detects and prevents prompt injection attacks by designing a secure execution layer around LLM operations. The work sits at the intersection of generative AI and software security, bringing established principles from the latter to protect emerging systems. SafeEar [37] proposes a method for detecting deepfake audio without accessing the actual speech content.TaskTracker[2] is a white-box defense, that centers on tracking the modelâs internal activations before and after processing external text.DataSentinel[38] detects prompt injection attacks by fine-tuning a language model with a game-theoretic mini-max optimization. It uses a detection instruction embedding a secret key to verify the integrity of incoming data. Beurer-Kellner et al. [39] propose a six architectural design patterns (action-selector, plan-then-execute, map-reduce, dual LLM, code-then-execute, and context-minimization) that secure LLM agents against attacks. Overall, most existing defenses are probabilistic and can be bypassed [4], [5]. Retrospective systems are needed to identify attackers, yet cross-service threat intelligence for large- scale prompt injection attacks correlation remains unexplored. To address this gap, techniques like SimHash can be leveraged. However, SimHash captures only syntactic similarity and fails to capture semantic similarity (Figures 5, 6, and 7). BinaryShieldis the first to focus on this gap, proposing a concrete system for cross-service threat intelligence and paving the way for future research in this domain. A. Discussion Our study has limitations similar to those in any research. Synthetic paraphrasing and word substitution may yield varying results with different LLMs and prompts. The effectiveness of theBinaryShieldâs PII redaction module depends on the accuracy of detection methods. Residual PII may remain in prompts. Our results rely on specific embedding models, Mod- ernBert (open-source) [18], [19] and OpenAI text-embedding- 3-large (proprietary). Thus, performance may vary with other models. We determined optimal Hamming distance thresholds through search on our datasets. However, these thresholds may not generalize to other attack distributions or organizational contexts. All experiments contain English prompts. Future work should explore cross-lingual attack detection and domain- specific technical prompts. We depend on computational non- invertibility and differential privacy guarantees. The evaluation does not assess resistance to reconstruction attacks, which represent an orthogonal concern to current focus of the paper. Differential Privacy. Noise can be added to the fingerprint through various differential privacy mechanisms [12]â[14], [40]â[57].BinaryShieldutilizes foundational work of Randomized Response mechanism [12]. Assessing the impact of other DP mechanisms and global differential privacy on the overall privacy guarantees is an important direction for future research. We did not formulate the privacy utility trade off as an optimization problem. Readers may consult existing literature on this topic [14], [52]â[54]. However, in our experiments, we empirically evaluated conditions ranging from extreme privacy, where accuracy approaches zero, to moderate and weak privacy. Each organization has specific privacy policies, and we do not recommend a particular differential privacy budget. Organizations should work with their privacy teams to select an appropriate budget based on their guidelines. V. CONCLUSION Regulatory boundaries prevent an enterpriseâs multiple LLM services from sharing prompt injection threat intelligence. This isolation means an attack blocked by one service can still succeed against another, as each service has its own defense mechanism. This creates a fundamental gap in the organizationâs collective defense capabilities. BinaryShield addresses this critical gap by introducing the first privacy-preserving fingerprinting mechanism specifi- cally designed for prompt injection threat intelligence. Through a carefully orchestrated pipeline combining PII redaction, semantic embedding, binary quantization, and randomized re- sponse,BinaryShieldgenerates privacy-preserving finger- prints that can be safely shared across compliance boundaries while preserving the semantic characteristics necessary for effective threat correlation. The implications of this work extend beyond immediate operational benefits. As LLM-based systems become critical infrastructure across finance, healthcare, and government services, the attack surface for prompt injection continues to expand exponentially. The emergence of autonomous agents and MCP servers further amplifies these risks, potentially enabling attacks that cascade from text manipulation to arbitrary code execution and system compromise.BinaryShield establishes the foundational system for collaborative defense against these evolving threats, creating a pathway toward industry-wide threat intelligence feeds analogous to existing malware signature ecosystems. 12 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) VI. LLM USAGE CONSIDERATIONS We utilized GPT-4o to systematically generate the attack variants (word-flipped and paraphrased) for our evaluation. This approach was necessary because no public dataset exists for cross-service threat intelligence analysis and enterprise data access is restricted. To address this gap, we used GPT- 4o to create a partially synthetic dataset from original prompt injection attacks. The generation was a one-time, offline process using efficient prompts (see Appendix Listing 1 and Listing 2) to minimize the computational footprint. While using a closed- source model affects exact reproducibility, our generation process is transparent, and the resulting dataset characteristics are replicable with other powerful LLMs. We also briefly discuss this limitation in Section IV-A. We also used LLMs to correct grammar mistakes, similar to a feature like Grammarly. The whole content in this study is written entirely by the authors, who take full responsibility for the paper. ACKNOWLEDGMENT We sincerely thank the SaTML reviewers for their valuable feedback. We are also deeply grateful to Yonatan Zunger, An- gela Argentati, Dharmin Shah, Sukirna Roy, Habiba Mohamed, Anandan Sundar, Leah Zulas, and the entire Microsoft AI Safety and Security organization for their support. REFERENCES [1] T. H. News. Zero-Click AI Vulnerability Exposes Microsoft 365 Copilot Data Without User Interaction â thehackernews.com. https: //thehackernews.com/2025/06/zero-click-ai-vulnerability-exposes.html. [Accessed 11-08-2025]. [2] S. Abdelnabi, A. Fay, G. Cherubin, A. Salem, M. Fritz, and A. Paverd, âGet My Drift? Catching LLM Task Drift with Activation Deltas,â in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025, p. 43â67. [3] S. Abdelnabi, A. Fay, A. Salem, E. Zverev, K.-C. Liao, C.-H. Liu, C.-C. Kuo, J. Weigend, D. Manlangit, A. Apostolov, H. Umair, J. Donato, M. Kawakita, A. Mahboob, T. H. Bach, T.-H. Chiang, M. Cho, H. Choi, B. Kim, H. Lee, B. Pannell, C. McCauley, M. Russinovich, A. Paverd, and G. Cherubin, âLLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge,â 2025. [Online]. Available: https://arxiv.org/abs/2506.09956 [4]E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. TramĂšr, âDefeating Prompt Injections by Design,â CoRR, vol. abs/2503.18813, March 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2503.18813 [5]M. Costa, B. Köpf, A. Kolluri, A. Paverd, M. Russinovich, A. Salem, S. Tople, L. Wutschitz, and S. Zanella-BĂ©guelin, âSecuring AI Agents with Information-Flow Control,â 2025. [Online]. Available: https://arxiv.org/abs/2505.23643 [6]V. G. Li, M. Dunn, P. Pearce, D. McCoy, G. M. Voelker, and S. Savage, âReading the tea leaves: A comparative analysis of threat intelligence,â in 28th USENIX Security Symposium (USENIX Security 19).Santa Clara, CA: USENIX Association, Aug. 2019, p. 851â867. [Online]. Available: https://w.usenix.org/conference/usenixsecurity19/presentation/li [7]âLlm01:2025 prompt injection,â Apr 2025. [Online]. Available: https://genai.owasp.org/llmrisk/llm01-prompt-injection/ [8] P.Walsh,âAISystemsAndVectorDatabasesAre GeneratingNewPrivacyRisksâforbes.com,âhttps: //w.forbes.com/councils/forbestechcouncil/2023/11/02/ ai-systems-and-vector-databases-are-generating-new-privacy-risks/, [Accessed 12-08-2025]. [9] H. Li, M. Xu, and Y. Song, âSentence embedding leaks more information than you expect: Generative embedding inversion attack to recover the whole sentence,â in Findings of the Association for Computational Linguistics: ACL 2023, 2023, p. 14 022â14 040. [10] A. Tragoudaras, T. Aslanidis, E. G. Lionis, M. Orozco GonzĂĄlez, and P. Eustratiadis, âInformation leakage of sentence embeddings via generative embedding inversion attacks,â in Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2025, p. 3234â3243. [11] OWASP2025,âLLM08:2025VectorandEmbed- dingWeaknesses,âhttps://genai.owasp.org/llmrisk/ llm082025-vector-and-embedding-weaknesses/,2025,[Accessed 11-08-2025]. [12]S. L. Warner, âRandomized Response: A Survey Technique for Eliminat- ing Evasive Answer Bias,â Journal of the American statistical association, vol. 60, no. 309, p. 63â69, 1965. [13]U. Erlingsson, V. Pihur, and A. Korolova, âRAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response,â in Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS â14.New York, NY, USA: Association for Computing Machinery, 2014, p. 1054â1067. [Online]. Available: https://doi.org/10.1145/2660267.2660348 [14] T.Wang,J.Blocki,N.Li,andS.Jha,âLocally Differentially Private Protocols for Frequency Estimation,â in 26thUSENIXSecuritySymposium(USENIXSecurity17). Vancouver, BC: USENIX Association, Aug. 2017, p. 729â745. [Online]. Available: https://w.usenix.org/conference/usenixsecurity17/ technical-sessions/presentation/wang-tianhao [15]G. S. Manku, A. Jain, and A. Das Sarma, âDetecting near-duplicates for web crawling,â in Proceedings of the 16th International Conference on World Wide Web, ser. W â07. New York, NY, USA: Association for Computing Machinery, 2007, p. 141â150. [Online]. Available: https://doi.org/10.1145/1242572.1242592 [16]Microsoft, âHome - microsoft presidio,â [Online; accessed 2025-08-24]. [Online]. Available: https://microsoft.github.io/presidio/ [17]X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, â"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models,â in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, ser. CCS â24.New York, NY, USA: Association for Computing Machinery, 2024, p. 1671â1685. [Online]. Available: https://doi.org/10.1145/3658644.3670388 [18]B. Warner, A. Chaffin, B. ClaviĂ©, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, G. T. Adams, J. Howard, and I. Poli, âSmarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference,â in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, Eds.Vienna, Austria: Association for Computational Linguistics, Jul. 2025, p. 2526â2547. [Online]. Available: https://aclanthology.org/2025.acl-long.127/ [19] W. Gill, J. Cechmanek, T. Hutcherson, S. Rajamohan, J. Agarwal, M. A. Gulzar, M. Singh, and B. Dion, âAdvancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data,â 2025. [Online]. Available: https://arxiv.org/abs/2504.02268 [20]W. Zhao, X. Ren, J. Hessel, C. Cardie, Y. Choi, and Y. Deng, âWildChat: 1M ChatGPT Interaction Logs in the Wild,â in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=Bl8u7ZRlbM [21]K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, âNot What Youâve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,â in Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, ser. AISec â23.New York, NY, USA: Association for Computing Machinery, 2023, p. 79â90. [Online]. Available: https://doi.org/10.1145/3605764.3623985 [22] A. Wei, N. Haghtalab, and J. Steinhardt, âJailbroken: How Does LLM Safety Training Fail?â in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=jA235JGM09 [23]A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, âUniversal and Transferable Adversarial Attacks on Aligned Language Models,â 2023. [Online]. Available: https://arxiv.org/abs/2307.15043 [24]Y. Zhang, N. Carlini, and D. Ippolito, âEffective Prompt Extraction from Language Models,â in First Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?id=0o95CVdNuz [25]G. Deng, Y. Liu, Y. Li, K. Wang, Y. Zhang, Z. Li, H. Wang, T. Zhang, and Y. Liu, âMASTERKEY: Automated Jailbreaking of Large Language 13 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) Model Chatbots,â in Proceedings 2024 Network and Distributed System Security Symposium, ser. NDSS 2024.Internet Society, 2024. [Online]. Available: http://dx.doi.org/10.14722/ndss.2024.24188 [26]Y. Huang, S. Gupta, M. Xia, K. Li, and D. Chen, âCatastrophic Jailbreak of Open-source LLMs via Exploiting Generation,â in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=r42tSSCHPh [27]X. Liu, N. Xu, M. Chen, and C. Xiao, âAutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models,â in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=7Jwpw4qKkb [28]Y. Deng, W. Zhang, S. J. Pan, and L. Bing, âMultilingual Jailbreak Challenges in Large Language Models,â in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=vESNKdEMGp [29]P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, âJailbreaking Black Box Large Language Models in Twenty Queries,â in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025, p. 23â42. [30]Z. Yang, M. Backes, Y. Zhang, and A. Salem, âSOS! Soft Prompt Attack Against Open-Source Large Language Models,â CoRR, vol. abs/2407.03160, 2024. [Online]. Available: https://doi.org/10.48550/ arXiv.2407.03160 [31]M. Russinovich, A. Salem, and R. Eldan, âGreat, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack,â 2025. [Online]. Available: https://arxiv.org/abs/2404.01833 [32]B. Bullwinkel, M. Russinovich, A. Salem, S. Zanella-Beguelin, D. Jones, G. Severi, E. Kim, K. Hines, A. Minnich, Y. Zunger et al., âA Representation Engineering Perspective on the Effectiveness of Multi- Turn Jailbreaks,â arXiv preprint arXiv:2507.02956, 2025. [33]A. Zou, L. Phan, J. Wang, D. Duenas, M. Lin, M. Andriushchenko, J. Z. Kolter, M. Fredrikson, and D. Hendrycks, âImproving Alignment and Robustness with Circuit Breakers,â in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available: https://openreview.net/forum?id=IbIB8SBKFV [34]J. Yi, Y. Xie, B. Zhu, E. Kiciman, G. Sun, X. Xie, and F. Wu, âBenchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models,â in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, ser. KDD â25.New York, NY, USA: Association for Computing Machinery, 2025, p. 1809â1820. [Online]. Available: https://doi.org/10.1145/3690624.3709179 [35] Z. Yuan, Z. Xiong, Y. Zeng, N. Yu, R. Jia, D. Song, and B. Li, âRigorLLM: Resilient Guardrails for Large Language Models against Undesired Content,â in Forty-first International Conference on Machine Learning, 2024. [Online]. Available: https: //openreview.net/forum?id=QAGRPiC3FS [36] K. Hines, G. Lopez, M. Hall, F. Zarfati, Y. Zunger, and E. Kiciman, âDefending Against Indirect Prompt Injection Attacks With Spotlighting,â in Proceedings of the Conference on Applied Machine Learning in Information Security (CAMLIS 2024), Arlington, Virginia, USA, October 24-25, 2024, ser. CEUR Workshop Proceedings, R. Allen, S. Samtani, E. Raff, and E. M. Rudd, Eds., vol. 3920. CEUR-WS.org, 2024, p. 48â62. [Online]. Available: https://ceur-ws.org/Vol-3920/paper03.pdf [37] X. Li, K. Li, Y. Zheng, C. Yan, X. Ji, and W. Xu, âSafeEar: Content Privacy-Preserving Audio Deepfake Detection,â in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, ser. CCS â24.New York, NY, USA: Association for Computing Machinery, 2024, p. 3585â3599. [Online]. Available: https://doi.org/10.1145/3658644.3670285 [38]Y. Liu, Y. Jia, J. Jia, D. Song, and N. Z. Gong, âDataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks,â in 2025 IEEE Symposium on Security and Privacy (SP).IEEE, 2025, p. 2190â2208. [39]L. Beurer-Kellner, B. Buesser, A.-M. Cre ̧tu, E. Debenedetti, D. Dobos, D. Fabian, M. Fischer, D. Froelicher, K. Grosse, D. Naeff, E. Ozoani, A. Paverd, F. TramĂšr, and V. Volhejn, âDesign Patterns for Securing LLM Agents against Prompt Injections,â 2025. [Online]. Available: https://arxiv.org/abs/2506.08837 [40]A. Bhowmick, J. Duchi, J. Freudiger, G. Kapoor, and R. Rogers, âProtection Against Reconstruction and Its Applications in Private Federated Learning,â arXiv preprint arXiv:1812.00984, 2018. [41]O. Feyisetan and S. Kasiviswanathan, âPrivate Release of Text Embedding Vectors,â in Proceedings of the First Workshop on Trustworthy Natural Language Processing, 2021, p. 15â27. [42] S. Vadhan and T. Wang, âConcurrent Composition of Differential Privacy,â in Theory of Cryptography Conference, 2021, p. 582â604. [43]R. S. Carvalho, T. Vasiloudis, and O. Feyisetan, âBRR: Preserving Privacy of Text Data Efficiently on Device,â arXiv preprint arXiv:2107.07923, 2021. [44]D. Bollegala, S. Otake, T. Machide, and K.-i. Kawarabayashi, âA Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings,â in Findings of the Association for Computational Linguis- tics: IJCNLP-AACL 2023 (Findings), 2023, p. 65â79. [45] C. Xie, Z. Lin, A. Backurs, S. Gopi, D. Yu, H. A. Inan, H. Nori, H. Jiang, H. Zhang, Y. T. Lee et al., âDifferentially Private Synthetic Data via Foundation Model APIs 2: Text,â in International Conference on Machine Learning. PMLR, 2024, p. 54 531â54 560. [46]C. Meehan, K. Mrini, and K. Chaudhuri, âSentence-level Privacy for Document Embeddings,â in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, p. 3367â3380. [47] H. Asi, V. Feldman, and K. Talwar, âOptimal Algorithms for Mean Estimation under Local Differential Privacy,â in International Conference on Machine Learning. PMLR, 2022, p. 1046â1056. [48]M. Du, X. Yue, S. S. Chow, and H. Sun, âSanitizing Sentence Embeddings (and Labels) for Local Differential Privacy,â in Proceedings of the ACM Web Conference 2023, 2023, p. 2349â2359. [49]A. Bittau, U. Erlingsson, P. Maniatis, I. Mironov, A. Raghunathan, D. Lie, M. Rudominer, U. Kode, J. Tinnes, and B. Seefeld, âProchlo: Strong Privacy for Analytics in the Crowd,â in Proceedings of the 26th Symposium on Operating Systems Principles, ser. SOSP â17. New York, NY, USA: Association for Computing Machinery, 2017, p. 441â459. [Online]. Available: https://doi.org/10.1145/3132747.3132769 [50]G. Fanti, V. Pihur, and Ă. Erlingsson, âBuilding a RAPPOR with the Unknown: Privacy-Preserving Learning of Associations and Data Dictionaries,â Proceedings on Privacy Enhancing Technologies, vol. 3, p. 41â61, 2016. [51]J. Hsu, M. Gaboardi, A. Haeberlen, S. Khanna, A. Narayan, B. C. Pierce, and A. Roth, âDifferential Privacy: An Economic Method for Choosing Epsilon,â in 2014 IEEE 27th Computer Security Foundations Symposium. IEEE, 2014, p. 398â410. [52]J. P. Near and C. Abuah, Programming Differential Privacy, 2021, vol. 1. [Online]. Available: https://programming-dp.com/ [53]F. Fioretto, P. Van Hentenryck, and J. Ziani, âDifferential Pri- vacy Overview and Fundamental Techniques,â arXiv preprint arXiv:2411.04710, 2024. [54]C. Dwork, A. Roth et al., âThe Algorithmic Foundations of Differential Privacy,â Foundations and trendsÂź in theoretical computer science, vol. 9, no. 3â4, p. 211â407, 2014. [55]B. Balle and Y.-X. Wang, âImproving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising,â in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. PMLR, 10â15 Jul 2018, p. 394â403. [Online]. Available: https://proceedings.mlr.press/v80/balle18a.html [56]B. Ding, J. Kulkarni, and S. Yekhanin, âCollecting Telemetry Data Privately,â in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30.Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.c/paper_ files/paper/2017/file/253614bbac999b38b5b60cae531c4969-Paper.pdf [57]Y. Hu, F. Wu, Q. Li, Y. Long, G. M. Garrido, C. Ge, B. Ding, D. Forsyth, B. Li, and D. Song, âSoK: Privacy-Preserving Data Synthesis,â in 2024 IEEE Symposium on Security and Privacy (SP), 2024, p. 4696â4713. [58]L. Jiang, K. Rao, S. Han, A. Ettinger, F. Brahman, S. Kumar, N. Mireshghallah, X. Lu, M. Sap, Y. Choi, and N. Dziri, âWildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models,â in Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A.Fan,U.Paquet,J.Tomczak,andC.Zhang,Eds., vol. 37.Curran Associates, Inc., 2024, p. 47 094â47 165. [Online]. Available: https://proceedings.neurips.c/paper_files/paper/ 2024/file/54024fca0cef9911be36319e622cde38-Paper-Conference.pdf [59]W. Luo, S. Ma, X. Liu, X. Guo, and C. Xiao, âJailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks,â in First Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?id= GC4mXVfquq 14 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) [60]J. Piet, X. Huang, D. Jacob, A. Chow, M. Alrashed, G. Zhao, Z. Hu, C. Sitawarin, B. Alomair, and D. Wagner, âJailbreaksovertime: Detecting jailbreak attacks under distribution shift,â in Proceedings of the 18th ACM Workshop on Artificial Intelligence and Security, ser. AISec â25. New York, NY, USA: Association for Computing Machinery, 2026, p. 230â241. [Online]. Available: https://doi.org/10.1145/3733799.3762981 APPENDIX A. Threat Model An organization offers several siloed LLM services, each compartmentalized with distinct stacks and compliance bound- aries. An adversary interacts with these services via standard prompt interfaces (Figure 1). The adversary has no access to internal artifacts (such as logs, fingerprints, or defense details) and cannot tamper with compliance boundaries. The adversary sends prompt-injection attack variants to exploit vulnerabilities across these services. InBinaryShield, if a prompt injection attack is detected in one of these services, the goal is to share this threat intelligence with other peer services while preserving privacy.BinaryShieldshares a privacy-preserving fingerprint of the detected attack, which peer services can correlate in a privacy-preserving manner, thereby improving organization-wide visibility of attacks. B. Motivating Example The motivation in context of compliance boundaries is discussed in earlier sections (Section I and Section I). Here is another motivating example to demonstrateBinaryShield effectiveness. Consider a cloud provider offering generative AI services to multiple independent enterprise customers, hereafter referred to as Company A and Company B. Note in this scenario companies are customers of the cloud provider. Due to strict regulatory requirements and contractual obligations, these two customers operate in completely separate compliance boundaries. Their data cannot be shared, and their logging systems are entirely siloed. Now suppose a malicious actor launches prompt injection attacks against both companiesâ AI services. Because the data between Company A and Company B cannot be correlated or shared directly, each companyâs security team operates in isolation. They may detect anomalies independently, but they have no way to identify that the same attacker is targeting both services, nor can they assess the full scale and sophistication of the attack campaign. This is whereBinaryShieldprovides value. By gen- erating privacy-preserving fingerprints of detected malicious prompts, BinaryShield enables the cloud provider to cor- relate threat intelligence across these two isolated compliance boundaries without exposing the original prompts or violating privacy regulations. Understand this adversarial prompt: <original prompt>. Now generate synonyms for the following words: <words to change>. Return the JSON with the field â words_changes_toâ containing the synonyms in order. Listing 1: Template for generating adversarial prompt variants via lexical substitution. TABLE I: Performance comparison on the WildJailbreak dataset. MetricDense (No Privacy) BinaryShieldGap Accuracy@199.25%93.82%5.43% Accuracy@299.70%97.54%2.16% Accuracy@399.70%98.54%1.16% Accuracy@599.90%99.3%0.6% 10k20k30k40k50k60k70k80k90k100k Corpus Size 0 2 4 6 8 10 Accuracy Gap (%) (Lower is Better) Gap@1Gap@2Gap@3Gap@4Gap@5 Fig.13:Accuracy@kgap(DenseEmbeddings- BinaryShield) across corpus sizes. Rapid narrowing for higher k demonstrate that binary quantization plus local DP introduce a constant, not compounding, utility cost. Paraphrase the following adversarial prompt while maintaining the attack intent and purpose. Original prompt: <original prompt>. Return the JSON with the field âgenerated_responseâ. Listing 2: Template for generating adversarial prompt variants via full-sentence paraphrasing. C. Synthetic Variant Attack Potency and Evaluation on Wild- Jailbreak Dataset We conduct an evaluation to test the attack potency of our synthetic variants, specifically examining whether they maintain attack potency comparable to original attacks. We conduct a study comparing 500 original attack prompts [17] and their synthetic variants (V_Paraphrase) using GPT-4.1-nano combined with Azureâs Prompt Shields for attack detection in an enterprise setting. Original attacks achieve an 80.4% detection rate, while synthetic variants achieve a 78.0%. These near-identical detection rates demonstrate that the synthetic variants preserve adversarial characteristics comparable to the original attacks. Several jailbreak datasets exist [17], [58]â[60]. We evaluate BinaryShieldon [17] in Section I and on WildJail- break [58] here similar to Section I-C. Table I confirms BinaryShieldâs robustness on WildJailbreak. It achieves 93.82% Accuracy@1 on a 100K corpus of benign and adver- sarial prompts, retaining 94.5% of the non-private baseline, consistent with our earlier results (Figures 10 & 13). 15 Accepted at the 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 10k20k30k40k50k60k70k80k90k100k Corpus Size 10 1 10 2 10 3 Storage Size (MB) (Lower is Better) 256.88 491.25 725.62 960.00 1194.38 1428.75 1663.12 1897.50 2131.88 2366.25 4.01 7.68 11.34 15.00 18.66 22.32 25.99 29.65 33.31 36.97 Dense Embedding BinaryShield Fig.14:Storagesizecomparisondemonstrating BinaryShieldâs 64x reduction compared to dense embeddings. Dense embeddings scale from 256.88 MB at 10K entries to 2,366.25 MB at 100K entries, while BinaryShieldrequires only 4.01 MB to 36.97 MB respectively,enablingcost-effectivelong-termthreat intelligence retention. D. Storage Efficiency The storage requirements present an equally compelling case forBinaryShieldâs architecture, as illustrated in Figure 14. Dense embeddings consume 256.88 MB for 10K entries, growing linearly to 2,366.25 MB at 100K entries. This repre- sents a substantial infrastructure burden when organizations must maintain threat intelligence databases spanning billions of historical queries across multiple compliance boundaries. BinaryShieldreduces storage requirements by a factor of 64, requiring only 4.01 MB for 10K entries and 36.97 MB for 100K entries. Our reported storage figures use 64-bit floating- point precision. Storage requirements would be halved with the more commonly used 32-bit precision. This 64x reduction directly translates from the binary quantization process, where each embedding dimension is compressed from a floating-point value to a single bit, followed by the addition of noise through randomized response, which preserves the binary nature. 16