Paper deep dive

Quantization Enables Private Dense Retrieval against Malicious Service Providers

Louis Tremblay Thibault, Sofiane Azogagh, Marc-Olivier Killijian, Ulrich Aïvodji

Year: 2026Venue: arXiv preprintArea: cs.CRType: PreprintEmbeddings: 107

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Dense retrieval, the key component of Retrieval Augmented Generation (RAG), retrieves the most relevant documents by comparing dense vector representations of queries and passages from a large corpus. In privacy-sensitive applications, the server observes the query and controls which evidence is returned, creating both confidentiality and integrity risks. We formulate private dense retrieval as providing query privacy and retrieval integrity against a malicious server, and develop a two-round cryptographic protocol that provides both guarantees. Our protocol reduces private and verifiable retrieval to multiplication of a committed matrix by an encrypted vector and uses low-bit quantization to make this computation practical. We evaluate the resulting trade-off between cryptographic cost, retrieval quality, and downstream RAG accuracy across six embedding models, four language models, and corpora of up to 2.68 million passages. Our results show that, with a clipped quantizer, three-bit quantization largely preserves retrieval quality and downstream accuracy, while a private query over a corpus the size of a clinical reference requires one to three minutes of server time. These results suggest that private dense retrieval is already practical for moderately sized, privacy-sensitive corpora when minute-scale latency is acceptable.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

103,786 characters extracted from source content.

Expand or collapse full text

Quantization Enables Private Dense Retrieval against Malicious Service Providers Louis Tremblay Thibault Sofiane Azogagh École de technologie supérieure and Mila Eurecom Marc-Olivier Killijian Ulrich Aïvodji Université du Québec à Montréal École de technologie supérieure and Mila Abstract Dense retrieval, the key component of Retrieval Augmented Generation (RAG), retrieves the most relevant documents by comparing dense vector representations of queries and passages from a large corpus. In privacy-sensitive applications, the server observes the query and controls which evidence is returned, creating both confidentiality and integrity risks. We formulate private dense retrieval as providing query privacy and retrieval integrity against a malicious server, and develop a two-round cryptographic protocol that provides both guarantees. Our protocol reduces private and verifiable retrieval to multiplication of a committed matrix by an encrypted vector and uses low-bit quantization to make this computation practical. We evaluate the resulting trade-off between cryptographic cost, retrieval quality, and downstream RAG accuracy across six embedding models, four language models, and corpora of up to 2.68 million passages. Our results show that, with a clipped quantizer, three-bit quantization largely preserves retrieval quality and downstream accuracy, while a private query over a corpus the size of a clinical reference requires one to three minutes of server time. These results suggest that private dense retrieval is already practical for moderately sized, privacy-sensitive corpora when minute-scale latency is acceptable. 1 Introduction Dense retrieval is a central paradigm for large-scale information access. Bi-encoder models (Karpukhin et al., 2020; Reimers & Gurevych, 2019; Ni et al., 2022) represent documents and queries as vectors in a shared embedding space and answer queries by nearest-neighbor search. This is the mechanism underlying semantic search, open-domain question answering and the retrieval stage of retrieval-augmented generation (RAG) (Lewis et al., 2020). For exact retrieval by inner-product scoring, the interaction can be expressed as a matrix-vector product: the server holds an embedding matrix ∈ℝm×nA ^m× n, where each row represents a document, and the user submits a query embedding ∈ℝnx ^n. In such a setting, retrieval reduces to computing the score vector Ax and fetching the highest-scoring documents. However, this interaction exposes query embeddings to the server in the clear, potentially revealing sensitive information about the user’s intent. In particular, these embeddings can be linked across queries and inverted to recover information about the underlying text (Morris et al., 2023; Song & Raghunathan, 2020). Consider clinical decision support, where AI-assisted synthesis of the medical literature is increasingly common11 1 39%39\% of US physicians report incorporating summaries of medical research and standards of care into practice, up from 13%13\% two years earlier (American Medical Association, 2026). and physicians submit queries to tools such as UpToDate and OpenEvidence, which report reaching over three million clinicians worldwide and a majority of US physicians, respectively (Wolters Kluwer, 2026a; NBC News, 2026). Every such query is visible to the provider, to a compromised insider or to anyone who breaches the service or its logs.22 2 Medical IT providers are attractive targets for cyberattacks in practice, as in the 2026 Cegedim Santé incident that exposed 15 million medical files (Agence France-Presse, 2026). By cross-referencing with a clinician’s appointment schedule, an adversary could link individual patients to the topics their physician searched and reconstruct a working diagnosis. Such linkage can disclose protected health information subject to the HIPAA Privacy Rule (Office for Civil Rights, 2002). Beyond query privacy, the client also needs guarantees on the integrity of the retrieved evidence. Indeed, the server currently decides what evidence reaches the user or the downstream language model. One that silently rescores documents or substitutes the returned passage can steer diagnoses or generated answers, and an encrypted-but-unverified protocol gives the clinician no way to notice. Existing systems provide one of these two guarantees but not both, and those that encrypt the query assume an “honest-but-curious” server, which contradicts the threat model that motivates encryption in the first place (§3). Retrieval integrity also bounds indirect prompt injection (Greshake et al., 2023) through the retrieval channel: once a corpus has been audited and committed, neither the provider nor an attacker with write access to the corpus can slip adversarial instructions into the passages that reach the reader, since the client accept only content consistent with the audited digest (§2). Figure 1: Private dense retrieval. Round 1 returns encrypted scores, from which the client selects i∗i^*; Round 2 fetches i∗D_i^*. Each proof is verified against the committed A or D before decryption. Background. Homomorpic encryption (HE) (Gentry, 2009) is a form of encryption that allows computations on encrypted data without decrypting it first. The party holding the encryption key can decrypt the result of the computation, but the party performing the computation learns nothing about the underlying data. Over nearly two decades, HE has evolved from a theoretical topic to a vast field of privacy-preserving applications. For example, HE has been used to enable Private Information Retrieval (PIR) (Chor et al., 1998), where a client can retrieve an item from a server’s database without revealing to the server which item it is has retrieved. It is crucial to understand that homomorphically encrypted ciphertexts must contain noise for security. If not carefully managed, noise accumulation over homomorphic operations can lead to incorrect computation results. As such, smaller noise allows for more operations over the ciphertexts, but guarantees a lower security level. This trade-off is central to our work and we perform in §4.2 the careful balancing act of choosing cryptographic parameters that provide adequate levels of both security and correctness for the tasks at hand. Our approach. Our starting point is the observation that the two stages of dense retrieval, scoring and document fetch, reduce to a product between a public matrix held by the server (e.g., provider) and a private, encrypted vector held by the client (e.g., clinician). We instantiate both with a single maliciously secure primitive, verifiable matrix-vector multiplication under HE (Tremblay Thibault et al., 2026). The server cryptographically commits to a matrix, multiplies its committed matrix by the client’s encrypted vector and returns the encrypted result with a succinct proof, which the client verifies before decrypting. The primitive guarantees integrity by construction, and we show in §2 that verifying before decrypting also closes the reaction-attack channel (Chillotti et al., 2016) that honest-but-curious designs leave open. The resulting two-round protocol, illustrated in Figure 1, is exact (every document is scored, with no clustering or hashing approximation), secure against a malicious server (any manipulation of scores or of the returned document is detected except with negligible probability), client-lightweight (per round the client encrypts one vector, verifies one proof and decrypts one result) and modular (both rounds use one primitive behind one interface, so any faster instantiation may be swapped in directly). Setting. The naive way to hide a query is to send the corpus to the client and have it search locally. That option is not available here: the clinical references that physicians consult are proprietary and licensed per seat, so the provider grants query-time access rather than a copy. As such, the setting we consider is a server-held corpus queried by a client that must not reveal its queries. We target deferred-response settings, such as literature requests to medical librarians or tumor-board preparation, where users expect a researched answer rather than an immediate response and minute-scale latency is acceptable; we evaluate the protocol’s cost in this regime (§4.3). Contributions. 1. We formalize private dense retrieval as requiring both query privacy and retrieval integrity against a malicious server, and reduce it to two instances of verifiable matrix-vector multiplication (§2). We give a two-round protocol realizing this functionality, with exact retrieval and a verify-before-decrypt mechanism that provably blocks reaction attacks (§2). 2. Using the protocol as an instrument, we experimentally locate the operating point of private dense retrieval at b=3b=3 bits of quantization: retrieval quality and downstream RAG accuracy hold there for six encoders and four readers, on corpora of up to 2.682.68M passages (§4.1, §4.4). 3. We measure the cost of a private, verifiable query in time and dollars at corpus sizes up to 10610^6 chunks, and show that bit-width buys performance, with a ≈2×≈ 2× increase between b=2b=2 and b=3b=3 (§4.2, §4.3). At b=3b=3 a corpus the size of a clinical reference is served in one to three minutes, while the literature behind it takes hours to serve. 2 Private Dense Retrieval The setting comprises two interactive parties. The server holds a corpus of m documents, represented by an embedding matrix ∈ℤpm×nA _p^m× n (row i is the quantized embedding of document i) and a content matrix ∈ℤpm×ℓD _p^m× (row i is the content of document i, padded to length ℓ ). The client holds a private query embedding ∈ℤpnx _p^n. Embeddings are quantized from ℝR to ℤpZ_p by scalar quantization (Appendix A.6). This is further discussed in §4. This abstracts a common deployment pattern in RAG and enterprise search: the client trusts the embedding model that produced x, but does not trust the retrieval service with the confidentiality of the query or the integrity of the returned evidence. Definition 1 (Private Dense Retrieval). A private dense retrieval protocol allows the client to (1) obtain the similarity scores =⋅∈ℤpms=A·x _p^m, and (2) retrieve the document i∗D_i^* for i∗=arg⁡maxi⁡sii^*= _is_i, subject to: 1. privacy: the server learns no information about x or i∗i^* beyond public parameters; and 2. integrity: the client accepts only scores and documents consistent with the server’s committed database, except with negligible probability. The server is actively malicious: it may deviate arbitrarily from the protocol to learn the query or to alter the result. As in standard PIR, we do not require database privacy against the client; our confidentiality goal concerns the client’s query. The definition extends to top-k retrieval (§2). One primitive suffices. Both requirements of Definition 1 reduce to verifiable multiplication of a public plaintext matrix by an encrypted vector. The primitive provides privacy (ciphertexts reveal nothing about the encrypted vector by semantic security under the GLWE assumption), soundness (no server strategy makes the client accept anything but the product with the committed matrix, except with negligible probability) and succinctness (digest and proof are small, and verifying is far cheaper than computing the product). Formal definitions are in Appendix A.4; Tremblay Thibault et al. (2026) give a concrete instantiation, which our protocol treats as a module. Improvements to the primitive thus transfer directly to our protocol. Protocol. The protocol runs two rounds of the same sub-protocol (Figure 1). Once per database, the server commits to A and D and publishes the digests, which remain fixed for the database’s lifetime. Updates require recomputing only the affected commitment, a cost amortized over many queries. In Round 1, the client sends 1←⁡(1,)c_1← Enc( psk_1,x) and the server returns 1′←⋅1c _1 ·c_1 with a proof π1 _1 against the committed A. From this the client decrypts the exact score vector and computes i∗←arg⁡maxi⁡sii^*← _is_i locally. In Round 2, the client sends 2←⁡(2,i∗)c_2← Enc( psk_2,e_i^*), for i∗∈0,1me_i^*∈\0,1\^m the the one-hot encoding of i∗i^*. The server returns ⊤⋅2D ·c_2 with a proof π2 _2 against the committed D, from which the client decrypts i∗D_i^*. Each round uses a fresh key and the client aborts unless verification succeeds. The rounds are linked only through i∗i^*, which is computed locally. The server sees two semantically secure ciphertext vectors under independent keys and learns no information about x or i∗i^* beyond public parameters. The client’s Round-1 work, namely downloading m ciphertext slots, decrypting them and scanning for the maximum, is linear in m. We discuss scaling in §5. Verification before decryption. That the client verifies before decrypting is essential to security. If it decrypted an unverified response, its observable behavior (abort, retry, or which document it fetches in Round 2) would become a function of the plaintext. This is the side channel that reaction attacks exploit (Chillotti et al., 2016). Verifying first makes that behavior depend only on the proof’s validity, which the server already knows. Theorem 1 (Informal; proof sketch in Appendix C). If the primitive satisfies the properties of §2, the protocol realizes Definition 1: an actively malicious server learns no information about x or i∗i^* beyond public parameters, and cannot cause the client to accept scores or a document inconsistent with the committed database, except with negligible probability. Top-k retrieval. RAG pipelines typically consume k>1k>1 passages. After Round 1 the client knows the full score vector, so it can select any k indices and fetch each with an independent Round-2 query, multiplying the second round’s workload by k. Batching the k queries into one server request multiplies communication by k but barely increases latency, since the online server work of Tremblay Thibault et al. (2026) is dominated by a step independent of the number of batched queries (Appendix D). 3 Related Work Table 1: Private and verifiable retrieval systems. Exactness: scores are exact inner products rather than approximate search. Malicious security: the stated privacy and/or integrity guarantees hold against an actively deviating server. Privacy Integrity Exactness Malicious security Tiptoe (Henzinger et al., 2023) ✓ × × × PIR-RAG (Wang et al., 2025) ✓ × × × PPMI (Bae et al., 2025) ✓ × ✓ × Compass (Zhu et al., 2024) ✓ × × × VeriRAG (Lin et al., 2026) × ✓ × ✓ zkRAG (Jiang et al., 2026) × ✓ × ✓ Ours ✓ ✓ ✓ ✓ Encrypted and verifiable retrieval. Table 1 situates our protocol relative to prior work. We take dense retrieval by maximum inner product search (Karpukhin et al., 2020; Reimers & Gurevych, 2019; Ni et al., 2022) as given and make the query private and the result verifiable. The natural tool for hiding the query is HE, as PPMI (Bae et al., 2025) does for private LLM interaction. Tiptoe (Henzinger et al., 2023) and PIR-RAG (Wang et al., 2025) instead search published cluster centroids locally and fetch documents by private information retrieval (PIR) (Chor et al., 1998), for which practical single-server lattice-based constructions include XPIR (Aguilar-Melchor et al., 2016). Other systems perform the search itself privately: SANNS (Chen et al., 2020), Servan-Schreiber et al. (2022), Compass (Zhu et al., 2024) and BiSON (Das et al., 2026) run approximate k-N under two-party computation or oblivious RAM, at second-to-millisecond latencies on million- to billion-scale corpora. These systems assume an honest-but-curious server, or two non-colluding servers, and do not provide integrity against an actively deviating server. In interactive encrypted protocols, such deviations can additionally enable reaction attacks that compromise privacy (Chillotti et al., 2016). Conversely, VeriRAG (Lin et al., 2026) and zkRAG (Jiang et al., 2026) prove in zero knowledge that the server ran an approximate search over a committed database, but send the query in the clear. Our work instead follows the verifiable-computation-over-ciphertext paradigm (Bois et al., 2021; Ganesh et al., 2023; Chatel et al., 2024; Tremblay Thibault et al., 2026), enabling query privacy and retrieval integrity while preserving exactness. Low-bit embeddings and the retrieval-metric/downstream gap. Several methods have been proposed in recent years for compressing embeddings for more efficient downstream tasks, including binary hashing (Yamada et al., 2021), jointly learned product quantization (Zhan et al., 2021), Matryoshka representation learning (Kusupati et al., 2022), int8-trained commercial encoders (Reimers, 2024) and quantization-aware open ones (Yu et al., 2024). Clipping the quantization range to a percentile of the values, rather than to their maximum, is standard in post-training quantization (Banner et al., 2019). That literature primarily optimizes memory footprint and search efficiency, with representation size scaling linearly with bit-width. In our encrypted setting, every bit consumes a fixed share of a noise budget that only larger cryptographic parameters can compensate for (§4.2), which imposes a different cost model. Separately, ranking metrics are known to predict end-task RAG accuracy poorly, as retrieval and generation quality decouple (Salemi & Zamani, 2024; Cuconasu et al., 2024). Section 4.4 studies this decoupling as a function of bit-width, which also sets cryptographic cost, allowing us to select parameters based on downstream rather than retrieval quality alone. 4 Experimental Analysis The protocol preserves exact dense retrieval over quantized embeddings: the only approximation is the b-bit quantization itself, while all subsequent arithmetic is computed exactly in ℤpZ_p, with p≥2NB2p≥ 2NB^2. In fact, the GLWE plaintext modulus p (cf. §A.2) must be large enough so that ℤpZ_p contains the quantized embeddings and their inner products, without overflow. A larger b requires a larger p, which drives up cryptographic parameters. Holding the retrieval task and security target fixed, the bit-width b is therefore the key parameter coupling retrieval quality and cryptographic cost. Our experiments therefore ask how far b can be reduced before the cryptographic savings cease to justify the loss in retrieval and downstream quality. We measure this trade-off holistically: §4.1 measures retrieval quality across bit-widths and corpus sizes, §4.2 gives the conversion from bit-width to cryptographic cost, §4.3 the concrete costs at scale and §4.4 the downstream performance. Corpora. We evaluate on two biomedical retrieval corpora, chosen to match the setting of §1: SciFact (Wadden et al., 2020), scientific claim verification over biomedical abstracts (m=5,183m=5,183 documents, 300300 test claims), and NFCorpus (Thakur et al., 2021), medical retrieval pairing nutrition and health queries with linked PubMed documents (m=3,633m=3,633, 323323 queries). Both are BEIR (Thakur et al., 2021) tasks, so floating-point baselines are comparable with published numbers. The retrieval unit of the protocol is a chunk of at most 1,5361,536 bytes: 512512 matrix cells each containing 33 bytes (cf. Table 3). This allows more than half of documents to fit into a single chunk, all the while keeping the cost of the second round low. We cut at sentence boundaries with the title repeated in every chunk. This accommodates the clinical references of §1, which are typically several pages long. The downstream evaluation of §4.4 adds BioASQ (Krithara et al., 2023) for biomedical factoid question answering, and the scale study of §4.3 uses Natural Questions (Kwiatkowski et al., 2019), which also serves §4.4 as an out-of-domain control. BioASQ and Natural Questions are passage collections already fitting within one chunk. Retrieval quality in §4.1 is measured on the chunked corpora, ranked per chunk and scored per document, so that the relevance labels and the floating-point baselines remain the published document-level ones; Appendix F gives the same table on whole abstracts. 4.1 Retrieval quality under quantization We evaluate six embedding models of similar performance on both corpora, so that training for low-bit readout is the variable. Two were trained for low-bit readout: Cohere embed-english-v3.0, trained for int8 and binary output (Reimers, 2024), and Snowflake arctic-embed-l-v2.0, whose training was quantization-aware (Yu et al., 2024; Snowflake, 2024). Four were not: bge-large-en-v1.5 (Xiao et al., 2023), e5-large-v2 (Wang et al., 2022), gte-large (Li et al., 2023) and mxbai-embed-large-v1 (Li & Li, 2023; Mixedbread AI, 2024). The last is a control: it advertises int8 and binary use without claiming quantization-aware training. For each b∈2,…,8b∈\2,…,8\ we run the full two-round protocol. Table 8 reports, for b∈2,3,4,8b∈\2,3,4,8\, the fraction of floating-point nDCG@10 (Järvelin & Kekäläinen, 2002; Manning et al., 2008) each encoder retains. Appendix F gives every bit-width on whole abstracts (Table 12) and top-1 agreement at b∈3,4b∈\3,4\ (Table 9). At a high level, the results show that down to b=4b=4, all encoders retain a high percentage of retrieval quality compared to their floating-point counterparts. Appropriate quantization. A b-bit symmetric quantizer maps a chosen magnitude, the scale τ, to the top level qmax=2b−1−1q_ =2^b-1-1 and rounds every coordinate to the nearest multiple of the step τ/qmaxτ/q_ , so every coordinate within [−τ,τ][-τ,τ] is off by at most half a step. Max-abs scaling, the usual choice, sets the scale to the largest coordinate of the corpus matrix. BERT-family encoders carry one coordinate holding over 5%5\% of the squared norm of every embedding, and that one coordinate makes the step up to 4×4× coarser than the other 10231023 need. Appendix F shows the effect: BGE-large clipped at b=3b=3 retains 9797 and 99%99\% of floating-point nDCG@10 on SciFact and NFCorpus, where max-abs retains 7777 and 73%73\% at b=3b=3 and only reaches 9797 and 96%96\% at b=4b=4. We thus use clipping quantization for our experiments. Quality at scale. A larger index gives every query more near-misses to lose to (Reimers & Gurevych, 2021), so we repeat the sweep on Natural Questions from 10410^4 passages to the full 2.682.68M with five encoders, one of them trained for low-bit readout (Table 13, Appendix F). At b=4b=4 four of the five retain at least 98.8%98.8\% of floating-point Recall@5 at every size, and e5 retains 97.3%97.3\% at 2.682.68M. At b=3b=3 arctic, BGE and mxbai retain at least 98.4%98.4\% while gte and e5 fall to 94.594.5 and 92.0%92.0\% at 2.682.68M. At b=2b=2 the loss grows with the corpus, from 11 to 4%4\% at 10410^4 to 1010 to 23%23\% at 2.682.68M. The lead of the quantization-aware encoder grows with the corpus: arctic retains 0.80.8 to 2.72.7 points more Recall@5 than the conventional four at 10410^4 and 2.12.1 to 12.812.8 points more at 2.682.68M. As such, retrieval quality places the operating point at b=3b=3. There, five of the six encoders lose 11 to 3%3\% of nDCG@10 (e5 loses 6%6\% on NFCorpus), and for arctic, BGE and mxbai the loss stays small as the corpus grows. 4.2 Quantization robustness lowers cryptographic cost In this section we formalize how the bit-width b sets the cryptographic parameters, and thus the cost, of Round 1. For both rounds, we fix the ciphertext modulus q=264−232+1q=2^64-2^32+1, at which the primitive of Tremblay Thibault et al. (2026) is most efficient, and target λ≥128λ≥ 128 bits of security and a decryption failure probability of 2−642^-64 per coefficient. Theorem 2 (Correctness of Round 1). Let ∈ℤm×NA ^m× N and ∈ℤNx ^N have entries of absolute value less than B, let p≥2NB2p≥ 2NB^2 and let ←⁡(,)c← Enc( psk,x) be a GLWE encryption with ciphertext modulus q, plaintext modulus p and Gaussian noise of standard deviation σ. For every r>0r>0, provided q/σ≥2rNBpq/σ≥ 2r N\,B\,p, each coordinate of ⁡(,⋅) Dec( psk,A·c) equals that of ⋅A·x over the integers, except with probability ε(r)≔2exp(−r2/2)/(r2π) (r) 2 (-r^2/2)/(r 2π). We defer the proof to Appendix A.3. In Round 1 the inner dimension N is the embedding dimension d=210d=2^10, the quantized entries lie in [−(2b−1−1),2b−1−1][-(2^b-1-1),2^b-1-1] so B=2b−1B=2^b-1, and we take p=2NB2=22b+9p=2NB^2=2^2b+9. At q=264−232+1q=2^64-2^32+1 and r=9.16r=9.16, for which ε⁡(r)≤2−64 (r)≤ 2^-64, Theorem 2 thus makes Round 1 correct provided σ≤σmax(b)σ≤ _ (b) where log2⁡σmax(b)=log2⁡q−log2⁡(2r)−12log2N−(b−1)−(2b+9)≈ 46.8−3b. _2 _ (b)\;=\; _2q- _2(2r)- 12 _2N-(b-1)-(2b+9)\;≈\;46.8-3b. (1) Every added bit of quantization thus costs three bits of noise budget, one through the entry bound B and two through the plaintext modulus p. Security pulls in the other direction and calls for more noise. It admits no closed form, so we evaluate it with the lattice estimator (Albrecht et al., 2015) at σ=σmax(b)σ= _ (b), for a ternary secret and GLWE dimensions 2102^10 and 2112^11 (Table 2). Dimension 2102^10, which matches the embedding dimension, reaches 128128 bits at b=2b=2 only; every b≥3b≥ 3 must double the GLWE dimension κdκ d, by taking d=211d=2^11 or rank κ=2κ=2. As such, the cryptography admits two regimes rather than a cost per bit: the smaller ring at b=2b=2, and one doubling, paid once, from b=3b=3 on. We measure what this doubling costs in §4.3. Table 2: Quantization-cryptography exchange rate: Round 1 parameters at q=264−232+1q=2^64-2^32+1, κ=1κ=1, ternary secret and 2−642^-64 failure probability. λ10 _10 and λ11 _11 are the lattice-estimator security at σ=σmax(b)σ= _ (b) for LWE dimensions d=210d=2^10 and 2112^11, and d the smallest dimension at 128128 bits. b p σmax _ λ10 _10 λ11 _11 d 2 2132^13 240.82^40.8 138 284284 2^10 33 2152^15 237.82^37.8 122122 253253 2112^11 44 2172^17 234.82^34.8 110110 227227 2112^11 55 2192^19 231.82^31.8 100100 206206 2112^11 88 2252^25 222.82^22.8 7979 161161 2112^11 4.3 Concrete costs of private dense retrieval In what follows we quantify the cost of a private and verifiable query, isolate the share of that price paid for malicious security and identify the corpus sizes and workflows the protocol can serve today. Measured cost. Table 3 reports the cost of the full protocol on the evaluation corpora and at scale, at both parameter sets of §4.2. At b=2b=2 the server time is of 1.71.7 s on SciFact and on NFCorpus, and of 2.52.5 s from b=3b=3 on, once the ring is doubled. At 10610^6 chunks the two are of 3.03.0 and 5.45.4 minutes. Round 2 multiplies ⊤∈ℤp512×mD _p^512× m where Round 1 multiplies ∈ℤpm×1,024A _p^m× 1,024. Both matrices grow linearly in m and the prover’s cost follows this trend. As such, Round 2 costs ≈0.5×≈ 0.5× Round 1 at every corpus size at b=2b=2. Per round, the client uploads ⌈N/d⌉(κ+1)dlog2q N/d (κ+1)d _2q bits of ciphertext and downloads ⌈m′/d⌉(κ+1)dlog2q m /d (κ+1)d _2q bits plus the proof, where m′m is the output length of that round. Over both rounds, communication is of 528528 KiB on SciFact and 496496 KiB on NFCorpus at b=2b=2, proofs included, and 2020 KiB more once the ring is doubled. Most of the server time is spent computing the proof of correct computation. On the SciFact Round-1 shape at d=210d=2^10, the matrix-vector product over ciphertexts takes 1515 ms without proof generation and 1.181.18 s with it. Detecting a malicious server rather than trusting it thus costs a factor of 7777. Table 3: Measured cost on the evaluation corpora and at scale, at b=2b=2 (Round 1 at d=210d=2^10) and from b=3b=3 on (Round 1 at d=211d=2^11); Round 2 runs at d=212d=2^12 with ℓ=512 =512 (Table 5). Server time is measured on 4848 AMD EPYC 9654 cores; client time is measured on a single core. Prices at the on-demand rate of a 4848-core hpc7a.24xlarge EC2 instance ($7.207.20/h). b=2b=2 / b≥3b≥3 Corpus Round 1: A Round 2: ⊤D Client Server $/query NFCorpus (5,7615,761 chunks) 5,761×1,0245,761× 1,024 512×5,761512× 5,761 510510 ms / 914914 ms 1.71.7 s / 2.52.5 s 0.0030.003 / 0.0050.005 SciFact (7,5107,510 chunks) 7,510×1,0247,510× 1,024 512×7,510512× 7,510 510510 ms / 914914 ms 1.71.7 s / 2.52.5 s 0.0030.003 / 0.0050.005 m=104m=10^4 104×1,02410^4× 1,024 512×104512× 10^4 907907 ms / 1.71.7 s 3.03.0 s / 5.05.0 s 0.0060.006 / 0.0100.010 m=105m=10^5 105×1,02410^5× 1,024 512×105512× 10^5 6.36.3 s / 12.512.5 s 21.721.7 s / 33.833.8 s 0.0430.043 / 0.0680.068 m=106m=10^6 106×1,02410^6× 1,024 512×106512× 10^6 50.450.4 s / 1.71.7 min 3.03.0 min / 5.45.4 min 0.3640.364 / 0.6500.650 The deployable frontier. Table 3 presents the cost of a private dense retrieval query under our protocol for both b∈2,3b∈\2,3\. By §4.1, b=3b=3 is the smallest bit-width that keeps retrieval quality, so the following assumes b=3b=3, i.e. the larger GLWE parameters. Today, UpToDate, the curated reference clinicians consult first, holds over 12,00012,000 topics (Wolters Kluwer, 2026b), each the length of a review article, i.e., of the order of 22 to 5×1055× 10^5 chunks in all. A private query on a corpus this size costs one to three minutes and is affordable today. On the other hand, PubMed, the literature behind UpToDate, holds over 4040 million abstracts (National Library of Medicine, 2026), one or two chunks each. A query on such a large corpus would take 44 to 66 hours of server time at this throughput and incur communication costs of approximately 11 GB. We judge this is too costly for the clinical setting, even in the deferred-response context. Two things can bring the cost down: faster cryptographic machinery, and encoders that keep their quality at b=2b=2. The former lies with the cryptographic community. The latter is a matter of representation learning, and by §4.2 it is worth the one ring doubling, i.e. the factor of 1.51.5 to 1.81.8 between the two columns of Table 3. As such, at b=3b=3 the protocol serves a corpus of a few hundred thousand chunks in one to three minutes and 10610^6 in five, which covers a curated clinical reference and not the literature behind it. 4.4 Downstream task quality under quantization In the RAG deployments we target, what matters is the accuracy of the answer a language model produces from the retrieved passages rather than the relevance ranking of these passages. The two need not degrade together, since a reader consuming k passages tolerates a reordering provided the evidence it needs appears somewhere in its context (Salemi & Zamani, 2024; Cuconasu et al., 2024). We measure downstream accuracy as a function of b and locate the smallest bit-width it tolerates. Methodology. We follow the standard end-task protocol for retrieval-augmented generation (Karpukhin et al., 2020; Lewis et al., 2020; Petroni et al., 2021): the retriever returns the top-k passages, a fixed reader answers from them, and only the retrieved set changes between the conditions we compare. A question-answering response is correct if it contains a reference answer (Mallen et al., 2023; Asai et al., 2024), and a claim-verification response if it matches the label. We test each bit-width against floating point with an exact McNemar test on paired per-query outcomes. Since a non-significant difference is not evidence of equivalence, we further test equivalence within ±5± 5 points by two one-sided tests (TOST) on the same paired differences (Appendix E). Setup. We use three endpoints and two encoders, one from each group of §4.1. SciFact is evaluated on its original task: the reader receives a claim and the top-k retrieved abstracts and outputs Support, Contradict or NoInfo, scored against the human labels on the 188188 claims with evidence in the corpus, at k=5k=5 and k=1k=1. BioASQ is biomedical factoid QA over a corpus of 35,45435,454 passages, scored by containment of a reference answer. Natural Questions is the out-of-domain control at scale, answered over the full 2.682.68M-passage corpus of §4.1. Each endpoint has 300300 queries. We test the performance limits of the reader by measuring its performance in the closed book setting, where it answers from its own knowledge with no passages, and in the gold context setting, where it receives the labelled relevant passages in place of the retrieved ones. Retrieval uses the clipped quantizer with BGE and arctic, and only the retrieved set varies with b, so any change in accuracy is attributable to quantization. The reader is phi-4 (1414B) (Abdin et al., 2024). Appendix E repeats every measurement with OLMo-2-7B (OLMo Team, 2025) and Qwen2.5 at 77B and 7272B (Qwen Team, 2025), and gives the prompts and corpora. Downstream quality holds to b=3b=3 and not to b=2b=2. Table 4 shows the result. On closed book, the reader reaches 19%19\% on BioASQ, 31%31\% on NQ and 59%59\% on SciFact, with gold passages 6868, 7474 and 81%81\%, and with floating-point retrieval it sits between the two, close to the gold row. The effects of quantization below are to be read against that span of 2323 to 4949 points. No cell shows a significant drop from floating point at b=4b=4 or b=3b=3, and every cell is equivalent to floating point within 55 points at both bit-widths. This holds at 2.682.68M passages. The reason is that the RAG endpoint consumes recall rather than rank. More precisely, at b=3b=3 the top-55 set of BGE on NQ agrees with the floating-point one on 82%82\% of positions, yet it contains a relevant passage for 70%70\% of questions, as in floating point. Further, §4.1 showed Recall@5 retained at every corpus size. The three other readers agree: over all four, 3232 of 3232 reader-endpoint-encoder combinations are equivalent to floating point at b=4b=4 and 3030 of 3232 at b=3b=3, with a single significant drop of 3.33.3 points (Table 6). At b=2b=2, accuracy drops significantly in 2020 of 3232 combinations, by 33 to 99 points, and only 99 remain equivalent to floating point within 55 points. Again, poor recall is responsible for this drop in performance. On NQ, the top-55 set of BGE contains a relevant passage for 60%60\% of questions at b=2b=2 against 70%70\% at b=3b=3, and the reader answers Unknown more often rather than answering wrongly. The loss grows with the corpus: Recall@5 retained at b=2b=2 falls from 9898 to 86%86\% for BGE and from 9999 to 90%90\% for arctic between 10410^4 and 2.682.68M passages (Table 13). Interestingly, the loss is uneven. It is largest at k=1k=1, where a single reordering at the top removes the only passage the reader sees, at 44 to 99 points for both encoders. At k=5k=5 it depends on the encoder: on SciFact, BGE loses 33 to 66 points where arctic loses at most 2.12.1, with no significant drop for any of the four readers. This shows that the loss at b=2b=2 is a property of how the encoder was trained. Table 4: Downstream accuracy versus bit-width with the clipped quantizer (reader phi-4, n=300n=300 queries per endpoint). SciFact: accuracy on the 188188 evidence-bearing claims; BioASQ and NQ: answer containment, NQ retrieving over the full 2.682.68M-passage corpus. Closed book: the reader sees no passages; gold context: it sees the labelled relevant passages (at most k). Bold marks a significant drop from floating point (exact McNemar, p<0.05p<0.05). Other readers: Appendix E. SciFact k=5k=5 SciFact k=1k=1 BioASQ k=5k=5 NQ k=5k=5, 2.682.68M b BGE arctic BGE arctic BGE arctic BGE arctic closed book 0.5850.585 0.5850.585 0.1870.187 0.3130.313 gold context 0.8140.814 0.8090.809 0.6800.680 0.7400.740 float 0.7340.734 0.7130.713 0.7660.766 0.7390.739 0.6330.633 0.6470.647 0.6430.643 0.6430.643 88 0.7340.734 0.7180.718 0.7710.771 0.7390.739 0.6330.633 0.6430.643 0.6570.657 0.6370.637 66 0.7290.729 0.7130.713 0.7710.771 0.7390.739 0.6330.633 0.6400.640 0.6500.650 0.6400.640 44 0.7230.723 0.7180.718 0.7710.771 0.7390.739 0.6370.637 0.6370.637 0.6500.650 0.6270.627 33 0.7070.707 0.7130.713 0.7610.761 0.7550.755 0.6430.643 0.6400.640 0.6230.623 0.6400.640 22 0.6760.676 0.6970.697 0.6860.686 0.6810.681 0.6130.613 0.6070.607 0.5730.573 0.6000.600 Chunking the SciFact corpus as in §4 costs the conventional encoder one bit and the quantization-aware one none (Table 7, Appendix E). With BGE at k=5k=5 the loss reaches 4.34.3 points at b=3b=3, where arctic stays within 55 points of floating point down to b=2b=2 on this smaller corpus. As such, downstream accuracy places the operating point of private RAG at b=3b=3 at every corpus size we measured, one bit above the b=2b=2 at which §4.2 halves the GLWE parameters required for sufficient security. That frontier bit is decided by the encoder: with a conventional encoder the loss at b=2b=2 is of 33 to 99 points, with a quantization-aware one it is within 2.12.1 points at k=5k=5 on SciFact. Whether private RAG runs on the smaller GLWE parameters is thus a property of the encoder. 5 Discussion Limitations. The protocol is a drop-in retrieval layer for deployments in which a licensed corpus is hosted by an untrusted service, the queries are sensitive and an answer is expected in minutes. Importantly, our contribution is not an end-to-end privacy-preserving RAG pipeline: we assume the embedding model and the readers run locally or in a trusted environment, and leave a fully private pipeline, which current technology does not yet allow, to future work. In terms of scale, three constraints bound our protocol. The first is the client: the Round-1 response, the Round-2 query and the client’s decryption and argmax are all linear in m. At m=106m=10^6, Table 3 gives one to two minutes of sequential client work and roughly 1616 MB in each direction. The client is kept single-threaded to model a computationally weak device. The second is the encoder, which decides whether a task can go below b=3b=3. Our evidence is a controlled comparison of six encoders, of which only two were trained for low-bit readout. What they change is the score margin relative to the rounding step (§4.1); whether a training objective can widen it enough for b=2b=2 at scale is an open problem we hand to representation learning. Very recent advances in this regard are promising (Wang et al., 2026). The third constraint is the cryptographic primitive: verifiable HE throughput limits web-scale deployment, but our modular design directly absorbs any improvement to it. 6 Conclusion We have formulated private dense retrieval as a problem of privacy and integrity against a malicious server. We have reduced the underlying computations to verifiable multiplication of a committed matrix by an encrypted vector and used the resulting protocol to measure what privacy costs. With a clipped quantizer, retrieval quality and downstream accuracy hold at b=3b=3 for six encoders and four readers on corpora of up to 2.682.68M passages. A private, verified query over a corpus the size of a clinical reference is then of one to three minutes. At 128128 bits of security, only b=2b=2 admits the smaller GLWE parameters; b≥3b≥ 3 costs a factor of 1.81.8 in server time. Which encoders keep their quality below three bits is set by their score margin relative to the quantization step, and only those trained for low-bit readout do so at b=2b=2. Lowering the cost of privacy in dense retrieval is thus not only a cryptography problem, but also one of representation learning. AI Use Statement In this work, we used generative AI tools for feedback on research methodology. We have not used generative AI tools to generate synthetic data sets, help develop theoretical models or conceptual frameworks, formulate mathematical claims, provide critical ingredients for proving mathematical claims, assist in the writing of proofs, propose or refine hypotheses, implement methods, assist with translation, clean and reformat datasets, support qualitative and thematic data analysis, or interpret results. Additionally, we used generative AI tools to edit software code, generate tables and identify relevant literature. We have reviewed all AI-assisted work. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. Reproducibility Statement All retrieval-side results use public corpora (SciFact, NFCorpus, Natural Questions, BioASQ) and publicly available encoders. The downstream readers are open-weight models run with structured decoding, so every number in §4.4 is reproducible without access to an API. The quantization code and the per-bit-width sweep outputs are available at https://github.com/tremblaythibaultl/pdr-artifact, and the protocol implementation at https://github.com/tremblaythibaultl/vpir. Prompts are fixed across bit-widths within a reader and are given in Appendix E. The cryptographic parameters, the correctness constraint and the security estimate are stated in closed form in §4.2 and Appendix A.2, so Table 2 is reproducible without running the protocol. Timings are single-node measurements on a named processor with a fixed thread count, and the benchmark binary asserts verification and exact decryption at every shape. Ethics Statement This work aims to reduce the exposure of sensitive queries and is motivated by a concrete disclosure risk in clinical decision support. The corpora we use are public research datasets and contain no patient data. We emphasize that query privacy against the retrieval provider does not guarantee privacy over the rest of the technological stack. A protocol such as ours could be invoked to argue compliance while leakage persists elsewhere. The guarantee is exactly that of Definition 1 and no broader. Further, the integrity guarantee is relative to a committed digest. Deploying the protocol without independent attestation of that digest by peers or auditors offers the appearance of verification without its substance. Acknowledgments The authors would like to thank Léo Gagnon, Romane Asselin and Charlie Gauthier for insightful discussions. This work was supported by the Fonds de recherche du Québec - Nature et technologies (FRQNT) through grant https://doi.org/10.69777/2006324. References Abdin et al. (2024) Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024. Agence France-Presse (2026) Agence France-Presse. Hackers steal medical details of 15 million in france, 2026. URL https://w.france24.com/en/live-news/20260227-hackers-steal-medical-details-of-15-million-in-france. Aguilar-Melchor et al. (2016) Carlos Aguilar-Melchor, Joris Barrier, Laurent Fousse, and Marc-Olivier Killijian. XPIR: Private information retrieval for everyone. 2016(2):155–174, April 2016. doi: 10.1515/popets-2016-0010. Albrecht et al. (2015) Martin R. Albrecht, Rachel Player, and Sam Scott. On the concrete hardness of learning with errors. J. Math. Cryptol., 9(3):169–203, 2015. URL http://w.degruyter.com/view/j/jmc.2015.9.issue-3/jmc-2015-0016/jmc-2015-0016.xml. American Medical Association (2026) American Medical Association. 2026 physician survey on augmented intelligence. AMA Center for Digital Health and AI, March 2026. URL https://w.ama-assn.org/system/files/physician-ai-sentiment-report.pdf. Asai et al. (2024) Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2024. Bae et al. (2025) Yubeen Bae, Minchan Kim, Jaejin Lee, Sangbum Kim, Jaehyung Kim, Yejin Choi, and Niloofar Mireshghallah. Ppmi: Privacy-preserving llm interaction with socratic chain-of-thought reasoning and homomorphically encrypted vector databases, 2025. URL https://arxiv.org/abs/2506.17336. Banner et al. (2019) Ron Banner, Yury Nahshan, and Daniel Soudry. Post-training 4-bit quantization of convolutional networks for rapid-deployment. In Advances in Neural Information Processing Systems, volume 32, 2019. Bois et al. (2021) Alexandre Bois, Ignacio Cascudo, Dario Fiore, and Dongwoo Kim. Flexible and efficient verifiable computation on encrypted data. In Public-Key Cryptography (PKC), p. 528–558, 2021. doi: 10.1007/978-3-030-75248-4_19. Chatel et al. (2024) Sylvain Chatel, Christian Knabenhans, Apostolos Pyrgelis, Carmela Troncoso, and Jean-Pierre Hubaux. VERITAS: Plaintext encoders for practical verifiable homomorphic encryption. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), p. 2520–2534, 2024. doi: 10.1145/3658644.3670282. Chen et al. (2020) Hao Chen, Ilaria Chillotti, Yihe Dong, Oxana Poburinnaya, Ilya Razenshteyn, and M. Sadegh Riazi. SANNS: Scaling up secure approximate k-nearest neighbors search. In 29th USENIX Security Symposium (USENIX Security), p. 2111–2128, 2020. Chillotti et al. (2016) Ilaria Chillotti, Nicolas Gama, and Louis Goubin. Attacking FHE-based applications by software fault injections. Cryptology ePrint Archive, Paper 2016/1164, 2016. URL https://eprint.iacr.org/2016/1164. Chor et al. (1998) Benny Chor, Oded Goldreich, Eyal Kushilevitz, and Madhu Sudan. Private information retrieval. Journal of the ACM, 45(6):965–981, 1998. doi: 10.1145/293347.293350. Cuconasu et al. (2024) Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. The power of noise: Redefining retrieval for RAG systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), p. 719–729, 2024. doi: 10.1145/3626772.3657834. Das et al. (2026) Sankha Das, Rohan Ravi, Nishanth Chandran, and Divya Gupta. BiSON: Billion-scale oblivious nearest-neighbor search in milliseconds. Cryptology ePrint Archive, Paper 2026/1343, 2026. URL https://eprint.iacr.org/2026/1343. Ganesh et al. (2023) Chaya Ganesh, Anca Nitulescu, and Eduardo Soria-Vazquez. Rinocchio: SNARKs for ring arithmetic. Journal of Cryptology, 36(4):41, 2023. doi: 10.1007/s00145-023-09481-3. Gentry (2009) Craig Gentry. Fully homomorphic encryption using ideal lattices. In Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing, STOC ’09, p. 169–178, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585062. doi: 10.1145/1536414.1536440. URL https://doi.org/10.1145/1536414.1536440. Greshake et al. (2023) Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023. URL https://api.semanticscholar.org/CorpusID:258546941. Henzinger et al. (2023) Alexandra Henzinger, Emma Dauterman, Henry Corrigan-Gibbs, and Nickolai Zeldovich. Private web search with tiptoe. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, p. 396–416, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400702297. doi: 10.1145/3600006.3613134. URL https://doi.org/10.1145/3600006.3613134. Järvelin & Kekäläinen (2002) Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422–446, 2002. doi: 10.1145/582415.582418. Jiang et al. (2026) Yanze Jiang, Xinyang Yang, Xuanming Liu, Yanpei Guo, and Jiaheng Zhang. zkRAG: Efficiently proving RAG retrieval in zero knowledge. Cryptology ePrint Archive, Paper 2026/709, 2026. URL https://eprint.iacr.org/2026/709. Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 6769–6781, 2020. Krithara et al. (2023) Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras. BioASQ-QA: A manually curated corpus for biomedical question answering. Scientific Data, 10(170), 2023. doi: 10.1038/s41597-023-02068-4. Kusupati et al. (2022) Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. Matryoshka representation learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, p. 30233–30249, 2022. Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466, 2019. Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474, 2020. Li & Li (2023) Xianming Li and Jing Li. Angle-optimized text embeddings. arXiv preprint arXiv:2309.12871, 2023. Li et al. (2023) Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023. Lin et al. (2026) Chenqi Lin, Yubo Cui, Zhelei Zhou, Cheng Hong, Yufei Wang, Zhaohui Chen, and Meng Li. VeriRAG: Efficient zero-knowledge proofs for verifiable retrieval-augmented generation. Cryptology ePrint Archive, Paper 2026/637, 2026. URL https://eprint.iacr.org/2026/637. Lyubashevsky et al. (2013) Vadim Lyubashevsky, Chris Peikert, and Oded Regev. On ideal lattices and learning with errors over rings. J. ACM, 60(6):43:1–43:35, 2013. doi: 10.1145/2535925. URL https://doi.org/10.1145/2535925. Mallen et al. (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9802–9822, 2023. Manning et al. (2008) Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Information Retrieval. Cambridge University Press, Cambridge, England, 2008. ISBN 978-0-521-86571-5. Mixedbread AI (2024) Mixedbread AI. mxbai-embed-large-v1 model card. https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1, 2024. Accessed September 2026. Morris et al. (2023) John Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M Rush. Text embeddings reveal (almost) as much as text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 12448–12460, 2023. National Library of Medicine (2026) National Library of Medicine. About PubMed, 2026. URL https://pubmed.ncbi.nlm.nih.gov/about/. Accessed September 2026. NBC News (2026) NBC News. Most U.S. doctors are quietly using this AI tool. few patients know about it, 2026. URL https://w.nbcnews.com/tech/tech-news/openevidence-ai-doctor-medical-physician-login-app-what-npi-uptodate-rcna341064. Ni et al. (2022) Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, et al. Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 9844–9855, 2022. Office for Civil Rights (2002) HHS Office for Civil Rights. Standards for privacy of individually identifiable health information. final rule, 2002. OLMo Team (2025) OLMo Team. 2 OLMo 2 furious. arXiv preprint arXiv:2501.00656, 2025. Petroni et al. (2021) Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 2523–2544, 2021. Qwen Team (2025) Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2025. Reimers (2024) Nils Reimers. Cohere int8 & binary embeddings: Scale your vector database to large datasets. Cohere Blog, 2024. URL https://cohere.com/blog/int8-binary-embeddings. Reimers & Gurevych (2019) Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 3982–3992, 2019. Reimers & Gurevych (2021) Nils Reimers and Iryna Gurevych. The curse of dense low-dimensional information retrieval for large index sizes. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), p. 605–611. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.acl-short.77. Salemi & Zamani (2024) Alireza Salemi and Hamed Zamani. Evaluating retrieval quality in retrieval-augmented generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), p. 2395–2400, 2024. doi: 10.1145/3626772.3657957. Servan-Schreiber et al. (2022) Sacha Servan-Schreiber, Simon Langowski, and Srinivas Devadas. Private approximate nearest neighbor search with sublinear communication. In IEEE Symposium on Security and Privacy (S&P), 2022. Snowflake (2024) Snowflake. snowflake-arctic-embed-l-v2.0 model card. https://huggingface.co/Snowflake/snowflake-arctic-embed-l-v2.0, 2024. Accessed September 2026. Song & Raghunathan (2020) Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (CCS), p. 377–390, 2020. doi: 10.1145/3372297.3417270. Thakur et al. (2021) Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663, 2021. Tremblay Thibault et al. (2026) Louis Tremblay Thibault, Michael Walter, and Jiapeng Zhang. Practical SNARGs for matrix multiplications over encrypted data. Cryptology ePrint Archive, Paper 2026/027, 2026. URL https://eprint.iacr.org/2026/027. Wadden et al. (2020) David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 7534–7550, 2020. Wang et al. (2025) Baiqiang Wang, Qian Lou, Mengxin Zheng, and Dongfang Zhao. Pir-rag: A system for private information retrieval in retrieval-augmented generation, 2025. URL https://arxiv.org/abs/2509.21325. Wang et al. (2026) Hongyu Wang, Shuming Ma, Lingxiao Ma, Lei Wang, Wenhui Wang, Li Dong, Shaohan Huang, Huaijie Wang, Jilong Xue, Ruiping Wang, Yi Wu, and Furu Wei. Bitnet: 1-bit pre-training for large language models. J. Mach. Learn. Res., 26(1), September 2026. ISSN 1532-4435. Wang et al. (2022) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022. Wolters Kluwer (2026a) Wolters Kluwer. Health system adoption of Wolters Kluwer UpToDate Expert AI now reaching approximately 2,500 U.S. hospitals and health systems, August 2026a. URL https://w.wolterskluwer.com/en/news/health-system-adoption-uptodate-expert-ai-reaching-2500-us-hospitals-health-systems. Wolters Kluwer (2026b) Wolters Kluwer. What is UpToDate, 2026b. URL https://w.wolterskluwer.com/en/solutions/uptodate/about. Accessed September 2026. Xiao et al. (2023) Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding. arXiv preprint arXiv:2309.07597, 2023. Yamada et al. (2021) Ikuya Yamada, Akari Asai, and Hannaneh Hajishirzi. Efficient passage retrieval with hashing for open-domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), p. 979–986, 2021. doi: 10.18653/v1/2021.acl-short.123. Yu et al. (2024) Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. Arctic-embed 2.0: Multilingual retrieval without compromise. arXiv preprint arXiv:2412.04506, 2024. Zhan et al. (2021) Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. Jointly optimizing query encoder and product quantization to improve retrieval performance. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM), p. 2487–2496, 2021. doi: 10.1145/3459637.3482358. Zhu et al. (2024) Jinhao Zhu, Liana Patel, Matei Zaharia, and Raluca Ada Popa. Compass: Encrypted semantic search with high accuracy. Cryptology ePrint Archive, Paper 2024/1255, 2024. URL https://eprint.iacr.org/2024/1255. Appendix A Preliminaries A.1 Homomorphic Encryption We rely on a linearly homomorphic encryption scheme based on the General Learning With Errors (GLWE) assumption. A.2 (G)LWE encryption Definition 2 (GLWE encryption scheme Lyubashevsky et al. (2013)). Let q be a prime ciphertext modulus, p<qp<q a plaintext modulus, κ the GLWE dimension parameter and d a power of two ring degree so that ℛq≔ℤ⁡[X]/⟨Xd+1⟩R_q [X]/ X^d+1 is a cyclotomic polynomial ring. We let χs _s and χe _e denote the secret key and error distributions over ℛqR_q respectively. Finally, we let U denote the uniform distribution, ℛpR_p denote the message space of the encryption scheme and Δ≔⌊q/p⌉ q/p . A GLWE encryption scheme ℰ≔(,,)E ( KGen, Enc, Dec) consists of the following algorithms: • ⁡(1λ) KGen(1^λ): Takes as input a security parameter 1λ1^λ and returns a sampled secret key s→←$χsκ s← -2.0mu $ \$ -0.52502pt$ $ \$ -0.52502pt$ $ \$ -0.36751pt$ $ \$ -0.26251pt$ _s^κ. • ⁡(s→,m) Enc( s,m): Takes as input a secret key s→ s and a message m∈ℛpm _p and returns a ciphertext c→∈ℛqκ+1 c _q^κ+1 computed as follows: a→←$(ℛq)κ,e←$χe,c→=(a→,⟨a→,s→⟩+e+Δm)∈ℛqκ+1. a← -2.0mu $ \$ -0.52502pt$ $ \$ -0.52502pt$ $ \$ -0.36751pt$ $ \$ -0.26251pt$U(R_q)^κ, e← -2.0mu $ \$ -0.52502pt$ $ \$ -0.52502pt$ $ \$ -0.36751pt$ $ \$ -0.26251pt$ _e, c=( a, a, s +e+ m) _q^κ+1. • ⁡(s→,c→) Dec( s, c): Takes as input a secret key s→ s and a ciphertext c→=(a→,b)∈ℛqκ+1 c=( a,b) _q^κ+1 and returns a message m∈ℛpm _p computed as follows: m=⌊b−⟨a→,s→⟩Δ⌉m= b- a, s This encryption scheme is secure under the GLWE assumption for appropriate choices of parameters. For a concrete parameter set, its security can be estimated using the widely used lattice estimator Albrecht et al. (2015). Table 5 gives the parameter sets we instantiate the protocol with, satisfying the correctness constraint of Theorem 2 with failure probability at most 2−642^-64 per coefficient and at least λ=128λ=128 bits of security for a ternary secret, as estimated with the lattice estimator Albrecht et al. (2015). Round 1 is shown at b=2b=2, the only bit-width the ring d=210d=2^10 admits, and at b=3b=3 and 44 with d=211d=2^11. The remaining rows of Table 2 follow the same pattern (rank κ=2κ=2 at d=210d=2^10 has the same LWE dimension and the same security, but the implementation we measure supports rank 11 only). In each row σ is set at the correctness ceiling of (1), so the security shown is the largest the row allows. Round 2 encrypts a one-hot vector over m chunks with B=p=224B=p=2^24, so its ceiling is log2⁡σmax=64−log2⁡(2r)−12log2m−48 _2 _ =64- _2(2r)- 12 _2m-48, i.e., 25.32^5.3 at m=213m=2^13 and 21.82^1.8 at m=220m=2^20. We use the minimal width σ=3.2σ=3.2, which is correct up to m=220m=2^20 chunks and gives 223223 bits at d=212d=2^12, the ring at which 128128 bits are reached at that width; beyond 2202^20 chunks, each bit removed from p buys four doublings of m. Table 5: GLWE parameters per round: ring dimension d, rank κ, ciphertext modulus q, plaintext modulus p, noise standard deviation σ and the resulting security parameter λ (ternary secret) and maximum failure probability Pr⁡[] [ fail] per coefficient. Round 2 is given for m≤220m≤ 2^20 chunks. Round d κ q p σ λ Pr⁡[] [ fail] Round 1 (b=2b=2) 2102^10 11 264−232+12^64-2^32+1 2132^13 240.82^40.8 138138 2−642^-64 Round 1 (b=3b=3) 2112^11 11 264−232+12^64-2^32+1 2152^15 237.82^37.8 253253 2−642^-64 Round 1 (b=4b=4) 2112^11 11 264−232+12^64-2^32+1 2172^17 234.82^34.8 227227 2−642^-64 Round 2 2122^12 11 264−232+12^64-2^32+1 2242^24 21.72^1.7 223223 2−642^-64 The above encryption scheme is additively homomorphic and allows for multiplication by plaintext elements, provided the noise remains “small”. More precisely, given two ciphertexts c→1=(a→1,b1) c_1=( a_1,b_1) and c→2=(a→2,b2) c_2=( a_2,b_2) encrypting messages m1,m2∈ℛpm_1,m_2 _p respectively, we have: • Addition: c→add=c→1+c→2=(a→1+a→2,b1+b2) c_add= c_1+ c_2=( a_1+ a_2,b_1+b_2) is a ciphertext encrypting madd=m1+m2m_add=m_1+m_2. • Scalar multiplication: For any plaintext y∈ℛpy _p, c→scal=y⋅c→1=(y⋅a→1,y⋅b1) c_scal=y· c_1=(y· a_1,y· b_1) is a ciphertext encrypting mscal=y⋅m1m_scal=y· m_1. These properties hold provided the accumulated noise in the resulting ciphertexts remains small enough to allow for correct decryption, i.e. ‖e‖∞≤Δ/2\|e\|_∞≤ /2. A.3 Proof of Theorem 2 Proof. Each score si=∑jAijxjs_i= _jA_ijx_j satisfies |si|≤N(B−1)2<NB2|s_i|≤ N(B-1)^2<NB^2, so the 2NB2−1≤p2NB^2-1≤ p values it can take are distinct modulo p. It remains to show that decryption returns simodps_i p. By linearity, ⋅A·c decrypts to ⋅modpA·x p provided the noise of every coefficient stays below Δ/2 /2 in absolute value. We neglect the rounding of Δ=⌊q/p⌉ = q/p , which shifts this threshold by at most p/2p/2, and write Δ/2=q/2p /2=q/2p. The ciphertext c carries N independent noise coefficients e1,…,eN∼⁡(0,σ2)e_1,…,e_N (0,σ^2), and the coefficient of ⋅A·c carrying sis_i has noise e′i=∑j±Aijeπ⁡(j)e _i= _j± A_ij\,e_π(j), for a permutation π and signs fixed by the negacyclic ring structure. As such, ei′e _i is Gaussian of variance σ2∑jAij2≤NB2σ2σ^2 _jA_ij^2≤ NB^2σ^2, and the Gaussian tail bound gives Pr[|ei′|>rNBσ]≤ε(r) [|e _i|>r NBσ]≤ (r). The inequality q/σ≥2rNBpq/σ≥ 2r NBp implies that rNBσ≤q/(2p)r NBσ≤ q/(2p), so every coordinate is correct except with probability ε⁡(r) (r). ∎ A.4 Secure Matrix-Vector Multiplication We assume the existence of an efficient protocol for verifiable plaintext matrix–encrypted vector multiplication. Such a primitive can be instantiated with e.g. Tremblay Thibault et al. (2026). The protocol operates as follows: 1. The client encodes its input =(x1,…,xN)∈ℤpNx=(x_1,…,x_N) _p^N into ring elements and encrypts each component, obtaining a ciphertext vector c which is sent to the server. 2. The server holds a plaintext matrix ∈ℤpm×NM _p^m× N. It computes the matrix-ciphertext product row-wise, yielding a result ciphertext vector ′c of dimension m, which is returned to the client. 3. The client decrypts ′c to obtain ⋅M·x. Definition 3 (Passive security). The protocol is passively secure if the server’s view (the ciphertext vector c) reveals no information about x beyond public parameters and dimensions. Definition 4 (Malicious security / Verifiability). The protocol is maliciously secure if, in addition to passive security, the client can verify that the server computed ⋅M·x correctly. That is, a malicious server cannot convince the client to accept an incorrect result ≠⋅y ·x except with probability ⁡(λ) negl (λ ). The malicious security property is achieved via a verifiable computation mechanism which allows the client to check the server’s computation without re-doing it. A.5 Embedding-Based Retrieval Modern retrieval systems represent documents and queries as dense vectors (embeddings) in ℝnR^n using neural network encoders (Reimers & Gurevych (2019); Ni et al. (2022)). Given a corpus of m documents, each document i is mapped to an embedding i∈ℝna_i ^n. These embeddings are collected as rows of a matrix ∈ℝm×nA ^m× n. For a query with embedding ∈ℝnx ^n, the relevance of each document is measured by the inner product ⟨i,⟩ _i,x , and the top-k documents with highest scores are retrieved. A.6 Quantized Embeddings To operate over ℤpZ_p rather than ℝR we use clipped symmetric scalar quantization. Let qmax=2b−1−1q_ =2^b-1-1 and let τ be the 9999th percentile of |||A| over all entries of the embedding matrix. Each coordinate a is mapped to clip⁡(⌊aqmax/τ⌉,−qmax,qmax)clip( a\,q_ /τ ,-q_ ,q_ ), so the 1%1\% of coordinates beyond τ saturate. A query vector is quantized the same way with its own percentile. Every entry lies in [−qmax,qmax][-q_ ,q_ ]. The usual max-abs rule maps the single largest coordinate to qmaxq_ . BERT-family encoders carry one coordinate holding 55 to 6%6\% of the squared norm of every embedding, so under max-abs that coordinate sets a step 4×4× coarser than the bulk of the coordinates need. Clipping at a percentile is standard in post-training quantization (Banner et al., 2019). Table 10 compares clipping with max-abs on every encoder. The 99.999.9th and 9595th percentiles land within a point of the 9999th on both corpora. Compatibility with secure computation. Quantized embeddings are natively integers in ℤpZ_p, requiring no floating-point emulation, and the inner product of quantized vectors is computed exactly by the protocol. How closely it preserves the floating-point ranking is the subject of §4.1 and §4.3. Appendix B Protocol Message Flow Figure 2 gives the message flow of the two-round protocol of §2. ClientServer1←$⁡(1λ) psk_1← -2.0mu $ \$ -0.52502pt$ $ \$ -0.52502pt$ $ \$ -0.36751pt$ $ \$ -0.26251pt$ KGen(1^λ)1←⁡(1,)c_1← Enc( psk_1,\,x)1c_11′←⋅1c _1 ·c_1π1←⁡(,1,1′) _1← Prove(A,c_1,c _1)(1′,π1)(c _1,\, _1)⁡(π1) Verify( _1)←⁡(1,1′)s← Dec( psk_1,c _1)i∗←arg⁡maxi⁡sii^*← _is_ilocal computation2←$⁡(1λ) psk_2← -2.0mu $ \$ -0.52502pt$ $ \$ -0.52502pt$ $ \$ -0.36751pt$ $ \$ -0.26251pt$ KGen(1^λ)2←⁡(2,i∗)c_2← Enc( psk_2,\,e_i^*)2c_22′←⊤⋅2c _2 ·c_2π2←⁡(⊤,2,2′) _2← Prove(D ,c_2,c _2)(2′,π2)(c _2,\, _2)⁡(π2);⁡(2,2′) Verify( _2);\; Dec( psk_2,c _2)Round 1Round 2 Figure 2: The private dense retrieval protocol. Round 1 retrieves encrypted similarity scores, Round 2 privately fetches the top document; both use the same verifiable matrix-vector sub-protocol, and the client verifies each proof before decrypting. Appendix C Security of the Protocol We sketch the proof of Theorem 1. Client privacy. In each round the server observes only fresh ciphertexts, which by semantic security of the GLWE scheme are computationally indistinguishable from encryptions of any other vector of the same dimension. The two rounds use independent keys, so there is no cross-round linkage: the server’s view in the two rounds is simulatable from public parameters alone. Because the client verifies each proof before decrypting, its observable behavior (continue or abort) is a function of the proof’s validity only, which the server can compute itself. As such, the client’s reactions leak nothing about the decrypted values, which rules out reaction attacks Chillotti et al. (2016). Integrity. By soundness of the underlying protocol, any server deviation in round i, be it substituting the matrix, corrupting the ciphertext or fabricating the proof, causes πi _i to fail verification except with negligible probability: Pr[(πi)=∧i≠i⋅i]≤(λ), [ Verify( _i)= Accept\; \;y_i _i·x_i ]≤ negl (λ ), where (1,1)=(,)(M_1,x_1)=(A,x) and (2,2)=(⊤,i∗)(M_2,x_2)=(D ,e_i^*). Each round is verified against its own commitment, so a union bound over the two rounds gives the claim for the composed protocol. Appendix D Batched Top-k Retrieval After Round 1, the client can identify the top-k indices i1∗,…,ik∗i_1^*,…,i_k^* and issue k independent Round-2 queries. Using a single k-hot vector instead does not work: ⊤⋅∑jij∗D · _je_i_j^* yields the sum of the k documents, from which the individual documents cannot be recovered. For moderate k, the k queries batch into a single matrix–matrix product ⊤⋅D ·E, where ∈0,1m×kE∈\0,1\^m× k has one-hot columns. Why latency does not grow with k. The verifiable matrix–vector multiplication of Tremblay Thibault et al. (2026) proves the relation ′=⋅c =M·c with a sum-check protocol compiled with a polynomial commitment scheme (PCS). Once per database, the server commits to a polynomial encoding of M. To answer a query, it runs the sum-check, which reduces the claim ′=⋅c =M·c to the evaluation of the committed polynomial at a single random point fixed during the protocol, and then produces a PCS evaluation proof for that point. The evaluation proof is the dominant cost factor: it accounts for over 95%95\% of the server time. Batching exploits the fact that the sum-check protocol is linear in the claim it proves. More precisely, given k claims j′=⋅jc _j=M·c_j about the same committed matrix, the verifier samples a random challenge γ and the prover proves the single claim ∑jγjj′=⋅∑jγjj _jγ^jc _j=M· _jγ^jc_j instead. If any of the k claims is false, the combined claim is false as well except with probability at most k/||k/|C| over the choice of γ, for C the challenge space. This additive soundness loss is negligible provided |||C| is large enough. The batched claim is a matrix–vector relation of the same shape as a single query. As such, it is proved with one run of the sum-check and one PCS evaluation proof, regardless of k. The server still computes the k products and the sum-check messages of the combined claim, whose cost grows with k, but the evaluation proof, which dominates, is generated once. The k answers are all returned since the client needs each document, which is why communication grows by a factor k whereas the proof is essentially of the same size as for a single query. Appendix E Readers, Prompts and Downstream Corpora Protocol description. We evaluate every endpoint, encoder and condition c∈closed,gold,float,8,6,4,3,2c∈\closed,gold,float,8,6,4,3,2\. The evaluation of §4.4 works as follows. (1) The corpus and the 300300 queries are embedded once in floating point. For a bit-width c=bc=b, both are quantized with the clipped quantizer of Appendix A.6. They are then scored in integers, as in Round 1, and the top-k passages are kept. Ties are broken toward the lower index. The float condition uses the floating-point scores. The gold condition uses the labelled relevant passages, at most k of them. The closed condition uses no passage. (2) We build one prompt from the template of the endpoint (below), with the passages in rank order. A prompt that is identical across conditions is answered once. (3) The reader answers with greedy decoding. Decoding is constrained to the three labels for claim verification and capped at 2424 tokens for QA. (4) A QA answer is correct if the normalised text of any reference answer appears in the normalised response. A claim label is correct if it equals the human label. SciFact accuracy is computed over the 188188 claims with evidence in the corpus. (5) We compare per-query correctness under each condition with the float condition on the same queries, using the tests described below. We record retrieval-side statistics alongside, namely top-k overlap with float and whether a relevant passage is in the top k. The prompts and k were fixed before any bit-width was run. Nothing is tuned on the test queries. Readers. The body reports phi-4 (1414B, MIT licence) (Abdin et al., 2024). Table 6 adds OLMo-2-7B-Instruct (OLMo Team, 2025), whose training data is fully open, and the Qwen2.5 instruction-tuned models at 77B and 7272B (Qwen Team, 2025). All are run locally with vLLM and greedy decoding. Under the one-word prompt, OLMo-2 mostly abstains on claim verification. It answers NoInfo on 87%87\% of SciFact prompts and never Contradict, so its absolute SciFact accuracy is low. It still reproduces the shape of the degradation curve, which is what we compare across readers. Prompts. The claim-verification prompt states the claim and lists the retrieved abstracts. It asks for exactly one word from Support, Contradict, NoInfo. Decoding is constrained to those three strings. As such, there are no parse failures and no exclusions. The QA prompt states the question and lists the retrieved passages. It asks for the answer in a few words, or for exactly Unknown if the passages do not contain it. Responses are capped at 2424 tokens. Prompts are fixed across bit-widths and encoders. Only the retrieved passages change. One SciFact k=5k=5 prompt exceeds OLMo-2’s 4,0964,096-token context and is excluded for that reader. BioASQ corpus. We build the corpus from BioASQ-QA (Krithara et al., 2023) (training11b, 4,7194,719 questions). For every question of every type, we group the evidence snippets by source document. We concatenate them into one passage per (question, document) pair. This gives 35,45435,454 passages with a median length of 220220 characters. Queries are a seeded sample of 300300 factoid questions with a non-empty exact_answer and at most three gold passages. Of these, 190190 have one gold passage, 5454 have two and 5656 have three. The cap keeps gold-in-top-k from being trivially high. The relevant set of a question is its own passages. A response is correct if the normalised text of any reference answer synonym appears in it. NQ at scale. The 300300 Natural Questions test queries are the 100100 of the original evaluation plus 200200 more. These 200200 are a seeded draw among the queries whose question matches an NQ-open answer list. Passages are retrieved from the full BEIR corpus of 2,681,4682,681,468 Wikipedia passages. We use the same embeddings as the scale run of Appendix F. Statistics. For every (reader, endpoint, encoder) and bit-width, we report the paired difference in accuracy from floating point, with a normal-approximation 95%95\% interval on the per-query differences. Table 6 gives these differences at b∈4,3,2b∈\4,3,2\. We test for a drop with an exact McNemar test on the discordant pairs. A non-significant McNemar test is not evidence of equivalence. As such, we also run two one-sided tests (TOST) with equivalence margins of 55 and 33 points. A bit-width is declared equivalent to floating point when both one-sided tests reject at p<0.05p<0.05, i.e. when the 90%90\% interval lies inside the margin. At b=4b=4, all 3232 combinations are equivalent within 55 points and 2323 are equivalent within 33. At b=3b=3, these counts are 3030 and 1717. At b=2b=2, they are 99 and 22. Table 6: Paired accuracy difference from floating point, in points with 95%95\% interval, at b=4b=4, 33 and 22. All readers, endpoints and encoders use the clipped quantizer (n=300n=300 queries, 188188 evidence-bearing claims for SciFact). ∗: significant drop, exact McNemar p<0.05p<0.05. †: equivalent to floating point within ±5± 5 points, TOST p<0.05p<0.05. “float” is the floating-point accuracy. Endpoint Encoder float b=4b=4 b=3b=3 b=2b=2 phi-4 (14B) SciFact k=5k=5 BGE 0.7340.734 −1.1-1.1 [−3.6,+1.5][-3.6,+1.5]† −2.7-2.7 [−5.4,+0.1][-5.4,+0.1]† −5.9-5.9 [−10.1,−1.6][-10.1,-1.6]∗ SciFact k=5k=5 arctic 0.7130.713 +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† +0.0+0.0 [−2.6,+2.6][-2.6,+2.6]† −1.6-1.6 [−4.7,+1.5][-4.7,+1.5]† SciFact k=1k=1 BGE 0.7660.766 +0.5+0.5 [−0.5,+1.6][-0.5,+1.6]† −0.5-0.5 [−2.3,+1.3][-2.3,+1.3]† −8.0-8.0 [−12.1,−3.8][-12.1,-3.8]∗ SciFact k=1k=1 arctic 0.7390.739 +0.0+0.0 [+0.0,+0.0][+0.0,+0.0]† +1.6+1.6 [−0.2,+3.4][-0.2,+3.4]† −5.9-5.9 [−10.1,−1.6][-10.1,-1.6]∗ BioASQ BGE 0.6330.633 +0.3+0.3 [−1.4,+2.1][-1.4,+2.1]† +1.0+1.0 [−0.7,+2.7][-0.7,+2.7]† −2.0-2.0 [−4.6,+0.6][-4.6,+0.6]† BioASQ arctic 0.6470.647 −1.0-1.0 [−2.5,+0.5][-2.5,+0.5]† −0.7-0.7 [−2.3,+0.9][-2.3,+0.9]† −4.0-4.0 [−6.6,−1.4][-6.6,-1.4]∗ NQ (2.682.68M) BGE 0.6430.643 +0.7+0.7 [−2.1,+3.4][-2.1,+3.4]† −2.0-2.0 [−5.3,+1.3][-5.3,+1.3]† −7.0-7.0 [−11.5,−2.5][-11.5,-2.5]∗ NQ (2.682.68M) arctic 0.6430.643 −1.7-1.7 [−3.6,+0.3][-3.6,+0.3]† −0.3-0.3 [−2.9,+2.2][-2.9,+2.2]† −4.3-4.3 [−8.4,−0.3][-8.4,-0.3] OLMo-2 (7B) SciFact k=5k=5 BGE 0.2130.213 −0.5-0.5 [−3.3,+2.2][-3.3,+2.2]† −1.6-1.6 [−3.9,+0.7][-3.9,+0.7]† −6.4-6.4 [−10.7,−2.0][-10.7,-2.0]∗ SciFact k=5k=5 arctic 0.1980.198 −1.1-1.1 [−3.2,+1.0][-3.2,+1.0]† −2.1-2.1 [−4.7,+0.4][-4.7,+0.4]† −2.1-2.1 [−5.4,+1.2][-5.4,+1.2]† SciFact k=1k=1 BGE 0.5050.505 +1.1+1.1 [−0.4,+2.5][-0.4,+2.5]† −1.1-1.1 [−3.1,+1.0][-3.1,+1.0]† −4.8-4.8 [−8.5,−1.1][-8.5,-1.1]∗ SciFact k=1k=1 arctic 0.4790.479 +0.0+0.0 [−1.5,+1.5][-1.5,+1.5]† +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† −3.7-3.7 [−7.1,−0.3][-7.1,-0.3] BioASQ BGE 0.5030.503 +1.0+1.0 [−1.5,+3.5][-1.5,+3.5]† +0.0+0.0 [−3.1,+3.1][-3.1,+3.1]† −5.0-5.0 [−9.3,−0.7][-9.3,-0.7]∗ BioASQ arctic 0.5200.520 −0.3-0.3 [−2.9,+2.2][-2.9,+2.2]† −1.3-1.3 [−3.9,+1.3][-3.9,+1.3]† −2.7-2.7 [−6.0,+0.7][-6.0,+0.7] NQ (2.682.68M) BGE 0.4970.497 +1.0+1.0 [−2.1,+4.1][-2.1,+4.1]† −1.3-1.3 [−4.9,+2.2][-4.9,+2.2]† −5.0-5.0 [−9.5,−0.5][-9.5,-0.5]∗ NQ (2.682.68M) arctic 0.5630.563 −0.3-0.3 [−2.5,+1.8][-2.5,+1.8]† −3.3-3.3 [−6.2,−0.4][-6.2,-0.4]∗ −8.3-8.3 [−12.2,−4.5][-12.2,-4.5]∗ Qwen2.5-7B SciFact k=5k=5 BGE 0.6810.681 −0.5-0.5 [−3.7,+2.6][-3.7,+2.6]† −1.6-1.6 [−4.7,+1.5][-4.7,+1.5]† −5.3-5.3 [−9.9,−0.7][-9.9,-0.7]∗ SciFact k=5k=5 arctic 0.6540.654 −0.5-0.5 [−2.3,+1.3][-2.3,+1.3]† +1.1+1.1 [−1.5,+3.6][-1.5,+3.6]† +0.0+0.0 [−3.3,+3.3][-3.3,+3.3]† SciFact k=1k=1 BGE 0.6650.665 +0.5+0.5 [−0.5,+1.6][-0.5,+1.6]† −1.1-1.1 [−3.1,+1.0][-3.1,+1.0]† −9.0-9.0 [−13.4,−4.7][-13.4,-4.7]∗ SciFact k=1k=1 arctic 0.6440.644 −0.5-0.5 [−1.6,+0.5][-1.6,+0.5]† +0.0+0.0 [−2.1,+2.1][-2.1,+2.1]† −5.9-5.9 [−10.3,−1.4][-10.3,-1.4]∗ BioASQ BGE 0.4770.477 −1.7-1.7 [−4.0,+0.7][-4.0,+0.7]† −1.3-1.3 [−3.9,+1.3][-3.9,+1.3]† −4.3-4.3 [−7.8,−0.8][-7.8,-0.8]∗ BioASQ arctic 0.4700.470 −0.3-0.3 [−2.3,+1.6][-2.3,+1.6]† −1.0-1.0 [−3.7,+1.7][-3.7,+1.7]† −1.7-1.7 [−4.8,+1.5][-4.8,+1.5]† NQ (2.682.68M) BGE 0.3530.353 +1.7+1.7 [−1.0,+4.4][-1.0,+4.4]† +1.7+1.7 [−1.6,+4.9][-1.6,+4.9]† −1.3-1.3 [−5.6,+2.9][-5.6,+2.9]† NQ (2.682.68M) arctic 0.4130.413 −1.7-1.7 [−3.8,+0.5][-3.8,+0.5]† −2.3-2.3 [−5.2,+0.5][-5.2,+0.5]† −5.3-5.3 [−9.4,−1.2][-9.4,-1.2]∗ Qwen2.5-72B SciFact k=5k=5 BGE 0.8300.830 +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† −2.7-2.7 [−5.4,+0.1][-5.4,+0.1]† SciFact k=5k=5 arctic 0.8190.819 +0.0+0.0 [+0.0,+0.0][+0.0,+0.0]† +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† −0.5-0.5 [−3.3,+2.2][-3.3,+2.2]† SciFact k=1k=1 BGE 0.8300.830 +0.5+0.5 [−0.5,+1.6][-0.5,+1.6]† −1.1-1.1 [−3.1,+1.0][-3.1,+1.0]† −9.0-9.0 [−13.7,−4.4][-13.7,-4.4]∗ SciFact k=1k=1 arctic 0.7930.793 +0.0+0.0 [+0.0,+0.0][+0.0,+0.0]† +0.0+0.0 [−2.6,+2.6][-2.6,+2.6]† −7.4-7.4 [−12.0,−2.9][-12.0,-2.9]∗ BioASQ BGE 0.5800.580 +0.0+0.0 [−0.9,+0.9][-0.9,+0.9]† +0.7+0.7 [−0.9,+2.3][-0.9,+2.3]† −1.7-1.7 [−4.5,+1.2][-4.5,+1.2]† BioASQ arctic 0.5970.597 +0.0+0.0 [−1.3,+1.3][-1.3,+1.3]† −0.3-0.3 [−1.5,+0.8][-1.5,+0.8]† −3.3-3.3 [−6.1,−0.6][-6.1,-0.6]∗ NQ (2.682.68M) BGE 0.5730.573 −1.7-1.7 [−4.0,+0.7][-4.0,+0.7]† −2.7-2.7 [−5.7,+0.4][-5.7,+0.4] −6.7-6.7 [−10.5,−2.8][-10.5,-2.8]∗ NQ (2.682.68M) arctic 0.5670.567 +0.3+0.3 [−1.4,+2.1][-1.4,+2.1]† +0.0+0.0 [−2.3,+2.3][-2.3,+2.3]† −5.0-5.0 [−8.7,−1.3][-8.7,-1.3]∗ Chunked SciFact. Table 7 gives the SciFact endpoint for phi-4 when the reader receives the top-k chunks of §4 instead of whole abstracts. Chunks are cut at sentence boundaries and hold at most 1,5361,536 bytes including the title (implem/scale/chunk_corpus.py). The 5,1835,183 abstracts give 7,5107,510 chunks of 1,0701,070 bytes on average. In floating point, a chunk of a gold abstract is in the top 55 for 79%79\% of claims with BGE and 77%77\% with arctic. For whole abstracts, these figures are 8080 and 79%79\%. NFCorpus, chunked the same way, gives 5,7615,761 chunks, and 44%44\% of its abstracts fit in one. In floating point, the reader does better on five chunks than on five abstracts, by 4.34.3 points with BGE and 2.12.1 with arctic. At k=1k=1 it does about one point worse, since a single chunk may leave out the evidence sentence. Under quantization, the two encoders separate one bit earlier than on abstracts. The reason is that chunks of one abstract are near-duplicates, and quantization reorders them. Judged per document, the ranking survives exactly as on abstracts (Table 8). The reader, judged per chunk, achieves top-1 agreement at b=3b=3 is 0.820.82 for BGE and 0.890.89 for arctic, against 0.870.87 and 0.930.93 per document (Appendix F). With BGE at k=5k=5, the loss reaches 4.34.3 points at b=3b=3 (p=0.008p=0.008). On abstracts, it was 2.72.7 points and not significant. With arctic at k=5k=5, every bit-width down to b=2b=2 stays within 55 points of floating point, as on abstracts. As such, chunking moves the operating point of the conventional encoder from b=3b=3 to b=4b=4. The quantization-aware encoder stays at b=2b=2. Table 7: SciFact with chunks against whole abstracts, reader phi-4. We give floating-point accuracy on the 188188 evidence-bearing claims. We also give the paired difference from floating point at b=4b=4, 33 and 22, in points with 95%95\% interval. ∗: significant drop, exact McNemar p<0.05p<0.05. †: equivalent within ±5± 5 points, TOST p<0.05p<0.05. Endpoint Encoder Unit float b=4b=4 b=3b=3 b=2b=2 SciFact k=5k=5 BGE abstracts 0.7340.734 −1.1-1.1 [−3.6,+1.5][-3.6,+1.5]† −2.7-2.7 [−5.4,+0.1][-5.4,+0.1]† −5.9-5.9 [−10.1,−1.6][-10.1,-1.6]∗ chunks 0.7770.777 −2.7-2.7 [−5.0,−0.4][-5.0,-0.4]† −4.3-4.3 [−7.1,−1.4][-7.1,-1.4]∗ −6.9-6.9 [−11.4,−2.5][-11.4,-2.5]∗ SciFact k=5k=5 arctic abstracts 0.7130.713 +0.5+0.5 [−1.3,+2.3][-1.3,+2.3]† +0.0+0.0 [−2.6,+2.6][-2.6,+2.6]† −1.6-1.6 [−4.7,+1.5][-4.7,+1.5]† chunks 0.7340.734 +1.6+1.6 [−0.7,+3.9][-0.7,+3.9]† +1.6+1.6 [−1.9,+5.1][-1.9,+5.1]† −1.1-1.1 [−5.2,+3.1][-5.2,+3.1]† SciFact k=1k=1 BGE abstracts 0.7660.766 +0.5+0.5 [−0.5,+1.6][-0.5,+1.6]† −0.5-0.5 [−2.3,+1.3][-2.3,+1.3]† −8.0-8.0 [−12.1,−3.8][-12.1,-3.8]∗ chunks 0.7550.755 −0.5-0.5 [−2.3,+1.3][-2.3,+1.3]† −2.1-2.1 [−4.7,+0.4][-4.7,+0.4]† −12.2-12.2 [−17.6,−6.9][-17.6,-6.9]∗ SciFact k=1k=1 arctic abstracts 0.7390.739 +0.0+0.0 [+0.0,+0.0][+0.0,+0.0]† +1.6+1.6 [−0.2,+3.4][-0.2,+3.4]† −5.9-5.9 [−10.1,−1.6][-10.1,-1.6]∗ chunks 0.7290.729 +0.5+0.5 [−1.8,+2.9][-1.8,+2.9]† +0.0+0.0 [−3.0,+3.0][-3.0,+3.0]† −5.9-5.9 [−10.6,−1.1][-10.6,-1.1]∗ Appendix F The Encoder Suite and the Scale Run In this appendix we give the retrieval-side numbers behind §4.1 and §4.3 for every encoder. We also compare them with the max-abs quantizer and explain why some encoders lose more quality than others. Encoders and prefixes. Each encoder is run with the instruction prefix its model card prescribes. bge-large-en-v1.5 and mxbai-embed-large-v1 prepend “Represent this sentence for searching relevant passages: ” to queries. e5-large-v2 prepends “query: ” and “passage: ”. arctic-embed-l-v2.0 prepends “query: ”. gte-large uses no prefix, and Cohere takes an input type instead. We run arctic-embed-l-v2.0 with a 512512-token window instead of its native 8,1928,192, to match the other encoders. Abstracts rarely exceed this window. The downstream evaluation of §4.4 uses the clipped quantizer and these prefixes. Retained quality under the clipped quantizer. Table 9 gives floating-point nDCG@10 for every encoder and corpus. It also gives the fraction of it retained at b=4b=4 and b=3b=3, and top-1 agreement with the floating-point ranking at the same bit-widths. Table 10 gives the same quantities under max-abs scaling next to the clipped values. Table 8: Retrieval quality retained of our two-round protocol under clipped quantization, as the percentage of floating-point nDCG@10 capped at 100100, for b∈2,3,4,8b∈\2,3,4,8\ on SciFact and NFCorpus, ranked per chunk and scored per document. SciFact NFCorpus low-bit conventional low-bit conventional b arctic Cohere BGE mxbai e5 gte arctic Cohere BGE mxbai e5 gte 22 92.292.2 89.789.7 86.486.4 88.588.5 82.282.2 86.686.6 88.288.2 88.688.6 85.985.9 86.586.5 78.878.8 83.683.6 33 98.798.7 99.199.1 97.497.4 98.798.7 97.397.3 97.597.5 98.298.2 99.399.3 99.099.0 97.497.4 94.194.1 97.697.6 4 99.999.9 99.499.4 99.799.7 99.499.4 100.0100.0 99.899.8 99.699.6 99.899.8 99.599.5 98.998.9 96.596.5 100.0100.0 88 99.999.9 99.599.5 99.999.9 100.0100.0 100.0100.0 99.899.8 99.799.7 100.0100.0 99.799.7 100.0100.0 98.798.7 100.0100.0 Table 9: The encoder suite under the clipped quantizer. Floating-point nDCG@10, the percentage of it retained at b=4b=4 and b=3b=3, and top-1 agreement with the floating-point ranking at the same bit-widths. ✓ : trained for low-bit readout. SciFact NFCorpus Encoder low-bit float ret. 44 ret. 33 top-1 44 top-1 33 float ret. 44 ret. 33 top-1 44 top-1 33 arctic-embed-l-v2.0 ✓ 0.7060.706 100.4100.4 99.499.4 0.960.96 0.930.93 0.3540.354 99.699.6 98.098.0 0.910.91 0.820.82 Cohere embed-v3 ✓ 0.7180.718 99.699.6 98.798.7 0.960.96 0.910.91 0.3870.387 99.599.5 98.998.9 0.860.86 0.780.78 bge-large-en-v1.5 × 0.7460.746 99.499.4 97.197.1 0.940.94 0.870.87 0.3810.381 99.599.5 98.698.6 0.830.83 0.710.71 mxbai-embed-large-v1 × 0.7390.739 100.3100.3 100.1100.1 0.930.93 0.890.89 0.3870.387 100.1100.1 98.998.9 0.840.84 0.730.73 e5-large-v2 × 0.7220.722 99.499.4 96.896.8 0.880.88 0.770.77 0.3720.372 96.396.3 90.890.8 0.790.79 0.640.64 gte-large × 0.7430.743 99.799.7 97.897.8 0.940.94 0.810.81 0.3830.383 100.0100.0 95.295.2 0.810.81 0.620.62 Table 10: The quantizer lever. Each cell reads max-abs scaling → clipping at the 9999th percentile. We give the percentage of floating-point nDCG@10 retained and top-1 agreement at b=4b=4 and b=3b=3. Nothing on the cryptographic side differs between the two columns of each pair. b=4b=4 b=3b=3 Encoder ret. (%) top-1 ret. (%) top-1 SciFact arctic-embed-l-v2.0 100.0→100.4100.0→ 100.4 0.93→0.960.93→ 0.96 97.6→99.497.6→ 99.4 0.83→0.930.83→ 0.93 Cohere embed-v3 100.0→99.6100.0→ 99.6 0.93→0.960.93→ 0.96 95.4→98.795.4→ 98.7 0.74→0.910.74→ 0.91 bge-large-en-v1.5 97.1→99.497.1→ 99.4 0.79→0.940.79→ 0.94 77.3→97.177.3→ 97.1 0.54→0.870.54→ 0.87 mxbai-embed-large-v1 99.5→100.399.5→ 100.3 0.81→0.930.81→ 0.93 82.3→100.182.3→ 100.1 0.57→0.890.57→ 0.89 e5-large-v2 96.3→99.496.3→ 99.4 0.79→0.880.79→ 0.88 82.4→96.882.4→ 96.8 0.56→0.770.56→ 0.77 gte-large 97.3→99.797.3→ 99.7 0.81→0.940.81→ 0.94 88.1→97.888.1→ 97.8 0.68→0.810.68→ 0.81 NFCorpus arctic-embed-l-v2.0 98.5→99.698.5→ 99.6 0.83→0.910.83→ 0.91 96.2→98.096.2→ 98.0 0.69→0.820.69→ 0.82 Cohere embed-v3 98.5→99.598.5→ 99.5 0.83→0.860.83→ 0.86 96.4→98.996.4→ 98.9 0.61→0.780.61→ 0.78 bge-large-en-v1.5 96.0→99.596.0→ 99.5 0.67→0.830.67→ 0.83 73.4→98.673.4→ 98.6 0.35→0.710.35→ 0.71 mxbai-embed-large-v1 96.2→100.196.2→ 100.1 0.63→0.840.63→ 0.84 77.4→98.977.4→ 98.9 0.37→0.730.37→ 0.73 e5-large-v2 93.0→96.393.0→ 96.3 0.65→0.790.65→ 0.79 78.4→90.878.4→ 90.8 0.37→0.640.37→ 0.64 gte-large 96.3→100.096.3→ 100.0 0.62→0.810.62→ 0.81 85.0→95.285.0→ 95.2 0.42→0.620.42→ 0.62 Why some encoders lose more than others. Quantization adds a small rounding error to every score. A query keeps its top-1 document if the gap between its two best scores is larger than this error. Rounding to a step s gives an error that is uniform on [−s/2,s/2][-s/2,s/2], whose variance is s2/12s^2/12. The error on a score therefore has standard deviation (s2+s2)/12 (s_A^2+s_x^2)/12, where s_A is the corpus step and s_x the query step. For each query, we divide the gap by this standard deviation. We then take the median over queries. Table 11 gives this ratio for every encoder at b=4b=4. The larger the ratio, the more often the top-1 document survives. Over the 1212 encoder-corpus pairs, the ratio and the measured top-1 agreement have Spearman correlation 0.940.94 at b=4b=4 and 0.990.99 at b=3b=3. The two encoders trained for low-bit readout have the largest ratios. BGE and mxbai come next, and e5 and gte have the smallest. The table also explains why clipping helps BGE and mxbai. Their largest coordinate holds over 5%5\% of the squared norm, against at most 1.9%1.9\% for the other encoders. Under max-abs scaling, this one coordinate sets the step. For BGE on SciFact, the step would be 0.305/7≈0.0440.305/7≈ 0.044 instead of the clipped 0.0110.011, i.e. about 4×4× coarser. Table 11: Score gap against rounding error at b=4b=4, clipped quantizer. max⁡|| |A|: largest corpus coordinate. s_A: clipped step. top dim.: share of the squared norm in the largest coordinate. gap: median gap between the two best floating-point scores. gap/noise: median ratio of that gap to the rounding error. top-1: top-1 agreement with floating point. Encoder Corpus max⁡|| |A| s_A top dim. (%) gap gap/noise top-1 arctic-embed-l-v2.0 SciFact 0.2000.200 0.01220.0122 0.50.5 0.05820.0582 11.7211.72 0.960.96 Cohere embed-v3 SciFact 0.2210.221 0.01250.0125 0.80.8 0.04450.0445 8.838.83 0.960.96 bge-large-en-v1.5 SciFact 0.3050.305 0.01100.0110 5.55.5 0.03190.0319 7.007.00 0.940.94 mxbai-embed-large-v1 SciFact 0.3110.311 0.01110.0111 5.45.4 0.03450.0345 7.737.73 0.930.93 e5-large-v2 SciFact 0.1700.170 0.01050.0105 1.91.9 0.01490.0149 3.483.48 0.880.88 gte-large SciFact 0.1720.172 0.01070.0107 1.71.7 0.01890.0189 4.384.38 0.940.94 arctic-embed-l-v2.0 NFCorpus 0.1940.194 0.01230.0123 0.50.5 0.02680.0268 5.225.22 0.910.91 Cohere embed-v3 NFCorpus 0.2160.216 0.01260.0126 1.01.0 0.02110.0211 4.194.19 0.860.86 bge-large-en-v1.5 NFCorpus 0.2870.287 0.01090.0109 5.75.7 0.01170.0117 2.642.64 0.830.83 mxbai-embed-large-v1 NFCorpus 0.2920.292 0.01100.0110 5.85.8 0.01350.0135 2.972.97 0.840.84 e5-large-v2 NFCorpus 0.1650.165 0.01050.0105 1.91.9 0.00630.0063 1.501.50 0.790.79 gte-large NFCorpus 0.1750.175 0.01070.0107 1.71.7 0.00730.0073 1.671.67 0.810.81 Table 12: Table 8 on whole abstracts instead of chunks. This checks comparability against published document-level baselines. We give the percentage of floating-point nDCG@10 retained under clipped quantization for each bit-width b. Trained for low-bit readout Conventional b arctic Cohere BGE mxbai e5 gte SciFact 22 91.891.8 92.192.1 87.087.0 90.490.4 84.084.0 87.687.6 33 99.499.4 98.798.7 97.197.1 100.1100.1 96.896.8 97.897.8 4 100.4100.4 99.699.6 99.499.4 100.3100.3 99.499.4 99.799.7 55 100.7100.7 99.899.8 99.099.0 99.999.9 101.3101.3 99.299.2 66 100.1100.1 99.799.7 99.499.4 100.0100.0 101.2101.2 99.799.7 77 100.2100.2 99.899.8 99.299.2 100.0100.0 100.9100.9 99.799.7 88 100.0100.0 99.799.7 99.199.1 100.2100.2 100.9100.9 99.899.8 NFCorpus 22 87.287.2 88.788.7 83.283.2 87.487.4 75.075.0 82.782.7 33 98.098.0 98.998.9 98.698.6 98.998.9 90.890.8 95.295.2 4 99.699.6 99.599.5 99.599.5 100.1100.1 96.396.3 100.0100.0 55 99.299.2 99.899.8 100.6100.6 100.7100.7 97.897.8 98.798.7 66 99.999.9 99.799.7 100.4100.4 101.0101.0 98.598.5 99.899.8 77 99.999.9 99.899.8 100.5100.5 100.7100.7 98.698.6 99.799.7 88 99.699.6 99.799.7 100.4100.4 100.8100.8 98.598.5 99.799.7 Chunk-level retrieval. We first judge the chunked SciFact corpus per document, as in Table 8. Top-1 agreement at b=4b=4 and b=3b=3 is then 0.940.94 and 0.870.87 for BGE and 0.960.96 and 0.920.92 for arctic. These are the same values as on whole abstracts (Table 12). We then judge it per chunk, counting every chunk of a relevant abstract as relevant. Top-1 agreement is then 0.920.92 and 0.820.82 for BGE and 0.930.93 and 0.890.89 for arctic. The difference comes from chunks of one abstract trading places under rounding. The reader of §4.4 sees this reordering, but a document ranking does not. Quality at scale. Tables 14 and 13 give the measurements behind the quality-at-scale paragraph of §4.1. They cover the five encoders of the suite that run locally, namely arctic, BGE, mxbai, e5 and gte. Cohere is hosted and was not run at this scale. The corpus is the BEIR Natural Questions passage collection (2,681,4682,681,468 passages) with its 3,4523,452 test queries. We draw subsets of 10410^4, 10510^5 and 10610^6 passages at random with a fixed seed. Each subset contains every passage relevant to any query. As such, nDCG is defined at every size. Embeddings are computed once per encoder on one H100 and stored in half precision. The clipped percentile is estimated from a random sample of 4,0964,096 rows of each subset. Quantized scores are computed in single precision. This is exact for b≤8b≤ 8, since every partial sum is an integer below 2242^24. We also check the scores against 64-bit integer arithmetic. Ties are broken toward the lower index. Recall at the reader’s k. The sweep above scores at k=10k=10. The readers of §4.4 consume five passages. As such, we repeated the sweep at k=5k=5 and added b=2b=2. Table 13 gives Recall@5 retained at every size. At b=4b=4, four of the five encoders retain at least 98.8%98.8\% of floating-point Recall@5 at every size. The fifth, e5, retains 97.3%97.3\% at 2.682.68M. At b=3b=3, arctic, BGE and mxbai retain at least 98.4%98.4\%. At 2.682.68M, gte and e5 fall to 94.594.5 and 92.0%92.0\%. At b=2b=2, the loss grows with the corpus. It goes from 11 to 4%4\% at 10410^4 to 1010 to 23%23\% at 2.682.68M. The lead of arctic over each conventional encoder also grows, from 0.80.8 to 2.72.7 points at 10410^4 to 2.12.1 to 12.812.8 points at 2.682.68M. This is the retrieval-side counterpart of the downstream drop of §4.4. On the 300300 NQ questions of that section, the top-55 set of BGE contained a relevant passage for 70%70\% of questions in floating point and 60%60\% at b=2b=2. This is 86%86\% retained, against 86.1%86.1\% measured here on all 3,4523,452 queries. Table 13: Recall@5 retained under the clipped quantizer versus corpus size on BEIR Natural Questions (3,4523,452 queries), at b∈2,3,4b∈\2,3,4\. Floating-point Recall@5 is given for reference. Generated by implem/analysis/scale_recall5_table.py. Recall@5 retained (%) Encoder m Recall@5 (float) b=2b=2 b=3b=3 b=4b=4 arctic-embed-l-v2.0 10410^4 0.9880.988 99.299.2 99.999.9 100.0100.0 10510^5 0.9670.967 97.097.0 99.999.9 99.999.9 10610^6 0.8480.848 93.193.1 99.599.5 99.799.7 2.68⋅1062.68·10^6 0.7370.737 89.989.9 99.099.0 99.899.8 bge-large-en-v1.5 10410^4 0.9820.982 97.797.7 99.999.9 100.1100.1 10510^5 0.9440.944 94.794.7 99.299.2 99.599.5 10610^6 0.7940.794 88.388.3 98.798.7 98.898.8 2.68⋅1062.68·10^6 0.6550.655 86.186.1 98.598.5 99.799.7 mxbai-embed-large-v1 10410^4 0.9830.983 98.498.4 99.999.9 100.1100.1 10510^5 0.9440.944 96.396.3 99.699.6 99.999.9 10610^6 0.8000.800 90.490.4 98.898.8 99.499.4 2.68⋅1062.68·10^6 0.6610.661 87.787.7 98.498.4 100.6100.6 e5-large-v2 10410^4 0.9880.988 96.596.5 99.599.5 99.899.8 10510^5 0.9610.961 92.192.1 98.598.5 99.599.5 10610^6 0.8470.847 82.782.7 95.895.8 98.398.3 2.68⋅1062.68·10^6 0.7450.745 77.177.1 92.092.0 97.397.3 gte-large 10410^4 0.9860.986 97.297.2 99.799.7 99.899.8 10510^5 0.9450.945 94.494.4 99.499.4 100.0100.0 10610^6 0.7950.795 85.985.9 96.396.3 99.999.9 2.68⋅1062.68·10^6 0.6580.658 82.682.6 94.594.5 99.699.6 Table 14: Quality versus corpus size on BEIR Natural Questions with the clipped quantizer. We give nDCG@10 retained at b=3b=3 and b=4b=4. We also give the smallest b that retains 99%99\% of floating-point nDCG@10, or >8>8 when no b up to 88 does. Generated by implem/analysis/scale_analysis.py. nDCG@10 ret. (%) required b Encoder m b=3b=3 b=4b=4 (top-k) arctic-embed-l-v2.0 10410^4 100.0100.0 99.999.9 33 10510^5 99.999.9 100.1100.1 33 10610^6 99.199.1 99.999.9 33 2.68⋅1062.68·10^6 98.598.5 99.799.7 44 bge-large-en-v1.5 10410^4 99.599.5 99.999.9 33 10510^5 99.099.0 99.699.6 33 10610^6 98.098.0 99.199.1 44 2.68⋅1062.68·10^6 98.198.1 99.399.3 44 mxbai-embed-large-v1 10410^4 99.899.8 100.0100.0 33 10510^5 99.599.5 100.0100.0 33 10610^6 98.998.9 100.1100.1 44 2.68⋅1062.68·10^6 98.398.3 99.999.9 44 e5-large-v2 10410^4 99.099.0 99.799.7 33 10510^5 97.797.7 99.399.3 44 10610^6 93.993.9 98.198.1 88 2.68⋅1062.68·10^6 91.991.9 97.497.4 >8>8 gte-large 10410^4 99.499.4 99.999.9 33 10510^5 98.498.4 99.999.9 44 10610^6 95.795.7 99.999.9 44 2.68⋅1062.68·10^6 94.094.0 100.6100.6 44

Instant research discovery