Paper deep dive
Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing
Alessio Langiu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%
Last extracted: 4/1/2026, 1:28:34 AM
Summary
The paper introduces the 'Inseparability Paradigm,' which posits that advanced context management and privacy management are mathematically dual projections of the same contextual compression operator. The authors propose a 'Privacy Guard' framework using an on-premise Small Language Model (SLM) to perform abstractive summarization and Automatic Prompt Optimization (APO). This system decomposes prompts into sub-tasks, re-routes high-risk queries to secure models, and uses a LIFO-based context compacting mechanism to reduce operational costs (OpEx) and eliminate sensitive inference vectors (Zero Leakage).
Entities (5)
Relation Signals (4)
Privacy Guard → utilizes → Small Language Model
confidence 98% · a holistic contextual observer powered by an on-premise Small Language Model (SLM)
Privacy Guard → eliminates → Sensitive Inference Vectors
confidence 95% · simultaneously eliminates sensitive inference vectors (Zero Leakage)
Privacy Guard → performs → Automatic Prompt Optimization
confidence 95% · performs abstractive summarisation and Automatic Prompt Optimisation (APO)
Privacy Guard → reduces → Operational Cost
confidence 92% · achieving a 45% blended OpEx reduction
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The large-scale adoption of Large Language Models (LLMs) forces a trade-off between operational cost (OpEx) and data privacy. Current routing frameworks reduce costs but ignore prompt sensitivity, exposing users and institutions to leakage risks towards third-party cloud providers. We formalise the "Inseparability Paradigm": advanced context management intrinsically coincides with privacy management. We propose a local "Privacy Guard" -- a holistic contextual observer powered by an on-premise Small Language Model (SLM) -- that performs abstractive summarisation and Automatic Prompt Optimisation (APO) to decompose prompts into focused sub-tasks, re-routing high-risk queries to Zero-Trust or NDA-covered models. This dual mechanism simultaneously eliminates sensitive inference vectors (Zero Leakage) and reduces cloud token payloads (OpEx Reduction). A LIFO-based context compacting mechanism further bounds working memory, limiting the emergent leakage surface. We validate the framework through a 2x2 benchmark (Lazy vs. Expert users; Personal vs. Institutional secrets) on a 1,000-sample dataset, achieving a 45% blended OpEx reduction, 100% redaction success on personal secrets, and -- via LLM-as-a-Judge evaluation -- an 85% preference rate for APO-compressed responses over raw baselines. Our results demonstrate that Token Parsimony and Zero Leakage are mathematically dual projections of the same contextual compression operator.
Tags
Links
- Source: https://arxiv.org/abs/2603.28972v1
- Canonical: https://arxiv.org/abs/2603.28972v1
Trouble viewing inline? Open PDF directly →
Full Text
45,460 characters extracted from source content.
Expand or collapse full text
Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing Alessio Langiu National Research Council of Italy - Institute of Marine Sciences (CNR-ISMAR) alessio.langiu@cnr.it April 1, 2026 Abstract The large-scale adoption of Large Language Models (LLMs) forces a trade-off between operational cost (OpEx) and data privacy. Current routing frameworks [18, 4] reduce costs but ignore prompt sensitivity, expos- ing users and institutions to leakage risks towards third- party cloud providers. We formalise the “Inseparability Paradigm”: advanced context management intrinsically coincides with privacy management. We propose a lo- cal “Privacy Guard”—a holistic contextual observer pow- ered by an on-premise Small Language Model (SLM)— that performs abstractive summarisation and Automatic Prompt Optimisation (APO) to decompose prompts into focused sub-tasks, re-routing high-risk queries to Zero- Trust or NDA-covered models. This dual mechanism si- multaneously eliminates sensitive inference vectors (Zero Leakage) and reduces cloud token payloads (OpEx Re- duction). A LIFO-based context compacting mechanism further bounds working memory, limiting the emergent leakage surface. We validate the framework through a 2x2 benchmark (Lazy vs. Expert users; Personal vs. Institu- tional secrets) on a 1,000-sample dataset, achieving a 45% blended OpEx reduction, 100% redaction success on per- sonal secrets, and—via LLM-as-a-Judge evaluation—an 85% preference rate for APO-compressed responses over raw baselines. Our results demonstrate that Token Parsi- mony and Zero Leakage are mathematically dual projec- tions of the same contextual compression operator. Keywords: Large Language Models, Privacy Guard, Zero-Trust Architecture, Token Parsimony, Emergent Leakage, Edge Computing. CCS Concepts: · Security and privacy → Privacy- preserving protocols; · Computing methodologies → Natural language processing. 1 Introduction The integration of Foundation Models into corporate, governmental, and local edge research workflows has highlighted a distinct dichotomy in LLM systems re- search. On one hand, MLOps research vigorously pur- sues dynamic routing to minimise the financial burdens arising from querying frontier models. On the other hand, the cybersecurity community emphasises the vulnerabil- ities of such models to training data extraction and the inference of personal or corporate attributes, proposing rigorous sanitisation mechanisms. However, these two formidable challenges are inextricably linked by a single common denominator: the context of the prompt. This paper bridges this gap by introducing a compre- hensive Context-Aware Routing framework based on a Dual-Vault architecture. We argue that the semantic com- pression and algorithmic decomposition of a prompt do not act solely as efficiency optimisers, but operate as a powerful Information Flow Control mechanism. Our ap- proach segregates knowledge into a Personal Vault (pro- tecting individual secrets, sensitive data, and unpublished ideas from both external entities and the institution’s in- ternal monitoring) and an Institutional Vault (protecting firm secrets, algorithms, and project details from external 1 arXiv:2603.28972v1 [cs.CR] 30 Mar 2026 leakage). Furthermore, we explicitly address the profound chal- lenge of “emergent leakage”: the inadvertent exposure of corporate or personal secrets during long conversational sessions. Even when users actively attempt to maintain confidentiality, the accumulated context allows external models to piece together and infer the underlying intel- lectual property or sensitive attributes. To counter this, a local “Privacy Guard” acts as a holistic contextual ob- server. Rather than relying solely on simple, easily by- passed regex patterns, it monitors the entire prompt his- tory and actively rewrites and splits complex prompts into parallel or sequential sub-tasks. This dynamic prompt decomposition focuses the model’s work and minimises the required working con- text, acting as an intrinsic filter for privacy and secrets. When emergent inference risks are identified, the sys- tem subsequently blocks them or re-routes the calls to secure, NDA-covered, or on-premise Zero-Trust models. This “dual-purpose” strategy ensures that sensitive infor- mation never leaves the trusted perimeter, whilst simul- taneously minimising the token payload sent to the more expensive cloud LLMs. Finally, we present a rigorous benchmark structure to quantify both economic savings (Token Parsimony) and sanitisation efficacy, including a LIFO-optimised context compacting mechanism, laying the groundwork for Zero-Trust and cost-effective LLM orchestrations. 2 State of the Art: Privacy, Rout- ing and Context Management in LLMs The large-scale adoption of Large Language Models (LLMs) in corporate and personal contexts has raised crit- ical and interconnected challenges regarding the optimi- sation of operational costs (OpEx) and privacy protection. This section analyses recent literature detailing the evolu- tion of dynamic routing techniques, the intrinsic vulnera- bilities of LLMs, and mitigation mechanisms, highlight- ing the current gap in the joint management of context, privacy, and costs. 2.1 Dynamic Routing and Cost Optimisa- tion The continuous querying of state-of-the-art proprietary models (e.g., GPT-4o, Claude 3.7 Sonnet, Gemini 1.5 Pro) entails prohibitive costs on a large scale. To mitigate this issue, research has heavily focused on model routing and cascading architectures. Chen et al. [4] with FrugalGPT introduced a pioneering framework that learns to route queries to a cascade of LLMs, balancing the inference budget and accuracy. More recently, RouteLLM [18] for- malised the training of predictive routers based on human preferences, whilst open-source routing frameworks like LiteLLM [3] and Semantic Router [2] have democratised API-level routing for cost management across multiple providers. However, routing decisions are purely driven by model performance metrics and economic thresholds, without any semantic filtering of the request itself. Current Limitation: Although these works effectively optimise the cost-quality trade-off, they implicitly assume a homogeneous level of confidentiality. Routing is de- cided solely based on task complexity, completely ignor- ing the sensitivity of the data contained within the prompt. 2.2 LLM Vulnerabilities: Direct and Emer- gent Leakage Sending rich contexts to cloud LLMs exposes them to severe data leakage risks. Nasr et al. [16] empirically demonstrated how it is possible to extract gigabytes of training data, including PII (Personally Identifiable Infor- mation), from aligned models in production. Beyond direct memorisation, Staab et al. [20] explored an even more insidious vulnerability: the inference of per- sonal attributes. Their study demonstrates that current LLMs can deduce sensitive information with high preci- sion from seemingly innocuous and unstructured text pro- vided in the prompt. This phenomenon of “emergent leak- age” over long conversational sessions proves that sim- ple anonymisation based on Named Entity Recognition (NER) is no longer sufficient to guarantee context privacy. Recent works by Zhang et al. [22, 6] further demonstrate that attackers can exploit the conversational memory and in-context learning mechanisms of GPT-class models to explicitly reconstruct entire previous private conversa- tions. Complementarily, Liu et al. [13] demonstrate that 2 adversaries can inject malicious instructions directly into LLM-integrated applications to hijack model behaviour, while Yang et al. [26] empirically show that multi-turn conversational sessions are particularly susceptible to tar- geted leakage extraction, underscoring the absolute neces- sity of client-side contextual sanitisation. 2.3 User Laziness and the Data Dumping Problem Recent industry analyses highlight a significant discrep- ancy between best practices in prompt engineering and the actual behaviour of non-expert users. As noted by White et al. [23], while experts tend to minimise context and structure their prompts, the average user adopts op- portunistic behaviours (“user laziness”). Employees fre- quently use LLMs as indiscriminate receptacles for entire documents, pasting unredacted PDFs, financial reports, or codebases—a phenomenon termed “over-prompting” or “data dumping”. This behaviour generates two major criticalities. Firstly, it creates an extreme exposure to accidental data leakage, as quantified by industry reports [7] highlighting the massive volume of confidential data routinely pasted into consumer AI tools. Secondly, providing unneces- sarily vast contexts drastically increases operational costs (OpEx) through wasted input tokens. Furthermore, exces- sive context degrades the model’s ability to retrieve accu- rate information, exacerbating the well-documented “Lost in the middle” phenomenon [12], where LLMs fail to ac- cess relevant data buried within massive prompt dumps. 2.4 Mitigation Mechanisms: Proxies and SLMs To counter privacy risks without relinquishing the ca- pabilities of foundation models, the industry has devel- oped privacy firewalls like NeMo Guardrails [17] and de- terministic regex-based libraries [15], which offer pro- grammable bounds on prompt safety. However, they often rely on static deterministic rules rather than dynamic se- mantic analysis. A promising alternative involves Local Proxies backed by Small Language Models (SLMs). The use of a small- scale local LLM acting as an active “Privacy Guard” is gaining traction, leveraging specialised models hosted on platforms like HuggingFace. For instance, models such as Phi-3-Mini [1] for rapid logical routing, Llama-Guard [8] for comprehensive safety evaluation, or GLiNER [27] for advanced, bidirectional on-premise PII scrubbing offer robust, low-latency alternatives to relying solely on ex- ternal cloud moderation. 2.5 Context Management and Prompt De- composition The most recent frontier of research begins to overlap prompt reduction techniques (Prompt Compression) with privacy preservation. Certain studies, such as the recent work by Choi et al. [5] on CompactPrompt, are starting to highlight how intentional manipulation of context can act not only as an efficiency optimiser but also as a ”lossy” privacy filter for Information Flow Control. Concurrently,promptdecompositiontechniques such as Decomposed Prompting [11] and Chain-of- Thought [21] have emerged to break down complex tasks into manageable sub-tasks. Systems like MemGPT [19] and the LLMLingua family [9, 25, 10] have demonstrated the phenomenal efficacy of context compacting and memory tiering in bounding the context window, with LLMLingua-2 achieving 2x–5x compression ratios via data distillation and LongLLMLingua specifically addressing the long-context degradation directly linked to the “Lost in the Middle” phenomenon [12]. However, their application as a privacy-enhancing filter—where context minimisation inherently limits the exposure sur- face of sensitive data by focusing the model’s work solely on the required sub-task—remains a largely unexplored, yet highly potent, paradigm. 2.6 The Challenge of Internal Specialised Models An alternative approach to privacy and cost management involves the internal training or fine-tuning of specialised LLMs (the “Build vs. Buy” paradigm) [28]. While host- ing fully custom models on-premise theoretically miti- gates data exfiltration risks, recent industry reports [14] highlight the prohibitive computational and financial ef- fort required to train models capable of rivalling frontier 3 capabilities. Furthermore, even if an institution achieves compara- ble quality at a specific point in time, it faces the in- surmountable challenge of the “improvement pace” [24]. Big tech cloud providers continuously release updated models, rapidly shifting the state-of-the-art and render- ing static internal models obsolete. Consequently, relying on the fine-tuning of internal models for general-purpose reasoning is often economically and practically unfeasi- ble. Our framework expressly does not address the opti- misation or training of internal models. Instead, it focuses entirely on the secure, context-aware routing of standard, pre-trained open or commercial models, assuming that ac- cess to compliant Tier 2 or Tier 3 providers can match frontier cloud capabilities without requiring internal train- ing cycles. 3 Theoretical Framework: The In- separability Theorem To formalise the “Inseparability Paradigm”, we introduce a theoretical model that mathematically correlates the context length (which determines OpEx) and the proba- bility of sensitive data leakage (Data Leakage). 3.1 Context Entropy and Leakage Proba- bility LetP be a prompt composed of a set of tokens such that T (P) represents the total length in tokens. We define E s (P) as the entropy of sensitive information (e.g., PII, industrial IP, emergent conversational secrets) contained inP , measured in bits of confidential information exposed to the Cloud provider. A standard routing system routes P to a cloud model M C by minimising the monetary cost, but fully transmit- ting the entire conversational history, whereby the leakage probability P leak is strictly proportional to E s (P). 3.2 Theorem 1 (Inseparability between Pri- vacy and Costs) We introduce a Semantic Compression and Decomposi- tion function C(P) → P ′ 1 ,P ′ 2 ,... executed by a local Small Language Model (SLM), configured to preserve the user’s intent whilst discarding superfluous informative de- tails and splitting the task into focused units. The function generates a new set of sub-prompts such that the compres- sion ratio k = P T(P ′ i ) T(P) is strictly less than 1. Given an abstractive semantic compression and de- composition function C(P) → P ′ that preserves the log- ical intent I(P) ≈ I(P ′ ), the decrease in cloud opera- tional cost ∆OpEx is monotonically and directly propor- tional to the decrease in exposed sensitive entropy ∆E s , such that: ∆OpEx∝ (1− k) =⇒ ∆E s ≥ γ(1− k) where γ is the average sensitive density factor of the orig- inal prompt. Proof (Sketch): Assuming that sensitive data pos- sess high Shannon entropy but low structural weight for the representation of a specific decomposed sub-task intent, an SLM trained for abstractive summarisation and task splitting deterministically tends to discard high- entropy tokens non-essential to the primary directive. Consequently, the drastic reduction in token cardinality T (P ′ ) ≪ T (P) forces the mathematical elimination of sensitive attack vectors: lim k→k min E s (P ′ ) = 0. Eco- nomic savings and privacy sanitisation therefore become the mathematical projection of the exact same operator. 4 System Architecture: The Holistic Observer 4.1 Assumptions and Computational Ca- pacity To contextualise the framework, we establish a founda- tional assumption regarding the scalability of edge and on-premise computing resources.The local “Privacy Guard” orchestrator, relying on 7B to 14B parameter models, can be effectively executed even on personal con- sumer hardware, standard desktop workstations, or af- fordable cloud instances (such as a single NVIDIA T4 GPU with 16GB VRAM, commonly available on free tiers). This ensures that the economic benefits of Token Parsimony are immediately accessible to individual users and researchers. 4 Figure 1: The Dual-Vault Architecture. A local SLM acts as an Orchestrator and Privacy Guard, decoupling intents from secrets and routing the sanitised sub-tasks to the appropriate Cloud Tier based on trust and risk levels. Conversely, for large-scale enterprise or institutional environments handling higher throughput and requiring Zero-Leakage guarantees via 32B+ parameter models or dedicated deterministic layers, the system gracefully scales. Institutions disposing of robust computational in- frastructure (e.g., on-premise GPU clusters) or sufficient budget for dedicated Private Virtual Servers (PVS) can seamlessly upgrade the local tier. This flexibility en- sures that the framework’s baseline performance can be achieved rapidly on minimal hardware, while its security boundaries and processing speed can be augmented pro- portionally with infrastructural investment. 4.2 Logical Data Segregation While our core focus is on prompt handling and routing, a realistic enterprise implementation inherently requires logical data segregation. We structure this segregation conceptually as two distinct vaults: • Personal Vault: Protects individual secrets, sensi- tive PII data, and unpublished personal ideas from external entities and the institution’s internal moni- toring mechanisms. • Institutional Vault: Protects firm secrets, propri- etary algorithms, and internal project details from external cloud leakage. This segregation allows the orchestrator to apply distinct handling rules depending on the provenance and sensitiv- ity of the data, but the fundamental scientific contribution remains the holistic handling of the context itself. 4.3 The Holistic Contextual Observer and Task Decomposer The implementation of the framework is articulated in a sequential, agent-driven processing pipeline: 1. Context Retriever and Task Decomposer (The Orchestrator): Retrieves references whilst enforc- ing strict segregation between the Vaults. It actively rewrites and splits the user’s overarching prompt into 5 focused sub-tasks via Automatic Prompt Optimisa- tion (APO). By dividing the workload, it minimises the active working context for each sub-task, natively filtering out unnecessary secrets that would other- wise be exposed in a monolithic prompt. 2. The Privacy Guard (Holistic Contextual Ob- server): Employs a local SLM to monitor the en- tire conversational context. It goes beyond determin- istic scrubbing (e.g., standard regex engines) to de- tect “emergent leakage” over long sessions, applying abstractive summarisation and identifying inference risks. 3. Payload Assembler & Zero-Trust Router: Assem- bles the final, decomposed sub-prompts and dynam- ically routes them across a hierarchy of trust. 4.4 Tiered Trust Routing Model The Privacy Guard acts as a dynamic gateway, escalating queries across a hierarchy of cloud providers based on the residual risk assessed post-sanitisation: • Tier 0 (Untrusted / Free Tier): Public, cost-free, or geopolitically untrusted models (e.g., foreign open APIs) with no data protection guarantees.Used strictly for fully sanitised, zero-entropy queries. • Tier 1 (Commercial Standard): Standard enter- prise APIs (e.g., standard USA-based models) used for low-risk queries where standard contractual pri- vacy (NDA) is sufficient. • Tier 2 (Geographically Compliant):Models hosted within specific jurisdictions (e.g., EU-based datacentres), guaranteeing compliance with re- gional regulations (e.g., GDPR, AI Act) and data sovereignty. • Tier 3 (Zero-Trust / On-Premise): Dedicated, single-tenant cloud VPS instances or entirely on- premise hardware managed exclusively by the insti- tution. Reserved for the highest risk category where emergent leakage cannot be mathematically miti- gated. 4.5 The 2x2 Threat and Behaviour Matrix To systematically classify the diverse input vectors and the corresponding privacy risks, the orchestrator models incoming requests across a 2x2 matrix that intersects user behaviour with the nature of the sensitive data: • User Profile:“Expert” (structured, zero-shot prompts that minimise context) vs. “Lazy” (unstruc- tured, verbose document dumps). • Secret Typology: “Personal” (health data, private emails, financial pins) vs. “Institutional” (propri- etary algorithms, IP addresses, cloud infrastructure keys). This matrix provides the foundational operational perime- ter for the Privacy Guard, defining the structural complex- ity of the prompt and the required sanitisation rigour. 4.6 Context Compacting and LIFO Mem- ory Management (Zero-Waste) To mitigate the saturation of the context window in prolonged interactions, we have implemented a Con- text Compacting mechanism based on LIFO (Last- In, First-Out) stacks and Persistent Project Memory. During extended sessions, the system autonomously con- denses the accumulated context into a dense abstract. The session bootstrap is explicitly optimised to load only the most recent and relevant LIFO entries, strictly bounding the working memory. This not only drastically cuts I/O costs (OpEx) but serves as a fundamental pri- vacy mechanism: by minimising the historical context provided to the model, we proportionally reduce the at- tack surface for emergent leakage. 5 Methodology: Benchmark Struc- ture To demonstrate the efficacy of the “Privacy Guard” as a holistic observer, we have structured a multi-metric benchmark to be executed via LLM-as-a-Judge method- ologies. 6 5.1 Metric 1: Token Parsimony (OPEX Re- duction) The objective is to demonstrate the net savings derived from context reduction executed locally compared to full- cloud routing. The dataset is evaluated across the afore- mentioned 2x2 Threat and Behaviour matrix (Section 4.5) to account for distinct prompt complexities.Success KPI: ∆Cost = Cost baseline − Cost guard > 0 with a targeted reduction > 60% specifically for the Lazy pro- file document dumps. 5.2 Metric 2: Sanitisation Efficacy (Zero Leakage) The objective is to demonstrate that the local agent ro- bustly blocks sensitive data leaks, including emergent in- ference, across all four quadrants of the test matrix. Cru- cially, in a strict Zero-Trust architecture, the “Leakage Rate” is measured exclusively as the failure of the local Privacy Guard (False Negatives). Any sensitive datum present in the outbound payload is classified as a crit- ical breach, irrespective of whether the external Cloud provider subsequently memorises, utilises, or discards the information. • Test Dataset: A controlled set of prompts mapped to the 2x2 matrix, containing injected secrets and long conversational histories susceptible to emergent in- ference. • Success KPI: True Positive Rate (Sanitisation): 100% across all four cases; Semantic Preservation: > 90%. 5.3 Metric 3: Answer Quality and Emer- gent Leakage Retention The objective is to ensure that aggressive semantic com- pression (APO) does not degrade the quality of the final response generated by the cloud model, and to measure the exact rate at which a cloud provider memorises and successfully extracts any secrets that bypass the sanitisa- tion layer. • Evaluation Method: LLM-as-a-Judge comparing the semantic quality of responses generated from raw prompts versus sanitised, compressed prompts, alongside an explicit extraction attack on the cloud model’s conversational memory. • Success KPI: Non-negative net quality score (APO response quality≥ baseline quality) and quantifica- tion of cloud memory retention on leaked secrets. 6 Empirical Results and Evaluation To validate the proposed framework and the efficacy of the Holistic Contextual Observer, we executed the struc- tured benchmark detailed in Section 5. 6.1 Large-Scale Benchmark: Token Parsi- mony vs. Leakage The dataset was dynamically generated using the Faker library, scaling the volume by 10x to produce 40 realistic, complex prompt samples containing a total of 140 dis- tinct injected secrets (60 Personal, 80 Institutional). The dataset was mapped across a 2x2 test matrix, evaluating user behaviour (Lazy vs. Expert) against secret typology (Personal vs. Institutional). 6.2 The Architectural Failure of Qwen 2.5 7B on Sanitisation Subjecting the qwen2.5-7b-instruct model to the 40-sample benchmark revealed a profound structural fail- ure in simultaneous task execution. While the model ex- celled at extracting logical intent (APO), it failed com- pletely at strict syntactic redaction, registering a 100% Leakage Rate (40 out of 40 cases leaked at least one se- cret). This large-scale failure empirically proves a key corol- lary of the Inseparability Theorem: for non-specialised models under 8B parameters, Semantic Compression (APO) and Syntactic Scrubbing (Redaction) are conflict- ing objectives. When processing complex, verbose con- texts, the model prioritises logical summarisation over punctual censorship, inadvertently allowing PII and API keys to slip into the “sanitised” payload. This reinforces the necessity of adopting models strictly aligned for safety (e.g., Llama 3.1 8B Instruct) or splitting the architecture 7 by delegating deterministic PII scrubbing to robust regex- based systems (e.g., deterministic regex scanners) before applying SLMs for OpEx reduction. 6.3 APO Stability and OpEx Reduction Despite the sanitisation failure, the local 7B model demonstrated exceptional and highly stable capabilities as an Automatic Prompt Optimiser (APO). Operating with a latency of just 0.67s (±0.17s) on a standard local hard- ware accelerator, it consistently condensed the context window: • Expert Profile (Structured Prompts): Token pay- load reduction between 13.9% and 27.7%. • Lazy Profile (Verbose Document Dumps): Ag- gressive compression yielding an OpEx reduction between 53.9% and 58.1% (±2.6 standard devia- tion). The blended average across the entire 40-sample dataset resulted in a net cloud cost reduction of 47.6%. This empirically proves that the deployment of an active local orchestrator effectively halves the Cloud API bill with minimal latency overhead, provided the data leakage vector is managed by a specialised or larger-parameter subsystem. 6.4 Cloud GPU Control Test (Colab T4) and The Decoupling Necessity To isolate hardware and engine dependencies, and to es- tablish a robust statistical confidence interval, the bench- mark was expanded to 1,000 samples and completed on a cloud NVIDIA T4 GPU instance (via Google Colab), querying the qwen2.5:7b model through the standard Ollama engine with 4-bit quantisation (Q4 0). The large-scale cloud results provided definitive empir- ical validation of the framework’s mechanics, revealing a profound vulnerability associated with unstructured text. The blended OpEx reduction remained exceptionally sta- ble at 45.0%, proving that the local model systematically discards nearly half of the payload as non-essential en- tropy across diverse user profiles. However, the Leakage Rate settled at 12.9% (420 out of 3250 secrets leaked). A granular analysis of the 2x2 matrix (see Figure 2) revealed that this failure is not uniformly distributed. The 7B model achieved a 100% redaction success rate (0 leaks) on all Personal se- crets (e.g., National Insurance numbers, medical data) and on Institutional secrets presented within concise, Expert- profile prompts. The redaction failure was exclusively and catastrophically concentrated in the Lazy / Institu- tional quadrant (33.6% leakage rate, 420/1250 secrets leaked), where cryptographic keys and IPs were buried deep within massive, unstructured server log dumps. Expert Personal Expert Institutional Lazy Personal Lazy Institutional 0 10 20 30 40 50 60 70 Percentage (%) 41.6% 28.2% 66.7% 30.8% 0.0%0.0% 0.1% 34.4% Token Parsimony vs Data Leakage by Quadrant OpEx Reduction (%) Leakage Rate (%) Figure 2: Token Parsimony vs Data Leakage by user pro- file and secret typology. The model achieves high Token Parsimony across the board but suffers from severe Leak- age strictly in the Lazy/Institutional quadrant. This finding empirically solidifies a core architectural conclusion: the dual objective of the “Privacy Guard” must be decoupled. While a 7B parameter model oper- ates flawlessly as an Automatic Prompt Optimiser (APO) for Token Parsimony, its capacity to maintain strict syn- tactic scrubbing falters severely under the weight of com- plex, verbose datasets (exacerbating the “Lost in the mid- dle” phenomenon [12]).To validate whether scaling the parameter count mitigates this vulnerability, an ex- tended benchmark was executed deploying a 30B param- eter model (qwen3-coder-30b) on the 50 most com- plex Lazy/Institutional samples. The 30B model yielded a severe 54.4% Leakage Rate (117 secrets leaked out of 215). This unequivocally demonstrates that scaling up to the 30B parameter class does not solve the “Lost in the middle” failure mode for syntactic scrubbing on unstruc- tured data. 8 To identify the theoretical bounds of LLM-based redac- tion without deterministic assistance, an exploratory parameter-scaling evaluation was conducted. We sampled open-weight models across increasing parameter tiers (8B, 30B, 32B, 70B, and 104B) using a highly complex, 60-line server log dump containing deeply buried secrets. Qwen 2.5 (7B) Llama 3.1 (8B) Qwen 3 Coder (30B) Qwen 2.5 Coder (32B) Llama 3.3 (70B) Command R+ (104B) 0 20 40 60 80 100 Leakage Rate (%) 100.0% 33.3% 54.4% 0.0%0.0% 33.3% Exploratory Leakage Rate by Model Parameter Scale (Lost in the Middle) Figure 3: Exploratory space analysis of Leakage Rate across parameter scales (8B to 104B). Results are based on limited sample sizes designed to identify structural vulnerabilities rather than provide exhaustive statistical baselines. While not statistically exhaustive, this preliminary space exploration (Figure 3) reveals a non-linear relation- ship between parameter count and syntactic scrubbing ef- ficacy. For instance, while the 32B (Qwen 2.5 Coder) and 70B (Llama 3.3) instruct models successfully achieved 0% leakage on the exploratory needle-in-a-haystack task, other models in the 8B, 30B, and even 104B classes ex- hibited significant leakage rates (ranging from 33.3% to 100%). To statistically validate the frontier model capacity, an extensive follow-up benchmark was executed locally de- ploying llama-3.3-70b-instruct at 6-bit quan- tisation across all 50 samples of the Lazy/Institutional quadrant. The 70B model achieved a 1.20% Leakage Rate (only 3 secrets leaked out of 250). This confirms that while transitioning to frontier models drastically improves semantic retention and approaches near-perfect redaction, the reliability of syntactic scrubbing remains heavily de- pendent on the specific architecture, context window me- chanics, and training alignment rather than raw parame- ter count alone. Consequently, to achieve a guaranteed 100% True Positive Rate for redaction in enterprise envi- ronments, the architecture still strictly necessitates a ded- icated deterministic layer (e.g., deterministic regex scan- ners) acting as a rigid filter prior to the APO compression. 6.5 Task Decomposition and Active Routing OpEx Reduction To empirically evaluate the financial impact of active prompt decomposition, an additional benchmark simulat- ing massive system logs (approx. 11,300 input tokens per sample) was executed across 118 samples. The local 7B model was tasked with decomposing a complex user in- tent into three atomic sub-tasks: 1) extracting the critical root cause locally, 2) translating the extracted sentence lo- cally, and 3) routing only the final email generation task to the Tier-1 Cloud API. Baseline (Monolithic)Dual-Vault (Decomposition) 0 1 2 3 4 5 6 7 Projected Cloud Cost (USD) $6.71 $0.16 -97.5% OpEx OpEx Reduction via Task Decomposition (118 log samples) Figure 4: Projected vs actual OpEx reduction when com- paring a standard monolithic Cloud API call against active local prompt decomposition and selective routing. The baseline execution (routing the entire log dump to the Cloud API to fulfill all three intents simultaneously) resulted in a projected operational cost of $6.71. Con- versely, the active decomposition and routing approach processed the entire batch for a Cloud cost of just $0.16. This yielded a 97.54% total OpEx reduction (see Figure 4), definitively proving that offloading the heavy-lifting of information extraction to a local, air-gapped SLM before 9 querying commercial APIs mathematically reshapes the cost-efficiency curve of enterprise LLM deployments. 6.6 Semantic Quality and Emergent Leak- age Extraction To ensure that the drastic semantic compression (-45% OpEx) does not degrade the core utility of the Cloud model, a tertiary benchmark was executed using a 30B parameter model (qwen3-coder-30b) as the Tier- 1 Cloud API. An LLM-as-a-Judge evaluation was per- formed to compare the semantic quality of responses gen- erated from the raw, verbose prompts (Baseline) versus the sanitised, compressed prompts (Dual-Vault APO). Dual-Vault (APO) Wins 85.0% Baseline (Raw) Wins 10.0% Ties 5.0% Answer Quality Evaluation (LLM-as-a-Judge) Figure 5:LLM-as-a-Judge comparison of final re- sponse quality between raw Context (Baseline) and com- pressed/sanitised Context (Dual-Vault APO). The final results of this comparative analysis across a 40-sample evaluation dataset (Figure 5) revealed a counter-intuitive but highly significant phenomenon: the response generated via the Dual-Vault architecture was preferred by the Judge over the baseline in 85.0% of the cases (34 wins), with 5.0% ties and only a 10.0% pref- erence for the raw baseline (4 wins). This unequivocally demonstrates that aggressive semantic compression by the local 7B model not only preserves the core intent but ac- tively enhances the quality of the final Cloud response by stripping out conversational noise and focusing the Cloud model strictly on the technical requirements. 7 Analytical Projections and Archi- tectural Implications To complement the empirical benchmarks, we modelled three theoretical projections based on the observed data to illustrate the broader architectural implications of the framework. 7.1 LIFO Compacting and Multi-Turn Conversations In standard monolithic LLM interactions, conversational context grows linearly, causing the cloud operational cost to escalate quadratically (O(N 2 )) while continuously ex- panding the emergent leakage surface. By implementing a Last-In, First-Out (LIFO) memory stack, the local or- chestrator actively bounds the context window. This pro- jection demonstrates that LIFO compacting flattens the token cost to a constant O(1) per turn (Zero-Waste) and drastically suppresses the probability of exposing histor- ical secrets, structurally neutralizing emergent inference risks over prolonged sessions. 2468101214 Conversation Turn 0 10 20 30 40 50 60 Cumulative Cloud Input Tokens (Thousands) Cumulative OpEx Growth in Multi-Turn Sessions Standard (Monolithic Memory) LIFO Compacting (Zero-Waste) 2468101214 Conversation Turn 0 10 20 30 40 50 60 70 80 Emergent Leakage Probability (%) Risk of Exposing Historical Secrets Standard Memory LIFO Compacting Figure 6: Theoretical projection of OpEx and Emergent Leakage risk over a 15-turn conversation: Monolithic memory vs. LIFO Compacting. 7.2 Latency Overhead and Time-To-First- Token (TTFT) A common critique of local proxy models is the introduc- tion of latency. It is crucial to clarify that running a local SLM for token parsimony does not necessitate high-end 10 enterprise workstations or specialised professional hard- ware. The latency overhead modelled here assumes the use of a standard, latest-generation personal computer equipped with a consumer-grade gaming GPU (e.g., 8GB to 12GB VRAM), ensuring the setup is highly accessible. However, even on such standard hardware, cloud APIs consume substantial time processing massive input pay- loads (reading time). Our analytical model reveals that for prompts exceeding 10,000 tokens, the Dual-Vault ar- chitecture becomes strictly faster than direct cloud rout- ing. The time spent by the local consumer hardware to abstract and compress the prompt is overcompensated by the accelerated cloud reading time on the heavily reduced payload, effectively rendering the Privacy Guard Time- Neutral or latency-negative. 1k5k10k20k50k Initial Prompt Size (Tokens) 0 5 10 15 20 25 Time To First Token (Seconds) Dual-Vault becomes faster for context > 10k tokens Latency Overhead vs. Prompt Size Cloud-Only (Standard) Dual-Vault (Local APO + Cloud) Figure 7: Latency overhead simulation (Time-To-First- Token). The Dual-Vault architecture becomes faster than direct Cloud routing for contexts larger than 10k tokens due to massive payload reduction. 7.3 Hybrid Architecture: Deterministic Fil- tering plus SLM As demonstrated empirically, scaling the local SLM pa- rameter count from 8B to 70B resolves the “Lost in the middle” vulnerability, achieving 0% Leakage. How- ever, deploying 70B class models on-premise demands a 10x increase in VRAM infrastructure cost. A projected Hybrid Architecture—combining a deterministic offline scanner (e.g., regex-based PII engines or deterministic NER models like GLiNER) with a highly efficient 8B SLM—achieves the identical outcome (0% Leakage, 45% OpEx Reduction) at a fraction of the hardware cost, offer- ing a universally accessible “off-the-shelf” implementa- tion for enterprise edge nodes. SLM Only (8B Class) Presidio (Deterministic) Frontier LLM (70B Class) Hybrid (Presidio + 8B) 0 10 20 30 40 Percentage (%) 33.3% 0.0% 1.2% 0.0% 45.0% 0.0% 45.0%45.0% Hybrid Architecture: Deterministic Filtering + SLM Compression Leakage Rate (Lower is Better)OpEx Reduction (Higher is Better)Hardware Cost 0 2 4 6 8 10 12 Local Hardware VRAM Requirement (Relative) Figure 8: Comparison of local hardware cost vs. Leakage and Compression rates. The Hybrid architecture matches the 70B Frontier performance with 1/10th of the required VRAM. 8 Conclusion This paper has demonstrated that in a hybrid LLM ecosys- tem, cost management and privacy management are not opposing objectives, but synergistic levers controllable through rigorous context manipulation. While logical data segregation (Personal and Institutional Vaults) pro- vides a realistic enterprise structure, the core scientific ad- vancement lies in treating context management as privacy management. By actively decomposing prompts, man- aging memory via LIFO compacting, and employing a Tiered Trust Routing model guided by a holistic observer, institutions can simultaneously slash OpEx and decisively eliminate data exfiltration risks. 11 Acknowledgements The author used large language model assistance (Claude, Anthropic) for manuscript editing, bibliography verifica- tion, and development of benchmark scripts. All scientific claims, experimental design, results, and conclusions are solely the responsibility of the author. Data Availability and Reproducibility To facilitate independent verification and future re- search within the community, all materials associ- ated with the empirical validation of the “Insepa- rability Paradigm” have been made publicly acces- sible via https://github.com/alangiu-gif/ privacy-n-parsimony. The repository includes the complete Python execution scripts, the interactive Google Colab Notebooks (used for both the 2x2 Matrix scaling and the Task Decomposition benchmarks), and the dy- namically generated datasets containing the exact permu- tations of prompts and injected synthetic secrets. References [1] Marah Abdin et al.Phi-3 technical report: A highly capable language model locally on your phone. https://arxiv.org/abs/2404. 14219, 2024. [2] Aurelio AI. Semantic router: Superfast decision making for llms and agents. https://github. com/aurelio-labs/semantic-router, 2023. Accessed: 2024. [3] BerriAI. Litellm: Call all llm apis using the ope- nai format. https://github.com/BerriAI/ litellm, 2023. Accessed: 2024. [4] Lingjiao Chen, Matei Zaharia, and James Zou. Fru- galgpt: How to use large language models while re- ducing cost and improving performance, 2023. [5] Joong Ho Choi, Jiayang Zhao, Jeel Shah, Ritvika Sonawane, Vedant Singh, Avani Appalla, Will Flanagan, and Filipe Condessa. Compactprompt: A unified pipeline for prompt data compression in llm workflows. https://arxiv.org/abs/ 2510.18043, 2025. [6] Junjie Chu, Zeyang Sha, Michael Backes, and Yang Zhang. Reconstruct your previous conversations! comprehensively investigating privacy leakage risks in conversations with gpt models.In Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. arXiv:2402.02987. [7] Cyberhaven Research.11% of data pasted intochatgptisconfidential. https: //w.cyberhaven.com/blog/ 11-of-data-pasted-into-chatgpt-is-confidential/, 2023. Accessed: 2024. [8] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testug- gine, and Madian Khabsa.Llama guard: Llm- based input-output safeguard for human-ai con- versations. https://arxiv.org/abs/2312. 06674, 2023. [9] Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu.Llmlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:2310.05736, 2023. [10] Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dong- sheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LongLLMLingua: Accelerating and enhancing llms in long context scenarios via prompt compression. arXiv preprint arXiv:2310.06839, 2023. [11] Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular ap- proach for solving complex tasks. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR), 2023. arXiv:2210.02406. [12] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Sergey Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Asso- ciation for Computational Linguistics, 12:157–173, 2024. arXiv:2307.03172. 12 [13] Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu.Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499, 2023. [14] Nestor Maslej, Loredana Fattorini, Raymond Per- rault, Vanessa Parli, Anka Reuel, Erik Brynjolfs- son, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, et al. The artificial intelligence in- dex report 2024. Technical report, Stanford Institute for Human-Centered Artificial Intelligence (HAI), 2024. [15] Microsoft.Presidio:Data protection and de- identification sdk. https://github.com/ microsoft/presidio, 2019. Accessed: 2024. [16] Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ip- polito, Christopher A Choquette-Choo, Eric Wal- lace, Florian Tram ` er, and Katherine Bowman. Scal- able extraction of training data from (production) language models. In Proceedings of the Thirteenth International Conference on Learning Representa- tions (ICLR), 2025. arXiv:2311.17035. [17] NVIDIA. Nemo guardrails: An open-source toolkit for adding programmable guardrails to llm-based conversational applications. https://github. com/NVIDIA/NeMo-Guardrails, 2023. Ac- cessed: 2024. [18] Isaac Ong, Amey Hoon, Fanyin He, Xinyi Chen, Dacheng Li, Joseph E. Gonzalez, Ion Stoica, et al. Routellm:Learning to route llms with prefer- ence data. https://github.com/lm-sys/ RouteLLM, 2024. [19] Charles Packer, Vivian Fang, Shishir G Patil, Kevin Lin, Sarah Wooders, and Joseph E Gonzalez. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023. [20] Robin Staab, Mark Vero, Mislav Balunovi ́ c, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. In Proceedings of the Twelfth International Con- ference on Learning Representations (ICLR), 2024. arXiv:2310.07298. [21] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, 2022. [22] Rui Wen, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks against in- context learning. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communica- tions Security (CCS), 2024. arXiv:2409.01380. [23] Jules White, Quchen Fu, Sam Hays, Michael Sand- born, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. A prompt pattern catalog to enhance prompt engineer- ing with chatgpt. arXiv preprint arXiv:2302.11382, 2023. [24] David Gray Widder, Sarah West, and Meredith Whittaker. Open (for business): Big tech, concen- trated power, and the political economy of open ai. Available at SSRN 4543807, 2023. [25] Zhuoshi Wu, Qianhui Wang, Huiqiang Jiang, Menglin Ouyang, Zhenghao Zhang, Xufang Wang, Yuqing Lu, Chin-Yew Lin, Jingren Guo, Yuqing Yang, et al. LLMLingua-2: Data distillation for efficient and faithful task-agnostic prompt compres- sion. In Findings of the Association for Computa- tional Linguistics (ACL), 2024. arXiv:2403.12968. [26] Zheng Yang, Vikram Bhatt, Tong Yu, Ethan Tan, Saumitra Chaudhuri, Zhen Lin, Mengdi Xu, and David Clifton. Prompt leakage effect and defense strategies for multi-turn llm interactions. arXiv preprint arXiv:2404.16251, 2024. [27] Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois.Gliner: Generalist model for named entity recognition using bidirectional trans- former. https://arxiv.org/abs/2311. 08526, 2023. 13 [28] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Be- ichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023. 14