Paper deep dive
StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge
Anning Hu, Yangyu Li, Yuqi Huang, Shengchao Hu, Junchi Yan
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific model's scale and training and requiring repeated effort when the judge is updated or replaced. We propose StableJudge, a lightweight defense framework that externalizes prompt-injection mitigation as a reusable pre-scoring capability. StableJudge centers on a compact 0.13B Sentry that localizes injected spans and performs targeted sanitization prior to evaluation, and is trained using adversarially evolving attacks to improve robustness beyond a fixed prompt set. To stabilize measurement, StableJudge derives deterministic, fine-grained scalar scores from the judge's output distribution. Across held-out scientific abstracts and out-of-distribution attack variants, StableJudge improves score recovery under attack while largely preserving general-purpose performance, demonstrating that robust LLM-based judging can be achieved by decoupling safety from judge scale via reusable defense capabilities. Code and data will be released.
Tags
Links
- Source: https://arxiv.org/abs/2610.16814v1
- Canonical: https://arxiv.org/abs/2610.16814v1
Trouble viewing inline? Open PDF directly ā
Full Text
48,017 characters extracted from source content.
Expand or collapse full text
StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-JudgeThanks: Corresponding author: yanjunchi@sjtu.edu.cn. Anning Hu Yangyu Li Yuqi Huang Affiliation: Shengchao Hu Junchi Yan Affiliation: Shanghai Jiao Tong University Abstract Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific modelās scale and training and requiring repeated effort when the judge is updated or replaced. We propose StableJudge, a lightweight defense framework that externalizes prompt-injection mitigation as a reusable pre-scoring capability. StableJudge centers on a compact 0.13B Sentry that localizes injected spans and performs targeted sanitization prior to evaluation, and is trained using adversarially evolving attacks to improve robustness beyond a fixed prompt set. To stabilize measurement, StableJudge derives deterministic, fine-grained scalar scores from the judgeās output distribution. Across held-out scientific abstracts and out-of-distribution attack variants, StableJudge improves score recovery under attack while largely preserving general-purpose performance, demonstrating that robust LLM-based judging can be achieved by decoupling safety from judge scale via reusable defense capabilities. Code and data will be released. 1 Introduction Large language models (LLMs) are increasingly deployed as judges that assign scores for academic screening, recommendation, and automated evaluation (Zhuang et al., 2025; Zheng et al., 2023; Liu et al., 2023a; Dubois et al., 2024). Because downstream rankings are often induced by sorting these scores, scoring integrity is central to system reliability under untrusted inputs. Our motivation comes from an LLM-based paper-search system, where retrieved papers become inputs to a scoring stage: the content being evaluated is also an entry point for instructions that target the evaluator. Figure 1: Overview of StableJudge for LLM-as-a-Judge. (A) Illustrative score-hijacking example. (B) Mutual redāblue training: localization misses guide new attacks, and effective attacks train Sentry. (C) Deployment: the same frozen Sentry sanitizes inputs across judge backends, aiming to recover each unchanged judgeās clean-input score. However, LLM-as-a-Judge pipelines are vulnerable to prompt injection: adversarial instructions embedded in evaluated content that aim to override the judgeās intended protocol (Perez and Ribeiro, 2022; Greshake et al., 2023). We focus on score-hijacking injections, where short directives coerce inflated scores and corrupt the scoring signal before ranking or downstream decisions. Existing mitigations include secure prompting, fine-tuning, preference optimization, and guardrails (Suri and McCrae, 2025; Xu and Parhi, 2025). When defense is tied to a judgeās weights or behavior, changing the backend entails renewed adaptation or validation (Liu et al., 2023b; Tan et al., 2026; Wee et al., 2025). We propose StableJudge, which externalizes prompt-injection mitigation as a compact callable capability. The central design choice is to remove adversarial instructions while retaining the evidence for evaluation, so that a trained defense can be reused when the judge changes. StableJudge follows a simple protocol: sanitize before scoring. Its 0.13B Sentry localizes injected spans and produces minimally modified text, which the judge evaluates using a continuous logit-based score readout. Sentry is trained through mutual redāblue evolution: detector feedback guides attack generation, and the resulting effective attacks improve the detector. After training, the same frozen Sentry is reused across judge backends. The main contributions of this work are as follows: 1. We externalize score-hijacking defense as a reusable pre-scoring tool. A fixed-input evaluation across three judge backends tests score recovery with the same frozen Sentry and sanitized texts. 2. We introduce the StableJudge Sentry, a compact (0.13B) taggerāverifier that localizes and sanitizes injected instructions, with bounded-window scanning for long documents. 3. We develop mutual redāblue evolution through miss-guided attack generation and score-impact filtering, and evaluate the resulting defense on held-out attacks and an independent attack source. 2 Related Work LLM-as-a-Judge. LLM-based judges enable scalable evaluation of open-ended outputs and are increasingly used to derive rankings over systems and candidates (Zheng et al., 2023; Liu et al., 2023a; Dubois et al., 2024). Prior work identifies systematic biases and reliability issues in judge behavior (Zheng et al., 2023; Ye et al., 2024). Beyond instance-level preferences, system-level ranking quality can diverge from per-instance judgment accuracy (Godfrey et al., 2025; Liu et al., 2025); JuStRank highlights this gap and motivates system-centric evaluation (Gera et al., 2025). Our work studies a complementary failure mode: adversarial content can hijack the scoring signal and corrupt induced rankings (Raina et al., 2024; Maloyan and Namiot, 2025). Prompt injection. Prompt injection attacks embed malicious directives in natural language inputs or context so that models follow attacker instructions instead of the intended protocol (Perez and Ribeiro, 2022; Zou et al., 2023). Indirect prompt injection extends this threat to LLM-integrated applications, where adversarial instructions may be planted in external resources processed at inference time (Greshake et al., 2023; Keuper, 2025). Recent evaluations and benchmarks systematize attack/defense protocols and quantify robustness across tasks and models (Liu et al., 2023b; De Stefano et al., 2024; Zhan et al., 2024; Nasr et al., 2025). Defenses. Defenses range from training models to respect instruction priority (instruction hierarchy) (Wallace et al., 2024) to deploying external guard models such as Llama Guard (Inan et al., 2023). Guardrails must also address over-defense, where benign inputs are falsely flagged due to trigger-word bias (Li and Liu, 2024). LLAMOS uses a prompted defense agent to purify adversarial text before it reaches the target LLM and studies attackādefense interaction (Lin et al., 2024). StableJudge instead trains a compact taggerāverifier for targeted span sanitization and evaluates whether the same frozen defense recovers clean-input scores across judge backends. Adversarial generation supplies task-specific supervision (Madry et al., 2018; Radharapu et al., 2023; Perez et al., 2022); the external tool interface supports reuse without retraining each backend (Schick et al., 2023; Yao et al., 2023; Karpas et al., 2022). 3 Method Figure 2: StableJudge Sentry. The tagger fuses semantic and pattern features to predict O/B/IO/B/I labels; only candidate spans with verifier probability vjā„Ļ=0.5v_jā„Ļ=0.5 are sanitized. Selected windows and character-offset mapping extend the same pipeline to long inputs. 3.1 Problem Setup and Objective Let x denote clean evaluated content (an abstract in the primary scoring task), and let sā”(x)āās(x) be the score assigned by a given judge under a fixed protocol (Section 3.6). In this work we focus on scoring: attack-induced score shifts and defense-induced score recovery. Score-hijacking prompt injection. An attacker injects a short payload p, a text span intended to redirect the judgeās scoring, into the evaluated content: x~=Injectā”(x,p). x=Inject(x,p). (1) The goal is to coerce an inflated score, i.e., sā”(x~)ā«sā”(x)s( x) s(x). Score recovery objective. StableJudge introduces a pre-evaluation defense D that sanitizes x~ x into xā=Dā”(x~)x^*=D( x), with the objective of recovering the same judgeās clean score. We define the attack shift and recovery residual as Īatkā(x,p)=sā”(Injectā”(x,p))āsā”(x), _atk(x,p)=s(Inject(x,p))-s(x), (2) and Īrecā(x,p)=sā”(Dā”(Injectā”(x,p)))āsā”(x). _rec(x,p)=s(D(Inject(x,p)))-s(x). (3) Our primary objective is to minimize |Īrec|| _rec| on effective attacks while preserving non-injected content. The judge, scoring prompt, and readout are held fixed within each clean/attacked/defended comparison. Score-noise calibration. To separate genuine manipulation from intrinsic judge noise, we calibrate a score variability scale Ļ by repeated scoring on clean inputs and set a conservative tolerance ĻµĪµ using the 3Ļ rule (Pukelsheim, 1994). For a given attacked example (x,p)(x,p), we treat the attack as effective if Īatkā(x,p)>Ļµ _atk(x,p)>Īµ, and the defense as successful if |Īrecā(x,p)|ā¤Ļµ| _rec(x,p)|ā¤Īµ. Ļµ=3āĻ.Īµ=3Ļ. (4) Continuous residuals complement thresholded success by quantifying the magnitude of score recovery. 3.2 StableJudge Overview StableJudge externalizes defense into a reusable pre-scoring module without modifying the judge (Figure 1). The Sentry localizes injected spans in x~ x and produces a minimally modified text xāx^*, which the judge scores through a fixed readout (Section 3.6). 3.3 StableJudge Sentry: Localize and Sanitize The StableJudge Sentry pairs two compact DistilBERT encoders (Sanh et al., 2019), a tagger and a verifier, totaling approximately 0.13B parameters. Bidirectional token representations suit localization, while the compact encoders keep the pre-scoring module lightweight. We first describe one inspected string t; for a single-window abstract, t=x~t= x and tā=xāt^*=x^*. Long inputs use the same procedure on selected windows. Injected directives coexist with the evidence the judge must assess, so rejecting an entire input would discard useful content. As illustrated in Figure 2, a high-recall tagger localizes suspect tokens, and a precision-oriented verifier checks candidate spans before removal. This decomposition makes fine-grained sanitization reusable across judges. 3.3.1 Token-Level Localizer (Tagger) The tagger performs token classification to localize injected spans. Let t be tokenized into (w1,ā¦,wn)(w_1,ā¦,w_n). Each token has a label yiāO,B,Iy_iā\O,B,I\, with prediction y^i y_i. Here O denotes a non-injection token, B marks the beginning of an injected span, and I marks subsequent tokens inside the same span. We use a compact bidirectional encoder to obtain contextual token embeddings hiāādh_i ^d. We fuse a lightweight pattern feature vector fā”(t)āā3f(t) ^3 as auxiliary input to these contextual representations. Let gā”(t)=MLPā”(fā”(t))g(t)=MLP(f(t)) be a feature embedding shared across tokens, and h~i=[hi;gā”(t)] h_i=[h_i;g(t)]. We compute token logits and predictions by a linear head, where āiāā3 _i ^3, āi,c _i,c is the logit for class c, and W,W,b are learnable parameters: āi=Wāh~i+,y^i=argā”maxcāO,B,Iā”āi,c. _i=W h_i+b, y_i= _cā\O,B,I\ _i,c. (5) The auxiliary vector fā”(t)f(t) contains three lightweight scalars: (i) Style consistency measures abrupt stylistic shifts by comparing sentence embeddings; we compute the minimum cosine similarity between the first sentence and the remaining sentences. (i) Instruction density measures how instruction-like the text is via an imperative-verb ratio over sentences. (i) Keyword frequency measures the density of a fixed set of injection-related keywords normalized by text length. These are soft contextual features, not deletion rules: instruction-dense text is removed only if the tagger proposes a span and the verifier accepts it. 3.3.2 Span Verification and Sanitization We convert the token-level tag sequence y^1:nāO,B,In y_1:nā\O,B,I\^n into candidate injected spans by grouping consecutive tokens predicted as B or I into contiguous segments. To map token segments back to the raw text, we use tokenizer-provided offset mappings (si,ei)(s_i,e_i), where (si,ei)(s_i,e_i) are the start/end character indices of token wiw_i in the original text t. For a candidate span covering a contiguous token index set jJ_j, we set aj=miniājā”sia_j= _i _js_i and bj=maxiājā”eib_j= _i _je_i. Each candidate span is thus represented as a character interval (aj,bj)(a_j,b_j) in t. We then apply a DistilBERT binary verifier to each candidate span to suppress false positives. It is trained on injected spans as positives and clean sentences as negatives. Let vjā[0,1]v_jā[0,1] be its injected-class softmax probability given the candidate spanās text. With a fixed Ļ=0.5Ļ=0.5, we retain a span only if vjā„Ļv_jā„Ļ: ā=(aj,bj):vjā„Ļ.R=\(a_j,b_j):v_jā„Ļ\. (6) After merging overlapping intervals, a minimal edit operation replaces each accepted span with a fixed placeholder (a redaction marker), leaving text outside these intervals unchanged: tā=Sanitizeā”(t,ā),t^*=Sanitize(t,R), (7) The verifier selects among tagger proposals. Long-context coverage via radar-guided scanning. To inspect long inputs beyond a single truncated prefix, we scan bounded head and tail regions, plus radar-guided interior windows around suspicious cues. Each selected window uses the same taggerāverifier; predicted offsets are mapped back to the original document before sanitization. This concentrates computation on selected regions while retaining unedited context (Figure 2). Figure 3: Mutual redāblue evolution. (a) Score-impact filtering selects effective attacks to train the tagger and verifier; the updated taggerās miss report guides the next generation round. (b) Examples of evolved prompt-injection payloads. 3.4 RedāBlue Adversarial Evolution Training StableJudge couples attack evolution with defense learning: Sentryās localization misses guide the next red-team search, and effective attacks supervise the next blue-team update (Algorithm 1). The red side evolves its attack population and search strategy; the blue side updates the tagger and verifier, with parameters Īø and ĻĻ. These two feedback directions form the mutual evolution loop. We initialize ā0=Summarizeā”(0)I_0=Summarize(P_0), a deterministic report of attack categories and representative payloads. At round k, report ākā1I_k-1 conditions category-preserving mutation and open-ended creation, producing candidates ~k P_k. After the blue update, FailureReportFailureReport groups payloads for which the current tagger finds no injected span into ākI_k. When no complete misses remain, it requests new categories. Thus feedback directs where to search, while score-impact filtering determines which attacks enter the pool. We retain an effective subset keffā~kP^eff_k P_k using the score-shift criterion in Equation 9, estimated on a training-side calibration subset V, and update the structured pool via kāOrganizeā”(kā1āŖkeff),P_k \! (P_k-1 ^eff_k ), (8) where Organizeā”(ā )Organize(Ā·) normalizes payload strings, removes duplicates, and maintains category labels. On the blue side, we assemble attacked examples by injecting payloads into clean texts with diverse surface realizations, extract pattern features, and train the tagger and verifier on the resulting supervision. Dataset splits use paper IDs rather than individual injected variants; SplitByIDSplitByID preserves those assignments across rounds. Effectiveness filtering. The red team retains a payload when its expected score shift over the clean abstract corpus exceeds the calibrated tolerance: xā¼ā[sā”(Injectā”(x,p))āsā”(x)]>Ļµ,E_x [s(Inject(x,p))-s(x) ]>Īµ, (9) where C is a clean abstract corpus. In practice, FilterĻµFilter_Īµ estimates this corpus-level expectation by the sample mean over a small calibration subset āV . This selects score-changing attacks in line with the score-hijacking objective. Algorithm 1 Redāblue adversarial evolution training 0: clean corpus C; seed payload pool 0P_0; initial tagger/verifier parameters (Īø0,Ļ0)( _0, _0); tolerance ĻµĪµ; rounds K 0: trained Sentry parameters (ĪøK,ĻK)( _K, _K) and payload pool KP_K 1: ā0āSummarizeā”(0)I_0 (P_0) 2: fix paper-ID partitions; choose V from training IDs in C 3: for iteration k=1k=1 to K do 4: Red: ~kstealthāMutateā”(kā1,ākā1) P_k^stealth (P_k-1;I_k-1) 5: ~kcreativeāCreateā”(ākā1) P_k^creative (I_k-1) 6: ~kā~kstealthāŖ~kcreative P_kā P_k^stealthāŖ P_k^creative 7: keffāFilterĻµā(~k,)P^eff_k _Īµ( P_k;V) 8: kāOrganizeā”(kā1āŖkeff)P_k (P_k-1 ^eff_k) 9: Blue: kāAssembleā”(,k)D_k (C,P_k) 10: (ktr,kva,kte)āSplitByIDā”(k)(D_k^tr,D_k^va,D_k^te) (D_k) 11: ĪøkāTrainTaggerā”(Īøkā1,ktr,kva) _k ( _k-1;D_k^tr,D_k^va) 12: ĻkāTrainVerifierā”(Ļkā1,ktr,kva) _k ( _k-1;D_k^tr,D_k^va) 13: ākāFailureReportā”(Īøk,keff)I_k ( _k,P^eff_k) 14: end for 3.5 Tool-Augmented Scoring Pipeline The scoring protocol invokes Sentry before every judge call: the judge receives only sanitized text xāx^*. After redāblue training, Sentry is frozen and reused across backends; only the judgeās scoring interface changes. Seen Attacks Unseen (Evolved) Attacks Model Params Char IoU ā Damage ā Char IoU ā Damage ā Clean FPR ā Gemini-3-Flash API 96.96% 12.75% 97.81% 13.75% 0.00% GPT-5.2 API 98.11% 15.25% 98.63% 15.50% 0.00% Claude-4.6-Sonnet API 73.43% 10.50% 68.65% 10.25% 0.00% Doubao-Seed-1.6 API 92.68% 10.75% 95.93% 13.00% 0.00% Qwen3-0.6B (thinking) 0.6B 6.09% 2.75% 5.85% 1.75% 6.18% Qwen3-1.7B (thinking) 1.7B 4.51% 0.00% 6.59% 0.00% 0.00% Qwen3-4B (thinking) 4B 56.07% 2.75% 60.50% 2.50% 0.00% Qwen3-8B (thinking) 8B 67.35% 8.00% 66.60% 10.50% 0.27% Qwen3-14B (thinking) 14B 71.32% 4.00% 72.45% 4.25% 0.00% Qwen3-30B 30B 78.03% 10.00% 81.37% 10.00% 0.00% Qwen3-32B (thinking) 32B 74.99% 6.00% 81.81% 6.75% 0.00% Qwen3-235B-A22B (thinking) 235B 83.25% 5.25% 85.43% 5.00% 0.00% Keyword Rules rules 7.20% 0.25% 7.17% 0.00% 0.27% StableJudge Sentry (Tagger-only) ā¼ 0.07B 98.48% 3.25% 99.18% 2.50% 0.27% StableJudge Sentry (Tagger+Verifier) 0.13B 99.21% 1.50% 99.97% 0.50% 0.00% Table 1: Prompt injection detection and localization on seen and unseen (evolved, held-out) attack distributions. We report mean character-level span IoU (ā ) between predicted removal spans and gold injected spans, attack-split damage rate (ā ), and clean false-positive rate (FPR, ā ). 3.6 Deterministic Logit-Based Scoring To stabilize scoring and improve score resolution, we compute the expected score from the judgeās distribution over score labels. The prompt ends at a fixed Final Score: suffix; score labels are evaluated at this slot, not extracted from free-form reasoning. Thinking is disabled for endpoint scoring. Let =1,ā¦,10S=\1,ā¦,10\ and let zuz_u be the log-probability of the complete label for score u, summing conditional token log-probabilities when the label spans multiple tokens (e.g., ā10ā). Access to these label probabilities is required. We normalize over the allowed labels and take their expectation: Ļā”(u) Ļ(u) =expā”(zu)āvāexpā”(zv),s^=āuāuā Ļ(u). = (z_u) _v (z_v), s= _u uĀ·Ļ(u). (10) For single-token labels, this is equivalent to normalizing their logits. The expectation provides finer-grained scores than greedy integer decoding, distinguishing inputs that receive the same integer score. Unless otherwise noted, we use s^ā(x) s(x) as the scalar score sā”(x)s(x) throughout the paper and omit the hat for simplicity. 4 Experiments (a) Payload embeddings across evolution rounds (t-SNE; van der Maaten and Hinton, 2008). (b) Score distributions before attack, under attack, and after StableJudge defense. Figure 4: Adversarial evolution and score recovery. Left: a qualitative projection of payload formulations across rounds. Right: the primary Qwen3-8B judgeās clean, attacked, and defended scores on the same 298 effective attacks; residuals quantify recovery toward its clean-input scores. Attack Removal Round Perfect Detected ā Missed ā Damage ā Round 0 0.00% 1.00% 99.00% 58.50% Round 1 98.25% 99.50% 0.50% 8.50% Round 2 98.00% 99.50% 0.50% 2.25% Round 3 99.25% 99.50% 0.50% 2.25% Round 4 99.50% 99.50% 0.50% 1.00% Round 5 99.50% 99.50% 0.50% 2.00% Table 2: Evolution of Sentry performance across redāblue adversarial training rounds. Seen Attacks Unseen Attacks System Recall ā Integrity ā Recall ā Integrity ā Tagger 97.75% 99.73% 99.50% 99.73% Sentry 96.50% 100.00% 100.00% 100.00% Table 3: Sentry pipeline ablation: tagger-only versus full localization and verification. Judge Attacked MAD Defended MAD Qwen3-8B (adapted) 1.040 0.422 Qwen2.5-7B-Instruct 0.880 0.055 Llama-3.1-8B-Instruct 0.390 0.131 Table 4: Score recovery on 400 fixed unseen inputs, with the same Sentry and sanitized texts. Mean absolute deviation (MAD) is relative to each judgeās clean score (ā ). Method Pearson (r) ā Kendall (Ļ) ā Greedy (Integer) 0.0213 0.0146 Logits (Expected) 0.2026 0.1052 Table 5: Agreement with human scores using two readouts of the same trained judge. Dimension Dataset Metric Qwen3-8B (Base) StableJudge (Ours) Ī Knowledge GPQA (Diamond) Accuracy ā 28.79% 28.79% +0.00% Reasoning GSM8K (5-shot) Exact Match ā 88.25% 86.81% -1.44% Instruction IFEval Inst. Strict Acc ā 48.32% 49.04% +0.72% Instruction IFEval Prompt Strict Acc ā 33.64% 35.12% +1.48% General MMLU Average Accuracy ā 74.72% 74.98% +0.26% STEM MMLU-STEM Accuracy ā 75.48% 76.40% +0.92% Social Science MMLU-Soc. Sci Accuracy ā 83.95% 84.30% +0.35% Humanities MMLU-Humanities Accuracy ā 65.65% 65.57% -0.08% Other MMLU-Other Accuracy ā 78.53% 78.53% +0.00% Table 6: General capability retention of the StableJudge scoring agent relative to base Qwen3-8B across knowledge, reasoning, and instruction-following benchmarks. Payload selection All Head Middle Tail Char. coverage Mean scans Cue-matched 98.89% 100.00% 96.67% 100.00% 98.17% 3.03 Table 7: Full-paper localization with cue-matched payloads. All/Head/Middle/Tail report detection rates; character coverage is mean gold-span recall. Mean scans counts inspected regions per attacked paper. 4.1 Experimental Setup Task, data, and threat model. We study an LLM-as-a-Judge pipeline that assigns scalar scores to abstract-only inputs drawn from a real-world submissionāreview corpus. For primary-judge adaptation, a teacher model rewrites full-paper reviews into abstract-conditioned versions, removing references to unavailable content while preserving stance and score. Our threat model is score-hijacking prompt injection: short instruction-like spans embedded in the evaluated text to coerce inflated scores. We split the corpus by paper ID into train/validation/test partitions, keeping all injected variants of a paper in the same partition. The defense objective is to recover each judgeās own clean-input score; agreement with human scores separately evaluates the scoring readout. Attacks, models, and metrics. We inject payloads into clean abstracts and distinguish seen attacks (held-out variants of families exposed during evolution) from unseen attacks (evolved payloads excluded from detector training). Unless stated otherwise, results use the unseen test set and the fixed tolerance in Eq. (4). Our defense is the StableJudge Sentry (0.13B) deployed as a pre-scoring tool, compared against large LLM-based detectors. The primary judge is our abstract-scoring-adapted Qwen3-8B; additional Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct backends are evaluated without Sentry-specific judge training. The detection benchmark contains 400 seen attacks, 400 unseen attacks, and 372 clean controls. For Table 2, all round checkpoints use the same fixed unseen test set. Character-level span IoU and text damage measure localization; paired score residuals measure end-to-end recovery; Pearson r and Kendall Ļ compare scoring readouts. Backend transfer holds the 400 attack inputs and Sentry outputs fixed. Appendix A provides detailed protocols and training settings. 4.2 Prompt Injection Detection We compare Sentry with LLM-based detectors that identify injections and propose removal spans, measuring character-level IoU against gold spans and original-text damage. Table 1 shows precise localization with the 0.13B Sentry: character IoU reaches 99.21%/99.97% on seen/unseen attacks, with 1.50%/0.50% damage and no observed clean false positives. These results support the taggerāverifier decomposition for removing localized instructions while retaining scoring evidence. Sentry outperforms the large LLM-based detectors in span overlap, supporting specialized token-level localization for precise sanitization with low damage to surrounding text. With Sentry frozen, a carrier-domain test on 805 clean AlpacaEval outputs and 200 injected outputs obtains 99.24% mean payload-token removal and 0.99% clean false positives (Appendix A.5), extending localization beyond scientific writing. Cue-triggered full-paper localization. On 30 full papers (20ā59 pages), we test 30 held-out payloads matching the scannerās fixed cue set. Head/middle/tail injections yield 90 attacked inputs, 30 per position. Table 7 shows 98.89% detection with 3.03 scanned regions per paper. Middle-position detection reaches 96.67%, supporting localization beyond document boundaries through cue-triggered windows. None of the 30 clean controls is edited. Appendix A.6 gives the window configuration and protocol. 4.3 RedāBlue Adversarial Training Starting from seed attacks, the red team generates variants guided by detector feedback, while the blue team learns from the expanded effective-attack pool. Figure 3(b) illustrates direct attacks, persona mimicry, and strategic ghost payloads. Figure 4(a) visualizes their embeddings across evolution rounds, with points colored by round to show changes in attack formulations. Evolution rounds and Sentry ablation. Table 2 tracks Sentry performance across redāblue training rounds. For Table 2, Perfect Removal denotes at least 95%95\% of injected content removed, Detected Removal denotes at least 10%10\% removed, and Missed denotes less than 10%10\% removed. Table 3 compares tagger-only inference with full Sentry, including region scanning and span verification. Recall measures attacks with at least 95%95\% removal; Integrity measures zero-edit inputs among the same 372 clean controls in both column groups. The full pipeline retains high attack-removal recall while leaving all clean controls unchanged, supporting targeted sanitization that preserves benign content. Appendix A.7 gives threshold and encoder controls. Independent attacker. Llama-3.1-8B-Instruct generates attacks without access to Sentry predictions or defended scores; 50 attacks screened as effective against Qwen2.5-7B-Instruct form a frozen evaluation set. Sentry brings 43/50 defended scores within ĻµĪµ of their clean counterparts, while editing 0/50 matched clean inputs. This tests transfer to an attack source outside redāblue training (Appendix A.4). 4.4 Score Recovery and Scoring Readout Frozen-Sentry backend transfer. With attacked inputs and sanitized texts fixed, frozen Sentry reduces mean absolute deviation from clean scores on all three backends (Table 5). Appendix A.3 gives per-backend effective-subset and thresholded results. Primary-judge score recovery. Figure 4(b) compares clean, attacked, and defended scores on the primary judgeās 298 effective attacks. Defense reduces mean absolute deviation from 1.10 to 0.36, with 69.5% of defended scores within ĻµĪµ. Mean scores are 7.82 (clean), 8.92 (attacked), and 7.47 (defended), with a mean signed residual of ā0.35-0.35. Scoring readout. With the judge and abstract inputs fixed, logit-based scoring improves Pearson r from 0.0213 to 0.2026 and Kendall Ļ from 0.0146 to 0.1052 against human paper-level scores (Table 5), demonstrating improved agreement through the readout alone. 4.5 General Capability Table 6 compares base Qwen3-8B with the aligned StableJudge scoring agent equipped with Sentry. General capabilities are largely retained: GPQA is unchanged, MMLU rises by 0.26 percentage points, and GSM8K falls by 1.44 points. 5 Conclusion StableJudge separates defense from judge choice through a compact, reusable Sentry. Span localization and verification preserve scoring evidence; mutual redāblue evolution improves attack coverage; a logit-based readout refines score resolution. Backend transfer, independent-attacker recovery, and cross-domain localization support this practical architecture for robust LLM-based scoring. Limitations Given the computational cost of repeated judge evaluations, our main scoring experiments focus on abstracts rather than full papers. The primary judge is trained on abstract-conditioned review supervision to align its scoring with the available input. Complementary full-paper experiments evaluate injection localization; extending end-to-end score recovery to full-paper judging is a natural direction for future work. Ethical considerations StableJudge aims to protect the integrity of LLM-based evaluation by mitigating score-hijacking prompt injection. Adversarial payloads serve defensive training and evaluation, but attack-generation techniques could also be misused to manipulate automated assessments. We recommend restricting their use to authorized robustness testing. In consequential applications such as paper screening, automated scores should support human judgment, with human oversight of decisions affecting individuals. Reuse of submissionāreview data should respect source access conditions and reviewer privacy. This study uses existing text corpora and model-generated attacks and involves no new human-subject experiments. References De Stefano et al. (2024) G. De Stefano, L. Schƶnherr, and G. Pellegrino Rag and roll: an end-to-end evaluation of indirect prompt manipulations in LLM-based application frameworks. arXiv preprint arXiv:2408.05025. Cited by: Ā§2. Dubois et al. (2024) Y. Dubois, B. Galambosi, P. Liang, and T. B. Hashimoto Length-controlled alpacaeval: a simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475. Cited by: Ā§1, Ā§2. Gera et al. (2025) A. Gera, O. Boni, Y. Perlitz, R. Bar-Haim, L. Eden, and A. Yehudai JuStRank: benchmarking LLM judges for system ranking. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria, p. 682ā712. Cited by: Ā§2. Godfrey et al. (2025) C. Godfrey, P. Nie, N. Ostapuk, D. Ken, S. Gao, and S. Inati Likert or not: llm absolute relevance judgments on fine-grained ordinal scales. arXiv preprint arXiv:2505.19334. Cited by: Ā§2. Greshake et al. (2023) K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz Not what youāve signed up for: compromising real-world LLM-integrated applications with indirect prompt injection. arXiv preprint arXiv:2302.12173. Cited by: Ā§1, Ā§2. Inan et al. (2023) H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa Llama guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674. Cited by: Ā§2. Karpas et al. (2022) E. Karpas, O. Abend, Y. Belinkov, B. Lenz, O. Lieber, N. Ratner, Y. Shoham, H. Bata, Y. Levine, K. Leyton-Brown, D. Muhlgay, N. Rozen, E. Schwartz, G. Shachaf, S. Shalev-Shwartz, A. Shashua, and M. Tenenholtz MRKL systems: a modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv preprint arXiv:2205.00445. Cited by: Ā§2. Keuper (2025) J. Keuper Prompt injection attacks on LLM generated reviews of scientific publications. arXiv preprint arXiv:2509.10248. Cited by: Ā§2. Li and Liu (2024) H. Li and X. Liu InjecGuard: benchmarking and mitigating over-defense in prompt injection guardrail models. arXiv preprint arXiv:2410.22770. External Links: 2410.22770 Cited by: Ā§2. Lin et al. (2024) G. Lin, T. Tanaka, and Q. Zhao Large language model sentinel: LLM agent for adversarial purification. arXiv preprint arXiv:2405.20770. Cited by: Ā§2. Liu et al. (2025) W. Liu, X. Ma, W. Sun, Y. Zhu, Y. Li, D. Yin, and Z. Dou ReasonRank: empowering passage ranking with strong reasoning ability. arXiv preprint arXiv:2508.07050. Cited by: Ā§2. Liu et al. (2023a) Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu G-eval: nlg evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, External Links: 2303.16634, Link Cited by: Ā§1, Ā§2. Liu et al. (2023b) Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong Formalizing and benchmarking prompt injection attacks and defenses. arXiv preprint arXiv:2310.12815. Cited by: Ā§1, Ā§2. Madry et al. (2018) A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, Cited by: Ā§2. Maloyan and Namiot (2025) N. Maloyan and D. Namiot Adversarial attacks on LLM-as-a-judge systems: insights from prompt injections. arXiv preprint arXiv:2504.18333. Cited by: Ā§2. Nasr et al. (2025) M. Nasr, N. Carlini, C. Sitawarin, S. V. Schulhoff, J. Hayes, M. Ilie, J. Pluto, S. Song, H. Chaudhari, I. Shumailov, A. Thakurta, K. Y. Xiao, A. Terzis, and F. TramĆØr The attacker moves second: stronger adaptive attacks bypass defenses against LLM jailbreaks and prompt injections. arXiv preprint arXiv:2510.09023. Cited by: Ā§2. Perez et al. (2022) E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, External Links: 2202.03286, Link Cited by: Ā§2. Perez and Ribeiro (2022) F. Perez and I. Ribeiro Ignore previous prompt: attack techniques for language models. External Links: 2211.09527 Cited by: Ā§1, Ā§2. Pukelsheim (1994) F. Pukelsheim The three sigma rule. The American Statistician 48 (2), p. 88ā91. External Links: Document Cited by: Ā§3.1. Radharapu et al. (2023) B. Radharapu, K. Robinson, L. Aroyo, and P. Lahoti AART: ai-assisted red-teaming with diverse data generation for new LLM-powered applications. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, Singapore, p. 380ā395. Cited by: Ā§2. Raina et al. (2024) V. Raina, A. Liusie, and M. Gales Is LLM-as-a-judge robust? investigating universal adversarial attacks on zero-shot LLM assessment. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 7499ā7517. External Links: Link Cited by: Ā§2. Sanh et al. (2019) V. Sanh, L. Debut, J. Chaumond, and T. Wolf DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. Cited by: Ā§3.3. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. DessƬ, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, External Links: 2302.04761 Cited by: Ā§2. Suri and McCrae (2025) O. F. K. Suri and J. McCrae Securing large language models (llms) from prompt injection attacks. arXiv preprint arXiv:2512.01326. Cited by: Ā§1. Tan et al. (2026) Q. Tan, X. Song, N. Cheng, N. Liu, X. Zhai, L. Hong, Y. Wang, Z. Xiang, and G. Yuan Q-realign: piggybacking realignment on quantization for safe and efficient llm deployment. arXiv preprint arXiv:2601.08089. Cited by: Ā§1. van der Maaten and Hinton (2008) L. van der Maaten and G. Hinton Visualizing data using t-SNE. Journal of Machine Learning Research 9 (86), p. 2579ā2605. External Links: Link Cited by: 4(a). Wallace et al. (2024) E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel The instruction hierarchy: training LLMs to prioritize privileged instructions. arXiv preprint arXiv:2404.13208. Cited by: Ā§2. Wee et al. (2025) S. Wee, S. Kim, H. Kim, K. Hwang, and N. Kwak Alignment-aware quantization for llm safety. In arXiv preprint arXiv:2511.07842, Cited by: Ā§1. Xu and Parhi (2025) W. Xu and K. K. Parhi A survey of attacks on large language models. arXiv preprint arXiv:2505.12567. Cited by: Ā§1. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, External Links: 2210.03629 Cited by: Ā§2. Ye et al. (2024) J. Ye, Y. Wang, Y. Huang, D. Chen, Q. Zhang, N. Moniz, T. Gao, W. Geyer, C. Huang, P. Chen, N. V. Chawla, and X. Zhang Justice or prejudice? quantifying biases in LLM-as-a-judge. arXiv preprint arXiv:2410.02736. Cited by: Ā§2. Zhan et al. (2024) Q. Zhan, Z. Liang, Z. Ying, and D. Kang InjecAgent: benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, p. 10471ā10506. External Links: Document, Link Cited by: Ā§2. Zheng et al. (2023) L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica Judging LLM-as-a-judge with MT-bench and chatbot arena. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), Track on Datasets and Benchmarks, External Links: 2306.05685 Cited by: Ā§1, Ā§2. Zhuang et al. (2025) Z. Zhuang, J. Chen, H. Xu, Y. Jiang, and J. Lin Large language models for automated scholarly paper review: a survey. Information Fusion, p. 103332. Cited by: Ā§1. Zou et al. (2023) A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. Cited by: Ā§2. Appendix A Supplementary Protocol and Transfer Experiments Judge / cohort N |Īatk|| _atk| Īrec _rec |Īrec|| _rec| Success Same fixed set of unseen inputs Qwen3-8B (adapted) 400 1.040 ā0.416-0.416 0.422 60.75% Qwen2.5-7B-Instruct 400 0.880 ā0.039-0.039 0.055 98.25% Llama-3.1-8B-Instruct 400 0.390 ā0.125-0.125 0.131 97.25% Effective attacks, selected separately for each judge Qwen3-8B (adapted) 298 1.103 ā0.350-0.350 0.359 69.46% Qwen2.5-7B-Instruct 358 0.937 ā0.038-0.038 0.055 98.04% Llama-3.1-8B-Instruct 151 0.622 ā0.090-0.090 0.103 99.34% Table 8: Score recovery using a frozen Sentry. |Īatk|| _atk|, Īrec _rec, and |Īrec|| _rec| are sample means. Success means |Īrec|ā¤0.407007| _rec|ā¤ 0.407007. Slice Clean / attacked Clean false positives Detection Mean removal Perfect removal All 805 / 200 0.99% 99.50% 99.24% 97.50% Instruction/list-heavy 309 / 78 1.62% 100.00% 99.93% 98.72% Code 165 / 42 0.00% 97.62% 96.51% 90.48% Recipe 65 / 14 0.00% 100.00% 100.00% 100.00% Table 9: Carrier-domain localization and preservation of clean text. False-positive rate is the fraction of clean inputs edited. Tagger encoder Perfect removal Clean false positives DistilBERT 98.80% 0.00% all-MiniLM-L6-v2 96.80% 1.61% Table 10: Controls on 250 cross-run attacks and 372 clean inputs. Perfect removal is at least 95% payload-token recall. A.1 Scope and score tolerance The primary task is recovery of the same judgeās clean-input score after an injected score-inflation attempt. We evaluate backend transfer, independent-attacker transfer, and carrier-domain localization with the same frozen Sentry. The score variability scale Ļ is estimated by repeated scoring on clean inputs, with the reference tolerance set to Ļµ=3āĻĪµ=3Ļ before attack-test evaluation. The reported scoring evaluations use Ļµ=0.407007Īµ=0.407007; additional backends retain this same numerical tolerance. Signed and absolute residuals complement thresholded success with continuous measures of score recovery. Localization metrics. Table 1 uses 400 seen attacks, 400 unseen attacks, and 372 clean controls. Let G and P denote the sets of gold-injection and predicted-removal character positions. Character IoU is |Pā©G|/|PāŖG||Pā© G|/|PāŖ G|, averaged over attacked inputs; damage is the fraction with any removal outside G. Clean false-positive rate counts clean inputs flagged or assigned removal spans. For the token-based removal controls, recall is the fraction of distinct lowercase payload words present in the removal text; perfect removal denotes at least 95% recall. Full-paper character coverage instead measures |Pā©G|/|G||Pā© G|/|G|. Localization metrics quantify the edits, while paired score residuals quantify their effect on scoring. A.2 Verifier, features, and sanitization The tagger and verifier use separate DistilBERT encoders; the approximately 0.13B parameter count refers to their combined deployment. Bidirectional contextual representations support token-level localization, while distillation provides a compact encoder. The verifier takes each proposed spanās text as input. Positive examples are injected spans and negative examples are clean sentences; its injected-class softmax probability is compared with the fixed threshold Ļ=0.5Ļ=0.5. The final tagger is trained on 2,860 training and 358 validation examples for three epochs, with batch size 16, learning rate 2Ć10ā52Ć 10^-5, maximum length 512, and seed 42. The verifier is trained for three epochs with batch size 32, learning rate 2Ć10ā52Ć 10^-5, and maximum length 128. Accepted overlapping intervals are merged before replacement. The primary and cross-backend scoring results replace verified spans with the fixed marker [REDACTED SECURITY THREAT]. The category report ākI_k is assembled deterministically from category labels and representative payloads. After a blue round, the current tagger probes retained payloads; a complete miss means that no injected span is proposed. These misses are grouped into the next report; when there are none, it requests exploration beyond existing categories. The report conditions red-side generation, while Eq. (9) controls admission to the effective pool. Pool maintenance normalizes and length-limits strings and removes duplicates. Category-conditioned variants retain their category, while creative generation can propose new categories. Novelty encourages diversity during generation, separately from score-shift filtering; the recorded configuration uses weight 0.2 and minimum novelty 0.08. A.3 Frozen-Sentry backend transfer All three backend evaluations use the same 400 held-out unseen inputs and frozen removal predictions. The primary Qwen3-8B judge is adapted to abstract scoring; the added judges are Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct. Sentry is neither retrained nor retuned for the added judges. We retain the original replacement protocol across these runs. Table 8 reports both the fixed cohort and effective subsets selected separately for each judge (Īatk>Ļµ _atk>Īµ). Mean absolute attack shift in the fixed cohort includes both upward and downward movements, while signed residuals indicate the direction of remaining score displacement. A.4 Independent attack generator Llama-3.1-8B-Instruct generates plausible academic/editorial score-hijacking payloads while avoiding overt injection cues, without access to StableJudgeās attack pool, category reports, Sentry predictions, or defended scores. We freeze 50 attacked inputs on different abstracts, selected by their initial score impact on Qwen2.5-7B-Instruct, and retain their 50 paired clean inputs. The Sentry and its predictions remain fixed. Using the short placeholder [REDACTED], 43/50 defended scores are within ĻµĪµ of the clean score, and none of the 50 clean inputs is edited. A.5 Carrier-domain transfer We use 805 AlpacaEval outputs as clean carrier texts and inject 200 randomly selected held-out evolved payloads into 200 of them, at prefix/middle/suffix positions (67/67/66). Only the benign carrier domain changes: the trained detector, attack pool, and thresholds are fixed. The task is span localization and preservation of clean content. Mean removal is payload-token recall; perfect removal means at least 95% recall. The listed content slices are disjoint subsets of the overall set. A.6 Selective full-paper scanning We sample 30 NeurIPS 2025 full papers (20ā59 pages; median 32 pages and 102,024 extracted characters) and 30 held-out evolved payloads matching the scannerās fixed cue set. Each paper receives an injection at head, middle, and tail, giving 90 attacked inputs and 30 clean controls. The detector, cue set, and thresholds remain fixed. The scanning configuration takes the first and last 2,000 characters, plus a window extending 500 characters on either side of each radar match. Each window is encoded with a maximum of 512 tokens. The eight case-insensitive cue patterns cover previous-instruction overrides, task/goal reassignment, text disregard, score-only requests, requests for score 10, system overrides, system-administrator impersonation, and explicit score assignments. Window size is bounded; the number of interior windows depends on cue matches. Reported scan counts include overlapping windows before the final span merge. Table 7 in the main text reports detection and character coverage; detection counts attacked inputs with an accepted span. Mean character-level IoU is 86.00%. A.7 Verifier threshold and encoder controls We hold the final-round training split, seed, tagger optimization settings, behavioral features, verifier, and evaluation manifest fixed. The encoder comparison changes only the tagger encoder. Both tagger encoders achieve high perfect-removal rates, with a lower clean false-positive rate for DistilBERT. Across Ļā0.3,0.5,0.7Ļā\0.3,0.5,0.7\, detection is 249/250, perfect removal ranges from 98.8% to 99.2%, and no clean false positives are observed among the 372 controls.