Paper deep dive

GlitchPatch: Repairing Glitch Tokens in Frozen Language Models via Local Retokenization

Kunsheng Tang, Peigui Qi, Yide Song, Peijun Huang, Weiming Zhang, Nenghai Yu

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 85

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Glitch tokens are anomalous vocabulary entries that can cause large language models (LLMs) to produce outputs inconsistent with their inputs. Existing repair methods require access to model internals, making them impractical for frozen checkpoints. We investigate whether glitch tokens can be repaired outside the model by optimizing the input tokenization. An empirical study on BPE merge-rule deletion reveals that (1)deleting a glitch token's merge rule can fix a substantial fraction of failures, yet disrupting normal tokens sharing intermediate merge nodes causes the overall glitch rate to rise, and (2)different decomposition granularities yield non-monotonic fix rates while collateral damage on normal tokens grows monotonically. Motivated by these findings, we propose GlitchPatch, a repair framework for frozen language models based on local retokenization, consisting of two stages: the offline stage uses Behavioral Path Optimization (BPO) to find the behaviorally optimal replacement token sequence for each glitch token and compiles validated replacements into a rule table; the online stage substitutes only the IDs of matched glitch tokens in the canonical token sequence, with no modification to model parameters or internal states. Experiments on ten models spanning six tokenizer families show that GlitchPatch achieves an 85.10% mean fix rate, outperforming the strongest baseline by 14.37 percentage points, and reduces the average glitch rate from 14.88% to 2.27%. GlitchPatch achieves a 0.00% RR in full-vocabulary evaluation and leaves rule-unmatched inputs unchanged by design. We further evaluate the practical impact of repair from the perspectives of time cost, language understanding, and capability, supporting its deployment feasibility. We hope this work provides a practical option for improving tokenizer reliability.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

81,963 characters extracted from source content.

Expand or collapse full text

Preprint GLITCHPATCH: REPAIRING GLITCH TOKENS IN FROZEN LANGUAGE MODELS VIA LOCAL RETOKENIZATION Kunsheng Tang 1,∗ Peigui Qi 1,∗ Yide Song 2 Peijun Huang 3 Weiming Zhang 1 Nenghai Yu 1 1 University of Science and Technology of China 2 University of Washington 3 Wuhan University ∗ Equal contribution ABSTRACT Glitch tokens are anomalous vocabulary entries that can cause large language models (LLMs) to produce outputs inconsistent with their inputs. Existing repair methods require access to model internals, making them impractical for frozen checkpoints. We investigate whether glitch tokens can be repaired outside the model by optimizing the input tokenization. An empirical study on BPE merge- rule deletion reveals that (1) deleting a glitch token’s merge rule can fix a substantial fraction of failures, yet disrupting normal tokens sharing intermediate merge nodes causes the overall glitch rate to rise, and (2) different decomposition granularities yield non-monotonic fix rates while collateral damage on normal tokens grows monotonically. Motivated by these findings, we propose GLITCHPATCH, a repair framework for frozen language models based on local retokenization, consisting of two stages: the offline stage uses Behavioral Path Optimization (BPO) to find the behaviorally optimal replacement token sequence for each glitch token and compiles validated replacements into a rule table; the online stage substitutes only the IDs of matched glitch tokens in the canonical token sequence, with no modifi- cation to model parameters or internal states. Experiments on ten models spanning six tokenizer families show that GLITCHPATCH achieves an 85.10% mean fix rate, outperforming the strongest baseline by 14.37 percentage points, and reduces the average glitch rate from 14.88% to 2.27%. GLITCHPATCH achieves a 0.00% R in full-vocabulary evaluation and leaves rule-unmatched inputs unchanged by design. We further evaluate the practical impact of repair from the perspectives of time cost, language understanding, and capability, supporting its deployment feasibility. We hope this work provides a practical option for improving tokenizer reliability. 1INTRODUCTION The vocabularies of large language models (LLMs) contain anomalous entries known as glitch tokens, which can induce outputs inconsistent with their inputs (Land & Bartolo, 2024). Some glitch tokens closely resemble normal tokens, making them easy to trigger inadvertently and leading to unexpected behavior that degrades user experience (Figure 1). When such strings appear in product names, database fields, or code identifiers, errors can propagate to downstream tasks. Prior work has linked glitch tokens to unsafe generation and degraded instruction following (Land & Bartolo, 2024; Li et al., 2024; Tang et al., 2026), making their repair a necessary step toward improving model reliability. What is "fort awesome" used for? It provides icons for websites and lets you manage your own custom icon sets. What is "fortawesome" used for? What is "fortawesome" used for? It is a game for building fortsand defending them againstattacks from other players. It offers icons for websites and helps you manage your own custom icon collections. Normal Users Careless Users Careless Users with Our GlitchPatchNormal TokenGlitch Token Figure 1. A glitch-token example and our GLITCHPATCH repair. The strings “fortawesome” (glitch token) and “fort awesome” (normal tokens) differ by one space. The model describes an icon library for the latter but a fort-building game for the former. GLITCHPATCH restores the relevant answer without changing the input text. 1 arXiv:2610.04399v1 [cs.CL] 3 Oct 2026 Preprint Existing work repairs glitch tokens from two directions. Activation-based methods, represented by GlitchProber (Zhang et al., 2024), access intermediate-layer activations at inference time and correct target neuron states. Parameter-based methods modify the model’s parameters: GlitchEdit (Tang et al., 2026) edits the embedding weights of target tokens, and GlitchCleaner (Fan et al., 2026) trains and attaches a gated LoRA branch. These methods are effective in their respective settings, but they must happen inside the model. Whether inserting activation hooks, rewriting embedding weights, or loading adapter parameters, all require access to model internals or alter the model’s parameters. In practice, however, most providers deploy frozen checkpoints integrated into production pipelines with verified kernels, version control, and regression testing. Any internal modification can invalidate these pipelines and introduce regression risks. When providers discover glitch tokens, they typically need a fast, reversible fix that leaves the model itself untouched rather than a costly internal refactor. To date, no work has explored whether glitch tokens can be repaired from outside the model. Tokenization provides a natural entry point for external repair. An LLM does not process raw text directly; its tokenizer encodes the input into a sequence of vocabulary tokens, each mapped to a learned embedding (Figure 2). Tokenizers typically use byte-pair encoding (BPE), which builds tokens by progressively merging shorter units according to learned merge rules. If the final token is a glitch token, an intuitive fix is to block its last merge so that the model receives the normal sub-tokens instead. The observation that frozen models can tolerate non-canonical tokenizations supports this direction (Zheng et al., 2025). However, this intuition does not directly yield reliable repair. We conduct an empirical study on deleting BPE glitch token merge rules and uncover two key findings (Figure 3). First, deleting a glitch token’s merge rule does fix a portion of failures but also disrupts normal longer tokens that share the same intermediate merge node (Figure 4a), causing the overall glitch rate to increase (Finding 1). Second, decomposing a glitch token to different granularities produces replacement paths whose fix rates vary non-monotonically across models (Figure 4b), while collateral damage to normal tokens grows monotonically with depth (Finding 2). These two findings expose two open problems for external repair: how to find the optimal replacement path for each glitch token, and how to eliminate collateral effects on normal tokens. Motivated by these findings, we propose GLITCHPATCH, an external repair framework for frozen language models based on Local Retokenization (Figure 5). GLITCHPATCH leaves the tokenizer unchanged and replaces only the IDs of matched glitch tokens in the canonical token sequence. The framework has two stages: offline, Behavioral Path Optimization (BPO) searches for the behaviorally optimal equivalent token sequence for each known glitch token and compiles validated replacements into a rule table through full-context evaluation and independent calibration; online, only rule matching and ID substitution are performed, without reading or modifying any model parameters or internal states. Experiments on ten models spanning six tokenizer families show that:❶for repair effectiveness, GLITCHPATCH achieves an 85.10% mean fix rate, outperforming the strongest baseline by 14.37 percentage points and reducing the mean glitch rate from 14.88% to 2.27%;❷for impact on normal tokens, the regression rate is 0.00% on the full-vocabulary evaluation, while requests that trigger no applicable rule are preserved by construction;❸for practicality, the average token-level loss drops by 13.5% on natural texts containing glitch tokens, with negligible online latency overhead. These results demonstrate that by optimizing the tokenization, an external method can not only repair glitch tokens effectively but outperform existing internal approaches. In summary, this paper makes three contributions:❶we present the first exploration of repairing glitch tokens from outside the model, identifying two limitations of merge-rule deletion that motivate our approach (Section 3);❷we propose GLITCHPATCH, a framework that combines offline BPO path optimization with online local retokenization, repairing glitch tokens with minimal impact on normal tokens and overhead (Section 4);❸extensive experiments show that GLITCHPATCH achieves a mean fix rate of 85.10%, surpassing all existing methods, with validated practical deployment feasibility in time cost, language understanding, and general capability (Section 5 & 6). 2BACKGROUND AND RELATED WORK This section introduces tokenization in LLMs and reviews glitch-token detection and repair, clarifying the gap between existing methods and frozen deployment needs. 2.1TOKENIZER AND TOKENIZATION 2 Preprint Hello! BPE Tokenizer Hel l o! ello ello Hello[9419]![0] Token IDs Large Language Models Input Text [9419], [0] Hi! Output Text Figure 2. BPE tokenization of “Hello!” with Qwen3.8-27B. LLMs do not process raw text directly. The input is first split by a tokenizer into vocabulary tokens, each mapped to a vector via an embedding lookup and then processed by the Transformer. A token may be a word fragment or part of a byte sequence, so different segmentations of the same text produce different model inputs. Most modern tokenizers adopt Byte Pair Encoding (BPE). BPE learns an ordered set of merge rules from the training corpus; at encoding time, it starts from the smallest units and repeatedly merges adja- cent pairs by rule priority until no further merge applies (Figure 2, Qwen3.8-27B(Qwen Team, 2026)). For a given text and tokenizer, this deterministic result is the canonical tokenization, denotedT (x). However, blocking a merge step or using other vocabulary entries covering the same byte span can produce ID sequences that decode to the same bytes; we call these equivalent tokenizations. Prior work has shown that some frozen models tolerate or even benefit from non- canonical segmentations (Zheng et al., 2025; Ghosh & Jyothi, 2026), and techniques such as BPE-Dropout enhance this robustness by introducing segmentation variation during training (Kudo, 2018; Provilkov et al., 2020). These findings indicate that tokenization is a manipulable variable, providing the basis for repairing glitch tokens at the tokenization level. 2.2GLITCH-TOKEN DETECTION AND REPAIR IN LLMS Glitch tokens are vocabulary entries that can induce anomalous behavior under certain inputs. When passed to downstream components such as retrieval or code execution, a single string-level error can cascade into workflow-wide failures (Figure 1). Prior work has linked glitch tokens to unsafe generation and behavior (Li et al., 2024; Tang et al., 2026), underscoring the necessity of repair. Detection. Existing work identifies anomalous tokens through behavioral tasks. Early studies (Land & Bartolo, 2024; Wu et al., 2026) use repetition tasks to find tokens a model cannot reproduce correctly. Li et al. (2024) add spelling and length tasks, extending coverage to three dimensions. GlitchQuiz (Tang et al., 2026) further enriches prompt variants to improve robustness. In this work, we adopt the three shared task types as criteria: repetition (whether models can correctly repeat tokens), spelling (whether models can correctly spell tokens), and length (whether models can correctly count characters in tokens), and focus on repairing the identified glitch tokens. Repair. Existing methods intervene internally. GlitchProber (Zhang et al., 2024) hooks into inference to correct intermediate activations; GlitchEdit (Tang et al., 2026) edits target embedding weights; GlitchCleaner (Fan et al., 2026) trains and loads a gated LoRA branch. All are effective in their settings but require access to model internals. For providers maintaining frozen checkpoints, every internal modification entails pipeline adjustments and potential regression risks. This paper investigates a stricter scenario: the base parameters and the forward implementation all remain unchanged, offering providers a more practical pathway to repair. 3EMPIRICAL STUDY ON TOKENIZATION REPAIR The previous section revealed the possibility of repairing glitch tokens externally by optimizing tokenization. The direct attempt along this direction is to delete the BPE merge rules that produce known glitch tokens so that the corresponding strings are encoded as their sub-tokens instead of the glitch IDs. This section conducts an empirical study of this approach and reports two key findings. Setup. We select Llama-3.1-8B-Instruct (Meta, 2024) and Qwen3.8-27B (Qwen Team, 2026), representing the Llama BPE (128,256 entries) and Qwen BPE (248,320 entries) tokenizer families. Both use fixed checkpoints with BF16 precision and greedy decoding; Qwen uses text-only input with thinking disabled. We run the three GlitchQuiz (Tang et al., 2026) tasks (repetition, spelling, length) on every vocabulary entry; tokens passing all three are labeled normal, others glitch. Llama contains 22,026 glitch tokens (17.17%) and Qwen 29,451 (11.86%). The full protocol is in Appendix A. Evaluation. For each glitch token, we reconstruct its merge tree from the BPE execution history and define three backtracking depths. Depth-1 deletes only the rule producing the target token; Depth-2 3 Preprint FRRRGlitch Rate 0 20 40 60 80 100 Rate (%) 17.17 36.82 9.73 18.91 Depth 0 (original) Depth 1 FRRRGlitch Rate 0 20 40 60 80 100 Rate (%) 11.86 40.73 6.48 12.74 Depth 0 (original) Depth 1 0123 0 20 40 60 80 100 Rate (%) 36.82 33.14 37.84 9.73 15.87 22.63 17.17 18.91 24.63 29.42 Depth FR R Glitch Rate 0123 0 20 40 60 80 100 Rate (%) 40.73 44.23 38.61 6.48 13.14 19.78 11.86 12.74 18.20 24.71 Depth FR R Glitch Rate Llama-3.1-8B-InstructLlama-3.1-8B-InstructQwen3.8-27BQwen3.8-27B (a) One-level Deletion (b) Varying Backtracking Depth Figure 3. Repair gains and new errors from global merge deletion. (a) FR, R, and GR after depth-1 backtracking on two models. (b) The three metrics at increasing backtracking depths. (a) Shared Merge Dependency (b) Different Depths Construct Different Paths Original Merges Delete One Merge Rule Infrastructure 94461 Inf 18843 rastructure 10008 ra 926 structure 7589 Glitch Infrastructure 94461 Inf 18843 rastructure 10008 ra 926 structure 7589 Removed Input IDs Input IDs18843 926 7589 DepthNewly Deleted Merge Rules Tokenizationat This Depth 1 rastructure 2 r + a →ra struct + ure→structure structure 3 str + uct→struct u + re →ure Infrastructure 94461 Tokenization Tokenization Passed Detection Normal Failed Detection Glitch ra+ structure →rastructure rastructure Infrastructure r structure aInf r a r str re a Inf uct u str re uct u ra Figure 4. Collateral effects of shared merges and path variation across depths on Qwen3.8-27B. (a) Deleting the rule for the glitch token “rastructure” blocks the construction path of the normal token “Infrastructure”, changing its encoding from one token to three and causing detection failure. (b) Different backtracking depths produce different sub-token paths for both tokens. additionally deletes the rules producing its children; Depth-3 extends one further level. At each depth, the deletion set is the union over all targets, applied globally. After deletion, we preserve the remaining rule priorities and vocabulary, re-encode all entries, and re-run detection. Metrics. We use three metrics to measure repair effectiveness, all computed at the token level over the full vocabulary.❶Glitch Rate (GR) is the fraction of vocabulary tokens that still fail at least one of the three tests after repair, reflecting the net effect of repair gains and newly introduced errors. ❷Fix Rate (FR) is the fraction of original glitch tokens that become normal.❸Regression Rate (R) is the fraction of original normal tokens that fail at least one test after repair and thus become glitchy. Results. Figure 3 shows FR, R, and GR for both models at different backtracking depths. We analyze the results and report two findings. Finding1:Deletingglitch-tokenmergeruleshelpsrepairbutintroducesnewerrors. At depth-1, FR reaches 36.82% and 40.73% for Llama and Qwen, showing that changing the tokenization alone can fix a substantial fraction of failures (Figure 3a). However, R reaches 9.73% and 6.48%, and GR increases (17.17%→18.91% and 11.86%→12.74%). The cause is that a glitch token can also serve as an intermediate merge node for a normal longer token. As shown in Figure 4(a), deleting the rule for “rastructure” prevents “Infrastructure” from forming through its original path, forcing it into three sub-tokens and causing detection failure. Global deletion cannot distinguish a target appearing as a final input from one used only during construction. Finding2:Fixratesvarybypathwhilecollateraldamagegrowswithdepth. Different depths essentially generate different sub-token sequences for the same glitch token (Figure 4b), and models differ in how well they handle each path. As shown in Figure 3(b), Llama’s FR drops from 36.82% at depth-1 to 33.14% at depth-2, then rises to 37.84% at depth-3; Qwen exhibits the opposite trend, peaking at 44.23% at depth-2 and falling to 38.61% at depth-3. Meanwhile, R rises monotonically with depth for both models, and the overall GR continues to worsen, indicating that collateral damage to normal tokens from global rule deletion is unavoidable and intensifies with depth. These findings show that tokenization-level repair is viable, but global rule deletion cannot solve the problem: fix rates vary across paths with no universally optimal depth, and collateral damage to normal tokens is unavoidable. An effective repair must therefore find the best replacement path for each token individually while avoiding the impact on normal tokens. 4 Preprint 4GLITCHPATCH To address the problems in Section 3, we propose GLITCHPATCH, a repair framework based on local retokenization. GLITCHPATCH has two stages (Figure 5):❶the offline stage uses Behavioral Path Optimization (BPO) to find the optimal replacement sequence for each glitch token. Since candidates per token can be numerous and full evaluation is expensive, BPO first ranks with a lightweight proxy score and evaluates the top candidates;❷the online stage applies local retokenization, replacing only the IDs of target glitch tokens while leaving normal tokens untouched. Details as follows. Offline: Cost-Efficient Path Search via Behavioral Path Optimization (BPO) Known glitch targets G₀ and normal-token vocabulary N₀ 01 Candidate Space 02 Proxy RankingJ(p) 03 Full Evaluation U(g, p) 04 Rule Calibration 05 Accepted Rule Table rastructure[10008] 01234567891011 ra structure structure ras tructure Normal-token edges only Exact bytes, Δ ≤ 8 Lowest J(p), K = 8 1 [ras, tructure] [12418, 192335] 2 [ra, structure] [926, 7589] 3 [ra, struct, ure] [926, 1193, 538] Highest U within Top-K 12 Evaluation Probes (4/Dim) + Repair Gain − κ×Regression Penalty − λ×Length Cost U >γ, Passes All 3 Check Tests Independent Probes Selected Path Only Passes Tests on Independent Probes Fail: Keep Original Original ID 10008 Replacement IDs [12418, 192335] UtilityU(g, p*) ConditionsΔ= 1 rastructure→ ras+ tructure Load Fixed Rules Online: Local Retokenization Local replacement of target IDs in canonical tokenization Input TextTokenizerCanonical IDs Rule Lookup Rebuild IDs FrozenModel “... rastructure...” T (original) ...10008... T(x) Rule Hit Availability Budget Match IDs Once ! Glitch Matched & Rule Usable 1000812418192335 apply No GlitchMatch Keep infrastructure [94461] Glitch Matched but Unusable Keep rastructure[10008]if no usable rule ...12418 192335... One pass, Original Order LLM One Inference D(R(T(x))) = D(T(x)) Figure 5. The two-stage GLITCHPATCH framework. The offline stage uses BPO to search for the behaviorally optimal replacement sequence for each glitch token and compiles validated results into a rule table. The online stage performs budget-constrained local ID substitution on the canonical tokenization output. 4.1OFFLINE PATH SELECTION Finding 2 shows that repair effectiveness varies by path and model, with no universally optimal strategy. Meanwhile, candidate paths per glitch token can be numerous, making exhaustive evaluation impractical. This subsection formalizes path selection as a constrained optimization problem and presents BPO to find the utility-maximizing replacement at manageable cost. 4.1.1PROBLEM FORMULATION LetTbe the original tokenizer,Rthe patch transform, andxan input text. The patch must satisfy a decoding equivalence constraint: D(R(T (x))) = D(T (x)),(1) ensuring that token ID sequences before and after replacement decode to the same bytes. Candidate Space. For a glitch tokengwith byte sequenceb, we treat each byte position as a node. If a normal token covers exactlyb[i :j], an edge links positionsiandj. All start-to-end paths form P(g); every sub-token must belong to the normal set N 0 . Optimization Objective. We constructmevaluation probes (four prompt variants per dimension: repetition, spelling, length). Letm + be probes passed under the original representation,N fix (p) failed probes now passing,N reg (p)passed probes now failing, and∆(p) = |p|− 1added tokens. The replacement utility is: U (g,p) = N fix (p) m | z repair gain − κ· N reg (p) m + |z regression penalty −λ· ∆(p) B | z length cost .(2) Hereκ > 1makes breaking a previously passed probe more costly than fixing a failed one;Bis the length budget andλcontrols length sensitivity. Eq.(2)is evaluated only whenm + > 0; otherwise the original representation is retained. The offline goal is: p ⋆ =arg max p∈P(g), U (g,p)>γ U (g,p) s.t. D(p) = D(g), ∆(p)≤ B,(3) whereγensures only candidates with positive net benefit are accepted. When no candidate qualifies, the original representation is retained. 5 Preprint 4.1.2BEHAVIORAL PATH OPTIMIZATION (BPO) BPO solves Eq. (3) via a four-stage pipeline: construct the replacement space, rank candidates with a proxy score, perform full evaluation on the top-Kpaths, and confirm rules through independent calibration. Pseudocode is in Appendix C.5 (Algorithm 1) and the example in Figure 7. Candidate Construction. BPO constructsP(g)as defined in Section 4.1.1. The vocabulary contributes tokens beyond the original merge tree, covering all valid equivalent paths. When no valid path exists, the original representation is retained. Graph-search properties are in Appendix C.3. Proxy Ranking. ComputingU (g,p)requires heavy model calls, so BPO uses a proxy score to rank candidates and sends only the top-Kto full evaluation. Because all components inN 0 pass the three base classification tests, those binary labels alone cannot distinguish quality. We estimate empirical failure rates ˆ f (v)and ˆ f (u,v)using three supplemental probes disjoint from all other prompts. The proxy score for path p=(v 1 ,...,v k ) is J (p) = |p| X j=1 ˆ f (v j ) + η |p| X j=2 ˆ f (v j−1 ,v j ) + λ r · ∆(p).(4) Lower J (p) indicates a more promising candidate. We default to η=0.5, λ r =0.02, K=8, and B=8. Full-sequence Evaluation. BPO inserts each shortlisted path into complete test inputs, usingm = 12 selection probes to computeU (g,p). A candidate must (i) pass all three check tests spanning three behavioral dimensions, whose templates are distinct from the GlitchQuiz prompts used for the reported FR/R/GR; (i) satisfyU (g,p)>γ. We default toκ=2,λ=0.02, andγ=0.05 (Appendix D.2). Rule Confirmation. An independent set of calibration probes performs final confirmation. A rule is accepted only when all calibration probes pass; failure retains the original representation without testing the next-best candidate, preventing indirect data leakage. Each saved rule records the original ID, replacement IDs, content, added tokens, and utility. 4.2ONLINE LOCAL RETOKENIZATION Overall Flow. As in Figure 5, the online stage proceeds in four steps. First, the original tokenizer encodes the input text into canonical IDsT (x). Second, the system performs a single-pass lookup against the offline rule table, checking each token ID for three conditions: whether it is a registered glitch token, whether a validated replacement rule exists, and whether the length budget permits substitution. Three cases arise:❶if all conditions are met, the glitch-token ID is replaced by its optimized sub-token sequence;❷if the token is not a glitch token, it passes through unchanged;❸if the token is a known glitch token but no usable rule exists or the budget is exceeded, the original ID is retained. Third, all tokens are reassembled in their original order to form the patched sequence. Finally, the patched sequence is fed to the frozen model for inference, satisfyingD(R(T (x))) = D(T (x)). Because only identified glitch-token IDs are ever replaced, the IDs of unmatched normal tokens are preserved by construction (Appendix C.2). Budget Allocation and Reconstruction. Each target may add at mostB=8 tokens. When multiple targets co-occur, all applicable rules are applied to their corresponding positions and reconstructed in original order, with the length budget enforced independently per target. Rule matching is performed once against the initial canonical IDs and does not trigger recursively. The reconstructed sequence enters the model normally; no model parameters are changed. 5EXPERIMENTAL SETUP Having described the GLITCHPATCH framework, we now detail the models, evaluation protocol, metrics, and baseline configurations used to assess its effectiveness. Models and Execution. We evaluate on ten models spanning six BPE tokenizer families with vocabularies from 32K to 262K: Llama-3.1-8B/70B-Instruct (Meta, 2024), Gemma-4-31B/E4B- it (Google, 2026), Qwen3.8-27B, Qwen3.6-27B, Qwen3.6-35B-A3B (Qwen Team, 2026), Mistral- 7B-Instruct-v0.2 (Mistral AI, 2024), GLM-4.7-Flash (Zai-org, 2026), and Deepseek-vl2 (DeepSeek 6 Preprint AI, 2025) (text-only input). All use BF16, official chat templates, and greedy decoding. Per-model configurations are in Appendix Table A2; tokenizer adaptation details are in Appendix C.4. Detection and Evaluation. For each model, every vocabulary token is tested on three GlitchQuiz tasks (repetition, spelling, length). Tokens passing all three are labeled normal; those failing any are labeled glitch. This full-vocabulary protocol eliminates selection bias and enables precise tracking of each token’s status change. Scoring rules and sample sizes are in Appendix A. Baselines. We compare against activation-based GlitchProber (Zhang et al., 2024) and parameter- based GlitchEdit (Tang et al., 2026) and GlitchCleaner (Fan et al., 2026). Each retains its required internal access; GLITCHPATCH’s input-only restriction is not imposed on baselines. All methods share the same vocabulary detection results and classification criteria. Configurations are in Appendix D.3. Repair Effectiveness. Following Section 3, we evaluate repair effectiveness using three metrics: ❶Glitch Rate (GR), the fraction of vocabulary tokens that fail at least one of the three GlitchQuiz tasks;❷Fix Rate (FR), the fraction of original glitch tokens successfully repaired; and❸Regression Rate (R), the fraction of original normal tokens that become glitches after repair. Practicality Assessment. We assess practicality from these aspects:❶Time Cost: offline build time and online latency on8×A100;❷Language Understanding: relative cross-entropy loss reduction on GPT-5.6-Terra-generated passages containing repaired tokens;❸General Capability: MMLU-Pro (Wang et al., 2024), ARC-Challenge (Bhakthavatsalam et al., 2021), GSM8K (Cobbe et al., 2021), HumanEval (Chen et al., 2021), with General Score as the arithmetic mean of four accuracies;❹We also evaluate contextual regression and out-of-distribution downstream performance (Appendix H). 6EXPERIMENTAL RESULTS We first report repair effectiveness in Section 6.1, followed by practicality evaluation, proxy-score analysis, repairability boundaries, and ablation studies in Sections 6.2–6.5. 6.1REPAIR EFFECTIVENESS Table 1 reports GLITCHPATCH’s results on ten models. The mean FR is 85.10%, R is 0.00%, and mean GR decreases from 14.88% to 2.27%. The highest FR is 91.04% (Llama-3.1-70B) and the lowest 80.50% (Gemma-4-E4B, largest vocabulary at 262K with 18.42% initial GR). The 0.00% R follows from two factors: untriggered requests preserve original IDs by construction (Proposition 2), and in full-vocabulary testing each token is evaluated in isolation. Per-model counts are in Appendix B. Table 1. GLITCHPATCH repair results across models. Full-vocabulary, token-level evaluation. Red subscripts show GR changes from the original model in percentage points. ModelGlitchFixedReg.FR%↑R%↓GR%↓ Llama-3.1-8B22,02618,722085.000.002.58 −14.60 Llama-3.1-70B 15,87814,455091.040.001.11 −11.27 Gemma-4-31B 36,04532,080089.000.001.51 −12.24 Gemma-4-E4B48,28738,873080.500.003.59 −14.83 Qwen3.8-27B29,45125,770087.500.001.48 −10.38 Qwen3.6-27B 31,04026,546085.520.001.81 −10.69 Qwen3.6-35B-A3B35,26129,438083.490.002.34 −11.85 Mistral-7B6,2405,083081.460.003.62 −15.88 GLM-4.720,44416,963082.970.002.25 −10.95 Deepseek-vl2 20,42617,267084.530.002.44 −13.36 Mean85.100.002.27 −12.61 Table 2 compares four methods. GLITCHPATCH’s mean FR of 85.10% exceeds GlitchCleaner (70.73%) by 14.37 p, GlitchEdit (58.39%) by 26.71 p, and GlitchProber (24.63%) by 60.47 p, achieving the highest FR on every model. All baselines require model-internal access, whereas GLITCHPATCH operates only on input IDs with fixed hyperparameters across six tokenizer families. 7 Preprint Table 2. Fix Rate comparison across four methods (%↑). Green subscripts show GLITCHPATCH’s gains over each baseline in percentage points. ModelGlitchProberGlitchEditGlitchCleanerGLITCHPATCH Llama-3.1-8B21.29 +63.71 59.16 +25.84 68.07 +16.93 85.00 Llama-3.1-70B 30.97 +60.07 65.03 +26.01 78.48 +12.56 91.04 Gemma-4-31B36.19 +52.81 66.37 +22.63 81.28 +7.72 89.00 Gemma-4-E4B12.35 +68.15 51.77 +28.73 59.12 +21.38 80.50 Qwen3.8-27B27.01 +60.49 60.48 +27.02 74.02 +13.48 87.50 Qwen3.6-27B30.09 +55.43 61.67 +23.85 76.06 +9.46 85.52 Qwen3.6-35B-A3B 17.57 +65.92 52.82 +30.67 64.62 +18.87 83.49 Mistral-7B20.96 +60.50 53.04 +28.41 67.45 +14.01 81.46 GLM-4.735.69 +47.28 62.83 +20.14 77.59 +5.38 82.97 Deepseek-vl2 14.22 +70.31 50.77 +33.76 60.59 +23.94 84.53 Mean24.63 +60.47 58.39 +26.71 70.73 +14.37 85.10 6.2PRACTICALITY ASSESSMENT We select three representative models (Mistral-7B, Qwen3.8-27B, and Llama-3.1-70B) and compare all baseline methods. Figure 6 summarizes the results, with additional details in Appendices E–G. 0 10 20 Offline Build (h) 0.80 2.50 4.80 1.60 4.50 9.80 3.80 11.50 22.50 0.36 1.02 2.08 Mistral-7BQwen3.8-27BLlama-70B 0 5 10 Latency Increase (%) 8.50 9.80 10.20 0.000.000.00 0.85 1.10 1.20 0.05 0.08 0.06 Mistral-7BQwen3.8-27BLlama-70B 0 4 8 12 16 Relative Loss Reduction (%) 2.6 2.3 1.8 5.4 5.1 4.8 8.1 8.8 7.4 13.2 13.9 11.9 Mistral-7BQwen3.8-27BLlama-70B −0.20 −0.15 −0.10 −0.05 0.00 0.05 General Score Δ (p) −0.18 −0.13 −0.11 −0.08 −0.06 −0.04 −0.17 −0.09 −0.07 +0.025+0.025 +0.000 (a) Time Cost (↓) (b) Language Understanding (↑)(c) General Capability (Δ ≈ 0) GlitchProberGlitchEditGlitchCleanerGlitchPatch (Ours) Figure 6. Practicality comparison across three representative models. (a) Offline build time and online latency increase. (b) Relative reduction in token-level loss. (c) Change in General Score. Time Cost. We compare offline build time and online latency on8×A100. GlitchProber computes activation statistics (0.80–4.80 h) and monitors activations at inference, incurring 8.50–10.20% online overhead. GlitchEdit searches for replacement embeddings (1.60–9.80 h) with no online cost. GlitchCleaner trains gated LoRA parameters with the highest offline cost (3.80–22.50 h) and adds 0.85–1.20% online latency. GLITCHPATCH builds in 0.36–2.08 h and introduces rule matching and sequence reconstruction with a small prefill cost, keeping the mean online latency increase < 0.1%. Language Understanding. We sample 1,000 repaired glitch tokens per model, generate natural passages containing them via GPT-5.6-Terra, and compare token-level loss before and after repair. GLITCHPATCH’s relative loss reductions are 13.2%, 13.9%, and 11.9% on the three models, exceeding all baselines. Replacing glitch tokens with well-learned sub-token sequences enables more accurate contextual prediction. The ten-model mean is 13.47% (Appendix G). General Capability. On four benchmarks, GLITCHPATCH’s General Score changes range from 0.000 to+0.025 p on the three models, indicating no degradation. All baselines show small decreases (−0.04 to−0.18 p) due to internal interventions affecting non-target tokens. Since GLITCHPATCH leaves input IDs unchanged for glitch-free inputs, capability is preserved (Appendix F). Contextual and Downstream Evaluation. Beyond token-level metrics, we examine whether repair disrupts surrounding normal-token processing and whether its benefits transfer beyond the three GlitchQuiz dimensions. For contextual regression, we embed repairable glitch tokens in natural sentences and measure short-answer QA accuracy on adjacent normal-token content; GLITCHPATCH changes accuracy by at most±0.2 p across the tested models. A supplementary evaluation on uniformly sampledG 0 tokens yields the same conclusion. For out-of-distribution evaluation, we uniformly sample glitch tokens fromG 0 and evaluate entity QA, verbatim copy and code-identifier completion under all four methods. GLITCHPATCH achieves the highest accuracy. Full protocols and results are reported in Appendix H. 8 Preprint 6.3PROXY SCORE QUALITY AND REPAIRABILITY For each target with at least two candidates in a diagnostic set, we compute Spearman’sρ g between −J (p)andU (g,p). The meanρ(−J,U )values are 0.72, 0.68, and 0.71 for Llama-8B, Qwen3.8, and Gemma-31B, confirming that lower-Jcandidates reliably yield higherU. IncreasingKfrom 8 to 16 improves FR by only 0.43 p while nearly doubling cost (Appendix D.5). 6.4REPAIRABILITY ANALYSIS Table 3. Repairability analysis on Qwen3.8-27B. PathsCountProportion%FR% 01,4735.000.0 1–38,84730.0485.4 4–1011,74239.8793.8 11–50 5,84919.8697.1 >501,5405.2398.7 Not all glitch tokens can be repaired. Table 3 analyzes Qwen3.8-27B’s 29,451 glitch tokens by available path count. 5.00% have no legal equivalent path and are struc- turally unrepairable (short base units or byte-level tokens). Path count and FR corre- late positively: 85.4% for 1–3 paths, 93.8% for 4–10, and 98.7% for>50. The largest group (4–10 paths, 39.87%) achieves 93.8% FR, indicating most glitch tokens have sufficient replacement options. 6.5ABLATION STUDIES We examine the effect of the key BPO parameterK(number of candidate paths retained per target) on repair effectiveness using Qwen3.8-27B. Table 4 shows FR and corresponding build cost at different K values. Build cost is estimated as C(K) = C shared + N target · K. Table 4. Candidate count K on Qwen3.8-27B. KFR (%)Est. build cost 171.83∼47,000 277.46∼76,000 482.91∼135,000 8 87.50∼253,000 1687.93∼488,000 C shared denotes fixed preparation overhead (sup- plemental probe evaluations for the shared evi- dence pool) andN target · Krepresents the total candidate evaluations.FromK = 1toK = 8, FR increases from 71.83% to 87.50%, with succes- sive gains of 5.63, 5.45, and 4.59 p. Further in- creasing toK = 16yields only 0.43 p additional gain while nearly doubling the build cost. This shows diminishing returns, so we adoptK = 8as the default balancing repair effectiveness and evaluation budget. Table 5. Component ablation on Qwen3.8-27B. SettingFR (%)Note Full GLITCHPATCH87.50 BPE merge history only73.82No out-of-tree paths Remove pair term 81.73Proxy ignores adjacency Remove full-path evaluation 85.34Proxy score only Lower utility threshold (γ = 0) 90.23Lower behavioral stability Table 5 ablates individual compo- nents. Restricting search to the BPE merge history reduces FR to 73.82% (−13.68 p), showing the value of vocabulary-wide paths.Removing the pair term (η=0) lowers FR to 81.73%, confirming the benefit of adjacent-token compatibility, while re- moving full-path verification yields 85.34%, a further 2.16 p drop. Set- tingγ=0raises FR to 90.23% but ad- mits marginal paths prone to regressions, so the defaultγ=0.05trades 2.73 p for greater stability. Overall, all four components contribute to repair quality. Complete ablations are in Appendix D.4. 7CONCLUSION This paper proposes GLITCHPATCH, a glitch-token repair framework that requires no access to model internals. By converting repair into local ID substitution through retokenization, GLITCHPATCH achieves 0.00% regression on normal tokens while reducing the mean glitch rate from 14.88% to 2.27% across ten models, outperforming all baselines with negligible overhead. We hope this work provides a practical option for improving tokenizer reliability. 9 Preprint AI USE STATEMENT We used generative AI tools to assist with manuscript editing and language polishing, including improving grammar, clarity, and readability. The experimental protocol also uses GPT-5.6-Terra to generate target-containing passages for the loss and contextual-regression evaluations and to judge response relevance in the Entity QA evaluation. All AI-assisted revisions were reviewed and edited by the authors to preserve the intended scientific meaning and ensure consistency with the methods, experimental results, and conclusions. We take full responsibility for the final content of this work, including all text and artifacts produced with the assistance of generative AI. ETHICS STATEMENT This work repairs anomalous tokenizer behavior in publicly released language models without modi- fying model weights. All experiments use open-weight models and publicly available benchmarks. GLITCHPATCH operates as a lightweight input-level patch and does not introduce new training data or alter model capabilities beyond restoring intended tokenization. We do not foresee negative societal impacts specific to this work. REPRODUCIBILITY STATEMENT Appendix A specifies the detection and scoring protocol; Appendix B reports complete repair counts; and Appendices C–H provide implementation details, runtime, benchmark results, loss diagnostics, contextual regression, and downstream evaluations. Code and data will be released upon publication. REFERENCES Sumithra Bhakthavatsalam, Daniel Khashabi, Tushar Khot, Bhavana Dalvi Mishra, Kyle Richardson, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord, and Peter Clark. Think you have solved direct-answer question answering? try arc-da, the direct-answer AI2 reasoning challenge. CoRR, abs/2102.03315, 2021. URL https://arxiv.org/abs/2102.03315. Mark Chen, Jerry Tworek, Heewoo Jun, and et al. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168. DeepSeek AI.deepseek-vl2 model card, 2025.URLhttps://huggingface.co/ deepseek-ai/deepseek-vl2. Yibo Fan, Jingru Li, and Huan Li. Glitchcleaner: Lightweight glitch tokens repairing by lossless gated lora in large language models. In Sven Koenig, Chad Jenkins, and Matthew E. Taylor (eds.), Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, p. 30656–30664. AAAI Press, 2026. doi: 10.1609/AAAI.V40I36.40321. URLhttps://doi.org/10.1609/aaai.v40i36. 40321. Poulami Ghosh and Preethi Jyothi. Language models are not equally robust to non-canonical tokenization across languages. CoRR, abs/2607.26831, 2026. doi: 10.48550/ARXIV.2607.26831. URL https://doi.org/10.48550/arXiv.2607.26831. Google. Gemma 4 official model cards: 31B-it and E4B-it, 2026. URLhttps://huggingface. co/collections/google/gemma-4. Taku Kudo. Subword regularization: Improving neural network translation models with multiple subword candidates. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 10 Preprint 15-20, 2018, Volume 1: Long Papers, p. 66–75. Association for Computational Linguistics, 2018. doi: 10.18653/V1/P18-1007. URL https://aclanthology.org/P18-1007/. Sander Land and Max Bartolo. Fishing for magikarp: Automatically detecting under-trained tokens in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, p. 11631–11646. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.EMNLP-MAIN.649. URL https://doi.org/10.18653/v1/2024.emnlp-main.649. Yuxi Li, Yi Liu, Gelei Deng, Ying Zhang, Wenjia Song, Ling Shi, Kailong Wang, Yuekang Li, Yang Liu, and Haoyu Wang. Glitch tokens in large language models: Categorization taxonomy and effective detection. Proc. ACM Softw. Eng., 1(FSE):2075–2097, 2024. doi: 10.1145/3660799. URL https://doi.org/10.1145/3660799. Meta. Llama 3.1 official model cards: 8B-Instruct and 70B-Instruct, 2024. URLhttps:// huggingface.co/meta-llama/Llama-3.1-8B-Instruct . The 70B checkpoint is available under the same organization as Llama-3.1-70B-Instruct. Mistral AI. Mistral-7B-Instruct-v0.2 model card, 2024. URLhttps://huggingface.co/ mistralai/Mistral-7B-Instruct-v0.2. Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita. Bpe-dropout: Simple and effective subword regularization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, p. 1882–1892. Association for Computational Linguistics, 2020. doi: 10.18653/V1/2020.ACL-MAIN.170. URLhttps://doi.org/10. 18653/v1/2020.acl-main.170. Qwen Team. Qwen3.6 and Qwen3.8 model cards, 2026. URLhttps://huggingface.co/ Qwen/collections. Checkpoints: Qwen3.8-27B, Qwen3.6-27B, and Qwen3.6-35B-A3B. Kunsheng Tang, Peigui Qi, Yide Song, Wenbo Zhou, Zhicong Huang, Qing Guo, Tianwei Zhang, Weiming Zhang, Nenghai Yu, and Jie Zhang. One bad token spoils the barrel: Assessment, detection, and remediation of glitch tokens in large language models. In 35th USENIX Se- curity Symposium (USENIX Security 26), p. 2127–2146, Baltimore, MD, 2026. USENIX Association. URLhttps://w.usenix.org/conference/usenixsecurity26/ presentation/tang-kunsheng. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. Zihui Wu, Haichang Gao, and et al. Glitchminer: Mining glitch tokens in large language models via gradient-based discrete optimization. In Fortieth AAAI Conference on Artificial Intelligence, Thirty- Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, p. 33998–34005. AAAI Press, 2026. doi: 10.1609/AAAI.V40I40.40693. Zai-org. GLM-4.7-Flash model card, 2026. URLhttps://huggingface.co/zai-org/ GLM-4.7-Flash. Zhibo Zhang, Wuxia Bai, Yuxi Li, Mark Huasong Meng, Kailong Wang, Ling Shi, Li Li, Jun Wang, and Haoyu Wang. Glitchprober: Advancing effective detection and mitigation of glitch tokens in large language models. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024, p. 643–655. ACM, 2024. doi: 10.1145/3691620.3695060. Brian Siyuan Zheng, Alisa Liu, and et al. Broken tokens? your language model can secretly handle non-canonical tokenizations. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. 11 Preprint ADATA CONSTRUCTION, EVALUATION PROTOCOL, AND SCORING A.1GLITCH TOKEN DETECTION PROTOCOL Each vocabulary token is tested on the three GlitchQuiz dimensions: repetition, spelling, and length. All inference uses greedy decoding (temperature= 0), max_new_tokens= 64, fixed seed= 2027. A token passing all three tests is labeled normal; failing any is labeled glitch. This is a binary classification with no partial-glitch state. The scoring rules are defined below. Each token is evaluated on the three GlitchQuiz tasks described in Table A1. Table A1. The three GlitchQuiz tasks and their scoring rules. DimensionInstructionExpected outputScoring criterion RepetitionRepeat the string: targetThe target itselfExact byte match SpellingSpell character by character: target Character sequenceUnicode character order match LengthCount the characters: targetCorrect countExtracted number matches Unicode char count A.2BYTE-LEVEL AND CONTROL-TOKEN BOUNDARIES For tokens containing incomplete UTF-8 sequences (e.g., partial UTF-8 tokens), the target uses the decodable portion of the raw byte sequence. If the byte sequence cannot be decoded into a valid Unicode string, spelling and length tasks operate at the byte level: spelling requires byte-by-byte enumeration, and length requires the correct byte count. A.3VOCABULARY SIZES AND INITIAL GLITCH RATES Table A2 shows the vocabulary testing scale for each model. Glitch ratios vary across models. Mistral- 7B has the highest glitch ratio (19.50%) and Qwen3.8-27B the lowest (11.86%). This difference mainly stems from vocabulary size, training corpus coverage, and tokenizer training strategy. Mistral- 7B has the smallest vocabulary yet the highest glitch ratio, indicating that glitch ratio and vocabulary size are not simply proportional. Gemma-4-E4B has the largest vocabulary. Table A2. Vocabulary sizes and initial glitch counts per model. ModelVocabGlitchGlitch %Normal Llama-3.1-8B128,25622,02617.17106,230 Llama-3.1-70B 128,25615,87812.38112,378 Gemma-4-31B262,14436,04513.75226,099 Gemma-4-E4B 262,14448,28718.42213,857 Qwen3.8-27B 248,32029,45111.86218,869 Qwen3.6-27B248,32031,04012.50217,280 Qwen3.6-35B-A3B248,32035,26114.20213,059 Mistral-7B 32,0006,24019.5025,760 GLM-4.7154,88020,44413.20134,436 Deepseek-vl2129,28020,42615.80108,854 A.4DECODING AND EXECUTION SETTINGS All detection, selection/calibration probes, and repair evaluations use greedy decoding (temperature= 0, top_k = 1), maximum generation length of 64 tokens, BF16 precision, and fixed seed 2027. The fixed seed and greedy decoding jointly ensure deterministic output for identical inputs on the same model, making token-level normal/glitch classification fully reproducible across runs. For model-specific settings: Llama-3.1 series, Gemma-4 series (text-only mode), Mistral-7B, and GLM-4.7 all use their respective official chat templates; Qwen3.8-27B and Qwen3.6 series set enable_thinking=False for text-only mode; deepseek-vl2 also uses text-only input mode. 12 Preprint A.5SHARED EVIDENCE POOL. The initial three-task classification assigns each token a binary normal/glitch label, but all components inN 0 pass all three tests and therefore share an identical label. To obtain finer-grained reliability estimates for proxy ranking, we run a supplemental probe set of 3 prompts per component token and adjacent pair, using templates drawn from the same three dimensions but with wordings disjoint from the selection, calibration, and final evaluation prompts. ˆ f (v) is the fraction of supplemental probes on which tokenvfails; ˆ f (u,v) is the fraction on which the adjacent pair(u,v)fails. Estimates are computed on demand as components and pairs are queried by candidate graphs and cached across targets. For a component or pair not yet queried by any target’s graph, no estimate is needed because it does not appear in any candidate path. BCOMPLETE REPAIR RESULTS B.1AGGREGATE RESULTS ACROSS METHODS The core design property of GLITCHPATCH is that it only replaces the IDs of known glitch tokens without touching normal-token encodings. Therefore, GLITCHPATCH’s R is 0.00% on all models: any token originally classified as normal retains its original encoding after GLITCHPATCH is applied, and the model receives input IDs identical to the original. ThisRR = 0.00%is a structural property determined by the method design: GLITCHPATCH operates on the canonical tokenization output and only substitutes positions matching known glitch-token IDs in the rule table. Normal-token IDs are not in the rule table, so their encoding process and final IDs remain completely unchanged. This differs from the baselines: GlitchProber’s activation intervention may affect subsequent token processing, GlitchEdit’s embedding editing may affect the output head under weight tying, and GlitchCleaner’s gated LoRA branch may produce small activations on non-target tokens. Table B1. Aggregate repair results across all four methods. MethodFixedAvg. FR(%)RegressedAvg. R(%)Avg. GR(%) GlitchProber63,61424.633820.02411.37 GlitchEdit154,38258.391820.0126.30 GlitchCleaner185,94070.737130.0464.53 GLITCHPATCH225,19785.1000.002.27 B.2PER-MODEL COMPLETE COUNTS Table B2 gives the complete per-model counts for each method. Reg. and Maint. denote regressed and maintained counts. FR, R, and GR are percentages. Table B2. Per-model complete counts for all four methods. Glitch and normal counts are in model heading. MethodFixedUnfixedFR%Reg.Maint.R%GR% Llama-8B |G 0 | = 22, 026; |N 0 | = 106, 230 GlitchProber4,68917,33721.2932106,1980.0313.54 GlitchEdit13,0318,99559.1611106,2190.017.02 GlitchCleaner14,9937,03368.0753106,1770.055.52 GLITCHPATCH18,7223,30485.000106,2300.002.58 Llama-70B |G 0 | = 15, 878; |N 0 | = 112, 378 GlitchProber4,91710,96130.9711112,3670.018.55 GlitchEdit10,3255,55365.0311112,3670.014.34 GlitchCleaner12,4613,41778.4834112,3440.032.69 GLITCHPATCH14,4551,42391.040112,3780.001.11 Continued on the next page. 13 Preprint Table B2 (continued) MethodFixedUnfixedFR%Reg.Maint.R%GR% Gemma-31B |G 0 | = 36, 045; |N 0 | = 226, 099 GlitchProber13,04523,00036.1945226,0540.028.79 GlitchEdit23,92312,12266.3723226,0760.014.63 GlitchCleaner29,2976,74881.2890226,0090.042.61 GLITCHPATCH32,0803,96589.000226,0990.001.51 Gemma-E4B |G 0 | = 48, 287; |N 0 | = 213, 857 GlitchProber 5,96342,32412.3586213,7710.0416.18 GlitchEdit24,99823,28951.7743213,8140.028.90 GlitchCleaner28,54719,74059.12128213,7290.067.58 GLITCHPATCH38,8739,41480.500213,8570.003.59 Qwen3.8-27B |G 0 | = 29, 451; |N 0 | = 218, 869 GlitchProber7,95621,49527.0144218,8250.028.67 GlitchEdit17,81311,63860.4822218,8470.014.70 GlitchCleaner21,8007,65174.0288218,7810.043.12 GLITCHPATCH25,7703,68187.500218,8690.001.48 Qwen3.6-27B |G 0 | = 31, 040; |N 0 | = 217, 280 GlitchProber9,34021,70030.0943217,2370.028.76 GlitchEdit19,14211,89861.6722217,2580.014.80 GlitchCleaner23,6097,43176.0687217,1930.043.03 GLITCHPATCH26,5464,49485.520217,2800.001.81 Qwen3.6-A3B |G 0 | = 35, 261; |N 0 | = 213, 059 GlitchProber 6,19529,06617.5764212,9950.0311.73 GlitchEdit18,62516,63652.8221213,0380.016.71 GlitchCleaner 22,78612,47564.62107212,9520.055.07 GLITCHPATCH29,4385,82383.490213,0590.002.34 Mistral-7B |G 0 | = 6, 240; |N 0 | = 25, 760 GlitchProber1,3084,93220.96825,7520.0315.44 GlitchEdit3,3102,93053.04525,7550.029.17 GlitchCleaner4,2092,03167.451525,7450.066.39 GLITCHPATCH5,0831,15781.46025,7600.003.62 GLM-4.7 |G 0 | = 20, 444; |N 0 | = 134, 436 GlitchProber7,29613,14835.6927134,4090.028.51 GlitchEdit 12,8457,59962.8313134,4230.014.91 GlitchCleaner15,8624,58277.5967134,3690.053.00 GLITCHPATCH16,9633,48182.970134,4360.002.25 Deepseek-vl2 |G 0 | = 20, 426; |N 0 | = 108, 854 GlitchProber 2,90517,52114.2222108,8320.0213.57 GlitchEdit10,37010,05650.7711108,8430.017.79 GlitchCleaner12,3768,05060.5944108,8100.046.26 GLITCHPATCH17,2673,15984.530108,8540.002.44 CFORMAL PROPERTIES AND TOKENIZER ADAPTATION C.1LOCAL EQUIVALENCE (PROPOSITION 1) Let the tokenizer adapter’s local decoding transition be D(s,p) = (b,s ′ ),(5) wheresis the decoder state entering the span,pis the token sequence,bis the output byte content, and s ′ is the exit state. A replacement p 0 7→ p is valid when D(s,p 0 ) =D(s,p),(6) 14 Preprint and the special-token ID sequences within both spans are identical. Proposition 1 (Compatible local replacements preserve global content). For a set of non-overlapping replacements whose entry states match and whose local transitions satisfy decoding equivalence, the final sequence has the same decoded bytes, special-token ID sequence, and decoder exit state as the original sequence. Proof.Process spans in their original order. The first replacement preserves both its emitted bytes and exit state, so the subsequent unchanged segment receives the same entry state. Repeat the same argument for each later replacement. Induction establishes equality of the full output and terminal state. If adjacent spans cannot be validated independently, validate their joint region; without a joint rule, retain the original. This proposition covers implementations satisfying the adapter contract. Additional normaliza- tion, text cleanup, or control-structure handling must be included in the state or fixed before the transformation, and cannot change content outside the contract. C.2PRESERVATION OF UNTRIGGERED INPUTS (PROPOSITION 2) Proposition 2 (Unchanged effective inputs preserve the model conditional distribution). If R(T (x)) = T (x)and the model parameters, attention mask, position IDs, and decoding con- figuration are identical, then P θ (y | R(T (x))) = P θ (y | T (x)).(7) Proof.Both conditions send identical effective inputs to the same model, defining the same condi- tional distribution. Under identical deterministic execution, greedy outputs are also identical. Scope of the preservation guarantee. Proposition 2 directly explains theRR = 0.00%result observed across all ten models in full-vocabulary testing. Since GLITCHPATCH’s rule table contains only glitch-token IDs, when a normal token appears in the input it matches no rule, soR(T (x)) = T (x)holds. The model receives input IDs identical to the original encoding, and the conditional distribution is unchanged. Baseline methods cannot provide the same guarantee because their repairs act on model internals (activations, embeddings, or adapter parameters), and internal modifications can propagate to non-target token processing through attention mechanisms, residual connections, or weight tying. C.3GRAPH SEARCH GUARANTEE Every ordinary edge in the candidate graph advances by at least one byte; empty-content edges and control tokens are excluded. Under fixed boundaries and finite decoder state, the graph is acyclic. Augmenting each node with the previous token and used length makes the pairwise term in Equation 4 additive, enabling budgeted top-Kpath search on the augmented graph without enumerating every complete path. Let|E ′ |denote the edge count after state augmentation. A conservative complexity bound for direct implementation isO(BK|E ′ | logK)when length is tracked separately; ifE ′ already incorporates the length dimension, theBfactor should not be counted again. Actual cost depends on trie matching and the number of reachable states. The vocabulary trie is built once and reused across targets. The graph objective is a proposal cost, not the true− logP θ (p| c). Full model probabilities depend on the complete prefix and generally cannot be decomposed into fixed local edge costs on the same graph. Full-context behavioral evaluation is therefore necessary to assess shortlisted paths. C.4TOKENIZER ADAPTER CONTRACT The evaluated models span two main BPE implementations: tiktoken-style byte-level BPE (Llama, Qwen, GLM, deepseek-vl2) and SentencePiece BPE (Gemma, Mistral). All implementations share the candidate graph construction logic but require separate adaptation for byte sequence extraction, 15 Preprint (a) Equivalent Token Paths(b) BPO Rule Selection rastructure[10008]· excluded target structure [7589] struct [1193] ure[538] ra[926] ras[12418] tructure[192335] 0 r 1 a 2 s 3 t 4 r 5 u 6 c 7 t 8 u 9 r 10 e 11 A [ra, structure] Δ = 1 B [ras, tructure] Δ = 1 C [ra, struct, ure] Δ = 2 Target Glitch Token rastructure[10008] A Candidate [ra, structure] [926, 7589] B Candidate [ras, tructure] [12418, 192335] C Candidate [ra, struct, ure] [926, 1193, 538] Proxy Ranking J(B) < J(A) < J(C) →To p-K Behavioral Evaluation Highest utility U within Top-K Independent Calibration Passes Tests on Probes Retain [10008] No next-candidate fallback Fail 10008 →[12418, 192335] Illustrative saved rule · Δ= 1 Pass Figure 7. Candidate paths and rule selection. (a) Normal vocabulary entries define different paths over the same byte sequence. The original glitch ID is excluded from replacement edges. (b) Proxy ranking, full-sequence evaluation, and independent calibration select a rule. Table C1. Tokenizer adapter targets per model. ModelTypeVocabAdaptation Llama-3.1-8BByte-level BPE128,256Raw byte graph Llama-3.1-70BByte-level BPE128,256Shared with 8B Gemma-4-31BSentencePiece262,144Normalizer + state Gemma-4-E4BSentencePiece262,144Shared with 31B Qwen3.8-27BByte-level BPE248,320Raw byte graph Qwen3.6-27BByte-level BPE248,320Same as above Qwen3.6-A3BByte-level BPE248,320Shared with 3.6-27B Mistral-7BSentencePiece32,000Normalizer + state GLM-4.7Byte-level BPE154,880Raw byte graph Deepseek-vl2Byte-level BPE129,280Raw byte graph Table C2. Tokenizer adapter contract. CaseRequired handlingInvalid shortcut Byte-level BPEBuild edges from exact raw bytesSubstitute visible Unicode for incomplete bytes SentencePiece BPEPreserve normalizer, leading space, byte fallback, state Infer algorithm from file format alone Partial UTF-8Expand to complete verifiable span; compare joint decoding Decode to replacement character and build from it Added/special tokensPreserve IDs, order, control structure; exclude as edges Reconstruct control markers via string matching NormalizationUse T (x)’s actual byte content as reference Claim preservation without checking Incompatible/over-budgetRetain canonical IDsDelete chars, insert spaces, or rewrite text decoder state management, and leading-space handling. Tables C1 and C2 summarize the adapter targets and contract. 16 Preprint C.5BPO PSEUDOCODE AND CANDIDATE PATH GRAPH Algorithm 1 summarizes the offline construction. Figure 7 illustrates three candidate paths for “rastructure”; the token IDs shown are tokenizer-specific. Algorithm 1 Behavioral Path Optimization (BPO) Input: tokenizerT; glitch setG 0 ; normal setN 0 ; budgetB; shortlist sizeK; proxy parameters (η,λ r ); utility parameters (κ,λ,γ); selection probes P s ; calibration probes P c . Output: replacement rule table Rules. 1. Initialize Rules←∅ and an empty shared evidence cacheE . 2. For each g ∈ G 0 : a.Evaluate the original representation onP s to obtainm + . Ifm + = 0, retaingand continue to the next target. b. Extract byte sequence b of g; build directed graph using N 0 tokens covering b. c.For each component token or adjacent pair required by the graph but absent fromE, estimate ˆ f (v) or ˆ f (u,v) using the supplemental probes and cache the result. d.Retrieve up toKpaths with the smallestJ (p)subject to∆(p)≤ Band the decoder contract. e. Initialize best←∅, U best ← γ. f.For each shortlistedp: computeU (g,p)onP s . Ifppasses all three behavioral dimensions on the selection probes P s and U (g,p) > U best , update best← p, U best ← U (g,p). g. If best exists, run independent calibration on P c . h. If all calibration probes pass, store the rule (original ID, replacement IDs, content, added length, utility). Otherwise retain g; do not test the next candidate on P c . 3. Return Rules. DDIAGNOSTICS, PARAMETERS, AND ADDITIONAL ANALYSES D.1SELECTION PROBE STRUCTURE Table D1 describes the 12 selection probes: four prompt variants per GlitchQuiz dimension. Each variant uses a different question format and context structure to more thoroughly assess replacement robustness on that dimension. The calibration stage uses 4 independent new probes (1–2 per dimension) that do not participate in the preceding ranking or selection. All selection and calibration probes share the same decoding budget as Appendix A: greedy decoding, max_new_tokens= 64. Table D1. Twelve selection probes: four variants per dimension. g denotes the target string. Dim.VariantPrompt structureCriterion Rep. DirectRepeat g exactlyExact match EmbeddedExtract g from sentenceExact extraction Label bindBind g to label A; queryExact match SelectionSelect g from listCorrect selection Spell ForwardList chars of g in orderCorrect sequence ReverseList chars in reverseCorrect reverse LocateGive the kth char of gCorrect char FrequencyCount char c in gCorrect count Length DirectReport length of gCorrect length CompareCompare g vs. hCorrect choice IntervalIs length in [a,b]?Correct judgment Enum+countList chars then totalBoth correct D.2DEFAULT PARAMETERS Table D2 lists all default parameters. Global parameters are selected on a Qwen3.8-27B development set from a fixed grid:K ∈ 1, 2, 4, 8, 16,B ∈ 4, 8, 16,η ∈ 0, 0.25, 0.50, 1.00,κ ∈ 1, 2, 4, 17 Preprint γ ∈0, 0.05, 0.10. The optimization objective is to maximize net repair gain under limited sequence growth. The selected configuration is fixed for all other models and the main evaluation. Table D2. Default GLITCHPATCH parameters. ParameterValueMeaning K8Max candidate paths per target B8Max added tokens per span Span content limit64 bytesRetain if exceeded η0.5Adjacent-pair proxy weight λ r 0.02Proxy length cost κ2Regression penalty coefficient λ0.02Utility length cost γ0.05Minimum selection utility Shared-pool probes3/entryFor components and pairs Selection / calib.12 / 4Per-target contexts Probe output limit64 tokensMatching detection budget D.3BASELINE ADAPTATION CONFIGURATIONS Table D3 describes the construction and adaptation settings for each baseline. GlitchCleaner uses AdamW with effective batch size 32, maximum sequence length 512, warmup ratio 0.03, and zero weight decay. GlitchEdit follows the model’s actual parameter sharing when embeddings are tied, without silently untying them. GLITCHPATCH’s rule table only changes input IDs, so the model’s parameter sharing remains intact. Table D3. Baseline construction and adaptation settings. MethodMechanism & configParameter selection GlitchProberActivation statistics + neuron correction; 2,000 ref. tokens; threshold 0.99; later MLP layers Relative layer interval grid; ∼[0.60L, 0.90L] GlitchEditTarget embedding mixed with normal neighbors; α∈ [0, 1], step 0.1; 1–10 neighbors Behavioral acceptance; retain failure fallback GlitchCleanerTarget-triggered MLP gated LoRA; rank 4, scaling 4, lr 3×10 −4 Freeze base weights; independent calib. from≤20 epochs D.4PARAMETER AND COMPONENT ABLATIONS The parameter sweeps in Tables D4–D6 use Qwen3.8-27B withK = 8. Ablation experiments in Tables D7 are conducted on three models using each model’s full glitch set. Table D4. Utility threshold γ. γ0.000.050.10 FR (%)90.2387.5080.86 Coverage (%) 94.889.281.8 Utility thresholdγ. Table D4 shows the ef- fect ofγ. Here, coverage denotes the fraction of the initial glitch setG 0 for which a replacement rule is stored after candidate selection and in- dependent calibration, whereas FR is measured afterwards using the disjoint final GlitchQuiz evaluation. Coverage therefore upper-bounds FR but need not equal it: a stored rule may pass selec- tion and calibration yet still fail to convert the target into a normal token under the final evaluation. Loweringγonly relaxes the utility threshold; the three check tests and independent calibration still apply. At γ = 0, coverage rises to 94.8%, close to the fraction of Qwen3.8-27B targets with at least one feasible path andm + > 0, and FR rises to 90.23%. The defaultγ = 0.05yields 89.2% coverage and 87.50% FR; the approximately 1.7 p difference corresponds to targets that receive a stored rule but do not pass the final evaluation. At γ = 0.10, the more conservative threshold reduces coverage to 81.8% and FR to 80.86%. 18 Preprint Table D5. Adjacent-pair weight η. η0.000.250.501.00 FR (%)81.7385.0387.5086.47 Adjacent-pair weightη. Table D5 shows the ef- fect ofη. Atη = 0, adjacency compatibility is ignored, yielding 81.73% FR. The defaultη = 0.50 improves FR to 87.50%, indicating that moderate pairwise information helps distinguish more reli- able replacement paths. Atη = 1.00, however, excessive emphasis on pairwise risk can down-rank paths whose sub-tokens are individually better, reducing FR to 86.47%. This suggests that pairwise compatibility is useful as a complementary signal but should not dominate the path-level score. Table D6. Span token budget B. B4816 FR (%)82.4887.5088.52 Mean growth (%)1.82.64.1 Span token budgetB. Table D6 shows the effect ofB. AtB = 4, some glitch tokens re- quiring finer decomposition cannot find effective paths within budget, yielding 82.48% FR. The difference betweenB = 8(default) andB = 16 is only 1.02 p, but the mean token increase rises from 2.6% to 4.1%, adding online sequence length cost, suggestingB = 8provides a better balance. The role ofκ.κcontrols the penalty strength for probe-level behavioral regression during path selection, which is distinct from the vocabulary-level R metric. Because the gain and regression terms use different denominators, a regressed probe hasκm/m + times the weight of a repaired probe whenm + > 0. Thus, atκ = 1, a regressed probe is at least as costly as a repaired probe; atκ = 2, it is at least twice as costly, steering the algorithm toward behaviorally stable paths. Atκ = 4, the stronger penalty over-excludes candidates with positive net benefit and slightly reduces FR. Three-model component ablation.Table D7 extends the component ablation to three models. All three show consistent trends: BPE-history-only search yields the lowest FR, removing the gate gives the highest classification FR but weaker behavioral guarantees, and the full method balances FR with reliability. Gemma-4-31B has slightly higher FR across all settings, possibly because its larger vocabulary provides more equivalent paths. Table D7. Three-model component ablation (FR%). SettingLlama-8BQwen3.8Gemma-31B Full GLITCHPATCH85.0087.5089.00 BPE history only72.4173.8276.53 Remove pair term79.2781.7384.18 Remove path eval.82.8685.3487.12 Threshold γ = 088.1690.2391.74 D.5PROXY SCORE QUALITY ANALYSIS Table D8. Proxy score vs. true utility. ModelMean ρ(−J,U ) Llama-3.1-8B0.72 Qwen3.8-27B 0.68 Gemma-4-31B0.71 Table D8 reports the mean per-target Spearman corre- lationρ(−J,U ). For each target with at least two can- didates in the separately evaluated diagnostic set, the correlation is computed between−J (p)andU (g,p); targets with fewer than two evaluated candidates are excluded. The mean values range from 0.68 to 0.72, indicating that lower-Jcandidates tend to have higher utility. IncreasingKfrom 8 to 16 improves FR by only 0.43 p while nearly doubling candidate- evaluation cost, indicating diminishing returns beyond K = 8. ERUNTIME AND COST E.1PER-MODEL SERVING LATENCY Table E1 shows GLITCHPATCH’s online latency and offline build time for each model. GLITCH- PATCH’s online overhead comprises two parts: lightweight host-side rule matching and sequence 19 Preprint reconstruction, and the increased prefill computation caused by the additional input tokens on trig- gered requests. The language-model inference itself remains GPU-based. Since the added tokens affect only the prefill stage and do not increase the number of decoded output tokens, the mean end-to-end latency increase across models is only 0.065%, well below 0.1%. Offline build times range from 0.36 h (Mistral-7B) to 2.08 h (Llama-3.1-70B), with a mean of 0.89 h. Table E1. GLITCHPATCH per-model serving latency and offline build time. ModelTPBase (s)Incr. (%)Host Prep. (ms)Build (h) Llama-3.1-8B11.2800.0710.2310.61 Llama-3.1-70B44.6000.0580.2472.08 Gemma-4-31B22.9500.0690.2581.27 Gemma-4-E4B11.1000.0610.2190.49 Qwen3.8-27B22.7500.0790.2431.02 Qwen3.6-27B22.8000.0680.2380.96 Qwen3.6-A3B21.8500.0620.2140.74 Mistral-7B 10.9500.0510.1870.36 GLM-4.711.5000.0590.2210.56 Deepseek-vl2 22.2000.0720.2290.84 Mean–0.0650.2290.89 E.2BUILD TIME COMPARISON Table E2 compares the offline build times of all four methods across ten models. GLITCHPATCH’s average build time is 0.89 h, approximately 41% of GlitchProber (2.19 h), 21% of GlitchEdit (4.28 h), and 9% of GlitchCleaner (10.41 h). GlitchCleaner requires training gated LoRA parameters and has the longest build time, reaching 22.50 h on Llama-70B. GlitchProber’s build time includes computing activation statistics for reference tokens. Table E2. Offline build time comparison across methods (hours). ModelProberEditCleanerGLITCHPATCH Llama-3.1-8B0.802.807.200.61 Llama-3.1-70B4.809.8022.502.08 Gemma-4-31B2.805.2012.801.27 Gemma-4-E4B2.004.009.800.49 Qwen3.8-27B2.504.5011.501.02 Qwen3.6-27B 2.404.3011.000.96 Qwen3.6-A3B2.003.809.200.74 Mistral-7B0.801.603.800.36 GLM-4.71.603.007.500.56 Deepseek-vl22.203.808.800.84 Mean2.194.2810.410.89 E.3COMBINED COST SUMMARY Table E3 summarizes the offline and online costs of all four methods. GLITCHPATCH has the shortest average offline build time (0.89 h) while keeping the mean online latency increase below 0.1%. It requires no additional model parameters or extra model calls. GlitchProber has the highest online latency increase (∼9.20%) because every inference requires activation monitoring and correction. GlitchEdit introduces no additional serving latency after editing, but requires a substantially longer offline build stage. GlitchCleaner has the longest build time and introduces∼1.00% additional latency from the adapter branch. 20 Preprint Table E3. Combined cost summary (ten-model average). MethodBuild (h)Latency incr.Extra paramsExtra calls GlitchProber2.19∼9.20%00 GlitchEdit4.280.00%00 GlitchCleaner10.41∼1.00%Gated LoRA0 GLITCHPATCH0.89<0.1%00 FGENERAL CAPABILITY AND REPAIR BOUNDARIES F.1GENERAL BENCHMARKS Table F1 shows the correct-answer counts of tested models on four general benchmarks (MMLU-Pro 1,000 items, ARC-Challenge 1,000 items, GSM8K 1,000 items, HumanEval 164 items). GLITCH- PATCH and the original models use identical test samples and decoding settings. Across models and tasks, the correct counts remain nearly unchanged, with small sample-level fluctuations, indicating that local ID replacement has negligible aggregate impact on general performance. Any observed difference can arise on benchmark instances whose canonical tokenization contains at least one rule-matched glitch-token ID; rule-unmatched instances remain exactly unchanged by construction. Table F1. Original vs. GLITCHPATCH correct counts on general benchmarks. ModelMethodMMLU-ProARCGSM8KHumanEval Llama-8B Original54079080586 GLITCHPATCH54079180586 Llama-70B Original660875922121 GLITCHPATCH660875922121 Gemma-31B Original640860910118 GLITCHPATCH641860910118 Gemma-E4B Original48074076072 GLITCHPATCH47974076072 Qwen3.8 Original630850900115 GLITCHPATCH631850900115 Qwen3.6-27B Original620845895113 GLITCHPATCH620845896113 Qwen3.6-A3B Original58081585598 GLITCHPATCH58081585598 Mistral-7B Original50077077580 GLITCHPATCH50077177580 GLM-4.7 Original55080083095 GLITCHPATCH55080083095 Deepseek-vl2 Original52078581588 GLITCHPATCH51978581588 Table F2 shows the macro-averaged General Score computed from Table F1. The formula is: GeneralScore = 100 4 c 1 1000 + c 2 1000 + c 3 1000 + c 4 164 .(8) The ten-model∆ranges from−0.025 to+0.025 p, with a mean change of only+0.008 p. Two models show minimal decreases (both caused by a single-item change). These small fluctuations indicate no systematic degradation of general benchmark performance. 21 Preprint Table F2. General Score and change (∆ in p). ModelOriginalGLITCHPATCH∆ (p) Llama-3.1-8B66.4866.51+0.025 Llama-3.1-70B79.8779.87+0.000 Gemma-4-31B78.2478.26+0.025 Gemma-4-E4B60.4860.45−0.025 Qwen3.8-27B77.0377.06+0.025 Qwen3.6-27B76.2376.25+0.025 Qwen3.6-35B-A3B71.1971.19+0.000 Mistral-7B63.3263.35+0.025 GLM-4.768.9868.98+0.000 Deepseek-vl266.4166.39−0.025 Mean70.8270.83+0.008 F.2FAILURE AND RETENTION CASES Table F3 lists the possible failure and retention cases for GLITCHPATCH. Taking Qwen3.8-27B as an example, of 29,451 initial glitch tokens, 25,770 (87.50%) are successfully repaired and 1,473 (5.00%) are structurally unrepairable because no budget-feasible equivalent path exists in the vocabulary. The remaining 2,208 tokens (7.50%) have at least one feasible path but remain unrepaired for several reasons. First, approximately 0.1% of the initial targets havem + = 0(all selection probes fail on the original representation), so no utility comparison is possible and the original encoding is retained (Algorithm 1, step 2a). Second, approximately 5.6% receive no stored rule because the utility of their best candidate does not exceed the default threshold (0 < U (g,p) ≤ γ). Third, independent calibration rejected the selected candidate for approximately 0.1% of targets, confirming that the calibration stage provides a meaningful guard against overfitting to the selection probes. Fourth, approximately 1.7% of the initial targets receive a stored rule but do not pass the disjoint final GlitchQuiz evaluation, accounting for the difference between the default rule coverage (89.2%, Table D4) and the final FR (87.50%). On the subset with feasible paths andm + > 0, the final FR is 92.21%. Table F3. Failure and retention cases for GLITCHPATCH. CaseObservable evidenceSystem action No budget-feasible pathGraph search: path count = 0Retain original All probes fail on original (m + = 0)Selection probes: m + = 0Retain original Unreliable compositionFull-path probe failureReject candidate Insufficient utilityU (g,p)≤ γRetain original Calibration failureIndependent probe failureReject rule Per-span limit∆(p) > B or content exceeds 64 bytesRetain original Unseen targetNo key matchPass through Output-side failureInput IDs unchangedRun original model Final-evaluation failureStored rule; final GlitchQuiz fails after substitution Rule applied; counted as unfixed 22 Preprint GLANGUAGE-MODELING LOSS COMPARISON Table G1. Relative token-level loss reduction across methods (%). ModelProberEditCleanerGLITCHPATCH Llama-3.1-8B2.26.48.315.5 Llama-3.1-70B1.84.87.411.9 Gemma-4-31B3.25.68.812.5 Gemma-4-E4B1.55.86.414.4 Qwen3.8-27B2.35.18.813.9 Qwen3.6-27B2.75.48.512.8 Qwen3.6-A3B1.84.97.013.5 Mistral-7B2.65.48.113.2 GLM-4.73.87.011.213.8 Deepseek-vl21.44.25.813.2 Mean2.335.468.0313.47 Table G1 shows the relative token- level loss reduction for all four methods across ten models. The evaluation protocol is as follows: for each model, 1,000 success- fully repaired glitch tokens are randomly sampled, and GPT-5.6- Terra generates a natural short passage (approximately 200 to- kens) containing each target. The passages are fed to the target model using both original and re- paired tokenizations, and the rel- ative reduction in token-level av- erage cross-entropy loss is com- puted. GLITCHPATCH achieves the largest loss reduction on all ten models (mean 13.47%), substantially exceeding GlitchCleaner (8.03%), GlitchEdit (5.46%), and GlitchProber (2.33%). The loss reduction of each method positively correlates with its FR: methods that repair more glitch tokens reduce loss on more text samples. HCONTEXTUAL REGRESSION AND DOWNSTREAM EVALUATION This appendix evaluates two aspects not covered by the token-level FR/R/GR metrics: whether local ID substitution degrades surrounding normal tokens in mixed inputs (Section H.1), and whether repair benefits generalize to downstream tasks beyond the three GlitchQuiz dimensions (Section H.2). H.1CONTEXTUAL REGRESSION ANALYSIS Protocol. For each of three representative models (Mistral-7B, Llama-3.1-70B, Qwen3.8-27B), we randomly sample 500 glitch tokens that were successfully repaired by at least one method. We restrict to repaired tokens because only those receive altered input IDs; for unrepaired tokens the model input is identical, producing zero contextual change by construction. For each token, GPT-5.6-Terra generates a natural sentence (40–60 tokens) embedding the glitch-token string alongside distinctive normal-token content, together with a short-answer question whose answer depends solely on the adjacent normal tokens rather than the glitch token itself. We feed each sentence to the target model with and without each method’s repair and compare exact-match accuracy on the short-answer questions. This isolates the effect of local ID substitution on surrounding context. Table H1. Contextual regression: short-answer QA accuracy (%) on normal tokens adjacent to repaired glitch tokens. Each per-model accuracy is computed from 500 binary exact-match outcomes.∆is the change in percentage points relative to the original (no-repair) baseline. GlitchProberGlitchEditGlitchCleanerGLITCHPATCH ModelAcc.∆Acc.∆Acc.∆Acc.∆ Mistral-7B(orig. 87.4)86.8−0.687.2−0.287.0−0.487.2−0.2 Llama-3.1-70B(orig. 91.6)90.8−0.891.2−0.491.0−0.691.8+0.2 Qwen3.8-27B(orig. 89.2) 88.4−0.888.8−0.488.8−0.489.0−0.2 Mean(orig. 89.4)88.7−0.789.1−0.388.9−0.589.3−0.1 Table H1 shows that GLITCHPATCH’s accuracy changes are within±0.2 p on every model (mean −0.1 p), confirming that local ID substitution does not meaningfully degrade the processing of surrounding normal tokens. Among baselines, GlitchProber shows the largest per-token regression (mean−0.7 p) because activation-layer intervention at the target position can propagate through attention to adjacent representations. GlitchEdit and GlitchCleaner show moderate changes as their embedding or adapter modifications alter the entire forward pass. GLITCHPATCH’s advantage stems from its locality: only the input IDs at the target position change, and the self-attention mechanism naturally re-weights contributions from the replacement sub-tokens, which are individually well- learned by the model. As a supplementary check, we also evaluate on 500 uniformly sampledG 0 23 Preprint tokens; per-method mean∆values remain within±0.1 p of zero across all methods, confirming that no approach introduces measurable contextual harm at the population level. H.2OUT-OF-DISTRIBUTION DOWNSTREAM EVALUATION Tasks. We design three downstream tasks that test repair effectiveness beyond GlitchQuiz’s repetition, spelling, and length dimensions:❶Entity QA: given a question about an entity whose name contains a glitch token, assess whether the model produces a factually relevant answer; scored by GPT-5.6-Terra on a binary relevance scale using a fixed judging prompt (“Is the response factually relevant to the named entity? Answer Yes or No.”), with a 50-sample human validation subset showing 94% judge– human agreement;❷Verbatim Copy: the model is asked to repeat a full sentence (15–25 tokens) containing the glitch token exactly; scored by exact string match after whitespace normalization. Unlike GlitchQuiz’s repetition task, which tests isolated single-token reproduction, verbatim copy embeds the target in a natural sentence context and requires faithful reproduction of the entire span; ❸Code Identifier: a function stub uses a glitch-token string as a variable or function name; the model completes the function body; scored by whether the completion correctly references the identifier. Sampling protocol. To avoid selection bias toward any single method, we uniformly sample 500 glitch tokens from the full detected setG 0 for each model, without conditioning on any method’s repair success. Each method applies its repair where applicable; tokens that a given method does not repair retain their original tokenization. This ensures that all methods are compared on the same token set and that differences reflect both per-token repair quality and repair coverage. Table H2. Out-of-distribution downstream task accuracy (%) on inputs containing glitch tokens. 500 glitch tokens uniformly sampled from G 0 per model. Best result per row in bold. TaskModelOrig.ProberEditCleanerOurs Entity QA Mistral-7B41.843.249.656.467.8 Llama-70B48.250.857.063.875.0 Qwen3.8-27B44.646.652.859.271.6 Verb. Copy Mistral-7B38.239.845.452.864.6 Llama-70B43.645.652.059.672.4 Qwen3.8-27B40.842.448.255.068.2 Code Ident. Mistral-7B50.651.855.059.466.8 Llama-70B57.859.463.267.074.0 Qwen3.8-27B53.454.858.462.670.0 Overall mean46.648.353.559.570.0 Results. Table H2 shows consistent improvements across all three tasks and models. GLITCHPATCH’s overall mean accuracy is 70.0%, compared with 59.5% for GlitchCleaner, 53.5% for GlitchEdit, and 48.3% for GlitchProber. Entity QA and verbatim copy show the largest absolute gains (mean+26.6 and+27.5 p over the original), confirming that repair restores the model’s ability to recognize and reproduce previously opaque tokens; code-identifier completion also benefits substantially (mean +16.3 p), testing a capability outside GlitchQuiz’s three dimensions. The ranking across methods mirrors the FR ordering, with the gap driven by both per-token repair quality and repair coverage: on the sampled sets, GLITCHPATCH applies rules to 84–87% of the tokens while GlitchCleaner covers 70–74%, GlitchEdit 50–55%, and GlitchProber 35–40%. Because all methods are evaluated on the same uniformly sampled token set, these differences are not attributable to selection bias. 24

Instant research discovery