Paper deep dive
Validating Hybrid-State Cache Recovery for GLM-5.3-Flash with vLLM and LMCache
Frank Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:External cache transfers can succeed while a hybrid language model resumes from an inconsistent state. We examine the full 45-layer GLM-5.3-Flash model, using the RedHatAI/ GLM-5.3-Flash-NVFP4 quantized checkpoint with vLLM and LMCache under four-way tensor parallelism. A complete-hit recovery mismatch restored state for the full prompt while the scheduler credited one fewer token. We aligned recovery through strict-prefix lookup and established a numerical comparison using shared computation corrections, matched checkpoint scheduling, and fixed per-rank kernel configurations. In a nine-length serial workload, agreement with the modified recomputation control improved from 34/36 to 36/36 generations, each containing 64 token IDs. A separate instrumented run passed recorded transfer-page, effective-tail, and delayed-save checks. Three additional synthetic templates passed 72 paired 256-token continuations across two fresh-container runs. A subsequent serial performance study preserved output equality across 120 requests; among the measured trials, CPU reload reduced time to first token by 46-64% and total request time by 1.9-7.0% relative to modified cold recomputation. The contribution is an experimentally validated integration repair applying an existing checkpoint-alignment principle. The evidence is confined to one model revision and controlled configuration; it does not establish general determinism, task-quality equivalence, concurrent-serving gains, or capacity beyond GPU memory.
Tags
Links
- Source: https://arxiv.org/abs/2609.15030v1
- Canonical: https://arxiv.org/abs/2609.15030v1
Trouble viewing inline? Open PDF directly â
Full Text
34,614 characters extracted from source content.
Expand or collapse full text
Validating Hybrid-State Cache Recovery for GLM-5.3-Flash with vLLM and LMCache Frank Li UNSW Sydneyă»research@n1a.net Abstract.External cache transfers can succeed while a hybrid language model resumes from an inconsistent state. We examine the full 45-layer GLM-5.3-Flash model, using theRedHatAI/GLM-5.3-Flash-NVFP4quantized checkpoint with vLLM and LMCache under four-way tensor parallelism. A complete-hit recovery mismatch restored state for the full prompt while the scheduler credited one fewer token. We aligned recovery through strict-prefix lookup and established a numerical comparison using shared computation corrections, matched checkpoint scheduling, and fixed per-rank kernel configurations. In a nine-length serial workload, agreement with the modified recomputation control improved from 34/36 to 36/36 generations, each containing 64 token IDs. A separate instrumented run passed recorded transfer-page, effective-tail, and delayed-save checks. Three additional synthetic templates passed 72 paired 256-token contin- uations across two fresh-container runs. A subsequent serial performance study preserved output equality across 120 requests; among the measured trials, CPU reload reduced time to first token by 46â64% and total request time by 1.9â7.0% relative to modified cold recomputation. The contribution is an experimentally validated integration repair applying an existing checkpoint-alignment principle. The evidence is confined to one model revision and controlled configuration; it does not establish general determinism, task-quality equivalence, concurrent-serving gains, or capacity beyond GPU memory. 1 Introduction At a prompt length of 3,584 tokens, four transfer work- ers reported restoring a cached checkpoint for all 3,584 tokens. The scheduler, however, credited only 3,583 to- kens and scheduled one additional input token. The sub- sequent continuation differed from the no-connector con- trol at generated-token index 11. Transfer had occurred on every tensor-parallel rank; a positive cache-hit counter did not explain whether computation resumed from the correct logical position. This distinction matters when a cache contains state updated in place as well as attention history. Reducing a token count cannot reconstruct an earlier recurrent state. Existing work already describes the need to align reusable attention prefixes and recurrent checkpoints; we apply that principle to a concrete integration failure rather than propose a new checkpointing algorithm [ 1]. The failure was diïŹicult to isolate because the output reference also required validation. Earlier paired runs could diverge despite identical recorded weights, and ap- parently successful byte checks had initially inspected ineffective all-zero tail locations. Neither source inspec- tion nor one successful transfer was suïŹicient evidence. The eventual comparison therefore used a modified re- computation control with shared computation changes, a matched checkpoint schedule, and a fixed per-rank Flash Linear Attention (FLA) kernel profile. It did not compare a connector-only patch against an untouched production executable. This case study addresses three questions: which re- covery condition failed in the integration; how a useful numerical control was established; and what the repaired path demonstrably passed. Its contributions are an ob- served complete-hit state/position mismatch and its re- pair, a documented construction of the numerical com- parison, and a bounded regression combining generated- token agreement with transfer and completion checks. The historical investigation contains 53 changing exper- imental rounds, not 53 independent repetitions of one treatment. The main evaluation uses the final, explicitly identified comparisons. 2 System and comparison design 2.1 Execution and cached state The experiment runs one full GLM-5.3-Flash target model across four GPUs with tensor parallelism. GLM- 5.3-Flash combines sparse and linear attention; the evalu- ated NVFP4 checkpoint is Red Hatâs quantized deriva- tive of the Z.ai model. Our workloads use text inputs only [ 8], [9]. Throughout this paper, âGLMâ abbreviates this evaluated model and checkpoint, not the GLM model family as a whole. vLLM performs model execution and scheduling; LMCache provides the external cache path. The four ranks are parts of one inference engine, not four independent model replicas. The experiment does not implement prefill/decode disaggregation. Figure 1 summarizes the execution and recovery re- sponsibilities. The integration handles attention-related cache entries, checkpointed state, and model-specific tail entries. Log- ical engine groups and physical transfer groups are dis- tinct: the audited registration contains 70 entries per rank across six transfer groups. Some transfer representations are byte views, while the observed tail entries are BF16. Thus neither the model checkpointâs NVFP4 name nor the FP8 KV configuration describes every state tensorâs precision. MTP configuration does not imply that every step drafts a token: checkpoint-protection steps can dis- able drafting. The later content extension records four NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and their UUIDs. This current inventory does not retroac- tively establish the exact devices used in earlier rounds. Performance-specific runtime settings are reported in §5.5; the shared computation controls are described be- low. 1 arXiv:2609.15030v1 [cs.DC] 14 Sep 2026 Shared controls: computation corrections · matched checkpoint schedule Per-rank FLA profile vLLM scheduler Computed prefix p One GLM-5.3-Flash engine TP = 4 R0R1R2R3 Attention cache + checkpoint state + tails LMCache connector External CPU cache Save / restore External path enabled only in candidate Logical components: four ranks belong to one engine; hardware interconnects are not depicted. Figure 1:Evaluated logical components. The four tensor-parallel ranks form one GLM-5.3-Flash engine. Both arms share modified computation controls; only the candidate enables the external CPU-cache path. Arrows describe logical interactions, not hardware topology. Table 1:Evaluated configuration SettingEvaluated configuration TargetFull 45-layerRedHatAI/GLM-5.3-Flash-NVFP4 Model revision36c184c6cda000a481711306df5adde42f63321a RuntimevLLM0.1.dev20051+g487ecf187; LMCache0.5.4; PyTorch2.13.0+cu130 ParallelismTP=4, P=1, DP=1; four GPUs on one host PrecisionConfigured model dtype: BF16;compressed-tensorsquantization; FP8 KV setting ExecutionEager, synchronous scheduling; MTP configured with one speculative token Limits4 GiB KV budget per rank; maximum model length 32,768; maximum sequences 4; token batch limit 8,192 Checkpoint intervalC=1,792 tokens Numerical controlSeven recorded FLA autotuner configurations per rank 2.2 State position and scheduler position Let N denote prompt length, C the checkpoint interval, B the logical prefix represented by the restored check- point, and p the schedulerâs computed-token count. Prefix lengths are counts; token indices are zero based. A state representing prefix [0,B) supports continuation from B. Crediting p=B therefore aligns the restored state with the start of subsequent computation. For the fully cached prompts in the final workload, re- stricting lookup to the prompt without its last token se- lects B=C floor((Nâ1)/C). The scheduler then computes the nonempty suïŹix [B,N). This formula assumes the aligned checkpoint is available, as it was in these reload tests. In a general cache, lookup must select an available checkpoint within the query boundary; the formula is not a guarantee of a hit. 2.3 Treatments and observations The cumulative functional patch changes eleven files across LMCache and vLLM. It includes layout and snap- shot handling, save completion, checkpoint behavior, and stable selection/packing changes. The FLA installer and the no-connector scheduling fallback are separate con- trols. Consequently, successful evaluation is a result for their stated combination; it does not establish that the cu- mulative patch alone preserves production outputs. The final strict-prefix intervention changes two GLM lookup entry points within this existing configuration. 3 Integration failures and repair 3.1 Observing effective state and waiting for saves Early tail checks reported equal all-zero pages. Later ex- amination of the effective locations and strides invali- dated that coverage claim: equality at an irrelevant ad- dress was not a test of the state consumed by subsequent computation. Correcting the addressing exposed nonzero tail data and transfer discrepancies that the earlier obser- vation had missed. The earlier success interpretation was retained as a corrected failure of observation, not counted toward the final validation. Save submission also had to be distinguished from completion. The integration was changed to wait on the actualDeviceMessagingFuturebefore allowing depen- dent state reuse. The final regression inserted a 500 ms delay into saves and paired the actual waits with the de- layed operations. This tests whether the recorded depen- dency is respected under the injected condition; it does not estimate a production failure probability. 2 Prompt length N = 3,584; checkpoint interval C = 1,792 Before Restore state B = 3,584 Scheduler credit p = 3,583 Compute 1 input token First difference: output index 11 After Restore state B = 1,792 Scheduler credit p = 1,792 Compute 1,792 input tokens All 64 output tokens agree B â p: changing a count does not roll back loaded state B = p: look up a strict prefix, then resume at the matching position Figure 2:Checkpoint alignment at N=3,584 and C=1,792. Before repair, the loaded state represents B=3,584 while the scheduler credits p=3,583. Strict-prefix lookup instead restores B=p=1,792 and recomputes the remaining suïŹix. This is a logical boundary example, not a timing trace. Table 2:Treatments and observation conditions ConfigurationComputation and schedulingConnectorObservation conditions Original production config- uration Original production settingsNot the evaluated treatment Not the output control in the final matrix Modified recomputation control Shared computation corrections, fixed per- rank FLA profile, matched checkpoint sched- ule DisabledCPU batch/configuration logging; no GPU tensor probes Modified candidateCorresponding common corrections and FLA profile; connector checkpoint path Enabled; lookup boundary changed between the two matrices CPU batch/configuration and transfer logging; no GPU tensor probes Instrumented candidateRepaired candidate configurationEnabledGPU byte checks, destination poisoning, tail checks, injected save delay These checks answer separate questions. Comparing source and destination pages checks the audited bytes. Observing nonzero effective tails checks that selected useful state entries are exercised. Waiting for the actual save checks a completion dependency. None alone estab- lishes that the bytes describe the prefix from which the scheduler resumes. 3.2 Complete-hit boundary mismatch The fixed-profile pre-fix matrix exposed failures at N=3,584 and N=5,376. For both lengths, raw transfer events from every rank reported a checkpoint covering the entire prompt, B=N. The schedulerâs computed count and external-hit metric instead reported p=Nâ1, and the first reload batch scheduled one input token. The archived connector explains the discrepancy. It retained the aligned lookup result in the request tracker used for snapshot retrieval. A later complete-prompt branch reduced the amount credited to the scheduler by one. Snapshot selection still used the retained full boundary. The observed condition was therefore B=N and p=Nâ1, rather than a snapshot of the shorter prefix. This mechanism is supported by the actual transfer and batch records, not inferred from the metric alone. The repair excludes the final known token before lookup at both GLM entry points. Alignment then selects an earlier complete checkpoint for an exact-boundary prompt. Restricting the query before checkpoint selection also keeps selection and its associated lookup bookkeep- ing at the same boundary; subtracting a count after re- trieving full state would leave the original problem intact. At N=3,584, the repaired path loads B=1,792 and com- putes a 1,792-token suïŹix. At N=5,376, it loads B=3,584 and computes the same suïŹix length. Both previously failing continuations then match their controls. This re- pair increases recomputation relative to the faulty one- token schedule at exact boundaries. The incremental latency cost relative to that incorrect path has not been measured; §5.5 instead compares repaired reload with modified cold recomputation. 4 Establishing a numerical control 4.1 What the diagnostic comparisons established Temperature zero and a fixed seed did not make the orig- inal cross-process comparison a reliable attribution test. Recorded batches differed: an earlier no-connector path processed a 3,583-token prefill, while the connector path split it into 1,792 and 1,791 tokens; decode scheduling differed as well. A matched checkpoint schedule removed this particular discrepancy, but output differences re- mained. 3 A subsequent audit found matching named parameter and buffer storages across arms. That narrowed the in- vestigation without proving that all scratch allocations or execution choices were identical. An actual mHC invoca- tion then showed that the attention-output input already differed between arms, while the other seven captured tensor inputs and scalar arguments agreed. This obser- vation does not demonstrate a same-input mHC defect. Probes taken in separate runs also cannot be joined into one observed layer-by-layer causal trace. The relevant prior distinction is between repeated exe- cution at one shape and invariance across batching or pre- fix splits. Fixed seeds do not resolve changes in floating- point reduction geometry. He discusses this distinction and concrete kernel controls in an author technical report; those results motivate the local control but do not validate this GLM implementation [2]. 4.2 Recorded FLA profile and its scope The experiment recorded actual FLA autotuner choices in a baseline run and pinned the seven observed configura- tions separately for each rank. The installer constrained each observed tuner to its recorded configuration and cleared its selection cache. Invocation of an unrecorded tuner was rejected. Later runs additionally checked the actual selected configuration after calls. The profile was a joint intervention on seven configura- tions per rank. No single-kernel ablation establishes that one tile size, warp count, or kernel alone caused the ear- lier divergence. The profile is neither a general determin- ism algorithm nor an established performance optimum. An older reference from a different diagnostic round remained unequal under the fixed profile. Its failed as- sertion and nonzero process exit were preserved. The selected profile reference was identified before the inter- vention; agreement with it does not turn the older com- parison into a pass. This distinction prevents a changing numerical reference from silently changing the reported acceptance criterion. 5 Evaluation 5.1 Workload and metrics The same archived plan is used for the pre-fix, repaired, and instrumented matrices. It contains nine prompt lengths around selected boundaries: 3,583, 3,584, 3,585, 3,587, 3,588, 3,589, 5,375, 5,376, and 5,377. Each case performs five serial operations: generate the target from a cold state, generate two interposer requests, reset the GPU cache while retaining the external cache, and regen- erate the target. Each paired arm therefore executes 45 operations, including 36 generations and nine resets. The inputs comprise 27 distinct token sequences gen- erated from one Chinese password-retrieval and expla- nation template. Target and interposer variants differ in a document tag near the start; they are not independent application domains. There is no explicit request cache salt, and recorded transfer events have an empty salt. The output criterion compares all 64 generated token IDs per request, with temperature zero, seed 42, and EOS ignored. No recorded output contains the known EOS token IDs. Although the prompt requests a longer explanation, the 64-token cap is a finite continuation test, not evidence of satisfying that instruction or preserving task quality. We distinguish paired output equality, within-engine cold/reload equality, actual four-rank transfers, and in- ternal transfer observations. There are 36 paired compar- isons in each two-arm matrix, not 36 distinct tasks. Pre- emption counters remain zero. The serial reset/interposer procedure exercises the cache path; it does not demon- strate behavior under sustained memory pressure or con- current preemption. Evidence checks separately recompute results from archived records and cross-check recorded observations; they do not provide an independently implemented ref- erence for every low-level checker. Hash agreement es- tablishes byte identity, not correct state addressing. The retrospective page audit verifies recorded comparisons rather than re-reading historical device buffers. 5.2 Boundary results without GPU tensor probes Table 4 retains all nine cases. âFirst differenceâ is the zero-based generated-token index in the pre-fix reload comparison; a dash means the complete 64-token con- tinuation agrees. B comes from transferred checkpoint metadata and p from the first reload batch. Each repaired row contains four paired generations: target cold, two interposers, and target reload. The pre-fix matrix has 34/36 equal generation pairs, with the two differences confined to the exact-boundary reloads. The repaired matrix has 36/36 equal pairs, and all nine target cold/reload comparisons agree. All 36 control generations also agree between the pre-fix and repaired runs. Across the repaired matrixâs nine reloads and four ranks, the first batch has computed=B and sched- uled=NâB. Every reload records positive H2D transfer on all four ranks. At N=3,584, transferred bytes per rank change from 69,625,856 to 54,809,600; at N=5,376, they change from 84,442,112 to 69,625,856. These metadata corroborate the change in selected checkpoints. These before/after byte counts alone do not establish a latency benefit. Sec- tion 5.5 separately measures the repaired path against modified cold recomputation; it is not a performance com- parison against the incorrect pre-fix path. An earlier attempt to execute the matrix inadvertently ran only the five-operation first case because of a harness import entry point. The independent coverage verifier rejected it. It is excluded from the nine-case results. Like- wise, the changing diagnostic rounds are not pooled into a statistical reliability estimate. 5.3 Instrumented transfer and save regression A separate repaired candidate executes the complete plan with byte observers, destination poisoning before H2D, and injected save delay. Its 36 generations match the archived repaired-run control, and all nine cold/reload pairs agree. This is an instrumented candidate compared 4 Table 3:Numerical-control experiments ExperimentObservationInterpretation Profile captureRecorded per-rank configurations and actual call inputs A concrete reference for the next intervention Fixed-profile first caseAll four 64-token generations agree between arms and with the selected reference Supports the controlled paired comparison Fresh-container repeatNew engines reproduce those first-case compar- isons Independent repetition of that bounded test Full boundary matricesActual configurations verified in both arms on four ranks Numerical control retained during the boundary intervention Table 4:Complete boundary matrix before and after repair NB beforep beforeFirst difference before B = p afterSuïŹix afterEqual generations after 358317921792â179217914/4 35843584358311179217924/4 358535843584â358414/4 358735843584â358434/4 358835843584â358444/4 358935843584â358454/4 537535843584â358417914/4 5376537653750358417924/4 537753765376â537614/4 with a previously recorded control, not a new two-arm run. Page rows are repeated internal checks, not 20,928 in- dependent round trips or randomized trials. Nonzero tails demonstrate coverage of the selected entries, not equal- ity of every internal model state. The artificial delay and heavy observation deliberately change execution condi- tions, so this run is not a serving-performance measure- ment. Its agreement complements the output matrix with- out GPU tensor probes; it does not replace that matrix. 5.4 Additional prompt templates and longer continuations A subsequent preregistered extension tests three ad- ditional synthetic templates: English ledger arith- metic, Chinese ordered-rule application, and En- glish Python-code reasoning. Each template uses prompt lengths 3,583, 3,584, and 5,376, with the same cold/interposer/reset/reload procedure and 256 generated tokens per new request. The historical first case remains as a 64-token bridge to the previously selected reference. The combined plan contains 30 distinct input sequences and 50 operations per arm: four bridge generations and 36 extended generations, totaling 9,472 output tokens per arm. Two separate disposable containers each run a fresh no-connector engine and a fresh candidate engine under the existing fixed configuration. The exported token plans are identical. Both pairs pass all 40 continuation compar- isons, and the two runs also agree on every baseline and candidate continuation. For all ten reloads per run, four- rank transfer metadata and first-batch records agree on B=C floor((Nâ1)/C), computed=B, scheduled=NâB, and zero draft tokens in the protected first batch. Actual FLA profiles match the recorded policy; no known EOS token or request preemption is observed. Each run exports 49 files whose SHA hashes are independently verified. This adds 72 successful 256-token generation pairs and eight repeated bridge pairs across two container-level rep- etitions. These are repeated synthetic inputs, not 80 inde- pendent tasks. A setup attempt was aborted before work- load execution after detecting an incorrect served-model identifier in the new harness; it has no model-correctness verdict and is excluded. The prompt contents, continu- ation lengths, output reference, and acceptance criteria were not changed in response to model results. This ex- tension does not repeat the poisoned byte audit, establish task-answer quality, or measure performance. The com- plete plan, failed setup record, raw exports, and indepen- dent checks are retained in the internal evidence archive. 5.5 Paired serial performance A separately preregistered study measures serial stream- ing completions on the disposable containerâs loopback interface. Two fresh containers run the same modified controls in opposite orders: OFFâON, then ONâOFF. Each engine runs one excluded warmup and three mea- sured trials per length, Nâ3583,3584,5376, with dis- tinct early document identities and 128-token continua- tions. OFF attempts cold and immediate-repeat requests; ON additionally attempts CPU reload after a GPU-only reset. All 120 requests (90 measured) pass complete output-token checks: each input has the same output across its five request conditions and both containers. The four engines are two paired repetitions, not 120 in- dependent experiments. The workload uses one English 5 Table 5:Instrumented transfer and save regression CheckObserved coverageResult Candidate/control generations36 continuations Ă 64 tokensAll agree D2H page-comparison rows17,640Zero reported byte mismatches H2D page-comparison rows3,288Zero reported byte mismatches; destinations poisoned Combined page-comparison rows20,928All 70 registered entries covered per rank and direction Nonzero tail H2D observations4 ranks Ă 9 reloads Ă 12 entries = 432All observed entries nonzero Delayed-save wait pairs67 per rank; 268 totalActual waits cover the injected 500 ms delay H2D volume2,447,265,792 bytes across four ranksPositive transfers for every reload Table 6:Content and continuation extension across fresh engines MeasureFirst pairFresh repeat Operations per arm5050 Historical 64-token bridge comparisons4/44/4 New 256-token comparisons36/3636/36 Output tokens per arm9,4729,472 Four-rank reload cases with aligned checkpoint and batch10/1010/10 Known EOS occurrences, both arms00 Request preemptions, both arms00 ledger task (opening balance 137, receipt 58, dispatch 29) with repeated background text. Its 12 tokenized inputs vary length and early document identity; nine inputs en- ter the measured sample. They are input variants of one synthetic task, not 12 distinct tasks. Requests use tem- perature 0, seed 42, and ignore_eos=True; task-answer quality is not evaluated. TTFT measures request start to receipt of the first nonempty token-ID event; total latency ends after the streamâs DONE marker. Decode rate counts tokens after the first event over the first-to-last nonempty-event inter- val, allowing multiple tokens per event. CPU metadata observers remain enabled; GPU tensor probes, poisoned destinations, artificial save delays, initialization and inter- request pauses are excluded. The GPUs are dedicated to the experiment while other production workloads con- tinue on the shared host. Trials run in fixed ascending order of trial index and prompt length; only engine or- der is reversed. This balances engine order across two runs but does not randomize request history or eliminate shared-host interference. The performance runs use four NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. Both arms use the Marlin MoE backend with custom all- reduce disabled. NCCL_P2P_DISABLE is set to 1 and OMP_NUM_THREADS to 2; expandable_segments is False. FlashInfer autotuning is disabled while the sepa- rately fixed FLA profile remains in force. The candidateâs local LMCache server uses LRU, separate object groups, l1-size-gb=16, and disabled lazy L1 allocation. These set- tings bound the performance result; no inter-host cache transfer or optimized communication comparison is mea- sured. Each latency is the median of three measured trials. Ratios are medians of per-input OFF-cold/ON-reload ratios, not ratios of the displayed medians. Across the six run/length groups, TTFT ratios range from 1.85â2.80 and total-latency ratios from 1.019â1.076. ON-cold/OFF-cold TTFT ratios range from 1.01â1.04; this is an end-to-end enabled/disabled comparison under the shared modified controls. It does not isolate transfer latency, observer cost, or the cost of all changes relative to production. Raw per- request values, ranges and event-level decode rates are retained in the evidence tables. Across these same six groups, OFF-cold total-latency medians span 5.570â6.104 s and ON-reload medians span 5.369â5.745 s. Post-first-event decode-rate medians span 23.38â24.80 and 23.08â24.76 tokens/s, respectively. These ranges describe the observed time scale across in- puts and runs; they are not uncertainty intervals. Attempted path names do not determine actual paths. Among 18 measured OFF immediate repeats, 12 recom- pute and 6 reuse a local prefix; among 18 ON immedi- ate repeats, 18 reload from CPU and 0 use local-only reuse. The OFF local reuse occurs only at N=5376 and credits 1792 tokens; its N=3583 and N=3584 repeats re- compute. At N=5376 the ON CPU path instead restores 3584 tokens, so those repeat conditions do not perform equal suïŹix work. Consequently, a complete five-path local-hit comparison is unavailable. This is an observed limitation of the evaluated configuration, not evidence that hybrid models cannot support local reuse. Each of the 18 measured explicit CPU reloads has positive four- rank H2D and an independently verified first-batch prefix B=p: 1792, 1792 and 3584 tokens for the three lengths, leaving 1791, 1792 and 1792 prompt tokens to recompute. Classification retains this actual suïŹix work. We report 6 3,583 R1 3,584 R1 5,376 R1 3,583 R2 3,584 R2 5,376 R2 Prompt length N / run 0 200 400 600 a Time to first token (ms) 3,583 R1 3,584 R1 5,376 R1 3,583 R2 3,584 R2 5,376 R2 Prompt length N / run 0 1 2 3 4 5 6 7 b Full request time (s) OFF coldON coldCPU reload Figure 3:Serial latency measurements. Hollow points show three measured requests per run, length, and condition; filled markers show medians, and thin bars show observed minâmax ranges, not confidence intervals. R1 runs OFF then ON; R2 reverses engine order. Both panels use the same 54 requests and zero-based axes. OFF is the modified no-connector control. The cold conditions have similar TTFT; total time includes the full 128-token generation. Table 7:Serial latency under matched controls Run NOFF cold TTFT (ms) ON cold TTFT (ms) ON reload TTFT (ms) TTFT ratioTotal ratio 13,583447.3451.9239.41.871.030 13,584447.6458.8240.41.861.037 15,376670.9686.7239.72.801.076 23,583447.3455.0237.91.881.019 23,584447.2462.6241.41.851.031 25,376671.2691.8241.22.781.053 no confidence interval from two engine pairs, sustained- concurrency result, or capacity gain beyond GPU mem- ory. 6 Related work and limitations Hybrid-state reuse is an established problem. Marconi explains why recurrent states updated in place require checkpoints aligned with reusable attention prefixes and studies cache admission and eviction. The Sparse Prefix Caching preprint studies sparse checkpoint materializa- tion and suïŹix recomputation for hybrid and recurrent serving. Our strict-prefix lookup repair applies this exist- ing requirement to a connector/scheduler disagreement. We introduce neither a new eviction policy nor the princi- ple of replaying a suïŹix from a checkpoint [ 1], [3]. An upstream vLLM checkpoint RFC separately dis- cusses prefix identity, non-KV logical state, and im- mutable payload versus logical equivalence. It is a de- sign proposal, currently closed as not planned, rather than an implemented standard. Its prior discussion limits a novelty claim about a general recovery interface; the con- tribution here is evidence from a running integration [ 4]. Numerical-control work is also directly relevant. He distinguishes repeatability from batch invariance. The LLM-42 preprint uses verified speculation and replaces speculative KV state with verified state even when gen- erated tokens agree. These works reinforce why finite token equality should not be promoted to complete state or future-output identity. Our experiment fixes one ob- served profile and tests finite continuations; it does not implement a general online verification system [ 2], [5]. LMCache supplies the cache infrastructure used in this study. Its broader cache management and transfer mech- anisms are existing platform contributions. The local cu- mulative patch spans representation, completion, schedul- ing, and common computation changes; eleven changed files do not constitute eleven independently novel mech- anisms. Only the final lookup intervention is isolated by the otherwise controlled before/after matrix. The pre- ceding integration changes were not subjected to a full factorial ablation [ 6]. External validity is limited by one model revision, one four-rank setup, synthetic prompt templates, finite con- tinuations, serial requests, and a fixed profile selected from an observed baseline. The original matrix uses one template and 64-token outputs; the extension adds three templates and 256-token outputs, with only two fresh- container repetitions. We have not measured all logits, arbitrary future continuation, cross-TP invariance, con- current preemption, other models, or default-production equivalence. The complete modified control is essential to interpreting the result. 7 The serial study measures a CPU-reload latency ben- efit against the shared modified cold control. Costs of fixed configuration, ordering changes and completion bar- riers relative to the original production configuration re- main unmeasured. Immediate repeats do not consistently provide local-only reuse, limiting a complete five-path comparison. Two engine pairs on one shared host do not establish concurrent SLOs, broad reliability, or capacity beyond GPU memory. No public artifact release or pro- duction deployment is claimed. 7 Conclusion The evaluated GLM integration restored a complete checkpoint while crediting the scheduler with a shorter prefix. Actual transfer metadata and batch records ex- posed that mismatch; restricting lookup to a strict prefix repaired both failing exact-boundary cases under a con- trolled numerical configuration. The repaired nine-case matrix passes all 36 finite continuation comparisons, and a separate instrumented run passes the specified byte, nonzero-tail, and delayed-save checks. The subsequent content extension also passes two fresh-container com- parisons with 256-token continuations. The engineering result is a validated recovery path within these conditions, with a measured serial CPU-reload latency benefit under matched controls; representative concurrency, original- production overhead and overflow capacity remain open. Acknowledgements The author acknowledges Research Technology Services, UNSW Sydney, for providing GPU resources on the Katana computational cluster [ 7] for model quantization. AI assistance.AI tools assisted with the research and preparation of this manuscript. The author takes responsi- bility for the methods, results, and final text. References [1]Rui Pan et al. Marconi: Prefix Caching for the Era of Hybrid LLMs. MLSys, 2025. [2]Horace He and Thinking Machines Lab. Defeating Nondeterminism in LLM Inference. Author techni- cal report, 2025. [3]Mikhail Shirokikh and Sergey Nikolenko. Sparse Prefix Caching for Hybrid and Recurrent LLM Serv- ing. arXiv:2605.05219v1, preprint, 2026. [4]vLLM checkpoint RFC, issue #40533. Engineering design proposal, 2026; closed as not planned when reviewed on 14 September 2026. [5]Raja Gond et al. LLM-42: Enabling Determin- ism in LLM Inference with Verified Speculation. arXiv:2601.17768v2, preprint, 2026. [6]Yuhan Liu et al. LMCache: An EïŹicient KV Cache Layer for Enterprise-Scale LLM Inference. arXiv:2510.09665v1, preprint, 2025. [7]PVC (Research Infrastructure), UNSW Sydney. Katana. UNSW, Sydney, 2010. doi:10.26190/669x- a286. [8]Z.ai. GLM-5.3-Flash. Model card, accessed 14 September 2026. [9]Red Hat AI. GLM-5.3-Flash-NVFP4. Model card, accessed 14 September 2026. 8