Paper deep dive
Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines
Bojie Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/7/2026, 3:50:28 AM
Summary
This paper introduces a fine-grained computation offload rerouting method that leverages existing server concurrency machinery to overlap offloads with other requests, avoiding stalls from context switches or busy-waiting. Validated across ten production servers, the approach requires only 22-138 lines of code and achieves 1.2-5.4x speedups, with a zero-edit LD_PRELOAD-based fiber runtime reaching up to 17.3x. It also provides a predictive model for speedup and code cost, and a correctness taxonomy with a transparent guard for run-to-completion atomicity hazards.
Entities (10)
Relation Signals (6)
Rerouting method â achieves â 1.2-5.4x speedup
confidence 95% ¡ this recipe takes 22-138 lines added, at most one modified, and recovers 1.2-5.4x on real hardware
Concurrency models â determines â speedup and code cost
confidence 94% ¡ the server's concurrency model determines what a synchronous offload costs and therefore what rerouting recovers
Hardware accelerators â sitson â critical path of online serving
confidence 93% ¡ Hardware accelerators now sit on the critical path of online servingâGPUs, FPGAs, and increasingly remote services
Fine-grained computation offload â causes â CPU stall
confidence 92% ¡ For fine-grained offloads (microseconds to a few milliseconds) the classic responses to the resulting stall both fail
LD_PRELOAD fiber runtime â enables â zero-edit rerouting
confidence 90% ¡ an LD_PRELOAD fiber runtime injects the reroute into an unmodified thread-per-connection binary
Transparent page-protection detector â guards â unlocked shared aggregates
confidence 88% ¡ a transparent page-protection detector guards exactly those, validated on stock Redis
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Hardware accelerators now sit on the critical path of online serving. GPUs, FPGAs, and increasingly remote services such as hardware security modules, post-quantum KEMs, and inference servers. For fine-grained offloads (microseconds to a few milliseconds) the classic responses to the resulting stall both fail: a context switch costs as much as the offload, and a busy-wait burns the core. Overlapping the offload with other requests is the fix, and prior systems obtain it by adding concurrency: an async-framework rewrite, a new runtime or dataplane OS, or a hand-tuned point integration. We observe that the concurrency already exists: serving concurrent requests is suspending and resuming them, so every server ships the machinery overlap needs. Overlap is then a routing problem, not a rewrite problem: submit the offload to an executor, suspend the request with the server's own deferred-response primitive, resume it on completion. Across ten off-the-shelf servers spanning every production concurrency model, this recipe takes 22-138 lines added, at most one modified, and recovers 1.2-5.4x on real hardware; the server's concurrency model and the offload's weight predict both numbers in advance, and the win is bounded by device throughput and the server's own overlap capacity. At the limit, an LD_PRELOAD fiber runtime injects the reroute into an unmodified thread-per-connection binary (17.3x) within a characterized envelope. Rerouting suspends run-to-completion atomicity; a measured taxonomy confines the hazard to unlocked shared aggregates, and a transparent page-protection detector guards exactly those, validated on stock Redis.
Tags
Links
- Source: https://arxiv.org/abs/2607.02630v1
- Canonical: https://arxiv.org/abs/2607.02630v1
Trouble viewing inline? Open PDF directly â
Full Text
52,804 characters extracted from source content.
Expand or collapse full text
Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines Bojie Li Pine AI Abstract Hardware accelerators now sit on the critical path of online servingâGPUs, FPGAs, and increasingly remote services such as hardware security modules, post-quantum KEMs, and inference servers. For fine-grained offloads (microseconds to a few milliseconds) the classic responses to the resulting stall both fail: a context switch costs as much as the offload, and a busy-wait burns the core. Overlapping the offload with other requests is the fix, and prior systems obtain it by adding concurrency: an async-framework rewrite, a new runtime or dataplane OS, or a hand-tuned point integration. We observe that the concurrency already exists: serving concurrent requests is suspend- ing and resuming them, so every server ships the machinery overlap needs. Overlap is then a routing problem, not a rewrite problem: submit the offload to an executor, suspend the request with the serverâs own deferred-response primitive, resume it on completion. Across ten off-the-shelf servers spanning every production concurrency model, this recipe takes 22â138 lines added, at most one modified, and recovers 1.2â5.4Ă on real hardware; the serverâs concurrency model and the offloadâs weight predict both numbers in advance, and the win is bounded by device throughput and the serverâs own overlap capacity. At the limit, anLD_PRELOADfiber runtime injects the reroute into an unmodified thread-per-connection binary (17.3Ă) within a characterized envelope. Rerouting suspends run-to-completion atomicity; a measured taxonomy confines the hazard to unlocked shared aggregates, and a transparent page-protection detector guards exactly those, validated on stock Redis. Code: https://github.com/19PINE-AI/transparent-offload Website: https://01.me/research/transparent-offload 0.00.51.01.52.02.53.03.54.0 speedup over synchronous offload (Ă, real GPU AES) HAProxy Python Node.js Postgres MariaDB nginx memcached Go Redis Apache 2.10Ă 2.37Ă 2.54Ă 2.59Ă 2.59Ă 2.74Ă 2.93Ă 3.01Ă 3.01Ă 3.45Ă event loop loop + pool thread pool proxy per-conn DB Figure 1. The headline result. Rerouting the offload through each serverâs own concurrency recovers 2.1â3.5Ăacross ten servers (real GPU AES, 1 MiB; up to 5.4Ăat 8 MiB; the databases pipeline the op intra-query); zero-edit rerouting from outside the binary reaches 17.3Ă(§4). Color marks the concurrency model. 1 arXiv:2607.02630v1 [cs.DC] 2 Jul 2026 Fine-Grained Computation Offload in Tens of Lines2 recvpre accelerator busy postsend CPU idle (wasted) Synchronous offload: one request, CPU stalls A:recvA:preA: offloaded (parked)A:post BCB time Overlapped offload: CPU serves B, C while A's offload is in flight Figure 2. The problem. A serving request is receiveâpre-processâoffloadâpost-processâsend. Top: with a synchronous offload, the CPU stalls while the accelerator works. Bottom: overlap fills the gap with other requestsâ processing. 1 Introduction Hardware accelerators are now a standard part of online serving. GPUs, TPUs, and FPGAs accelerate inference, image processing, encryption, and compression, and a growing share of âaccelerationâ is in fact a remote call: to a hardware security module, a post-quantum key- encapsulation service [National Institute of Standards and Technology, 2024], or a co-located inference server [Crankshaw et al., 2017]. Across these cases the application has the same shape: receive a request, pre-process, invoke an intensive routine, post-process, respond. On an accelerator that routine becomes an offload: a submission to the device and a wait for its completion, lasting microseconds to a few milliseconds. This paper is about a deceptively simple question: what should the CPU do while it waits? The textbook answers fail exactly in this regime (Figure 2). Blocking lets the operating system run another thread, but a context switch costs several microseconds, comparable to the offload itself; this is the âkiller microsecondâ [Barroso et al., 2017]: the switch wastes CPU and adds a wakeup delay to the requestâs tail. Busy-waiting hides the wakeup but burns a core doing nothing. The only principled answer is to overlap the offload with the processing of other requests. Overlap requires concurrency at the offload point, and the systems community has treated that concurrency as something to be added: write the application against an asynchronous framework from the start [Provos and Mathewson, 2026, ScyllaDB, 2026], adopt a new runtime or dataplane operating system [Belay et al., 2014, Ousterhout et al., 2019, Fried et al., 2020], or hand-integrate one offload into one server [Jang et al., 2011, Intel Corporation, 2026]. All of these assume the server lacks what overlap needs. Our starting observation is that it already has it. Serving concurrent connections is sus- pending and resuming requests: an event loop parks a connection whenever it waits for a Fine-Grained Computation Offload in Tens of Lines3 socket; a worker pool parks whole requests in its queue; a goroutine scheduler parks tasks at every blocking call; a database parks each connection in its own backend. Every modern server therefore ships, in production quality, exactly the machinery that overlap requires. Hiding a fine-grained offload is not a rewrite problem but a routing problem: at the offload call site, submit the work to a background executor, suspend the request using the serverâs own deferred-response primitive, and resume it to reply on completion. We validate this recipe on ten off-the-shelf servers spanning every concurrency model in production use: Redis, Node.js, Python/asyncio, nginx, memcached, Apache, Go, HAProxy, PostgreSQL, and MariaDB. Across the ten, the change is 22â138 lines added and zero or one existing line modifiedâthe core of the Redis integration is a dozen lines (Listing 1)âand on real hardware it recovers 1.2â5.4Ăacross servers and offload weights (Figure 1). The cost is small for a structural reason, not a lucky one: the integration only bridges the offload to machinery that already exists. That structure makes the outcome predictable. Two properties of the deployment fix both the speedup and the code cost in advance: the serverâs concurrency model determines what a synchronous offload costs and therefore what rerouting recovers, and how many lines the reroute takes; the offloadâs weight relative to per-request CPU work determines whether overlap pays at all. The model locates every server we measured, including the ones where overlap does not help. Two questions remain, and they complete the paper. First, can the change be zero? When there is no source access at all, the libc symbol boundary is itself a suspension layer: anLD_PRELOAD fiber runtime injects the rerouting from outside an unmodified binary and reaches 17.3Ă on a thread-per-connection serverâwithin an envelope we characterize precisely, because transparency extends exactly as far as the behavior that flows through the interposed layer. Second, what does rerouting break? Suspending a handler mid-request forfeits the run-to- completion atomicity that event-driven servers silently rely onâa hazard we measure, scope, and guard. In summary, this paper makes three contributions: â˘An offload-rerouting method. Overlapping a fine-grained offload requires rerouting it through concurrency the server already has, not adding concurrency. One three-step method, instantiated on ten off-the-shelf servers with 22â138 lines added and at most one existing line modified, recovers 1.2â5.4Ă on real hardware across offload weights (§3, §6). ⢠A predictive model of speedup and code cost. Concurrency model and offload weight predict the win and the cost in advance (§2); at the zero-edit limit, an interposition-envelope principle and a syscall-profile classifier predict where transparent rerouting is possible at all (§4). â˘A correctness taxonomy and a transparent guard. Rerouting breaks run-to-completion atomicity; a measured taxonomy confines the hazard to unlocked shared aggregates, and a transparent conflict detector protects exactly those with no application changes (§5). 2 The Fine-Grained Regime and a Predictive Model The fine-grained regime.An accelerator turns a CPU-bound routine into a device operation: the CPU prepares an input buffer, submits it (a kernel launch, a DMA, or a network send), and Fine-Grained Computation Offload in Tens of Lines4 1 Îźs10 Îźs100 Îźs1 ms10 ms offload latency (log scale) fine-grained regime: offload scheduling cost OS context switch (~15 Îźs) GPU AES / small kernel (~1050 Îźs) compression / small inference (~0.10.5 ms) HSM sign ¡ PQC KEM / remote inference (~110 ms) Figure 3. The fine-grained regime. Accelerator offloads span microseconds to milliseconds. In the shaded band, the offload latency is comparable to an OS context switch, so blocking pays a switch and a wakeup that rival the work itself, and busy-waiting wastes a core. later collects the result. These round-trips span microseconds to milliseconds (Figure 3), but all are fine-grained relative to operating-system scheduling: the offload finishes on the timescale of a context switch, so the dilemma of §1 holds across the whole band. A taxonomy of concurrency models. The thesis says to reroute the offload through the server âs own concurrency, so the first question about any server is where that concurrency lives. Five models cover production practice. A single-event-loop server (Redis, Node.js, a Python asyncioservice) multiplexes all connections on one thread; its suspension primitive is the deferred reply. An event-loop-with-pool server (nginx, memcached) runs a few such loops plus a worker pool. A thread- or goroutine-pool server (Apache, Go) parks whole requests in pooled workers or runtime-scheduled tasks. A proxy (HAProxy) can route requests through a native offload engine to an external agent. A process- or thread-per-connection server (PostgreSQL, MariaDB) gives each connection its own backend, which the OS suspends and resumes wholesale. Prediction 1: concurrency model determines speedup and code cost. The same taxonomy predicts what a synchronous offload costs and therefore what rerouting recovers (Figure 4). The extreme case is the single event loop: one thread handles every connection, so a synchronous offload stalls all of them and throughput collapses to one request per offload latency while the hardware idlesâa pathology of structure, not capacityâand the few-line reroute is dramatic there. In an event loop with a pool, a synchronous offload stalls only one loop of several. A thread or goroutine pool overlaps offloads automatically: the reroute needs no asynchronous code at all, because parking the worker is the routing. For a proxy, the reroute is configuration plus an external agent. And a per-connection database already overlaps across connections for free (the OS runs the backends concurrently), leaving intra-query pipelining of a serial offload Fine-Grained Computation Offload in Tens of Lines5 single event loop sync offload stalls everything 2.43.0Ă Redis, Node, Python event loop + pool stalls one loop of N 2.72.9Ă nginx, memcached thread / goroutine pool pool overlaps the rest 3.03.5Ă (0 async code) Apache, Go proxy + agent agent offloads async 2.1Ă (C agent) HAProxy process-per- connection OS overlaps connections free intra-query pipelining Postgres, MariaDB concurrency model predicted rerouting win Win grows with offload weight; it pays only when the offload outweighs per-request CPU. Figure 4. Prediction 1: the concurrency model determines what a synchronous offload costs and therefore how much rerouting recovers, from dramatic (a blocked single loop) to automatic (a pool that overlaps for free) to intra-query (databases that already overlap across connections). Example servers are shown under each regime. loop as the only win. The code cost follows the same axis: where the server exposes a plugin API or a deferred- response primitive the reroute is purely additive; where it exposes none, the developer must add the suspend/resume transition to the request state machine by hand, which is where the line count comes from (§3). Prediction 2: offload weight determines whether overlap pays. The second property is independent of the server. Overlap reclaims the CPU time the offload would have wasted, so if per-request CPU work already rivals the offload there is little to reclaim; the win grows with the offloadâs weight until it hits one of two ceilings: the acceleratorâs throughput (overlap fills the device but cannot exceed it) or the serverâs own overlap capacity (the concurrency its machinery can keep in flight). Real speedups live between these bounds. Section 6 confirms both predictions, including a block-size sweep that traces the weight dependence on real hardware (Figure 11) and the servers where overlap correctly buys nothing. Together the two properties form a map: given a serverâs concurrency model and an offloadâs latency, an operator can predict before writing any code whether to bother, how large the win will be, and roughly how many lines it will take. 3 Offload Rerouting on Ten Servers The recipe is the thesis made concrete, and it is uniform across servers (Figure 5): 1. at the offload call site, submit the work to a background executor instead of waiting; 2. suspend the current request using the serverâs own deferred-response primitive; 3. resume it and send the reply when the offload completes. Listing 1 shows the recipe in Redis: the entire integration is an 83-line loadable module with Fine-Grained Computation Offload in Tens of Lines6 handler reaches the offload 1. submit to a background executor accelerator runs the offload 2. suspend the request (server's own primitive) loop serves other requests 3. resume + reply on completion The recipe: reroute the offload through the server's existing suspend/resume machinery 22138 lines added, at most 1 modified the machinery already exists; one only reroutes the offload. Figure 5. The rerouting method. While a request is suspended at its offload, the serverâs loop serves other requests. The edit is small because the suspend/resume machinery already exists. 1 /* accel.async: submit , suspend , resume on completion */ 2 int cmd_async(RedisModuleCtx *ctx , ...) 3 RedisModuleBlockedClient *bc = 4 RedisModule_BlockClient(ctx , reply_cb , 5 timeout_cb , NULL , 0); /* 2. suspend */ 6 enqueue(bc); /* 1. submit */ 7 return REDISMODULE_OK; /* loop serves other clients */ 8 9 void *worker(void *arg) /* pool thread */ 10 for (;;) job_t j = dequeue (); 11 accel_encrypt(j.buf , j.len); /* the offload */ 12 RedisModule_UnblockClient(j.bc , NULL); 13 /* 3. resume */ 14 Listing 1. The reroute in Redis (condensed from the 83-line moduleaccel_module.c).BlockClientis Redisâs own deferred-response primitive; the worker pool is the background executor. The command suspends before submitting, so the resume cannot race the suspension. zero edits to Redis itself, and its overlap core is the dozen lines of the listing. The synchronous variant of the same command simply callsaccel_encrypton the event-loop thread; the diff between the two is the reroute. Table 1 instantiates the recipe across the landscape, and the integration point follows the concurrency model exactly as §2 predicts. Servers with a plugin or extension API take the reroute with zero core edits: a Redis [Redis Ltd., 2026] module, an nginx [F5/NGINX, 2026] thread-pool add-on, an Apache [Apache Software Foundation, 2026] content handler, Post- greSQL [PostgreSQL Global Development Group, 2026] and MariaDB [MariaDB Foundation, 2026] user-defined functions, and a Node.js [OpenJS Foundation, 2026] N-API add-on over libuv [libuv contributors, 2026]. Servers backed by a language runtime, whatever their con- currency model, need almost no new code: the goroutine scheduler overlaps a Go [The Go Authors, 2026] handler âs plain blockingcgooffload with zero asynchronous code, and a Python asyncio[Python Software Foundation, 2026] service reroutes throughrun_in_executorin one line of handler code. A proxy reroutes in configuration: HAProxy [HAProxy Technologies, 2026a] streams each request over its native Stream Processing Offload Engine (SPOE) [HAProxy Technologies, 2026b] to a standalone offload agent (the proxy itself changes only configuration; Fine-Grained Computation Offload in Tens of Lines7 serverintegration point+linesmodspeedup Redis 6.0loadable module (BlockClient)8303.01Ă Node.js 22N-API add-on (libuv queue)3402.54Ă Python 3.10 run_in_executor2202.37Ă nginx 1.18add-on module (thread pool + aio)11202.74Ă memcached 1.6state-machine patch7012.93Ă Apache 2.4module (apxs), pooled workers2703.45Ă Go 1.18plain blocking cgo call2803.01Ă HAProxy 2.4SPOE + standalone C agent13802.10Ă PostgreSQL 14C extension (intra-query)4202.59Ă â MariaDB 10.6UDF (intra-query)3402.59Ă â Table 1. Offload rerouting on ten off-the-shelf servers (grouped by concurrency model: single event loop, event loop + pool, thread/goroutine pool, proxy, per-connection database). â+linesâ is integration code added; âmodâ is existing lines modified. Speedups are over the synchronous offload with a real GPU (1 MiB AES; â the databases pipeline the op intra-query: serial vs. pipelined offloads within one query). Seven are stock server binaries; for Node.js, Python, and Go the server is an idiomatic handler on the stock runtime. Measurement setup: §6. the 138-line C agent carries the offload). The lone case that touches existing code is memcached [memcached, 2026], which exposes no deferred-response primitive, so we add the suspend/resume transition to its connection state machine by hand (70 lines, one modified). It is the exception that proves the structural claim: the cost of the reroute is the distance to the serverâs nearest suspend/resume primitive. We report lines added separately from existing lines modified because they carry different maintenance costs: added lines live in a module or extension and survive server upgrades; modified lines are the invasive part. Figure 6 plots the whole study on one canvas: code cost against measured win, with the zero-edit limit of §4 at the origin. 4 The Zero-Edit Limit: Rerouting by Interposition Can the reroute cost zero lines? When there is no source access at allâa stock binary, a vendor blobâthe recipe cannot be applied from inside. But there is one suspension layer every dynamically linked binary passes through: the libc symbol boundary. Our transparent runtime is a shared library loaded byLD_PRELOADthat interposes the standard threading and I/O symbols. When the application creates a connection-handling thread, the runtime instead creates a fiberâa user-level thread with a register-only context switch of tens of nanoseconds [Li et al., 2023]âand multiplexes all fibers on one carrier OS thread (occupying a single core). Whenever a fiber would block, on socket I/O or on the offload, the runtime switches to another runnable fiber; a scheduler polls I/O readiness and offload completions and resumes the corresponding fiber, saving and restoring per-fiber libc state such aserrnoacross switches. The handler keeps its plain synchronous shape and never learns that its âthreadâ is a fiber: the recipeâs submit, suspend, and resume are injected from outside the binary (Figure 7). On its home ground this works well: on a synchronous thread-per-connection handler with a real GPU performing AES, the runtime overlaps 64 connectionsâ offloads on one carrier thread and reaches 17.3Ăthe throughput of the same binary busy-waiting on each offload, with results verified bit-for-bit (11.9Ăover a baseline that blocks in the driver and lets the OS Fine-Grained Computation Offload in Tens of Lines8 ~010100 lines added to the application (0138) 1 2 3 5 10 20 speedup over sync offload (Ă, real GPU) Redis Node.js Python nginx memcached Go Apache Postgres MariaDB HAProxy transparent (0 edits, own binary; real GPU AES) no gain single event loop event loop + pool thread / goroutine pool proxy (offload agent) per-connection DB (intra-query) Figure 6. The study in one plot: lines of integration code vs. speedup from rerouting a real-GPU AES offload on an idle device (1 MiB blocks; the databases pipeline the op intra-query). The few-line reroute clusters at 2â3.5Ă; the zero-edit transparent runtime reaches 17.3Ăin its niche (§4). Color encodes the concurrency model. carrier core (one OS thread) fiber A (conn 1) fiber B (conn 2) fiber C (conn 3) scheduler (epoll + poll device) accelerator device offload (submit + yield) completion (resume fiber) LD_PRELOAD interposes pthread_createfiber, read/write/pollyield, offloadyield. The application binary is unchanged. Figure 7. The zero-edit limit. AnLD_PRELOADlibrary turns each connection thread into a fiber on one carrier thread; a fiber yields at its offload (and at socket I/O) and the scheduler runs another until the offload completes. The binary is unchanged. On a synchronous thread-per-connection handler with a real GPU this overlaps 64 connectionsâ offloads for a 17.3Ăgain over a busy-wait synchronous baseline. overlap the 64 threads 1 ); a separately built DNN-inference server shows 11.8Ă. 1 The blocking-baseline comparison was measured while a co-tenant shared the GPU (both sides equally affected); the busy-wait comparison ran on an otherwise idle GPU. Fine-Grained Computation Offload in Tens of Lines9 Making stock binaries run under the runtime took a catalog of interposition engineeringâ most notably a glibc condition-variable symbol-versioning hazard that silently corrupts some binaries and that any threading-interposing tool must fixâwhich Appendix B records for practitioners. The interposition envelope. The more useful contribution is the limit. A preloaded library virtualizes exactly one layer, the libc symbol boundary, so its reach is the completeness of that layer: behavior that escapes the layer is out of reach, and there are exactly three ways to escape it (Figure 8). ⢠Below it. A storage engine such as InnoDB synchronizes with rawfutexsystem calls [Franke et al., 2002] issued directly, bypassingpthread; a fiber that blocks in a raw futex stalls the whole carrier, and under contention the server deadlocksâwe confirmed this with a live backtrace of the frozen carrier inside InnoDB. â˘Beside it. A managed runtime such as the JVM keeps âthe current threadâ in a native thread- local slot read inline from a CPU register, which symbol interposition cannot virtualize per fiber; the moment two fiberized JVM requests interleave, the runtime reads the wrong thread object and segfaults (the same mechanism applies to Go and .NET). ⢠Behind it. Overlap needs idle CPU to reclaim, and a thread-per-connection TLS terminator that spends real CPU on per-request crypto has noneâthe OS already overlaps its offloads across threads, while the single carrier serializes the crypto and pays an interposition tax on every yield, ending up 2â3Ă slower than native. These are properties of where an application places its behavior, not gaps in engineering; no interposition removes them. A syscall-profile classifier. The envelope is predictable from the outside: we profile the per-request blocking syscalls a server makes under load (Figure 9). Event-driven servers show anepoll-dominated profile and have no per-connection thread to fiberize: the runtime loads safely but never engages. Thread-per-connection servers that block at the libc layer show near-zero per-requestfutextraffic and are fiberizable. Engines that synchronize below libc are unmistakable: InnoDB issuesâź13,000 futex calls per request, some 500Ăa fiberizable server âs, alongside asynchronous kernel I/O (io_uring[Axboe, 2019]). A cheapstracetherefore tells an operator in advance whether zero-edit rerouting can work. The verdict frames the division of labor. Zero-edit rerouting serves a real but narrow niche: thread-per-connection servers with light per-request CPU and libc-level blocking, reached with no source access. By construction it cannot help event-driven serversâthe class where a synchronous offload is most catastrophic. For everyone else, the few-line recipe of §3 is the answer. 5 Correctness under Rerouting Rerouting has one cost that is easy to miss. A single-threaded server runs each handler to completion, so handlers observe shared state atomicallyâan invariant the code silently relies on. Suspending a handler at its offload breaks that invariant: if post-processing does a read-modify- write on shared stateâa counter, a cache entry, a key in a storeâtwo overlapped requests can Fine-Grained Computation Offload in Tens of Lines10 Application / runtime libc symbols the interposition layer Kernel / hardware Where transparent interposition reaches and the three walls raw syscall(futex) below libc (InnoDB) %fs TLS thread identity in native TLS register (JVM, Go) CPU 100% per-request CPU > offload (TLS crypto) Figure 8. The interposition envelope. A preloaded library virtualizes the libc symbol layer, and reaches exactly the behavior that flows through it. Behavior escapes in three ways: below the layer (rawfutex syscalls, InnoDB), beside it (thread identity in a native TLS register, the JVM and Go), and behind it (per-request CPU that already exceeds the offload). RedisstunnelmemcachedMariaDB (InnoDB) 10 1 10 2 10 3 10 4 per-request futex calls (log) sub-libc wall (+ io_uring) fiberizable Futex density predicts whether a binary can be fiberized fiberizable (libc-level blocking) event-driven (loads safely, no overlap) sub-libc wall Figure 9. Predicting the envelope. Per-requestfutexcalls separate thread-per-connection servers that block at the libc layer (fiberizable: stunnel) from engines that synchronize below it (InnoDB,âź13,000, some 500Ăhigher, plusio_uring). Event-driven servers (gray) have no per-connection thread to fiberize; their epoll profile flags them âloads safely, no overlap.â interleave their sequences and lose updates. On a stock Redis server whose module command reads a key, runs a real GPU offload, and writes the incremented value back, unprotected overlap silently loses tens of thousands of updates (Figure 10, left). The more latency overlap hides, the wider the race it opens. Fine-Grained Computation Offload in Tens of Lines11 A measured taxonomy of shared-state patterns. To scope the problem we enumerated the shared-state patterns that real offload-adjacent servers keep, and measured each under over- lapped execution in race-instrumented harnesses spanning crypto (OpenSSL) and compression (zlib). Three patterns are unconditionally safe: read-only shared state (model weights, dictionar- ies, lookup tables), per-connection state (which one handler per connection serializes for free), and lock-protected state (locks serialize regardless of scheduling). All measured zero conflicts. The only hazardous pattern is the fourth: unlocked shared mutable aggregatesâcounters, batch queues, cachesâwhere unlocked read-modify-writes lose updates essentially every time they collide. And that pattern is exactly the state a single-threaded server keeps unlocked because it trusts run-to-completion atomicityâthe very atomicity rerouting removes. The dividing line is the state pattern, not the application domain: bulk crypto, compression, hashing, and stateless inference land safe because the heavy routine is pure and their shared state is read-only or already locked. A transparent conflict detector.The safe patterns need nothing. For the residualâunlocked shared aggregatesâa transparent conflict detector restores correctness with no application change: it write-protects the shared-state pages, snapshots a version clock when a handler parks at its offload, and treats a write fault on a page modified during the park as a conflict; under enforcement it serializes only the conflicting handlers. On the stock-Redis workload it drives lost updates to zero while running 1.8Ăfaster than a coarse lock, which serializes the very offloads it protects; its overhead versus unprotected overlap is within measurement noise (24.7K vs. 24.1K req/s) (Figure 10, right). The detector is a property of the overlap mechanism, not of any one integration, so it applies at every point of the spectrum, including the zero-edit limit. 6 Evaluation This section validates the two predictions of §2 on real hardwareâPrediction 1 across the ten-server landscape, Prediction 2 by sweeping the offloadâs weightâand closes with latency under load. Experimental setup. All cross-server offloads run on real hardware, with no emulated latencies; the one exception is the open-loop latency study of Figure 12, which uses a controlled emulated offload and is labeled as such. Experiments run on a server with an NVIDIA RTX PRO 6000 (Blackwell) GPU. The GPU path performs AES [OpenSSL Project, 2026] on its own CUDA stream, polled bycudaEventQuery[NVIDIA Corporation, 2026], with the block size sweeping from 4 KiB (launch-bound) to 8 MiB (bandwidth-bound). GPU AES is a stand-in for offloads that beat the host CPU (a modern coreâs AES-NI rivals a GPU on this cipher); we use it because it gives a cleanly tunable offload weight. For the latency-bound remote class (HSM signatures, post-quantum KEMs, remote inference) we stand up a real TCP signer doing a genuine RSA-2048 signature per request. Servers are driven by standard load generators (redis-benchmark, ab). Validation of Prediction 1 across the server landscape.On a 1 MiB GPU AES offload (realistic bulk crypto, idle GPU) the reroute recovers the win the model predicts in every regime, with Fine-Grained Computation Offload in Tens of Lines12 naive (overlap) detector + enforce 0 5000 10000 15000 20000 25000 30000 lost updates 23,450 lost 0 correct Shared-state safety (stock Redis) high contention low contention 0 10 20 30 40 throughput (K req/s) 1.8Ă 1.7Ă Detector keeps offloads overlapped naive (unsafe) detector (overlapped) lock (serialized) Figure 10. Guarding the one hazardous pattern, measured on a stock Redis server with a real GPU offload. Left: unprotected overlap of a shared read-modify-write loses 23,450 updates. The page- protection detector flags 26,185 conflicts and, under enforcement, reduces lost updates to zero. Right: a coarse lock serializes all offloads (13.8K req/s), while the detector keeps non-conflicting offloads overlapped (24.7K req/s), 1.8Ăfaster (1.7Ăat low contention) and within noise of unprotected overlap (24.1K req/s). no failed requests (Figure 1, Table 1): single event loops and event-loop-with-pool servers cluster around 2.4â3Ă, thread and goroutine pools reach 3.0â3.5Ăwith no asynchronous code, and the per-connection databases recover 2.6Ăthrough the one channel the model leaves them, pipelining the offloads within a query rather than across connections. The cluster tops out at 2â3.5Ăbecause this offload is bandwidth-bound: once overlap saturates the device, its throughput is the ceiling. The proxy isolates where the win comes from: HAProxyâs gain rides entirely on the offload agent, so a 21-line GIL-bound Python agent gains nothing while a 138-line C pthread-pool agent recovers 2.1Ă on the same offload. Validation of Prediction 2: offload weight and the two ceilings. Sweeping the GPU AES offload by block size on a single event loop traces the weight dependence (Figure 11; full table in Appendix A): the same reroute gives essentially nothing for a light block (1.24Ăat 4 KiB, 46 Îźs, where the offload is on the order of the serverâs own per-request work) and rises to 5.41Ăat 8 MiB (2.3 ms), approaching the GPUâs bandwidthâthe first ceiling, a property of the device. A latency-bound offload lifts that ceiling because its device is not saturated: with our real RSA-2048 remote signer (821 Îźs round-trip) the same Python server reaches 3.53Ăâbut not the order of magnitude a deep queue could in principle give, because the bottleneck moves to the second ceiling, the serverâs own overlap capacity (throughput and round-trip latency imply theasyncio/GIL executor keeps onlyâź6 requests truly in flight). Real overlap is thus bounded at both ends; across our servers and offloads it lands at 1.2â5.4Ă. Latency under offered load.Overlap is a latency win too. By keeping the CPU busy during offloads, the overlapping path holds both median and tail (p99) latency low up to roughly four Fine-Grained Computation Offload in Tens of Lines13 4K8K16K32K64K128K256K512K1M2M4M8M AES block size (offload weight) 1 2 3 4 5 speedup (Ă) no benefit (offload per-request work) Overlap pays with offload weight (real GPU AES, single event loop) speedup (async/sync) 10 2 10 3 single-op GPU latency (Îźs) GPU latency Figure 11. Prediction 2, measured (real GPU, idle): overlap pays only when the offload outweighs per-request CPU work. On a single-event-loop server, the same reroute gives 1.24Ăfor a light 4 KiB block (46 Îźs) and 5.41Ăat 8 MiB (2.3 ms), approaching the GPUâs bandwidth. Right axis: measured single-op GPU latency. 200400 offered load (K req/s) 10 2 10 3 10 4 10 5 p50 latency (Îźs, log) ~4Ă the load at the same latency median (p50) 200400 offered load (K req/s) 10 2 10 3 10 4 10 5 p99 latency (Îźs, log) tail (p99) Overlap holds low latency to ~4Ă the offered load (open-loop, Poisson arrivals) overlap (fibers)block (thread pool) Figure 12. Latency under load (open-loop Poisson arrivals over a controlled 20 Îźs emulated of- fload, the one emulated experiment in the paper, used so offered load can be swept precisely; runtime/openloop.csv). Blocking on the offload drives both median (p50, left) and tail (p99, right) latency up at its saturation point near 103K req/s, while overlapping holds low latency to roughly 4Ă that offered load (knee near 410K req/s) before its own knee. times the offered load that blocking can sustain before its knee (Figure 12). Fine-Grained Computation Offload in Tens of Lines14 modification to an existing server breadth of servers it applies to This paper: rerouting via existing concurrency (0138 lines, 10 server types) transparent threading libs rewrite in an async framework point offload integrations (QAT, SSLShader) our transparent runtime (one corner of the spectrum) Positioning: low modification AND broad coverage Figure 13. Positioning. Existing approaches are either low-effort but narrow (threading libraries, of which our own transparent runtime is one) or high-effort and still narrow (point integrations, async- framework rewrites). Rerouting through the serverâs own concurrency reaches the desirable region: little modification to existing servers, across many server types. 7 Discussion and Limitations When rerouting does not pay. The model doubles as a stop sign. When per-request CPU already rivals the offload (Prediction 2âthe same condition that raises the transparent run- timeâs third wall), rerouting buys complexity for nothing. And an operator whose accelerator is already saturated should buy device bandwidth, not rewire the server: overlap recovers wasted utilization but never manufactures throughput. Scope of the conflict detector.The conflict detector is a targeted safety net for the one pattern rerouting endangers, not a general transactional memory; shared state outside the monitored segments, or correctness requirements beyond last-writer-wins, need stronger machinery. 8 Related Work Figure 13 places this work among prior approaches, which either add the concurrencyânew frameworks, runtimes, and operating systemsâor hand-integrate one offload into one server; we instead reuse the concurrency existing servers already have, across the whole landscape of server architectures, and predict the cost of doing so. Threads, events, and coroutines.How to hide I/O latency cheaply is an old debate between an event-driven camp that structures the server as a state machine over an event loop at the cost of âstack rippingâ [Adya et al., 2002] (Flash [Pai et al., 1999], SEDA [Welsh et al., 2001]) and a user-level-thread camp that keeps the synchronous style and yields at blocking points (Capriccio [von Behren et al., 2003], State Threads [State Threads Library, 2009],libtask[Cox, Fine-Grained Computation Offload in Tens of Lines15 2005], core-aware Arachne [Qin et al., 2018], and language-level threads such as goroutines [The Go Authors, 2026] and Java virtual threads [OpenJDK, 2023]). Our fiber runtime is in the second lineage with a fast register-only switch [Li et al., 2023], but the contribution is not another threading library: it is to overlap an accelerator offload in servers built either way, quantify the modification cost across real servers, and map where the transparent form demonstrably stops. Microsecond-scale systems.A body of work rebuilds the OS or runtime to run microsecond- scale tasks efficiently: dataplane operating systems such as IX [Belay et al., 2014], work- stealing schedulers such as ZygOS [Prekas et al., 2017], and core-reallocating runtimes such as Shenango [Ousterhout et al., 2019], Caladan [Fried et al., 2020], and Demikernel [Zhang et al., 2021], typically built over kernel-bypass or asynchronous I/O (DPDK [DPDK Project, 2026], io_uring[Axboe, 2019]). They attack the same killer microsecond problem [Barroso et al., 2017] but demand a new stack and rewritten applications. We instead ask how little it costs to overlap one offload inside existing, unmodified servers; the directions are complementary. Full hardware offload.Another line moves the entire datapath onto hardware: KV-Direct [Li et al., 2017] runs a key-value store in the NIC, ClickNP [Li et al., 2016] compiles network functions to an FPGA, iPipe [Liu et al., 2019] offloads application logic onto SmartNICs (and must itself schedule the offloaded tasksâ granularity, the dual of our problem), and GPUnet [Kim et al., 2014] lets GPU code drive the network. These eliminate the CPUâs role and require rebuilding the application around the accelerator. Our target is the opposite and far more common case: the application stays on the CPU and must hide one fine-grained stepâs latency with almost no change. Offload integrations and async frameworks. Closer to us, specific systems overlap spe- cific offloadsâTLS on GPUs (SSLShader [Jang et al., 2011]), QAT engines in nginxâs async paths [Intel Corporation, 2026], and inference servers that batch GPU work (RedisAI [RedisAI, 2022], Clipper [Crankshaw et al., 2017])âbut each is a point integration tuned to one server and offload. Async frameworks (libevent [Provos and Mathewson, 2026], libuv [libuv con- tributors, 2026], Seastar [ScyllaDB, 2026], async/await) and proxy offload engines (HAProxy SPOE [HAProxy Technologies, 2026b], Envoyext_proc[Envoy Project, 2026]) make overlap the default, but only if the application is written in them from the start. We instead inject overlap into existing servers and predict the win and the cost across the landscape. Conflict detection. Dirty tracking via page protection, software distributed shared mem- ory [Amza et al., 1996], and transactional memory [Herlihy and Moss, 1993] all detect or prevent conflicting concurrent writes. Our detector borrows the page-protection technique but is lightweight and specialized to the one hazard rerouting introduces: a read-modify-write split across an offload. Symbol interposition.The fragility of interposing versioned glibc symbols is folklore among practitioners. Our condition-variable finding (Appendix B) documents a concrete, reproducible instance and its fix. Fine-Grained Computation Offload in Tens of Lines16 9 Conclusion What should the CPU do while it waits for the accelerator? It should overlap the offload with other requestsâand it does not need a new framework, runtime, or operating system to do so, because every server that serves concurrent requests already contains the machinery overlap requires. Rerouting the offload through that machinery takes tens of lines in off-the-shelf servers across every concurrency model (1.2â5.4Ăon real hardware), is predictable in advance from that model and the offloadâs weight, can occasionally be done with zero lines from outside the binary (17.3Ă, within a precisely characterized envelope), and stays correct with one targeted guard for the run-to-completion atomicity it suspends. The result is a practical recipe, and a map, for hiding fine-grained accelerator-offload latency in the servers that run online services today. Acknowledgements This paper began as a draft written during the authorâs internship at Microsoft Research in 2017. Nine years later, with the help of Pine Copilot and Claude Code, the author finally brought it to completion. The work was produced using Pine Copilotâs voice-directed whisper coding workflow [Pine AI, 2026], in which the author specifies, discusses, and reviews the work by voice while a coding agentâClaude Code with Claude Opus 4.8 and Claude Fable 5âcarries out the planning, coding, experiments, and paper writing. The author thanks BSQL Networking for hosting the NVIDIA RTX PRO 6000 GPU. References Atul Adya, Jon Howell, Marvin Theimer, William J. Bolosky, and John R. Douceur. Cooperative task management without manual stack management. In Proceedings of the USENIX Annual Technical Conference (ATC), 2002. Cristiana Amza, Alan L. Cox, Sandhya Dwarkadas, Pete Keleher, Honghui Lu, Ramakrishnan Rajamony, Weimin Yu, and Willy Zwaenepoel. TreadMarks: Shared memory computing on networks of workstations. IEEE Computer, 29(2):18â28, 1996. Apache Software Foundation. Apache HTTP server. https://httpd.apache.org/, 2026. Jens Axboe. Efficient IO with io_uring. https://kernel.dk/io_uring.pdf, 2019. Luiz Barroso, Mike Marty, David Patterson, and Parthasarathy Ranganathan. Attack of the killer microseconds. Communications of the ACM, 60(4):48â54, 2017. Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. IX: A protected dataplane operating system for high throughput and low latency. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 49â65, 2014. Russ Cox. libtask: A coroutine library for C and Unix. https://swtch.com/libtask/, 2005. Daniel Crankshaw, Xin Wang, Giulio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. Clipper: A low-latency online prediction serving system. In Proceedings of the 14th Fine-Grained Computation Offload in Tens of Lines17 USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 613â627, 2017. DPDK Project. DPDK: Data plane development kit. https://w.dpdk.org/, 2026. Ulrich Drepper. How to write shared libraries.https://w.akkadia.org/drepper/dsohowto. pdf, 2011. Envoy Project. External processing filter (ext_proc).https://w.envoyproxy.io/docs/ envoy/latest/configuration/http/http_filters/ext_proc_filter, 2026. F5/NGINX. nginx: Http and reverse proxy server. https://nginx.org/, 2026. Hubertus Franke, Rusty Russell, and Matthew Kirkwood. Fuss, futexes and furwocks: Fast userlevel locking in Linux. In Proceedings of the Ottawa Linux Symposium (OLS), pages 479â495, 2002. Joshua Fried, Zhenyuan Ruan, Amy Ousterhout, and Adam Belay. Caladan: Mitigating interference at microsecond timescales. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 281â297, 2020. HAProxy Technologies. HAProxy: The reliable, high-performance TCP/HTTP load balancer. https://w.haproxy.org/, 2026a. HAProxy Technologies. Stream processing offload engine (SPOE) and protocol (SPOP).https: //w.haproxy.com/documentation/, 2026b. Maurice Herlihy and J. Eliot B. Moss. Transactional memory: Architectural support for lock- free data structures. In Proceedings of the 20th Annual International Symposium on Computer Architecture (ISCA), pages 289â300, 1993. Intel Corporation. Intel quickassist technology (Intel QAT).https://w.intel.com/content/ w/us/en/architecture-and-technology/intel-quick-assist-technology-overview. html, 2026. Keon Jang, Sangjin Han, Seungyeop Han, Sue Moon, and KyoungSoo Park. SSLShader: Cheap SSL acceleration with commodity processors. In Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2011. Sangman Kim, Seonggu Huh, Xinya Zhang, Yige Hu, Amir Wated, Emmett Witchel, and Mark Silberstein. GPUnet: Networking abstractions for GPU programs. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 201â216, 2014. Bojie Li, Kun Tan, Layong (Larry) Luo, Yanqing Peng, Renqian Luo, Ningyi Xu, Yongqiang Xiong, Peng Cheng, and Enhong Chen. ClickNP: Highly flexible and high performance network processing with reconfigurable hardware. In Proceedings of the 2016 ACM SIGCOMM Conference, pages 1â14, 2016. Fine-Grained Computation Offload in Tens of Lines18 Bojie Li, Zhenyuan Ruan, Wencong Xiao, Yuanwei Lu, Yongqiang Xiong, Andrew Putnam, Enhong Chen, and Lintao Zhang. KV-Direct: High-performance in-memory key-value store with programmable NIC. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP), pages 137â152, 2017. Bojie Li, Zihao Xiang, Xiaoliang Wang, Han Ruan, Jingbin Zhou, and Kun Tan. FastWake: Revisiting host network stack for interrupt-mode RDMA. In Proceedings of the 7th Asia-Pacific Workshop on Networking (APNet), pages 1â7. ACM, 2023. doi: 10.1145/3600061.3600063. libuv contributors. libuv: Cross-platform asynchronous I/O. https://libuv.org/, 2026. Ming Liu, Tianyi Cui, Henry Schuh, Arvind Krishnamurthy, Simon Peter, and Karan Gupta. Offloading distributed applications onto SmartNICs using iPipe. In Proceedings of the 2019 ACM SIGCOMM Conference, pages 318â333, 2019. MariaDB Foundation. MariaDB server: The open source relational database.https://mariadb. org/, 2026. memcached. memcached: A distributed memory object caching system.https://memcached. org/, 2026. National Institute of Standards and Technology. Module-lattice-based key-encapsulation mechanism standard. Technical Report FIPS 203, NIST, 2024. NVIDIA Corporation. CUDA C++ programming guide: Asynchronous concurrent execution. https://docs.nvidia.com/cuda/cuda-c-programming-guide/, 2026. OpenJDK. JEP 444: Virtual threads (project Loom). https://openjdk.org/jeps/444, 2023. OpenJS Foundation. Node.js: A JavaScript runtime built on V8. https://nodejs.org/, 2026. OpenSSL Project. OpenSSL: Cryptography and SSL/TLS toolkit.https://w.openssl.org/, 2026. Amy Ousterhout, Joshua Fried, Jonathan Behrens, Adam Belay, and Hari Balakrishnan. Shenango: Achieving high CPU efficiency for latency-sensitive datacenter workloads. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 361â378, 2019. Vivek S. Pai, Peter Druschel, and Willy Zwaenepoel. Flash: An efficient and portable web server. In Proceedings of the USENIX Annual Technical Conference (ATC), 1999. PineAI.PineAI:Themostnaturalhuman-computerinterfaceis yourvoice.Blogpost,2026.URLhttps://w.19pine.ai/blog/ pine-ai-the-most-natural-human-computer-interface-is-your-voice.Accessed 2026-07-02. PostgreSQL Global Development Group. PostgreSQL: The worldâs most advanced open source relational database. https://w.postgresql.org/, 2026. Fine-Grained Computation Offload in Tens of Lines19 George Prekas, Marios Kogias, and Edouard Bugnion. ZygOS: Achieving low tail latency for microsecond-scale networked tasks. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP), pages 325â341, 2017. Niels Provos and Nick Mathewson. libevent: An event notification library.https://libevent. org/, 2026. Python Software Foundation. asyncio: Asynchronous I/O in CPython.https://docs.python. org/3/library/asyncio.html, 2026. Henry Qin, Qian Li, Jacqueline Speiser, Peter Kraft, and John Ousterhout. Arachne: Core-aware thread management. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 145â160, 2018. Redis Ltd. Redis: An in-memory data store. https://redis.io/, 2026. RedisAI. RedisAI: A redis module for serving tensors and executing deep learning models. https://oss.redis.com/redisai/, 2022. ScyllaDB. Seastar: A high-performance shared-nothing asynchronous C++ framework.https: //seastar.io/, 2026. State Threads Library. State threads library for internet applications.http://state-threads. sourceforge.net/, 2009. The Go Authors. The Go programming language: Goroutines and the scheduler.https: //go.dev/, 2026. Rob von Behren, Jeremy Condit, Feng Zhou, George C. Necula, and Eric Brewer. Capriccio: Scalable threads for internet services. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP), pages 268â281, 2003. Matt Welsh, David Culler, and Eric Brewer. SEDA: An architecture for well-conditioned, scalable internet services. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP), pages 230â243, 2001. Irene Zhang, Amanda Raybuck, Pratyush Patel, Kirk Olynyk, Jacob Nelson, Omar S. Navarro Leija, Ashlie Martinez, Jing Liu, Anna Kornfeld Simpson, Sujay Jayakar, Pedro Henrique Penna, Max Demoulin, Piali Choudhury, and Anirudh Badam. The demikernel datapath OS architecture for microsecond-scale datacenter systems. In Proceedings of the 28th ACM Symposium on Operating Systems Principles (SOSP), pages 195â211, 2021. A AES Block-Size Sweep (detailed values) Table 2 gives the full per-size measurements behind the offload-weight curve of Figure 11, all on a real GPU on an idle device (no co-tenant compute), driving a single-event-loop server (Pythonasynciowith a 32-thread executor) at 50 concurrent clients. The AES block size is swept over consecutive powers of two from 4 KiB (launch-bound, tens of microseconds) to 8 MiB (bandwidth-bound,âź2.3 ms). For each we report the measured single-op GPU latency Fine-Grained Computation Offload in Tens of Lines20 (one offload in flight), the synchronous and asynchronous throughput, and their ratio; §6 interprets the curve. block sizeGPU latency (Îźs)sync (req/s)async (req/s)speedup 4 KiB46.4375046671.24Ă 8 KiB47.0368846501.26Ă 16 KiB51.4365745731.25Ă 32 KiB60.3350143711.25Ă 64 KiB70.2332643551.31Ă 128 KiB92.7297042181.42Ă 256 KiB126.5270641281.53Ă 512 KiB191.8245444941.83Ă 1 MiB361.6153836442.37Ă 2 MiB653.696628582.96Ă 4 MiB1188.150619373.83Ă 8 MiB2258.622712285.41Ă Table 2.Real-GPU AES block-size sweep on a single-event-loop server (idle GPU). Source: transparent-runtime/apps/aes_blocksize_py_results.csv. B Interposition Obstacles for the Transparent Runtime Running stock binaries under theLD_PRELOADfiber runtime of §4 surfaced five obstacles below the architectural walls. Each was resolved once and is recorded here because any threading-interposing tool will meet them. The condition-variable versioning hazard. The sharpest obstacle is a symbol-versioning haz- ard [Drepper, 2011]: the glibc condition-variable functions carry two incompatible ABIs under one name (GLIBC_2.2.5andGLIBC_2.3.2), so a naive interposer that defines an unversioned pthread_cond_signalbreaks the linkerâs version-matched relocation and silently corrupts some binaries while sparing others: in our sweep it crashes MariaDB at startup with a wild pointer and is tolerated by Redis, memcached, nginx, and stunnel (Figure 14). Because a transparent runtime cannot know in advance which binaries are susceptible, version-matched interposition is mandatory: export the interposed symbols at their exact version via a linker version script and resolve the real ones with dlvsym. Four lesser obstacles. â˘Alternate I/O entry points. A server may do its socket I/O throughrecv/send/poll rather thanread/write, so every blocking entry point the application can reach must be interposed to yield the fiber. â˘Real-thread identity.pthread_selfmust return the genuine OS thread identity, because the C libraryâs stack-bounds logic relies on it; returning a per-fiber value breaks libc internals. â˘Carrier re-establishment afterfork. A daemon that forks drops the carrier thread in the child; the carrier must be re-established before any fiber can run. Fine-Grained Computation Offload in Tens of Lines21 CRASHOKOKOKOK naive (unversioned) OKOKOKOKOK versioned (our fix) MariaDBRedismemcached nginx stunnel Interposing pthread_cond_* without version-matching Figure 14. A reusable interposition hazard. A naive unversioned interposer of the glibc condition variable silently corrupts some binaries (MariaDB crashes at startup) while sparing others; version- matching the interposed symbols fixes all of them. â˘Priority inversion. Pinning the carrier to a real-time priority can invert priorities against a lock holder running at normal priority, stalling the carrier.