Paper deep dive

Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training

Andrew Geyko, Marius Mosbach, André Brinkmann

Year: 2026Venue: arXiv preprintArea: cs.DCType: PreprintEmbeddings: 89

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone. Prior work has measured individual training steps of such models on phones, but not complete training runs, and not whether adapters trained on the device improve personalization. We present the first systematic characterization of a multi-billion-parameter LLM fine-tuned on a mobile device, covering memory, per-step time, thermal behavior, and energy. An iPhone 17 Pro can fine-tune a 3B-parameter LLM to a typical user within one battery charge, and the resulting adapters improve personalization as much as adapters trained on a server. Sustained training throttles the phone to about half its initial throughput, and none of the pausing or burst schedules we tested recovers it. Nearly all of each training step is spent in the frozen base model, most of it in the backward pass, which nine of the ten other runtimes we audited do not accelerate. Apple's MLX had a kernel for it that was never dispatched and was incorrect, and our repair, now merged upstream, trains an adapter 1.47x faster on a third less energy. On-device fine-tuning is feasible on current phones, and making it efficient requires runtimes and operating systems to treat training as a first-class workload.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

86,054 characters extracted from source content.

Expand or collapse full text

Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training CCS: Computing methodologies Machine learningCCS: Computer systems organization Embedded systems Andrew Geyko Affiliation: Saarland University, Saarbrücken, Germany email: ange00008@uni-saarland.de , Marius Mosbach Affiliation: Saarland University, Saarbrücken, Germany email: marius.mosbach@uni-saarland.de and André Brinkmann Affiliation: Saarland University, Saarbrücken, Germany email: andre.brinkmann@uni-saarland.de © none Abstract. Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone. Prior work has measured individual training steps of such models on phones, but not complete training runs, and not whether adapters trained on the device improve personalization. We present the first systematic characterization of a multi-billion-parameter LLM fine-tuned on a mobile device, covering memory, per-step time, thermal behavior, and energy. An iPhone 17 Pro can fine-tune a 3B-parameter LLM to a typical user within one battery charge, and the resulting adapters improve personalization as much as adapters trained on a server. Sustained training throttles the phone to about half its initial throughput, and none of the pausing or burst schedules we tested recovers it. Nearly all of each training step is spent in the frozen base model, most of it in the backward pass, which nine of the ten other runtimes we audited do not accelerate. Apple’s MLX had a kernel for it that was never dispatched and was incorrect, and our repair, now merged upstream, trains an adapter 1.47× faster on a third less energy. On-device fine-tuning is feasible on current phones, and making it efficient requires runtimes and operating systems to treat training as a first-class workload. Keywords: on-device training, LoRA, quantization, mobile GPU, thermal throttling, energy, measurement study 1. Introduction Figure 1. One complete per-user adapter run on an iPhone 17 Pro under each backward kernel. LLMs are now a standard component of mobile operating systems, and both iOS and Android ship an on-device model (Li et al., 2025a; Gemini Team, 2023). The same phones hold their users’ messages and emails, from which a model could learn, for example, to draft replies in the user’s own style (Zhang et al., 2025b). However, the models deployed on phones today do not learn from this data. They are trained once on public data and ship with identical weights to every user, so any personalization is limited to the user data that prompt retrieval can place in context at the time of a request (Salemi et al., 2024b; Salemi et al., 2024a). Fine-tuning on the history instead encodes the user’s patterns in the model’s weights without lengthening the prompt, with a benefit that grows with the length of the history and adds to that of prompt retrieval (Tan et al., 2024; Salemi and Zamani, 2025). Training has traditionally run on servers, but user history is sensitive, and even the gradients and activations that federated and split learning send off the device can be used to reconstruct it (Zhu et al., 2019; He et al., 2019; Pasquini et al., 2021). In contrast, training on the device keeps the history, and everything computed from it, on the phone. We present the first systematic characterization of a multi-billion-parameter LLM fine-tuned on a mobile device. Fine-tuning a model on one user’s history is small by server standards, a few thousand training steps over a few hundred records. Full fine-tuning keeps a gradient and two optimizer values for every parameter and needs several times the memory of inference, but three techniques from server-side training bring this training within a phone’s memory: low-rank adaptation (LoRA) (Hu et al., 2021) trains a small adapter while the base model stays frozen, quantization (Dettmers et al., 2023) stores weights in a lower precision, and gradient checkpointing (Chen et al., 2016) recomputes activations instead of storing them. However, sustained training of models with billions of parameters has not yet been studied on a phone. Prior work has measured only individual training steps of such models on a phone (Song and Tang, 2025; Park et al., 2026a; Tam et al., 2026), and has fine-tuned models of only about 1B parameters (Geng et al., 2025; Wang et al., 2025; Wang et al., 2026a), split larger models across several mobile devices (Chen et al., 2025c), or trained without backpropagation (Peng et al., 2024; Gao et al., 2025). A history of a few hundred records takes hours to train, and over that time the phone throttles as it heats and drains its battery. Furthermore, models are trained in runtimes and operating systems designed for inference, where each generated token takes milliseconds and each weight matrix is read in a single direction. We measure per-user training of a 3B-parameter model at 4-bit precision on an iPhone 17 Pro, whose GPU cores include matrix units (Apple Inc., 2025). The measurements run from the GPU kernels of one training step to the battery drain of a complete run and cover memory, the time decomposition of a step, sustained thermal behavior, and energy. We complement these measurements with an audit of the quantized backward pass in ten other runtimes and an evaluation of the resulting adapters on a personalization benchmark. Figure 1shows two complete runs of the same adapter, one before and one after a kernel repair described below. In both, throughput falls within minutes as the phone throttles and stays low for the hours the run takes. None of the pausing or burst schedules we tested recovers the lost throughput. Over complete runs, training time and energy are linear in the number of training tokens, about 0.01 seconds and 0.04 joules per token, so the cost of an adapter can be predicted before a run starts. An adapter over a 405-record history costs about half a battery charge, and a 987-record history exhausts a full charge before training completes. Adapters trained on the phone improve personalization scores on top of prompt retrieval and match adapters trained on server GPUs. Although adapters are small, every training step still runs the input through the frozen model and propagates gradients back through it (Chen et al., 2025b; Sung et al., 2022; Tenison et al., 2026). On the phone, the frozen model takes nearly all of each step, most of it in the backward pass, which reads the quantized weights in the opposite direction from inference. MLX (Hannun et al., 2023), the runtime we use, contained a matrix-unit kernel for this direction that had never been dispatched and was incorrect. The repaired kernel trains an adapter 1.47× faster, on a third less energy, with an unchanged loss trajectory, and every other end-to-end measurement in this paper uses it. Because runtimes are designed for inference, only one of the ten other runtimes we audited, ExecuTorch (Nachin et al., 2026; PyTorch Contributors, 2026), provides an accelerated kernel for this direction, added concurrently with this work. This paper makes the following contributions. • The first measurements of complete multi-billion-parameter training runs on a phone, covering training time, sustained throttling, adapter configuration, and energy (Section 6), and the memory and per-step breakdowns they rest on (Section 4, Section 5). • A diagnosis of the quantized backward pass that exposed an incorrect, never-dispatched kernel in MLX, and an audit showing that only one of ten other runtimes provides an accelerated kernel for this path (Section 5). • An evaluation showing that adapters trained on the phone improve personalization benchmark scores on top of prompt retrieval and match adapters trained on server GPUs (Section 7). • Recommendations for runtimes and operating systems that support on-device training (Section 9). []Our code and measurement data are available at https://github.com/gordofreemo/mobile_LoRA_ft. Section 2introduces personalization by fine-tuning, the techniques that fit training on a phone, and the iOS execution environment, and Section 3 describes the experimental setup. Section 4 measures memory, Section 5 the time within a training step and the kernel repair, and Section 6 complete adapter runs. Section 7 evaluates the adapters, Section 8 discusses related work, Section 9 discusses implications and limitations, and Section 10 concludes. 2. Background We describe how fine-tuning personalizes an LLM, the techniques that bring this training within a phone’s memory, and the constraints iOS places on an application that trains, since these determine the cost of the workload we measure. 2.1. LLM Fine-Tuning for Personalization Personalization adapts a model to a user based on that user’s past behavior. LaMP (Salemi et al., 2024b) is an established benchmark to measure the ability of a model to personalize its responses to users. It consists of seven prediction tasks, each pairing a query with the user’s history of similar items, which differentiates users from one another. For example, the rating task provides a user review and the user’s earlier reviews with ratings, and asks the model to predict what rating the user would give a product. A model can use the history in two ways. Retrieval selects the few past records most similar to the new item and places them in context. Fine-tuning instead trains on the history, so that the user’s patterns are encoded in the model’s weights. Retrieval requires no training and adapts to new records at inference time. Fine-tuning requires training but can draw on the entire history and is not limited by the model’s context length. Prior work has found that the benefit of fine-tuning grows with the length of a user’s history and adds to that of retrieval (Tan et al., 2024; Salemi and Zamani, 2025). Personalized fine-tuning demands two things from a model. The model must first learn the task, which requires a diverse training set from many users, and it must learn about the individual, which requires only the records from that user. Tan et al. (2024) separate the two into successive stages, each trained as an adapter, a small set of additional weights that is trained while the base model stays frozen and that can be added into the base weights afterwards. A task adapter is trained on a corpus drawn from many users, and is then merged into the base weights so that the result behaves as an ordinary model that knows the task. A per-user adapter is then trained on top of that model from a single user’s history, with each past record serving as one training example. The task stage needs data that no single device holds and can run once on a server. The per-user stage needs only the data the device already has. This division makes on-device fine-tuning feasible, and the per-user stage is the run this paper measures. 2.2. On-Device Training Training requires more memory than inference. The activations of every block stay resident until the backward pass consumes them, and every trained parameter carries a gradient and two optimizer values, three extra values per parameter. For a model of several billion parameters, full fine-tuning at 16-bit precision therefore requires several times the memory of inference. In order to fit training on-device, three techniques adopted from server-side training are required. Low-rank Adaptation. LoRA (Hu et al., 2021) reduces the number of trained parameters by keeping every weight matrix W frozen and adding a smaller weight matrix BABA to it, where B and A are thin matrices whose shared inner dimension, the rank, is small. Only B and A receive gradient and optimizer states, whose parameter counts amount to a fraction of a percent of the original model. Every training step still does a full forward pass through the base model, and a full backward pass still runs through W to carry the gradient to previous blocks. Only the gradient with respect to W itself disappears. LoRA therefore removes nearly all gradient and optimizer memory but only about a third of the arithmetic, the products that compute the gradient of each W (Hu et al., 2021; Chen et al., 2025b). Quantization. Quantization reduces the precision of every parameter, lowering the size of the base model. Each weight is stored as a low-precision integer together with a scale and offset that reconstruct the value. The scale and offset are shared by a group of G consecutive weights along the input dimension of the layer, where G is the group size. At 4 bits with groups of 64, the model occupies roughly a fourth of the original memory footprint of the base 16-bit model. The grouping also fixes how weights are laid out in memory, a fact that becomes important when the backward pass reads those weights in a different order (Section 5.2). Gradient Checkpointing. Checkpointing reduces the activation memory at the cost of recomputation. Instead of storing every block’s activations through the forward pass, the training loop stores only the inputs of selected blocks and recomputes the rest during the backward pass (Chen et al., 2016). The selection can be made per block or per group of blocks, and coarser groups store less but recompute more at once. Checkpointing trades peak memory for one extra forward pass per block, so it lengthens the backward pass by a fixed amount. 2.3. The iOS Execution Environment Our measurements use MLX (Hannun et al., 2023), Apple’s array framework for Apple silicon, which provides the training loop, the quantized weight format and the Metal kernels that execute on the GPU. The A19 Pro’s GPU cores include matrix units, which Apple calls Neural Accelerators (Apple Inc., 2025), and MLX ships kernels that run the quantized matrix multiplication on them. We call these the accelerated kernels, against the generic kernels that run on the GPU’s ordinary arithmetic units. iOS also constrains the control an application has over its execution. Four policies impact sustained on-device training. Memory. iOS enforces a per-process memory limit and terminates any process that exceeds it. On our device the effective limit lies near 6 GB6\,GB of the 12 GB12\,GB installed. Our application holds the increased-memory-limit entitlement (Apple Inc., 2026c), without which iOS terminates training at 750 tokens. Termination occurs at any point peak memory exceeds the limit, which for training occurs during the first backward pass. This limit sets the longest sequence a phone can train on. Frequency and thermal control. Dynamic voltage and frequency scaling on iOS is managed entirely by the system. An application cannot select a frequency or pin work to a core, so it can trade speed for temperature only by idling or through an entitlement that has the system limit its burst performance and keep it at a consistent level (Apple Inc., 2026d). The only thermal information exposed is ProcessInfo.thermalState, a four-level indicator without a temperature or a rate of change. Under sustained load the SoC heats and the system reduces its frequency to hold the package within its thermal envelope. Inference workloads seldom encounter this regime, since a single request completes within seconds and the device cools between requests. A training run lasts hours and spends almost all of them throttled, so the throttling behavior determines sustained throughput. Background execution. iOS grants GPU access to background work only through BGContinuedProcessingTask with a GPU resource entitlement (Apple Inc., 2026a), and documents that not all devices support this resource. On our device the scheduler reports no GPU resource, and an ordinary background processing task loses GPU access within seconds of the application leaving the foreground. Training must therefore run as a foreground application with the display active, so the display draws power throughout every run. Energy accounting. iOS exposes battery state to applications only as UIDevice.batteryLevel, with no access to current, voltage or the power management unit, and our device reported it in steps of five percent of a charge, apart from a final 1%. This is the only energy signal an application can read at run time. 3. Experimental Setup The previous section described the per-user training stage and the mobile environment training executes in. This section describes our configuration for our measurements of mobile training performance. All measurements are taken on an iPhone 17 Pro (model A3523, rated battery energy 55.6 kJ55.6\,kJ (Apple Distribution International Limited, 2025)) running MLX (Hannun et al., 2023) via mlx-swift, its Swift interface, inside a normal iOS application. We measure on iOS for two reasons. First, the iPhone 17 Pro’s A19 Pro is among the first phone SoCs with matrix units in the GPU cores (Apple Inc., 2025). Second, MLX offers a mature array framework for it, with quantized kernels, automatic differentiation and lazy evaluation in one code base, so the training loop runs in the same runtime as inference. The cost is that iOS restricts applications more than Android, whose foreground services keep work running in the background (Google, 2026a). Workload: The workload we measure is the per-user training stage on the LaMP dataset, with our specific model configuration listed in Table 1. We train on histories from the rating-prediction task of LaMP, in which the model predicts the 1-to-5 rating of a user’s product review. They belong to five of the 100 users with the longest histories among those the task adapter was not trained on, and hold 405, 448, 500, 550 and 987 records, spanning the range of those 100 users. The base model we use is SmolLM3-3B (Bakouch et al., 2025), chosen because it trains within the memory constraints at the sequence lengths our histories need and performs competitively on standard model benchmarks. SmolLM3-3B is stored at 4-bit precision with a group size of 64. We attach adapters of rank 8 to the query and value projections of every block. A run trains only these adapters, on weights into which the task adapter was merged before quantization. A training example consists of a history record transformed into the format of the task. The prompt carries up to four of the user’s earlier records, retrieved using BM25 (Robertson and Zaragoza, 2009), a lexical ranking function that scores a record by the query words it shares. An example of a query is shown in Figure 2. A training run makes three passes over the user’s history at a batch size of 1, meaning each training step processes a single example, since activation memory grows with the number of examples in a step. Examples are truncated at 1024 tokens, the word pieces a model reads text in, the largest length fitting in memory and covering 92% of the records in these histories. Section 7.4 evaluates adapter quality on shorter movie-tagging histories instead, with a median of 36 records, which train in a median of 3 minutes. Measurement: One training step consists of preparing the batch, constructing the graph, the forward pass, the backward pass, the optimizer step, and readback. We time each phase by requesting a result at its boundary, forcing an evaluation. Because evaluations prevent the runtime from overlapping work, we also time each step without them. Across the six sequence lengths and both kernels the six phases sum to 1.02 to 1.08× the fused step. Throughput is training tokens processed per second. Where every step has the same length, as in Section 4, Section 5, Table 5 and the probes of Section 6.3, steps per second and tokens per second differ by a constant factor, and we report ratios or step times. Kernel-level breakdowns come from GPU captures around each phase, analyzed later in Xcode’s Metal debugger. The debugger bounds the resources a capture can replay, so a capture of the backward pass samples 12 of the 36 blocks. Peak memory is read from the runtime allocator at the end of each measurement window. Energy usage is derived from the battery gauge, which reports charge in 5% increments. We report energy net of an idle baseline, the 0.26 W0.26\,W the phone draws with the screen on and no training. Unplugged runs start at 85% charge, except the 987-record run and one 550-record run, which start from a full charge. Runs whose thermal history could influence one another are separated by at least one hour, longer than the thermal recovery time we measured. The device ran without a case, screen on at a fixed brightness, with Wi-Fi on and cellular off. Table 1. On-device training configuration. Model SmolLM3-3B (Bakouch et al., 2025) Precision 4-bit affine, group size 64 Adapter LoRA, rank 8, query and value Adapted blocks 36 Optimizer AdamW (Loshchilov and Hutter, 2019), lr 10−510^-5, weight decay 0.01 Batch size 1 Gradient checkpointing per block Maximum sequence length 1024 tokens Steps 3 passes over the history, i.e. 3×3× its record count [system] The following are examples of this user’s past activity. Use them to match this user’s preferences and writing style. - Review: "What a great book with all the twists and turn ..." -- the user rated it 5/5 - Review: "..." -- the user rated it 4/5 (up to 4 retrieved records) [user] What is the score of the following review on a scale of 1 to 5? just answer with 1, 2, 3, 4, or 5 without further explanation. review: This book was fantastic!! Such a touching story of growing up ... [assistant] 5 Figure 2. One training example from the rating-prediction task, as the model sees it (shortened). For this user the median example is about 530 tokens and the longest 1190. 4. Memory Analysis of Fine-Tuning Training a 3B-parameter model on mobile devices is possible only because adapters, quantization, and gradient checkpointing bring the memory requirements of fine-tuning within the device’s constraints. This section measures how, and in which combination, to use the aforementioned techniques. Table 2reports peak memory usage by sequence length. Without gradient checkpointing, training SmolLM3-3B peaks at 2.3 GB2.3\,GB at 32 tokens and 4.7 GB4.7\,GB at 256, and iOS terminates the process on the first backward pass at 512 tokens, a length that 89% of the records in our histories exceed. Per-block checkpointing lowers peak memory by 22% at 32 tokens and 47% at 256, and reaches 1024 tokens at 4.1 GB4.1\,GB. iOS does not specify the memory limits for a process, but the same sweep over Qwen3-8B (Qwen Team, 2025) locates the limit. With per-block checkpointing it trains at sequence lengths from 32 to 256 tokens, peaking at 5.6 GB5.6\,GB, before iOS terminates it at 512. Without checkpointing it fits within the memory constraints at only 32 tokens, using 5.7 GB5.7\,GB, which is the largest usage that survived in any configuration and places the limit between it and the smallest terminated run, near 6 GB6\,GB. Since only 0.3% of the records in our histories fit in 256 tokens, Qwen3-8B cannot train on the histories users hold. Gradient checkpointing granularity is another lever to control memory usage. Figure 3 plots memory usage against checkpoint granularity and shows that per-block checkpointing yields the lowest memory footprint. With L blocks checkpointed in groups of K, the forward pass retains L/KL/K group boundaries and the backward pass materializes one group’s internal activations, so peak memory is M⁡(K)=aL/K+bK+cM(K)=aL/K+bK+c. A least-squares fit to the five groupings plotted in Figure 3 gives a=8.1 MBa=$8.1\,MB$ per retained boundary, b=282 MBb=$282\,MB$ of internal activations per block and c=3.61 GBc=$3.61\,GB$ for everything else, at 1024 tokens, and reproduces each measurement within 3 MB3\,MB. The minimum lies at K=aL/b≈1K= aL/b≈ 1. A block’s internal activation size exceeds its group boundary size by roughly the depth of the model, so the L L grouping that Chen et al. (2016) derive for layers of equal memory cost does not apply, and iOS terminates every grouping coarser than six blocks. None of the coarser groupings that fit trains faster than per-block checkpointing. Checkpointing recomputes each block’s forward pass exactly once during the backward pass, and at matched sequence length it lowers throughput to 0.77–0.85× that of training without checkpointing. Table 2. Peak memory (MB) by sequence length. “Killed” marks the first length at which iOS terminated the process. Model Checkpointing 32 64 128 256 512 1024 SmolLM3-3B none 2328 2651 3312 4655 killed — SmolLM3-3B per block 1827 1919 2109 2449 3179 4125 Qwen3-8B none 5671 killed — — — — Qwen3-8B per block 4712 4834 5052 5560 killed — Figure 3. Peak memory against checkpoint granularity in blocks per checkpoint, 1024 tokens. iOS terminated every grouping of nine or more blocks. 5. Time Breakdown and the Backward Path This section breaks down a single training step and finds that its backward pass takes twice as long as its arithmetic predicts. The cause is a defect in a kernel that nothing dispatched, and an audit shows that the kernel the backward pass needs is missing from nine of the ten other runtimes. 5.1. Per-Kernel Breakdown A training step consists of six phases, as described in Section 3. Before the repair of Section 5.3, at 250 tokens and above, the backward pass accounts for 78% of a step, the forward pass 19–22%, and batch preparation, graph construction, optimizer updates, and readback together account for 1–3% of a step. The optimizer step is the only phase that deals exclusively with adapter state, and it makes up less than 2% of training time. Within the two passes, most of the time is spent within the frozen base model, as shown by the per-kernel breakdown in Figure 4. Quantized matrix multiplication over the frozen weights is 86% of the forward pass and 90% of the backward pass. The adapter’s low-rank multiplications are 3% and 2%, respectively. Attention, normalization and rotary embedding together are under 1% of either pass. Thus, the time spent training at these sequence lengths is driven by the weight matrices, not attention. Adding together the adapter’s forward and backward products with the optimizer update gives 0.15 s0.15\,s out of a 3.55 s3.55\,s step at 250 tokens, roughly 4%. The remaining 96% of the time is spent on the frozen model, and quantized matrix multiplication alone takes 87% of the step. Parameter-efficient fine-tuning on mobile devices therefore reduces the memory of a training step, but concentrates the work within the frozen model. We note that the backward pass is more expensive than it should be. The backward pass computes one input-gradient per block plus one forward pass recomputation due to checkpointing, so arithmetic alone predicts the backward pass to take twice as long as the forward pass. At 250 tokens and above it takes 3.5–4.2× as long, and up to 4.4× at 100 tokens. Disabling checkpointing at 100 to 375 tokens, where training fits without it, separates the two terms. Recomputation costs the time of 0.7 to 0.9 forward passes, and the input-gradient products cost 3.4 to 3.6. The excess time is concentrated in the quantized matrix multiplication of the backward pass. Figure 4. Per-kernel share of GPU time within each phase of a training step before the kernel repair, at 250 tokens and at 500 for the optimizer update. affine_qmm_t and affine_qmm_n are the quantized products of the forward and backward directions, and affine_qmm_t also appears in the backward pass as checkpoint recomputation. 5.2. Backward Pass Diagnosis The forward and backward pass multiply by the same frozen weight matrices, so the excess time lies in how the backward pass reads them. A quantized weight stores a 4-bit code together with a scale and offset shared by a group of G=64G=64 consecutive weights along the input dimension of the layer. For a linear layer with input X∈ℝM×NX ^M× N and weights W∈ℝN×KW ^N× K, the forward pass Y=XWY=XW reads N weights down one column of W, which is N/GN/G whole groups. Writing g for the group containing index n and q for the stored codes, Ym,k=∑g(sg,k∑n∈gXm,nqn,k+βg,k∑n∈gXm,n).Y_m,k= _g (s_g,k _n∈ gX_m,n\,q_n,k+ _g,k _n∈ gX_m,n ). The inner sums run over integer codes, and the scale is applied once per group of 64 products. The backward pass computes the input gradient ∂X=∂YW⊤∂ X=∂ Y\,W and reads W the other way, as Figure 5 shows. One element ∂Xm,n=∑k∂Ym,kwn,k∂ X_m,n= _k∂ Y_m,k\,w_n,k reads one row of W, whose weights all lie in one group but carry a different scale sg,ks_g,k in every column, so the scale no longer factors out of the sum over k. It factors out along n instead, since the 64 rows of a group share their scales, so a kernel for this contraction scales a tile of ∂Y∂ Y once per group and reuses it across the group’s rows. The arithmetic is the same as in the forward pass, but the codes are packed along N, so this kernel reads them across packed words and needs a tiling of its own. The backward pass therefore needs a second kernel, and MLX has one. An accelerated kernel for this contraction exists in the source and is compiled into every binary, but no code path dispatches it. The backward pass instead runs a generic Metal kernel on the GPU’s ordinary arithmetic units, which makes it 1.65 to 2.13 times slower than with the accelerated kernel (Section 5.3). q11q_11q12q_12q13q_13q21q_21q22q_22q23q_23q31q_31q32q_32q33q_33q41q_41q42q_42q43q_43s0,1s_0,1β0,1 _0,1s1,1s_1,1β1,1 _1,1(a) forward: Ym,1Y_m,1, along NNone (s,β)(s,β) per groupq11q_11q12q_12q13q_13q21q_21q22q_22q23q_23q31q_31q32q_32q33q_33q41q_41q42q_42q43q_43s0,1s_0,1β0,1 _0,1s0,2s_0,2β0,2 _0,2s0,3s_0,3β0,3 _0,3(b) backward: ∂Xm,1∂ X_m,1, along KKone (s,β)(s,β) per group,shared across rows Figure 5. Group layout and the two contraction directions over the same stored W. Fill color identifies the group, and the outline marks the slice one output element reads. Drawn with N=4N=4, K=3K=3, G=2G=2, whereas G=64G=64 in our configuration. 5.3. Repairing the Backward Kernel We found two defects in the accelerated kernel at our group size of 64. It indexed the stored weights and scales with offsets computed for the transposed layout, and it did not bound its tile loads, so it read and wrote out of bounds whenever a matrix’s row count was not a multiple of the tile height. The repair corrects the offsets and leading dimension passed to the tile loader, bounds the loads as the forward kernel does, and dispatches the kernel in place of the generic one. []11 1 The repair was merged into MLX as pull request #4051 (https://github.com/ml-explore/mlx/pull/4051), and MLX dispatches the repaired kernel by default since release 0.32.3. The kernel entered MLX together with its support for the matrix units and was unchanged until this work. MLX’s quantized linear layer computes only the transposed product, so the forward direction was wired to the matrix units, while the backward direction, which training needs at every step, was not. Every defect reported against these kernels before ours concerned an inference path. We first alternate the generic and repaired kernels between successive steps at six sequence lengths from 50 to 1000 tokens, so that both see the same thermal state. The backward pass runs 1.65 to 2.13× faster and the whole step 1.45 to 1.68× faster, 1.55× on average per step. Trained unplugged from 85% charge with the same batch order under each kernel, a complete adapter over the 405-record history takes 2.55 h2.55\,h under the generic kernel and 1.74 h1.74\,h under the repaired one, a factor of 1.47, and draws 39.3 kJ39.3\,kJ instead of 26.2 kJ26.2\,kJ net of the idle baseline. Across the six lengths the backward pass now takes 1.8 to 2.2 times the forward pass, against 3.4 to 4.4 before the repair, so the generic kernel accounts for the whole excess over the factor of two that its arithmetic predicts. The repaired kernel agrees bit for bit with a dense matrix multiplication over the dequantized weights on the shapes where that reference runs on the matrix units, and to within 10−310^-3 relative error elsewhere. The two runs’ loss trajectories coincide, with per-report losses correlated at 0.9999 and mean loss over the final 50 steps of 0.7911 under the generic kernel and 0.7820 under the repaired one. The adapter’s share of the training step rises from 4% to 7%, with the frozen base still dominating. 5.4. ML Runtime Audit Every runtime that stores quantized weights needs this second kernel to compute the backward contraction without dequantizing the weights. We examined the source code of eleven inference and training runtimes, as it stood between 17 and 20 August 2026, and determined, for each, whether an accelerated quantized matrix multiplication exists for that contraction and whether a training loop dispatches it. We count a kernel as accelerated if it is a GPU kernel tiled for this contraction over the packed low-bit weights that, on a GPU with matrix units, runs on them, which excludes generic kernels such as the one MLX dispatched before the repair. Table 3shows the results. Six runtimes have no quantized backward pass at all, and where they support training they do so on floating-point weights. Three compute the input gradient by dequantizing the weights for a dense multiplication, ONNX Runtime and bitsandbytes (bitsandbytes Foundation, 2026) over the whole matrix and llama.cpp row by row on the CPU. On our device, dequantizing each weight matrix for a dense multiplication on the matrix units makes the backward pass 12 to 44% slower than the repaired kernel at 250 to 1024 tokens and raises peak memory by 1.1 to 1.5 GB1.5\,GB. One runtime, ExecuTorch (Nachin et al., 2026), added an accelerated backward kernel for quantized weights to its Vulkan and WebGPU backends in July 2026 (PyTorch Contributors, 2026), concurrently with this work. MLX was the only other runtime in which such a kernel existed, and it was never dispatched and incorrect until the repair. For inference, all eleven store quantized weights in the layout the forward pass reads, and all eleven have optimized that read. The contraction that runs the other way is needed only by training, which on-device runtimes built for inference have had little reason to accelerate. Table 3. Input-gradient path over quantized weights in eleven runtimes. Runtime Accelerated kernel Training path ExecuTorch (Nachin et al., 2026) yes accelerated kernel MLX (Hannun et al., 2023) yes, not dispatched generic kernel ONNX Runtime (ONNX Runtime developers, 2021) no dequantize, dense bitsandbytes (bitsandbytes Foundation, 2026) no dequantize, dense llama.cpp (Gerganov et al., 2023) no dequantize per row, CPU MNN (Jiang et al., 2020) no float weights only LiteRT (Google AI Edge, 2024) no float weights only TVM (Chen et al., 2018) no float weights only Core ML (Apple Inc., 2026e) no float weights only MLC-LLM (MLC team, 2025) no no training ncnn (Tencent, 2017) no no training 6. End-to-End Adapter Training Prior work has measured single training steps of multi-billion-parameter models on phones, but a per-user adapter takes thousands of steps, over which the phone throttles and drains its battery. We therefore train complete adapters on the repaired runtime over user histories of 405 to 987 records, and measure their training time (Section 6.1), the cost of the adapter configuration (Section 6.2), throughput under sustained load (Section 6.3), and energy (Section 6.4). 6.1. Time Because attention takes under 1% of a step at these lengths (Section 5.1), the cost of a run should depend on the number of training tokens it processes rather than on the number of records. Table 4 lists the five end-to-end runs ordered by token counts. Wall-clock time is linear in tokens. We fit training time as t=0.0112s⋅tokens+13st=0.0112\,s·tokens+13\,s, which has r=0.998r=0.998 and no residual above 6%, against r=0.931r=0.931 and a residual of 35% for a fit on record count. The 448-record history takes longer than the 500- and 550-record histories because its records are longer. A run’s duration can therefore be estimated from its token count before it starts, and the per-token rate of 0.011 to 0.012 seconds is the same across histories. Repeating the 405-record run three times gave 6872, 6948 and 6616 seconds, a spread of 4.9%. The three runs agree within 1% on their cold-start rate, the rate over their first ten steps, and differ in the level their throughput settles to under sustained load. The same history also trains faster unplugged. One run on battery power took 1.74 h1.74\,h, 5 to 10% less than the plugged runs, with a slightly slower cold start and a faster plateau, so the difference arises under sustained load. Table 4. Complete adapter training runs, plugged in, ordered by token count. Tokens count all passes over the history. Records Tokens (M) Time (s) s/token 405405 0.640.64 68126812 0.01060.0106 550550 1.111.11 12 44312\,443 0.01120.0112 500500 1.161.16 13 08013\,080 0.01130.0113 448448 1.271.27 14 92414\,924 0.01180.0118 987987 2.352.35 26 09926\,099 0.01110.0111 6.2. Adapter Configuration Our recipe fixes three properties of the adapter: its rank, the projections it attaches to, and the blocks it covers. Each is a knob to trade model quality against cost, and we seek to find which of them cost depends on. Table 5 varies each on a device heat-soaked by 45 minutes of continuous training and reports throughput, FLOP count, and memory usage relative to the original recipe (first row), with the depth rows compared against the recipe measured in the same session. Each cell is the mean of three measurements. Raising rank from 8 to 32 on the same two projections costs 8 to 10% of throughput while quadrupling the adapter’s parameters. The low-rank products add little arithmetic, and the number of kernels launched stays fixed. Changing which projections have adapters matters more, as each adds an extra kernel launch. Adapting all seven linear projections of a block instead of only the query and value projections cost 17 to 27% of throughput, and rank 64 on all seven projections at 512 tokens degrades throughput by 35% with 1.3 GB1.3\,GB of additional memory used. Even then, the adapter’s forward products are 4.3% of the frozen model’s arithmetic at 512 tokens. Rather, adapters of rank 8 or 64 fail to fill the GPU’s matrix units, resulting in low-rank products running an order of magnitude less efficiently per operation than the quantized base matrix multiplication. Adapter depth changes the amount of computation done, since backpropagation stops at the first block that holds a trainable parameter, as in layer freezing (Rücklé et al., 2021; Wang et al., 2023). Every block before the first trainable parameter runs its forward pass and nothing else. Adapting the last 18 of 36 blocks takes 0.60× the time of adapting all 36 at 512 tokens and 0.57× at 1024. Peak memory falls by at most 6%, since the frozen weights and the checkpointed activations of every block stay resident whether or not the block is adapted. Rank, projections, and depth also determine what the adapter can learn. We measured this for depth on a server, at 4-bit precision, with the per-user movie-tagging adapters of Section 7.3, over the 100 users and 3,302 queries evaluated there. Adapters on the last 24 blocks reach an accuracy of 0.476, against 0.488 on all 36, retaining 91% of the gain over the task adapter across queries and 66% of the mean gain per user. Adapters on the last 18 blocks reach 0.472, retaining 88% and 64%. Adapting the last 24 blocks raises throughput on the phone by 34 to 45% in Table 5 at a cost of 1.1 accuracy points. The rest of the paper keeps the adapter shape of the per-user recipe of Tan et al. (2024), so that its measurements describe the published adapter. Table 5. Cost of the adapter configuration at the throttled plateau. Adapter Throughput vs. recipe Peak Rank Proj. Blocks FLOPs 512 tok 1024 tok (MB) 8 q, v 36 0.1% — — 4185 32 q, v 36 0.3% −8%-8\% −10%-10\% 4252 8 all 7 36 0.5% −26%-26\% −17%-17\% 4341 32 all 7 36 2.1% −27%-27\% −18%-18\% 4859 64 all 7 36 4.3% −35%-35\% −19%-19\% 5552 8 q, v 30 0.1% +15%+15\% +18%+18\% 4009 8 q, v 24 0.1% +34%+34\% +45%+45\% 4077 8 q, v 18 0.1% +67%+67\% +77%+77\% 4040 8 q, v 12 0.1% +106%+106\% +134%+134\% 4020 8 q, v 6 0.1% +171%+171\% +243%+243\% 3927 6.3. Sustained Training Every measured training run spends most of its time in a throttled thermal state. Every one starts at a cold-start rate and loses about half of it, most of that within minutes, which is why a per-token rate measured in a run’s first minutes overstates what the run will achieve by about a factor of two. Figure 6shows the throughput of the five complete runs in training tokens per second. Whatever history it trains on, every plugged-in run settles to between 83 and 89 tokens per second, about half of the 166 tokens per second of a cold start, and none recovers for the rest of the run. A plateau that does not depend on the history is what makes the time of a run linear in its tokens (Section 6.1). Half of the throughput loss occurs within about six minutes, and throughput comes within 5% of its plateau after 32 to 38 minutes. The pattern is consistent with the SoC reaching its temperature limit and iOS lowering the clock frequency to hold it there, and the plateau is the rate the device sustains at that limit. Throttling slows all phases of a step alike. After a 45-minute soak, a measurement of the 500-token step that forces evaluation at every phase boundary (Section 3), and so idles the GPU between phases, finds it 1.41× slower than its cold rate, and the share of each of its six phases moves by at most 1.5 percentage points, so the breakdown of Section 5 holds throughout a run. Continuous training, which does not idle, is throttled more deeply, to 1.9 to 2.0× its cold-start step time in the runs of Figure 6 and to 2.17× in the session of Figure 7. Once training stops, single-step throughput recovers within minutes, but iOS’s thermal signal does not track it. Figure 7 probes throughput with a single step every two minutes after an hour of continuous training at 500 tokens that had slowed the step by 2.17× from its cold rate. Between probes the device is idle, so this is the recovery of an idle device. Throughput regains half of its loss within four minutes and 95% within six, the resolution of probes two minutes apart. The indicator ProcessInfo.thermalState, the only thermal information iOS exposes, does not return to nominal until 26 minutes have passed, more than four times longer than throughput takes to recover. Since throughput recovers within minutes, an application could try to train in bursts or to pace its steps and spend part of the run cool. We tested both on a heat-soaked device in thirteen interleaved twenty-minute blocks, alternating continuous training with a one-second pause after every step, a three-second pause, and ten minutes of training followed by two minutes of rest. Every schedule completed fewer steps per hour than continuous training, ranging from 0.91 to 0.92× with the one-second pause, 0.78 to 0.79× with the three-second pause, to 0.88 to 0.89× in bursts. Pausing does cool the device, as the three-second pause shortens the active step from 5.2 to 3.6 s3.6\,s, but each second of pause buys back about half a second of compute, and a burst that starts after two minutes of rest returns to the continuous step time within its first minutes. Training already runs at the power the chassis can dissipate. If power grows faster than linearly with frequency, as it does under voltage scaling, a burst at a higher frequency does less work per joule than continuous training at the throttled one, so time spent idle is not recovered. Figure 6. Throughput of the five complete adapter runs of Table 4 against wall-clock time. Faint traces are per-window values over ten steps, bold traces a centered rolling mean. Figure 7. Throughput recovery after an hour of continuous training, probed with a 500-token step every two minutes and normalized to the cold rate. Color is the value of ProcessInfo.thermalState at each probe. 6.4. Energy We repeated three of the histories unplugged to measure the energy of a complete adapter. Table 6reports the four runs. The 405-record history, started at 85%, completed after drawing 26.2 kJ26.2\,kJ net of the idle baseline, 47% of a charge. The 550-record history, also started at 85%, shut the device down at step 1590 of 1650 after drawing 43.9 kJ43.9\,kJ, 79% of a charge, and a second attempt shut down at step 1630. Started from a full charge, the same history completed after drawing 44.4 kJ44.4\,kJ, 80% of a charge. The 987-record history, started from a full charge, shut the device down at step 1640 of 2961 after drawing 51.6 kJ51.6\,kJ, 93% of the rated capacity. A full charge therefore gives training about 52 kJ52\,kJ, and the device shuts down at a reported 1%. Net power during training is 4.0 to 4.2 W4.2\,W across runs that differ in user history, duration and starting charge, and the generic kernel of Section 5.3 draws a similar 4.3 W4.3\,W, so the repair saves energy in proportion to the time it saves. With power constant, energy per token is constant too, 0.04 joules in every run, and the token count that predicts a run’s time in Section 6.1 predicts its energy as well. A full charge trains about 1.3 million training tokens. An adapter trained with our recipe thus costs about 0.01 seconds and 0.04 joules per training token, and three times that per token of history, over which the recipe makes three passes. Table 6. Energy to train one adapter unplugged, net of the idle baseline. Start Net % Net Steps Records % Hours kJ charge J/token W done 405405 8585 1.741.74 26.226.2 47.147.1 0.040.04 4.24.2 1215/1215 550550 8585 3.013.01 43.943.9 78.978.9 0.040.04 4.14.1 1590/1650 550550 100100 3.073.07 44.444.4 79.879.8 0.040.04 4.04.0 1650/1650 987987 100100 3.633.63 51.651.6 92.992.9 0.040.04 4.04.0 1640/2961 7. Personalization Quality This section evaluates the task adapter against prompt retrieval (Section 7.2), measures what a per-user adapter adds to the task adapter (Section 7.3), and repeats the per-user experiment with training and inference on the phone (Section 7.4). 7.1. Setup We use the seven LaMP (Salemi et al., 2024b) tasks selected by Tan et al. (2024), four classification tasks (citation identification, movie tagging, news categorization, rating prediction) and three generation tasks (news headline, scholarly title, and tweet paraphrase). Classification tasks are scored using accuracy, with mean absolute error (MAE) also reported for rating prediction, while generation tasks are scored using ROUGE-1 (Lin, 2004), which rates from 0 to 1 how many words the generated text shares with the text the user actually wrote. A task adapter is trained once on many users’ records to teach the model a task, and a per-user adapter is then trained on top of it from one user’s history (Section 2.1). The benchmark’s original test split contains one or two queries per user for most tasks, enough to measure whether a task adapter helps users in general. To measure a per-user adapter, we follow Tan et al. (2024) and evaluate on the users with the longest histories per task, who on three of the seven tasks contribute tens to hundreds of queries each. 7.2. Fine-Tuning vs. Retrieval We compare four ways of serving a user, including whether to fine-tune or not, and whether to use retrieval or not. The base model with no retrieval is the baseline. Retrieval places the four most relevant records in the prompt and requires no training. One task adapter is trained per task on the records of the training split users, so every evaluated user is unseen and served with either no retrieved history or with the four retrieved records. Results are shown in Table 7. Without any retrieved records, the task adapter outperforms the base model with retrieval on six of the seven tasks, by 3 to 19 points, with prompts two to five times shorter. The largest gain is on movie tagging, where accuracy rises from 0.278 with retrieval to 0.468 with the adapter. The exception is citation identification, in which the model must decide which of two references the user cited and the only signal is the user’s past papers. On this task the base model scores 0.500, exactly the same as randomly guessing, and the adapter alone reaches 0.536, below the 0.549 of retrieval. Fine-tuning and retrieval are complementary, and the adapter with retrieval is best on every task. Adding retrieval to the adapter raises citation identification from 0.536 to 0.716 and the other six tasks by 2 to 10 points. Retrieval on top of an adapter therefore trades a longer prompt and its computation for further quality. Table 7. Benchmark-wide evaluation on the LaMP test split at 4-bit precision, where each test item is one user. Retr. places the four most similar records in the prompt, and Adapter is the task adapter. Δ columns are relative to Base+Retr. Task Metric n Base Base+Retr. Adapter Adapter+Retr. Citation identification Acc↑ 2500 0.5000.500 0.5490.549 0.5360.536 −0.012-0.012 0.716 +0.167+0.167 Movie tagging Acc↑ 870 0.1360.136 0.2780.278 0.4680.468 +0.190+0.190 0.547 +0.269+0.269 News categorization Acc↑ 1274 0.2560.256 0.5420.542 0.6460.646 +0.104+0.104 0.741 +0.199+0.199 Rating prediction Acc↑ 2500 0.4370.437 0.6480.648 0.7120.712 +0.064+0.064 0.784 +0.136+0.136 MAE↓ 2500 0.6840.684 0.4420.442 0.3310.331 −0.111-0.111 0.243 −0.200-0.200 News headline ROUGE-1↑ 2376 0.1360.136 0.1480.148 0.1760.176 +0.028+0.028 0.211 +0.063+0.063 Scholarly title ROUGE-1↑ 2500 0.3310.331 0.3360.336 0.4550.455 +0.118+0.118 0.482 +0.146+0.146 Tweet paraphrase ROUGE-1↑ 1496 0.3950.395 0.4140.414 0.5160.516 +0.102+0.102 0.535 +0.120+0.120 7.3. Per-User Adaptation The task adapter in the previous section learns the structure of a task and nothing about the user. From the user’s own records, a per-user adapter is trained on top of it. Our second question is whether this adapter adds anything. Measuring this requires many queries from the same user, so we use the long-history users of Tan et al. (2024) and reproduce their pipeline, where each user’s most recent records are the test queries and the earlier ones form the history the adapter is trained on. Their pipeline uses its own task adapters, prompt format and a single retrieved record, so its scores are not comparable with those of Table 7. These comparisons run on a server at 4-bit, the precision the phone trains in. Table 8shows the results. Δ is the gain over all queries, which weights each user by the number of queries they contribute, and Δu _u is the mean of per-user gains, which weights every user equally. The per-user adapter improves the score on five of the seven tasks, but the gain is large only on movie tagging, where accuracy rises from 0.361 to 0.488, a Δ of 12.7 points and a Δu _u of 3.0. On the other two tasks with tens of queries per user the gain is below one point, with news categorization rising from 0.805 to 0.811 and news headline generation from 0.158 to 0.159. The four remaining tasks contribute about one query per user, and among them the per-user adapter lowers the MAE of rating prediction from 0.223 to 0.205 and raises citation identification by 1.5 points per user, while tweet paraphrase and scholarly title drop by less than one point. To check that the gain comes from the user’s own history and not just from the extra training, we also trained each user’s adapter on another user’s history instead of their own. That adapter adds 1.2 points over all queries and 1.0 per user, against 12.7 and 3.0 for the user’s own history, so most of the movie-tagging gain is specific to the user. Table 8. Per-user evaluation on the long-history users of Tan et al. (2024) at 4-bit precision, with the same queries in both arms. Task is a task adapter trained with their released code and evaluated with one retrieved record, and +User adds a per-user adapter trained with the recipe of Table 1. Task Metric Users Test q/user Task +User _u News categorization Acc↑ 49 123 0.8050.805 0.811 +0.006+0.006 +0.012+0.012 News headline ROUGE-1↑ 100 67 0.1580.158 0.159 +0.001+0.001 +0.001+0.001 Movie tagging Acc↑ 100 33 0.3610.361 0.488 +0.127+0.127 +0.030+0.030 Movie tagging, on phone Acc↑ 100 33 0.3660.366 0.486 +0.120+0.120 +0.027+0.027 Citation identification Acc↑ 100 1.2 0.634 0.634 0.0000.000 +0.015+0.015 Rating prediction MAE↓ 101 1.1 0.2230.223 0.205 −0.018-0.018 −0.025-0.025 Tweet paraphrase ROUGE-1↑ 100 1.1 0.535 0.5300.530 −0.005-0.005 −0.003-0.003 Scholarly title ROUGE-1↑ 101 1.1 0.494 0.4860.486 −0.008-0.008 −0.006-0.006 7.4. On-Device Adapters The per-user results of Section 7.3 were measured on a server, which uses the same 4-bit weights as the phone but different kernels and higher-precision arithmetic. To validate them, we repeated the movie tagging experiment entirely on the device, with the per-user adapter trained on the phone and all predictions generated there. Table 8lists the result as the “on phone” row, directly below the server’s movie-tagging row, over the same 100 users and 3,302 test queries. The phone-trained and the server-trained adapter agree within one accuracy point and differ on 7.8% of individual predictions. The per-user gain on the device (+12.0 over all queries, +2.7 per user) is close to that on the server (+12.7, +3.0). Training the 100 adapters took a median of 3 minutes per user, on histories of 21 to 774 records with a median of 36. A linear fit of their training times against tokens has a slope within 1% of the per-token rate of Section 6.1. The per-user gain grows with history length. Grouped by history length, users with fewer than 100 records gain 0 to 4 points on average over the task adapter, and the eight users with 100 or more gain 17. The difference between the on-device and the server-trained adapter does not vary with history length, so on-device training reproduces server training for short and long histories alike. 8. Related Work Profiling LLMs on mobile devices. MELT (Laskaridis et al., 2024) characterized on-device LLM inference across Android, iOS and Jetson, measuring throughput, memory, energy and thermal behavior. HeteroLLM (Chen et al., 2025a) and Hao et al. (2026) profile mobile GPUs and NPUs during inference and schedule work onto the matrix units they find idle. Thermal throttling of sustained inference has been measured on phones before (Laskaridis et al., 2024; Tummalapalli et al., 2026), and energy has been reduced through frequency selection (Zou et al., 2026; Zhang et al., 2025a) and workload-level knobs such as speculative decoding (Yang and Xia, 2026) for inference, and through frequency selection for fine-tuning on an embedded GPU (Park et al., 2026b). Our study asks the same questions of training on a platform that exposes no frequency control to applications. Training on mobile devices. PockEngine (Zhu et al., 2023) derives a training graph at compile time and prunes the backward pass. Sage, Melon and Mandheling (Gim and Ko, 2022; Wang et al., 2022; Xu et al., 2022) address the memory wall and integer training on mobile hardware. For LLMs, Song and Tang (2025) fit a 4B model into an iPhone’s memory with lazy weight loading and checkpointing, but measure only individual training steps on the phone, at 256 tokens, and evaluate training quality in a server-side simulation. MeSP (Park et al., 2026a) halves MeBP’s memory, again measuring single steps, on an iPhone 17 Pro with MLX. MobileFineTuner (Geng et al., 2025) trains models up to 1B on Android with a native stack and estimates energy from the platform’s power model, FBLayout (Tam et al., 2026) tunes memory layout for both passes on mobile GPUs, and EdgeTune (Wang et al., 2026a), noting that LoRA leaves about two-thirds of the fine-tuning arithmetic in the frozen backbone, removes low-yield adapter paths to reduce it on models of up to 1.1B parameters. Like server-side breakdowns (Zhu et al., 2026), our breakdown separates the adapter from the frozen base, and for a 3B model on a phone it places the adapter at 4–7% of the step, which bounds what adapter-side optimization can return, and identifies adapter depth as the placement choice that moves compute. PocketLLM (Peng et al., 2024) and MobiZO (Gao et al., 2025) train on the device without gradients through derivative-free optimization (Malladi et al., 2023), and MobiLLM and its successor (Li et al., 2025b; Yang et al., 2025) keep only the frozen forward pass on the device and train a side network on a server. CDCDA-PLM (Zhong et al., 2025) fine-tunes a per-user adapter on cloud-generated synthetic data, but runs the training on a workstation GPU and deploys the result to the device. We show that the full backward pass is affordable on the phone at 3B once its kernel is correct. Quantized Training. QLoRA (Dettmers et al., 2023) backpropagates through a frozen 4-bit base into 16-bit adapters. Its reference implementation (bitsandbytes Foundation, 2026) runs a fused 4-bit kernel in the forward pass for inputs of up to 1,536 tokens and dequantizes the weight to a dense matrix for the backward pass, so the reference implementation itself accelerates only the direction that inference needs. GPTQ and AWQ (Frantar et al., 2022; Lin et al., 2024) quantize for inference and fix the layout that makes the forward contraction cheap. The runtimes we audited in §5.4 inherit that layout. ggml (Gerganov et al., 2022), the tensor library of llama.cpp, has no mobile GPU kernel for the backward contraction over packed weights and dequantizes the matrix on the CPU once per gradient row instead. Concurrently with our work, ExecuTorch added 4-bit training kernels to its Vulkan and WebGPU backends, including the backward pass of the quantized linear layer and an AdamW optimizer step (PyTorch Contributors, 2026), whereas Nachin et al. (2026) validate on-device training only on a CIFAR-10 image classifier running on the CPU. We measure what such a kernel changes over complete training runs of a 3B-parameter model on a phone, and our audit shows that the gap it closes is common to runtimes designed for inference. On the server side, Chen et al. (2025b) make the complementary point that freezing the base removes only the products that compute its weight gradients, about a third of the arithmetic, and Sung et al. (2022) that the backward pass must still run through it. Personalization and privacy. Personalizing an LLM from user history is a growing field (Zhang et al., 2025b). LaMP (Salemi et al., 2024b) and OPPU (Tan et al., 2024) define the tasks and the two-stage adapter recipe we adopt, and Salemi and Zamani (2025) compare retrieval and fine-tuning for personalization on servers. Federated and split learning keep raw data local but send gradients or activations off the device (McMahan et al., 2017; Vepakomma et al., 2018; Chen et al., 2026; Geng et al., 2026), and inputs can be reconstructed from either (Zhu et al., 2019; He et al., 2019; Pasquini et al., 2021). Trusted execution on phones (Wang et al., 2026b) addresses the inverse problem, protecting a vendor’s model weights from the device owner. 9. Discussion and Limitations Cost levers. The cost of on-device training is decided in the frozen base, not in the adapter, since every step still runs a forward and a backward pass through the full model and the backward pass over quantized weights takes most of the step. The obstacle there is software rather than arithmetic, since the two contractions need different kernels, and the runtimes we audited either have no accelerated backward kernel or, in the case of MLX, had one that was never dispatched, except ExecuTorch, which added one concurrently with this work. Once dispatched, the backward pass runs at the cost its arithmetic predicts, and the MLX defect went unnoticed because no training workload or test exercised that kernel. What the application controls is the shape of the adapter, where rank changes the cost little while the number of adapted projections, the adapted depth, and the number of passes over the history change it substantially, and no schedule we tested lowers it. Limitations. Our measurements are of one device and one platform, and iOS is the restrictive one, since Android exposes a thermal headroom forecast (Google, 2026c), performance hint sessions (Google, 2026b) and foreground services that keep work running in the background (Google, 2026a), so our findings on scheduling and on the thermal signal are findings about iOS. What runtimes and operating systems should provide. A runtime that supports training needs the backward contraction over quantized weights as a kernel. Because the base is frozen, its layout can be chosen at load time for both contraction directions rather than inherited from the inference format. An operating system could support on-device training by letting it run while the device is charging and off screen, which removes the battery cost at the price of the 5 to 10% slowdown of our plugged-in runs. An adapter runs for hours without a deadline and can save its few megabytes of state at every step, like the photo indexing and backups that a phone already defers to charging periods, yet iOS grants such work budgets of minutes and no GPU (Apple Inc., 2026b). 10. Conclusion A phone can personalize its language model by training on the device. On an iPhone 17 Pro, per-user adapters for a 3B-parameter model at 4-bit precision improve personalization benchmark scores and match adapters trained on a server. Training time and energy are linear in the number of tokens in the user’s history, and a 405-record history costs about half a battery charge. Once adapters, quantization and checkpointing fit training into memory, its cost lies in the backward pass over the frozen quantized base model, which runs through kernels and weight layouts designed for inference. In MLX the accelerated kernel for this pass had never been dispatched and was incorrect, and nine of the ten other runtimes we audited lack such a kernel. Sustained training is further limited by the thermal envelope of the device and by an operating system that restricts GPU training to the foreground. We repaired the kernel in MLX. The weight layout, the testing of training paths in inference runtimes, and background execution remain open, and they will determine the cost of on-device training on future devices. Use of AI tools. We used Claude (Anthropic) to help write code, run experiments, search related work, and edit the text, and we checked and verified all of its output. References Apple Distribution International Limited (2025) Apple Distribution International Limited. 2025. Product Information Sheet, model A3523 (EPREL registration 2402615). https://eprel.ec.europa.eu/fiches/smartphonestablets20231669/Fiche_2402615_EN.pdf. European Product Registry for Energy Labelling, pursuant to Commission Delegated Regulation (EU) 2023/1669. Accessed 2026-09-11. Apple Inc. (2025) Apple Inc. 2025. Apple unveils iPhone 17 Pro and iPhone 17 Pro Max. https://w.apple.com/newsroom/2025/09/apple-unveils-iphone-17-pro-and-iphone-17-pro-max/. Apple Newsroom press release, 9 September 2025. Apple Inc. (2026a) Apple Inc. 2026a. BGContinuedProcessingTaskRequest.Resources.gpu. https://developer.apple.com/documentation/backgroundtasks/bgcontinuedprocessingtaskrequest/resources/gpu. Apple Developer Documentation. Accessed 2026-09-11. Apple Inc. (2026b) Apple Inc. 2026b. BGProcessingTask. https://developer.apple.com/documentation/backgroundtasks/bgprocessingtask. Apple Developer Documentation. Accessed 2026-09-11. Apple Inc. (2026c) Apple Inc. 2026c. com.apple.developer.kernel.increased-memory-limit. https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.kernel.increased-memory-limit. Apple Developer Documentation. Accessed 2026-09-11. Apple Inc. (2026d) Apple Inc. 2026d. com.apple.developer.sustained-execution. https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.sustained-execution. Apple Developer Documentation. Accessed 2026-09-24. Apple Inc. (2026e) Apple Inc. 2026e. Core ML. https://developer.apple.com/documentation/coreml. Accessed 2026-09-11. Bakouch et al. (2025) Elie Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patiño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédec, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaibhav Srivastav, Joshua Lochner, Xuan-Son Nguyen, Colin Raffel, Leandro von Werra, and Thomas Wolf. 2025. SmolLM3: smol, multilingual, long-context reasoner. https://huggingface.co/blog/smollm3. bitsandbytes Foundation (2026) bitsandbytes Foundation. 2026. bitsandbytes. https://github.com/bitsandbytes-foundation/bitsandbytes. Accessed 2026-09-11. Chen et al. (2026) Fahao Chen, Jie Wan, Peng Li, Zhou Su, and Dongxiao Yu. 2026. Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices. In Proceedings of the 21st European Conference on Computer Systems (EuroSys ’26). ACM. System named FLUX; arXiv:2508.19078. Chen et al. (2025b) Guanduo Chen, Yutong He, Yipeng Hu, Kun Yuan, and Binhang Yuan. 2025b. CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models. arXiv:2502.01378 [cs.LG] https://arxiv.org/abs/2502.01378 Chen et al. (2025a) Le Chen, Dahu Feng, Erhu Feng, Yingrui Wang, Rong Zhao, Yubin Xia, Pinjie Xu, and Haibo Chen. 2025a. Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference. In Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles (SOSP ’25). ACM, 359–374. doi:10.1145/3731569.3764808 Chen et al. (2018) Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, 578–594. https://w.usenix.org/conference/osdi18/presentation/chen Chen et al. (2016) Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training Deep Nets with Sublinear Memory Cost. CoRR abs/1604.06174 (2016). arXiv:1604.06174 http://arxiv.org/abs/1604.06174 Chen et al. (2025c) Yuhao Chen, Yuxuan Yan, Shuowei Ge, Yuyang Qin, Yue Zheng, Qianqian Yang, Shibo He, Zhiguo Shi, Jiming Chen, and Yuanchao Shu. 2025c. Confidant: Customizing Transformer-based LLMs via Collaborative Training on Mobile Devices. In Proceedings of the 31st Annual International Conference on Mobile Computing and Networking (MobiCom ’25). 483–497. doi:10.1145/3680207.3723487 Dettmers et al. (2023) Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). doi:10.52202/075280-0441 arXiv:2305.14314 [cs.LG] Frantar et al. (2022) Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. arXiv:2210.17323 [cs.LG] https://arxiv.org/abs/2210.17323 Gao et al. (2025) Lei Gao, Amir Ziashahabi, Yue Niu, Salman Avestimehr, and Murali Annavaram. 2025. MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 20217–20234. doi:10.18653/v1/2025.emnlp-main.1022 Gemini Team (2023) Gemini Team. 2023. Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 [cs.CL] https://arxiv.org/abs/2312.11805 Geng et al. (2026) Jiaxiang Geng, Yiyi Lu, Lunyu Zhao, Yan Gao, Nicholas D. Lane, and Bing Luo. 2026. EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints. arXiv:2605.08636 [cs.LG] https://arxiv.org/abs/2605.08636 Geng et al. (2025) Jiaxiang Geng, Lunyu Zhao, Yiyi Lu, and Bing Luo. 2025. MobileFineTuner: A Mobile-Native Framework for On-Device LLM Fine-Tuning in Real-World Embedded AI Applications. arXiv preprint arXiv:2512.08211 (2025). Gerganov et al. (2022) Georgi Gerganov et al. 2022. ggml: Tensor library for machine learning. https://github.com/ggml-org/ggml. Gerganov et al. (2023) Georgi Gerganov et al. 2023. llama.cpp. https://github.com/ggml-org/llama.cpp Gim and Ko (2022) In Gim and JeongGil Ko. 2022. Memory-efficient DNN training on mobile devices. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’22). ACM, 464–476. doi:10.1145/3498361.3539765 System named Sage. Google (2026a) Google. 2026a. Foreground services overview. Android Developers documentation. https://developer.android.com/develop/background-work/services/fgs Accessed September 2026. Google (2026b) Google. 2026b. Performance Hint API. Android Developers documentation. https://developer.android.com/games/optimize/adpf/performance-hint-api Accessed September 2026. Google (2026c) Google. 2026c. PowerManager.getThermalHeadroom. Android Developers documentation. https://developer.android.com/reference/android/os/PowerManager Accessed September 2026. Google AI Edge (2024) Google AI Edge. 2024. LiteRT: Google’s On-Device Runtime for High-Performance ML and GenAI. https://github.com/google-ai-edge/LiteRT. Hannun et al. (2023) Awni Hannun, Jagrit Digani, Angelos Katharopoulos, and Ronan Collobert. 2023. MLX: Efficient and flexible machine learning on Apple silicon. https://github.com/ml-explore Hao et al. (2026) Zixu Hao, Jianyu Wei, Tuowei Wang, Minxing Huang, Huiqiang Jiang, Shiqi Jiang, Ting Cao, and Ju Ren. 2026. Scaling LLM Test-Time Compute with Mobile NPU on Smartphones. In Proceedings of the 21st European Conference on Computer Systems, EuroSys 2026, McEwan Hall/The University of Edinburgh, Edinburgh, Scotland, UK, April 27-30, 2026, Antonio Barbalace, Luo Mai, Roxana Geambasu, and Peter R. Pietzuch (Eds.). ACM, 2157–2172. doi:10.1145/3767295.3769382 He et al. (2019) Zecheng He, Tianwei Zhang, and Ruby B. Lee. 2019. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference (San Juan, Puerto Rico, USA) (ACSAC ’19). Association for Computing Machinery, New York, NY, USA, 148–162. doi:10.1145/3359789.3359824 Hu et al. (2021) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL] https://arxiv.org/abs/2106.09685 Jiang et al. (2020) Xiaotang Jiang, Huan Wang, Yiliu Chen, Ziqi Wu, Lichuan Wang, Bin Zou, Yafeng Yang, Zongyang Cui, Yu Cai, Tianhang Yu, Chengfei Lv, and Zhihua Wu. 2020. MNN: A Universal and Efficient Inference Engine. In MLSys. Laskaridis et al. (2024) Stefanos Laskaridis, Kleomenis Katevas, Lorenzo Minto, and Hamed Haddadi. 2024. MELTing Point: Mobile Evaluation of Language Transformers. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom 2024, Washington D.C., DC, USA, November 18-22, 2024, Weisong Shi, Deepak Ganesan, and Nicholas D. Lane (Eds.). ACM, 890–907. doi:10.1145/3636534.3690668 Li et al. (2025a) Ethan Li, Anders Boesen Lindbo Larsen, Chen Zhang, et al. 2025a. Apple Intelligence Foundation Language Models: Tech Report 2025. arXiv:2507.13575 [cs.LG] https://arxiv.org/abs/2507.13575 Li et al. (2025b) Liang Li, Xingke Yang, Wen Wu, Hao Wang, Tomoaki Ohtsuki, Xin Fu, Miao Pan, and Xuemin Shen. 2025b. MobiLLM: Enabling LLM Fine-Tuning on the Mobile Device via Server Assisted Side Tuning. arXiv preprint arXiv:2502.20421 (2025). Lin (2004) Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013 Lin et al. (2024) Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. In Proceedings of Machine Learning and Systems 6 (MLSys 2024). arXiv:2306.00978 [cs.CL] https://proceedings.mlsys.org/paper_files/paper/2024/hash/42a452cbafa9d64e9ba4a95c1ef21-Abstract-Conference.html Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=Bkg6RiCqY7 Malladi et al. (2023) Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023. Fine-Tuning Language Models with Just Forward Passes. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 53038–53075. doi:10.52202/075280-2308 McMahan et al. (2017) Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA (Proceedings of Machine Learning Research, Vol. 54), Aarti Singh and Xiaojin (Jerry) Zhu (Eds.). PMLR, 1273–1282. http://proceedings.mlr.press/v54/mcmahan17a.html MLC team (2025) MLC team. 2023–2025. MLC-LLM. https://github.com/mlc-ai/mlc-llm Nachin et al. (2026) Mergen Nachin, Digant Desai, Sicheng Stephen Jia, Chen Lai, Mengwei Liu, Jacob Szwejbka, Raziel Alvarez, RJ Ascani, Dave Bort, Manuel Candales, et al. 2026. ExecuTorch - A Unified PyTorch Solution to Run AI Models On-Device. arXiv preprint arXiv:2605.08195 (2026). https://github.com/pytorch/executorch ONNX Runtime developers (2021) ONNX Runtime developers. 2021. ONNX Runtime. https://onnxruntime.ai/. Version: 1.x. Park et al. (2026a) JuneYoung Park, Yuri Hong, Seongwan Kim, and Jaeho Lee. 2026a. Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track). 906–916. doi:10.18653/v1/2026.acl-industry.62 Park et al. (2026b) Jurn-Gyu Park, Sanzhar Zholdybayev, Aidar Amangeldi, and Ademi Zhanuzakova. 2026b. Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices. arXiv:2607.05933 [cs.LG] https://arxiv.org/abs/2607.05933 Pasquini et al. (2021) Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi. 2021. Unleashing the Tiger: Inference Attacks on Split Learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (Virtual Event, Republic of Korea) (CCS ’21). Association for Computing Machinery, New York, NY, USA, 2113–2129. doi:10.1145/3460120.3485259 Peng et al. (2024) Dan Peng, Zhihui Fu, and Jun Wang. 2024. PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs. In Proceedings of the Fifth Workshop on Privacy in Natural Language Processing (PrivateNLP). Association for Computational Linguistics, Bangkok, Thailand, 91–96. https://aclanthology.org/2024.privatenlp-1.10/ PyTorch Contributors (2026) PyTorch Contributors. 2026. ExecuTorch pull requests 20943 and 20931. https://github.com/pytorch/executorch/pull/20943. Add the linear_q4gsw_backward kernel to the Vulkan backend (20943) and the q4gsw training kernels to the WebGPU backend (20931). Both merged 21 July 2026. Qwen Team (2025) Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https://arxiv.org/abs/2505.09388 Robertson and Zaragoza (2009) Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval 3, 4 (2009), 333–389. doi:10.1561/1500000019 Rücklé et al. (2021) Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2021. AdapterDrop: On the Efficiency of Adapters in Transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 7930–7946. doi:10.18653/v1/2021.emnlp-main.626 Salemi et al. (2024a) Alireza Salemi, Surya Kallumadi, and Hamed Zamani. 2024a. Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR ’24). Association for Computing Machinery, New York, NY, USA, 752–762. doi:10.1145/3626772.3657783 Salemi et al. (2024b) Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024b. Lamp: When large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 7370–7392. Salemi and Zamani (2025) Alireza Salemi and Hamed Zamani. 2025. Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models. In Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) (Padua, Italy) (ICTIR ’25). Association for Computing Machinery, New York, NY, USA, 286–296. doi:10.1145/3731120.3744595 Song and Tang (2025) Congzheng Song and Xinyu Tang. 2025. Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025 - Industry Track, Suzhou, China, November 4-9, 2025, Saloni Potdar, Lina Maria Rojas-Barahona, and Sébastien Montella (Eds.). Association for Computational Linguistics, 766–774. doi:10.18653/V1/2025.EMNLP-INDUSTRY.52 Sung et al. (2022) Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. 2022. LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2206.06522 [cs.LG] https://proceedings.neurips.c/paper_files/paper/2022/hash/54801e196796134a2b0ae5e8adef502f-Abstract-Conference.html Tam et al. (2026) Kahou Tam, Wei Niu, Yu Bao, Xiaomin Ouyang, ChengZhong Xu, and Li Li. 2026. FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs. In Proceedings of the 24th Annual International Conference on Mobile Systems, Applications and Services, MobiSys 2026, University of Cambridge, Cambridge, United Kingdom, June 21-25, 2026, Cecilia Mascolo and Fahim Kawsar (Eds.). ACM, 432–447. doi:10.1145/3745756.3809214 Tan et al. (2024) Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024. Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 6476–6491. doi:10.18653/v1/2024.emnlp-main.372 Tencent (2017) Tencent. 2017. ncnn: A High-Performance Neural Network Inference Framework Optimized for the Mobile Platform. https://github.com/Tencent/ncnn. Tenison et al. (2026) Irene Tenison, Stella Ahn, Miriam Kim, Ebtisam Alshehri, and Lalana Kagal. 2026. Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation. arXiv:2604.22783 [cs.LG] https://arxiv.org/abs/2604.22783 Tummalapalli et al. (2026) Pranay Tummalapalli, Sahil Arayakandy, Ritam Pal, and Kautuk Kundan. 2026. LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load. arXiv:2603.23640 [cs.DC] https://arxiv.org/abs/2603.23640 Vepakomma et al. (2018) Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. 2018. Split learning for health: Distributed deep learning without sharing raw patient data. CoRR abs/1812.00564 (2018). arXiv:1812.00564 http://arxiv.org/abs/1812.00564 Wang et al. (2025) Haoming Wang, Boyuan Yang, Xiangyu Yin, and Wei Gao. 2025. Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection. In Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’25). 154–168. doi:10.1145/3711875.3729132 Wang et al. (2022) Qipeng Wang, Mengwei Xu, Chao Jin, Xinran Dong, Jinliang Yuan, Xin Jin, Gang Huang, Yunxin Liu, and Xuanzhe Liu. 2022. Melon: breaking the memory wall for resource-efficient on-device machine learning. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’22). ACM, 450–463. doi:10.1145/3498361.3538928 Wang et al. (2026b) Xunjie Wang, Jiacheng Shi, Zihan Zhao, Yang Yu, Zhichao Hua, and Jinyu Gu. 2026b. TZ-LLM: Protecting On-Device Large Language Models with Arm TrustZone. In Proceedings of the 21st European Conference on Computer Systems (EuroSys ’26). ACM, 657–674. doi:10.1145/3767295.3769334 Wang et al. (2023) Yiding Wang, Decang Sun, Kai Chen, Fan Lai, and Mosharaf Chowdhury. 2023. Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing. In Proceedings of the Eighteenth European Conference on Computer Systems (EuroSys ’23). ACM, 851–866. doi:10.1145/3552326.3587451 Wang et al. (2026a) Zhenyu Wang, Rana Muhammad Shahroz Khan, Tianlong Chen, and Shahriar Nirjon. 2026a. EdgeTune: Efficient On-Device LLM Personalization at the Edge. In Proceedings of the 2026 ACM/IEEE International Conference on Embedded Artificial Intelligence and Sensing Systems (SenSys ’26). ACM, 421–437. doi:10.1145/3774906.3802769 Xu et al. (2022) Daliang Xu, Mengwei Xu, Qipeng Wang, Shangguang Wang, Yun Ma, Kang Huang, Gang Huang, Xin Jin, and Xuanzhe Liu. 2022. Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloading. In Proceedings of the 28th Annual International Conference on Mobile Computing and Networking (MobiCom ’22). ACM, 214–227. doi:10.1145/3495243.3560545 Yang and Xia (2026) Weisi Yang and Stephen Xia. 2026. PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling. In Proceedings of the 2026 ACM/IEEE International Conference on Embedded Artificial Intelligence and Sensing Systems (SenSys ’26). ACM. arXiv:2609.09662. Yang et al. (2025) Xingke Yang, Liang Li, Zhiyi Wan, Sicong Li, Xiaoqi Qi, Jiang Liu, Tomoaki Ohtsuki, Xin Fu, and Miao Pan. 2025. PAE MobiLLM: Privacy-Aware and Efficient LLM Fine-Tuning on the Mobile Device via Additive Side-Tuning. arXiv:2507.01216 [cs.LG] https://arxiv.org/abs/2507.01216 Zhang et al. (2025a) Zongpu Zhang, Pranab Dash, Y. Charlie Hu, Qiang Xu, Jian Li, and Haibing Guan. 2025a. Dissecting the Impact of Mobile DVFS Governors on LLM Inference Performance and Energy Efficiency. arXiv:2507.02135 [cs.OS] https://arxiv.org/abs/2507.02135 Zhang et al. (2025b) Zhehao Zhang, Ryan A. Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, Ruiyi Zhang, Jiuxiang Gu, Tyler Derr, Hongjie Chen, Junda Wu, Xiang Chen, Zichao Wang, Subrata Mitra, Nedim Lipka, Nesreen Ahmed, and Yu Wang. 2025b. Personalization of Large Language Models: A Survey. (2025). arXiv:2411.00027 [cs.CL] https://arxiv.org/abs/2411.00027 Zhong et al. (2025) Zhaofeng Zhong, Wei Yuan, Liang Qu, Tong Chen, Hao Wang, Xiangyu Zhao, and Hongzhi Yin. 2025. Towards On-Device Personalization: Cloud-device Collaborative Data Augmentation for Efficient On-device Language Model. arXiv preprint arXiv:2508.21313 (2025). Zhu et al. (2023) Ligeng Zhu, Lanxiang Hu, Ji Lin, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. 2023. PockEngine: Sparse and Efficient Fine-tuning in a Pocket. In Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO ’23). ACM, 1381–1394. doi:10.1145/3613424.3614307 Zhu et al. (2019) Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems 32 (2019). Zhu et al. (2026) Zhanda Zhu, Qidong Su, Yaoyao Ding, Kevin Song, Shang Wang, and Gennady Pekhimenko. 2026. LoRAFusion: Efficient LoRA Fine-Tuning for LLMs. In Proceedings of the 21st European Conference on Computer Systems (EuroSys ’26). ACM, 586–604. doi:10.1145/3767295.3769331 Zou et al. (2026) Bohua Zou, Nian Liu, Binqi Sun, Matteo Mascherin, Debayan Roy, Yutao Liu, Yu Peng, Ning Jia, and Haibo Chen. 2026. EnerInfer: Energy-Aware On-Device LLM Inference. arXiv:2606.23001 [cs.SE] https://arxiv.org/abs/2606.23001

Instant research discovery