Paper deep dive
MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge
Mehmet Emre Akbulut, Johannes Geier, Ulf Schlichtmann
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:On-device learning is necessary when the model encounters user-,sensor-, or environment-specific shifts after deployment. Although parameter-efficient fine-tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA) variants, enable efficient adaptation at the edge, the limiting resource for Convolutional Neural Network (CNN) adaptation is often not the number of trainable parameters but the activation state that must be retained until the backward pass. This paper introduces Memory-Floor LoRA (MemFLoRA), a low-rank CNN adapter built around a memory-first design principle rather than a direct application of transformer-oriented LoRA. Instead of merely reducing trainable weights, we define an activation-memory-floor criterion: trainable backward computations must not depend on full-width layer inputs. The resulting adapter freezes the down-projection, trains a scale-matched up-projection, and combines eval-mode backbone normalization with activation-minimal backward rules, reducing saved state to the low-rank branch. Evaluated on three Human Activity Recognition (HAR) datasets and two CNN backbones under subject, body-location, and sensor-placement shifts, MemFLoRA reduces saved-activation memory by 98.5-98.7% and peak training-state memory by 94.9-97.3% relative to full fine-tuning, while matching or exceeding CNN PEFT baselines.
Tags
Links
- Source: https://arxiv.org/abs/2610.08669v1
- Canonical: https://arxiv.org/abs/2610.08669v1
Trouble viewing inline? Open PDF directly →
Full Text
51,488 characters extracted from source content.
Expand or collapse full text
MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge Conference: 32nd Asia and South Pacific Design Automation Conference; January 25–28, 2027; Tokyo, Japan32nd Asia and South Pacific Design Automation Conference (ASPDAC ’27), January 25–28, 2027, Tokyo, JapanDOI: 10.1145/3840404.3848401ISBN: 979-8-4007-2923-2/2027/01CCS: Computing methodologies Neural networks Mehmet Emre Akbulut Affiliation: Technical University of Munich, Munich, Germany email: mehmet.akbulut@tum.de , Johannes Geier Affiliation: Technical University of Munich, Munich, Germany email: johannes.geier@tum.de and Ulf Schlichtmann Affiliation: Technical University of Munich, Munich, Germany email: ulf.schlichtmann@tum.de © c Abstract. On-device learning is necessary when the model encounters user-, sensor-, or environment-specific shifts after deployment. Although parameter-efficient fine-tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA) variants, enable efficient adaptation at the edge, the limiting resource for Convolutional Neural Network (CNN) adaptation is often not the number of trainable parameters but the activation state that must be retained until the backward pass. This paper introduces Memory-Floor LoRA (MemFLoRA), a low-rank CNN adapter built around a memory-first design principle rather than a direct application of transformer-oriented LoRA. Instead of merely reducing trainable weights, we define an activation-memory-floor criterion: trainable backward computations must not depend on full-width layer inputs. The resulting adapter freezes the down-projection, trains a scale-matched up-projection, and combines eval-mode backbone normalization with activation-minimal backward rules, reducing saved state to the low-rank branch. Evaluated on three Human Activity Recognition (HAR) datasets and two CNN backbones under subject, body-location, and sensor-placement shifts, MemFLoRA reduces saved-activation memory by 98.598.5-98.7%98.7\% and peak training-state memory by 94.994.9-97.3%97.3\% relative to full fine-tuning, while matching or exceeding CNN PEFT baselines. Keywords: LoRA, On-Device Learning, Edge AI, CNN †c-license: by 1. Introduction Artificial Intelligence (AI) at the edge has become increasingly important as intelligent sensing devices become deeply embedded in everyday life. Smartphones, wearables, and Internet of Things (IoT) platforms interact with their environments and generate large amounts of domain-specific data. This trend pushes AI computation closer to the data source, enabling lower latency, reduced communication costs, and improved privacy compared with sending all data to cloud servers (Zhu et al., 2023). Figure 1. Brief overview of MemFLoRA computation and saved-state graph, reducing persistent saved state from O(BCℓTℓ)O(B\,C_ \,T_ ) to O(BrTℓ)O(B\,r\,T_ ) plus a bit-packed ReLU mask.MemFLoRA computation and saved-state flow MemFLoRA computation and saved-state flow. An input feature tensor splits into a frozen backbone branch and a low-rank adapter branch. The backbone branch applies a frozen convolution followed by evaluation-mode batch normalization. The adapter branch applies a frozen pointwise down-projection initialized from a normal distribution, optionally applies bottleneck batch normalization, then applies a trainable up-projection and a fixed per-channel scale inherited from the source batch normalization. The two branches are added and passed through a rectified linear unit. The diagram shows that only the bottleneck activation, whose channel count equals the adapter rank, and a bit-packed rectified-linear-unit sign mask are retained. The full-width input, backbone activations, and batch-normalization states are dropped rather than persistently saved for backward computation. However, a fixed pretrained model may fail to provide robust predictions after deployment. For example, in sensor-based HAR, the target stream can differ from the source data due to subject-dependent motion patterns, sensor-placement deviations, device heterogeneity, and evolving user behavior (Sunaga et al., 2023; Nadalini et al., 2023). These changes can lead to distribution shifts or concept drifts and degrade the out-of-distribution performance of fixed pretrained models. Since the corrective data are generated locally and may be privacy-sensitive, adapting the model on-device is preferable to repeated cloud-side retraining. However, full fine-tuning updates all model parameters and incurs substantial memory, compute, and energy overhead, which can exceed the resources of embedded systems and motivate more selective adaptation strategies. Since its introduction (Hu et al., 2022), LoRA has become a widely used adaptation method for Large Language Models (LLMs), but its role in CNNs and extreme-edge learning scenarios remains much less explored. The standard reading of PEFT is that reducing the number of trainable parameters reduces the training-memory footprint. This assumption is often reasonable in LLMs, where optimizer state and trainable weight gradients dominate. However, for edge CNNs, memory is often dominated by the activation tensors that automatic differentiation retains for backward propagation, not by the optimizer state or the trainable parameter set (Cai et al., 2020). This mismatch indicates that parameter-efficient methods may still be memory-expensive, motivating MemFLoRA11 1 Code is available at https://github.com/mehmetemreakbulut/MemFLoRA.: a LoRA-style CNN adapter with an activation-memory floor. The design rule is that no trainable backward computation should require the full-width activation tensor. The adapter satisfies this rule through a frozen down-projection, a trainable up-projection scale-matched to the frozen backbone, and eval-mode backbone normalization. This restricts the saved state to the low-rank branch while preserving adaptation capacity. In summary, our main contributions are: • By decomposing the saved-tensor cost into three pillars, we define an activation-memory-floor criterion requiring that no persistent full-precision activation tensor used by trainable backward computations scales with layer width, and show that MemFLoRA satisfies it with rank-r saved state. Enforcing this floor reduces saved-activation memory by 98.598.5-98.7%98.7\% and total peak training-state memory by 94.994.9-97.3%97.3\% relative to full fine-tuning. • We introduce a memory-floor CNN adapter that combines a frozen down-projection, a scale-matched trainable up-pro-jection, eval-mode backbone normalization, and activation-minimal backward rules, together with a streaming-gradient (SG) variant that recovers the exact projection gradient without persistently saving the full-width activation. • Across three HAR datasets spanning subject, location, and sensor-placement shifts and two CNN backbones, we match or exceed all evaluated rank-dependent PEFT methods in mean Macro-F1 across ranks and backbones. 2. Background and Related Work Prior work on efficient adaptation has followed two complementary directions. LoRA-style PEFT methods restrict updates to low-rank branches, reducing trainable parameters and optimizer-state costs while remaining easy to integrate into pretrained models and, in many cases, mergeable into the backbone for inference. On-device training methods instead prioritize the dominant training-memory cost by reducing saved activations through selective updates and freezing. Our work bridges these directions by co-designing the adapter architecture and training graph around this activation-memory constraint, rather than treating activation reduction and adaptation as independent problems. 2.1. Low-Rank Adaptation LoRA introduced low-rank adaptation by injecting trainable low-rank updates into frozen pretrained weights, reducing trainable parameters while preserving accuracy (Hu et al., 2022). Most low-rank adaptation research focuses on transformer-based models and the adapter architecture. Work on CNNs is sparser and largely adopts the same parameter-centric perspective as its transformer-based counterparts (Ding et al., 2024; Kwak et al., 2026; Matsutani et al., 2025) rather than reconsidering the activation state retained during training. ConvLoRA (Aleem et al., 2024) combines LoRA with AdaBN (Li et al., 2018) for medical-image domain adaptation, whereas Convolution Meets LoRA (Zhong et al., 2024) integrates lightweight convolutions into LoRA experts without fine-tuning the convolution itself. LoRA-C (Ding et al., 2024) performs low-rank decomposition at the convolutional-layer level rather than the kernel level, targeting parameter-efficient adaptation for IoT devices. More recently, LoRA-Edge (Kwak et al., 2026) targets on-device CNN fine-tuning for human activity recognition under domain shift, using a tensor-train structure to reduce trainable parameters and improve convergence. Across this body of work, the shared objective is parameter efficiency, leaving the CNN setting’s dominant saved-activation cost largely unaddressed. LoRA-FA (Zhang et al., 2023) freezes the down-projection and trains only the up-projection. Theoretical work on asymmetric low-rank adapters further explains that freezing the input-side projection constrains updates to a projected subspace, while leaving the output-side factor trainable can remain expressive in high-dimensional regimes (Zhu et al., 2024). MemFLoRA inherits this asymmetric-trainability principle but extends it to CNN adaptation, where BatchNorm, activation backward, and convolutional execution introduce saved-state constraints absent from the transformer setting. Asymmetric trainability alone removes the full-width input saved for the trainable down-projection. In CNNs, however, full-width state can also be retained by train-mode BatchNorm, activation backward, and the frozen convolutional branch. 2.2. On-Device Training and Activation-Memory Several prior systems reduce training memory for on-device learning through quantization, sparse updates, rematerialization, or activation compression. TinyTL (Cai et al., 2020) showed that activations dominate on-device training memory. Building on this perspective, MCUNetV3 (Lin et al., 2022) combines quantization-aware scaling, sparse updates, and a tiny training engine to enable learning under extremely small memory budgets. POET (Patil et al., 2022) formulates rematerialization and paging schedules for training, while gradient checkpointing (Chen et al., 2016) reduces activation storage by recomputing intermediate activations during the backward pass. Activation-compression methods such as ActNN (Chen et al., 2021), GACT (Liu et al., 2022), and Few-bit Backward (Novikov et al., 2023) reduce the cost of storing activations by compressing or quantizing them. Checkpointing, rematerialization, and activation compression are orthogonal execution-level techniques: they reduce or reconstruct the state required by a given adapter but do not change its intrinsic backward dependencies. Our floor instead concerns adapter design, before such techniques are applied. MemFLoRA attains this floor without recomputation or compression, and these techniques can be applied on top of it for further savings. MemFLoRA combines asymmetric low-rank adaptation with the activation-centric perspective of on-device training. Unlike the execution-level techniques discussed above, it modifies the adapter’s intrinsic backward dependencies so that the persistent state required by trainable computations is bounded by the rank-r branch. To the best of our knowledge, MemFLoRA is the first LoRA-style CNN adapter explicitly designed around an activation-memory-floor criterion. The novelty lies in enforcing a graph-level activation-memory constraint across the frozen projection, adapter branch, backbone normalization, activation backward, adapter placement, convolutional geometry, and optionally, projection-gradient recovery. 3. Method We introduce a family of LoRA-style adapters for activation-memory-efficient CNN adaptation as illustrated in Figure 1. In transformers, LoRA is naturally applied to linear projections, where the main design variable is the rank of the injected update. In CNNs, however, the adapter interacts with convolutional kernel geometry, BatchNorm mode and statistics, activation functions, and fused Conv–BN–ReLU execution. Consequently, directly factorizing convolutional weights may reduce trainable parameters while still retaining full-width feature maps for the backward pass. We instead require that no trainable backward formula should require the full-width activation tensor. In the remainder of this section, we define the memory floor, derive the adapter form and placement from it, and show how each component follows from this constraint rather than being an independent design choice. 3.1. The Activation-Memory Floor Criterion Total training memory decomposes as (1) Mtrain=Mweights+Mgrad+Moptim+Msaved+Mbuf,M_train=M_weights+M_grad+M_optim+M_saved+M_buf, where the terms denote model weights, gradients, optimizer states, tensors saved during the forward pass for backward reuse, and temporary buffers, respectively. Among these, MsavedM_saved can dominate for edge CNNs (Figure 2). Parameter-efficient fine-tuning reduces MgradM_grad and MoptimM_optim but leaves MsavedM_saved essentially unchanged, which is why parameter efficiency does not imply memory efficiency. For input xℓ∈ℝB×Cℓ×Tℓx_ ^B× C_ × T_ , MsavedM_saved decomposes into three pillars, each retained by a distinct part of the backward graph: (i) activation/ReLU state: although ReLU backward requires only the activation sign, standard automatic differentiation typically retains a full-width tensor, O(BCℓTℓ)O(B\,C_ \,T_ ); (i) BatchNorm state: train-mode BatchNorm’s backward requires the normalized input, so full-width BN input is saved, O(BCℓTℓ)O(B\,C_ \,T_ ); (i) frozen-branch convolution state: the input of every convolution is saved, including the input of the adapter and the input of the frozen backbone. A standard low-rank convolutional adapter trains both factors and therefore its saved adapter input remains full-width, O(BCℓTℓ)O(B\,C_ \,T_ ), matching the asymptotic activation width of full fine-tuning. We define the activation-memory floor for single-pass LoRA-style CNN adapters as persistent saved state of O(BrTℓ)O(BrT_ ) for trainable backward computations, independent of CℓC_ . The ReLU gate is accounted for separately as a bit-packed mask, which is the minimal direct state for exact ReLU backward without recomputation. MemFLoRA satisfies this by replacing the dense saved state of the three pillars with the rank-r bottleneck and a bit-packed ReLU gate. Eval-mode BatchNorm removes Pillar (i): During adaptation, the BatchNorm statistics (μℓ,σℓ2)( _ , _ ^2) are frozen to target-calibrated values, and the affine parameters (γs,ℓ,βs,ℓ)( _s, , _s, ) are the fixed backbone scale and shift. BatchNorm then reduces to a per-channel affine map (2) BNseval(z)=as,ℓ⊙z+bs,ℓ;as,ℓ=γs,ℓσs,ℓ2+ε;bs,ℓ=βs,ℓ−as,ℓ⊙μs,ℓBN^eval_s(z)=a_s, z+b_s, ;\;a_s, = _s, _s, ^2+ ;\;b_s, = _s, -a_s, _s, Because (2) is a fixed affine transform, its backward pass saves no activation. Eval-mode BatchNorm is therefore a precondition for the floor rather than an optional choice. Bit-packed sign mask reduces Pillar (i): the backward pass of a fused Conv–BN–ReLU operator stores the ReLU sign at one bit per element rather than a full-precision tensor. Frozen down-projection removes Pillar (i): We adapt each convolution with a frozen down-projection PℓP_ and a trainable up-projection UℓU_ . If PℓP_ were trainable, its gradient has the outer-product form ∇Pℓ∝gℓxℓ⊤∇ P_ \; \;g_ \,x_ which would require retaining the full-width input xℓx_ , of size O(BCℓTℓ)O(B\,C_ \,T_ ), violating the floor. After freezing PℓP_ , UℓU_ requires only the bottleneck activation qℓ=Pℓxℓ∈ℝB×r×Tℓq_ =P_ \,x_ ^B× r× T_ reducing the saved adapter state to O(BrTℓ)O(B\,r\,T_ ). At r=1r=1, this becomes O(BTℓ)O(B\,T_ ) and is independent of CℓC_ . Frozen-projection asymmetry is thus a requirement of the floor, not a stylistic choice. MobileNetV2 peak training-memory decomposition Figure 2. Peak training-state memory decomposition of MobileNetV2 at rank-2 on Opportunity with a batch size of 64. Labels report total peak memory, with saved-activation memory in parentheses. MobileNetV2 peak training-memory decomposition Stacked horizontal bars compare peak training memory for MobileNetV2 on Opportunity at rank two and batch size sixty-four. The five bars represent Full Fine-Tuning, LoRA-C, LoRA-Edge, MemFLoRA, and MemFLoRA-SG. Each bar is divided into model state, optimizer state, and live training state. Live training state occupies most of the memory for Full Fine-Tuning, LoRA-C, and LoRA-Edge. Total peak memory and saved-activation memory, both in megabytes, are 666.64 and 639.56 for Full Fine-Tuning, 592.61 and 583.05 for LoRA-C, 491.22 and 481.79 for LoRA-Edge, 17.83 and 8.40 for MemFLoRA, and 26.30 and 8.40 for MemFLoRA-SG. The MemFLoRA bars are substantially shorter than the three conventional training methods, while MemFLoRA-SG has a slightly larger total than MemFLoRA because of its additional replay state. 3.2. Adapter Form and Placement We adapt each convolution with a branch consisting of a frozen down-projection PℓP_ and a trainable up-projection UℓU_ : (3) Δℓ=Uℓ(Pℓxℓ)=Uℓ(qℓ), _ =U_ (P_ \,x_ )=U_ (q_ ), As shown in Section 3.1, freezing PℓP_ places this branch at the memory-floor: the gradient of UℓU_ depends only on the rank-r bottleneck qℓq_ , never on the full-width input xℓx_ . To preserve the activation-memory floor, we implement each Conv–BN–ReLU site as a single activation-minimal operator rather than exposing the convolution output, BN output, and ReLU preactivation as separate saved tensors. Under this memory constraint, the adapter branch is added to the fused Conv–BN output: (4) yℓ=φ(BNseval(Wℓ0xℓ)+Δℓ)y_ = \! (BN_s^eval\! (W^0_ \,x_ )+ _ ) Here, Wℓ0W_ ^0 is the frozen source convolution, φ is the activation function, and Δℓ _ is the low-rank adapter update. Because eval-mode BatchNorm is affine (Equation (2)), injecting the adapter before normalization is algebraically equivalent to (5) φ(BNseval(Wℓ0xℓ+Δℓ))=φ(BNseval(Wℓ0xℓ)+as,ℓ⊙Δℓ). \! (BN_s^eval(W_ ^0x_ + _ ) )= \! (BN_s^eval(W_ ^0x_ )+a_s, _ ). 3.3. Scale-Matching the Adapter Branch The forward rule in (4) adds the adapter output Δℓ _ to the normalized backbone output BNseval(Wℓ0xℓ)BN_s^eval(W^0_ x_ ). The backbone term is scaled per channel by as,ℓa_s, per (2); the adapter term, at raw scale, is not. This subsection shows that matching the per-channel scale of the two branches is critical under the evaluated initialization and optimization protocol, examines the two ways to supply it, and derives our choice. Writing the pre-activation as (6) zℓ=as,ℓ⊙(Wℓ0xℓ)+bs,ℓ+Uℓ(qℓ),z_ =a_s, (W^0_ x_ )+b_s, +U_ (q_ ), the bottleneck qℓ=Pℓxℓq_ =P_ x_ is a frozen random projection of the layer input; its per-channel magnitudes are uncontrolled and are not on the scale of the normalized backbone term. When the two branches are mismatched in scale, the adapter updates are poorly conditioned across channels, the adapter output rapidly grows out of scale with the normalized backbone term, and the ReLU gate destabilizes. Empirically, a raw-scale memory-floor adapter fails to adapt at all (Section 4, Table 4). The adapter branch, therefore, requires per-channel scale-matching to the backbone before summation. Option A: trainable bottleneck normalization. One way to supply the missing scale is to normalize the bottleneck before UℓU_ with a trainable BatchNorm BNr,ℓBN_r, over the r bottleneck channels, (7) ΔℓBNr=Uℓ(γr,ℓ⊙qℓ−μr,ℓσr,ℓ2+ε+βr,ℓ), _ ^BN_r=U_ \!\ ( _r, q_ - _r, _r, ^2+ + _r, ), with batch statistics (μr,ℓ,σr,ℓ2)( _r, , _r, ^2) and trainable affine (γr,ℓ,βr,ℓ)( _r, , _r, ). This normalization places every bottleneck channel at unit scale and prevents the collapse of (6). Batch-dependent statistics add variation under small adaptation batches and discard the per-channel bottleneck magnitudes used by the trainable spatial kernel. Option B: inherited scale. The equivalence in (5) identifies a natural per-channel scale directly: it is the eval-mode BatchNorm output scale as,ℓa_s, , the same constant that the fused block already applies to the backbone branch. Scaling the adapter output by as,ℓa_s, yields (8) yℓ=φ(BNseval(Wℓ0xℓ)+as,ℓ⊙Uℓ(qℓ))\;y_ = \! (BN_s^eval\! (W^0_ x_ )+a_s, U_ (q_ ) )\; Unlike BNr,ℓBN_r, , the factor as,ℓa_s, is a fixed, precomputed, per-channel constant, so it introduces no batch-statistic noise and no additional saved state, and it leaves the bottleneck magnitudes intact for UℓU_ . Choice. Under the matched initialization and geometry presented in Section 3.4, the fixed scale as,ℓa_s, outperforms the trainable normalization BNr,ℓBN_r, (Table 4): both prevent the collapse, but as,ℓa_s, avoids the statistical noise caused by BNr,ℓBN_r, and preserves the bottleneck signal. Although BNr,ℓBN_r, is initialization-agnostic and mitigates the effect of projection, we adopt (8) and use no bottleneck normalization. 3.4. Convolutional Geometry The adapter is defined by a frozen down-projection PℓP_ and a trainable up-projection UℓU_ , but it does not yet specify which factor carries the spatial kernel. For a linear layer, this question does not arise; for a convolution, it is a design choice determining whether the adapter can re-learn temporal or spatial filters for the target domain. Two parameterizations are on the memory floor, since both keep the bottleneck qℓq_ at rank r. Frozen-kernel parameterization. The down-projection PℓP_ carries the k-tap spatial kernel and is frozen; the trainable UℓU_ is a pointwise (1×11× 1) channel mixer: (9) Pℓ∈ℝr×Cℓ×k,Uℓ∈ℝCout×r×1,#(Uℓ)=rCout.P_ ^r× C_ × k, U_ ^C_out× r× 1, \#(U_ )=r\,C_out. Here, the spatial filters are fixed, so the adapter can only re-mix a fixed set of filter responses; it cannot form new temporal filters. Trainable-kernel parameterization. The down-projection is a frozen pointwise projection and the trainable up-projection carries the full k-tap spatial kernel: (10) Pℓ∈ℝr×Cℓ×1,Uℓ∈ℝCout×r×k,#(Uℓ)=rCoutk.P_ ^r× C_ × 1, U_ ^C_out× r× k, \#(U_ )=r\,C_out\,k. The trainable spatial kernel operates directly on the bottleneck magnitudes qℓq_ , which are the same magnitudes that Section 3.3 showed must be preserved rather than normalized away. Parameter cost and the floor. The trainable-kernel parameterization costs a factor of k in trainable parameters, rCoutkr\,C_out\,k versus rCoutr\,C_out. This cost falls on the optimizer state MoptimM_optim, not on the activation floor: the saved adapter state is O(BrTℓ)O(B\,r\,T_ ) in both parameterizations, since qℓ=Pℓxℓq_ =P_ x_ is rank-r regardless of where the kernel sits. For edge CNNs, where Msaved≫MoptimM_saved M_optim (see Figure 2), the k×k× parameter increase is a subdominant cost paid to recover adaptation capacity at the same activation floor. Initialization. Because PℓP_ is frozen, its initialization fixes the subspace in which the adapter operates and the per-channel scale of the bottleneck. We initialize Pℓ∼(0,1/r)P_ (0,1/r), for which (11) [Var(qℓ,j)]=1rtr(Σxℓ).E\! [Var(q_ ,j) ]= 1r\,tr( _x_ ). This expected variance is identical across the r bottleneck channels, where Σxℓ=Cov(xℓ) _x_ =Cov(x_ ). This produces a variance-balanced bottleneck compatible with the scale-matched adapter in (8). The trainable up-projection is zero-initialized, Uℓ=0U_ =0, so the adapter is inactive at initialization and adaptation starts exactly from the source model. The scale mismatch of Section 3.3 therefore acts on the gradient rather than the forward pass. We examine sensitivity to this initialization in Section 4. Table 1. Headline Macro-F1 comparison at 50 adaptation steps. Mean ± sample standard deviations across 20 seeds are reported. Dataset Backbone Zero Shot Full FT Bias-Tuning BN-Tuning r LoRA-C LoRA-Edge MemFLoRA MemFLoRA-SG Opportunity MobileNetV2 2 0.698 ± 0.037 0.730 ± 0.032 0.747 ± 0.025 0.758 ± 0.028 0.585 ± 0.088 0.820 ± 0.025 0.632 ± 0.054 0.668 ± 0.043 4 0.728 ± 0.032 0.760 ± 0.027 0.764 ± 0.028 0.774 ± 0.024 8 0.756 ± 0.028 0.789 ± 0.025 0.771 ± 0.028 0.785 ± 0.023 T-ResNet 2 0.746 ± 0.044 0.710 ± 0.052 0.768 ± 0.046 0.773 ± 0.040 0.511 ± 0.087 0.847 ± 0.026 0.655 ± 0.054 0.684 ± 0.052 4 0.773 ± 0.039 0.746 ± 0.046 0.793 ± 0.034 0.797 ± 0.035 8 0.801 ± 0.030 0.774 ± 0.038 0.812 ± 0.033 0.815 ± 0.031 RealDisp MobileNetV2 2 0.882 ± 0.005 0.892 ± 0.004 0.891 ± 0.004 0.896 ± 0.004 0.791 ± 0.000 0.894 ± 0.013 0.835 ± 0.003 0.865 ± 0.005 4 0.890 ± 0.005 0.898 ± 0.004 0.888 ± 0.010 0.897 ± 0.006 8 0.897 ± 0.003 0.903 ± 0.006 0.885 ± 0.010 0.895 ± 0.007 T-ResNet 2 0.738 ± 0.009 0.631 ± 0.005 0.742 ± 0.017 0.768 ± 0.015 0.526 ± 0.000 0.891 ± 0.009 0.599 ± 0.005 0.620 ± 0.005 4 0.787 ± 0.010 0.686 ± 0.004 0.787 ± 0.010 0.815 ± 0.009 8 0.827 ± 0.010 0.742 ± 0.006 0.828 ± 0.011 0.847 ± 0.011 RealWorld MobileNetV2 2 0.802 ± 0.044 0.817 ± 0.046 0.874 ± 0.038 0.882 ± 0.035 0.610 ± 0.087 0.911 ± 0.026 0.751 ± 0.067 0.825 ± 0.049 4 0.846 ± 0.037 0.866 ± 0.035 0.886 ± 0.035 0.892 ± 0.034 8 0.880 ± 0.032 0.889 ± 0.034 0.896 ± 0.034 0.901 ± 0.032 T-ResNet 2 0.689 ± 0.096 0.632 ± 0.108 0.826 ± 0.065 0.847 ± 0.052 0.449 ± 0.124 0.903 ± 0.028 0.554 ± 0.134 0.631 ± 0.125 4 0.779 ± 0.067 0.717 ± 0.085 0.859 ± 0.047 0.871 ± 0.044 8 0.847 ± 0.043 0.799 ± 0.056 0.875 ± 0.045 0.883 ± 0.040 3.5. Projection-Gradient Recovery MemFLoRA freezes PℓP_ to preserve the memory floor, but this restricts adaptation to a fixed projected subspace. As an optional extension, MemFLoRA-SG uses rematerialization, an execution-level technique orthogonal to the adapter design, to update PℓP_ without persistently retaining the full-width input xℓx_ . Let gℓ=∂ℒ/∂qℓg_ = /∂ q_ , where qℓ=Pℓxℓq_ =P_ x_ . Then (12) ∂ℒ∂Pℓ=∑b,tgℓ,b,txℓ,b,t⊤. ∂ P_ = _b,tg_ ,b,tx_ ,b,t . Standard backpropagation retains xℓx_ until this outer product is formed. MemFLoRA-SG stores only the rank-r gradient gℓg_ , replays the current adapted network under no_grad, and immediately contracts each rematerialized xℓx_ with gℓg_ (Chen et al., 2016). The input is released immediately after its layerwise contraction, so full-width activations do not accumulate across layers and only one such buffer is live at a time. MemFLoRA-SG therefore preserves rank-bounded persistent backward state, O(BrTℓ)O(B\,r\,T_ ), at the cost of one additional forward pass and a layer-local full-width temporary buffer included in the reported peak-memory accounting. 4. Experimental Results We follow the experimental protocol and application scope established by LoRA-Edge (Kwak et al., 2026), the most directly related on-device CNN adaptation baseline, while extending the evaluation across three ranks and 20 independent seeds, with detailed training-state memory accounting and additional rank-independent baselines. We evaluate T-ResNet (Ridnik et al., 2021) and MobileNetV2 (Sandler et al., 2018) on the three HAR benchmarks summarized in Table 2, spanning complementary deployment shifts. Opportunity uses Leave-One-Subject-Out (LOSO), with one subject as target and the others as source; RealWorld uses Leave-One-Location-Out (LOLO), with one sensor location as target; and RealDisp uses ideal placement as source and self placement as target. Target data are divided into 80% adaptation and 20% held-out test data, with no test data used for adaptation or AdaBN calibration. Table 2. Datasets, domain shifts, and preprocessing settings. Dataset Shift Subj. Act. Hz Win./Str. Opportunity (Chavarriaga et al., 2013) Subject 4 17 30 60/30 RealWorld (Sztyler and Stuckenschmidt, 2016) Body location 15 8 50 500/250 RealDisp (Banos et al., 2014) Sensor placement 17 33 50 250/125 All methods use Adam (Kingma and Ba, 2014) with a learning rate of 0.0010.001, a batch size of 64, and 50 adaptation steps to match the evaluation protocol of LoRA-Edge (Kwak et al., 2026). We adapt all convolutional layers in T-ResNet and the pointwise convolutions in MobileNetV2, and evaluate ranks r∈2,4,8r∈\2,4,8\. We report macro-F1 over 20 seeds. For each random seed, the source model is independently pretrained for 20 epochs; all adaptation runs use a weight decay of 0.00050.0005 and no learning-rate scheduler. MemFLoRA-SG replay uses the same mini-batch and unchanged parameters and completes before the optimizer step. We compare against both rank-independent and rank-dependent adaptation baselines. The rank-independent baselines are Zero-shot, which performs no target adaptation, Full Fine-Tuning (Full FT) which updates the entire model, Bias-tuning (Ben Zaken et al., 2022) and BN-tuning. The rank-dependent baselines are LoRA-C (Ding et al., 2024) and LoRA-Edge (Kwak et al., 2026), two CNN-oriented LoRA baselines for edge adaptation. 4.1. Comparative Memory Analysis Table 3 reports training-memory scaling at r=2r=2 across batch sizes for T-ResNet and MobileNetV2, while Figure 2 visualizes the B=64B=64 peak-memory decomposition. Although the other methods reduce the number of trainable parameters, their memory remains dominated by saved activations. At B=64B=64, MemFLoRA reduces saved-activation memory relative to Full FT by 98.5%98.5\% on T-ResNet and 98.7%98.7\% on MobileNetV2, with corresponding peak training-state reductions of 94.9%94.9\% and 97.3%97.3\%. MemFLoRA-SG retains rank-bounded persistent state but incurs a replay buffer, yielding peak-memory reductions of 92.1%92.1\% and 96.1%96.1\%, respectively. For fairness, all rank-dependent methods use activation-aware implementations that retain only tensors required by their adapter gradients. The batch-size sweep further shows that MemFLoRA’s saved-state advantage persists as activation memory grows. Notably, its peak memory at B=64B=64, 3.233.23 MB on T-ResNet and 17.8317.83 MB on MobileNetV2, is comparable to LoRA-Edge at B=1B=1, with 2.932.93 MB and 17.0917.09 MB, respectively. Thus, under a similar memory budget, MemFLoRA supports a 64×64× larger adaptation batch. Table 3. For T-ResNet and MobileNetV2, batch-size entries report peak/saved-activation memory [MB]; optimizer-state memory is reported separately. Method B1B_1 B8B_8 B32B_32 B64B_64 Opt. [MB] T-ResNet Full FT 9.02/0.90 13.84/7.09 35.08/28.33 63.40/56.65 4.49 LoRA-C 5.51/3.10 11.53/9.12 32.20/29.79 59.76/57.35 0.10 LoRA-Edge 2.93/0.60 7.05/4.72 21.19/18.86 40.04/37.71 0.02 MemFLoRA 2.44/0.02 2.50/0.11 2.81/0.42 3.23/0.83 0.08 MemFLoRA-SG 2.50/0.02 2.78/0.11 3.74/0.42 5.01/0.83 0.10 MobileNetV2 Full FT 43.20/16.12 109.00/81.91 346.93/319.85 666.64/639.56 17.97 LoRA-C 27.17/17.61 90.00/80.44 305.40/295.84 592.61/583.05 0.30 LoRA-Edge 17.09/7.66 69.77/60.34 250.39/240.96 491.22/481.79 0.16 MemFLoRA 9.63/0.20 10.54/1.11 13.66/4.23 17.83/8.40 0.16 MemFLoRA-SG 9.91/0.20 11.73/1.11 17.97/4.23 26.30/8.40 0.16 4.2. Results Across Adaptation Budgets Table 1 reports macro-F1 under a tight 50-step adaptation budget across six dataset–backbone settings and three ranks. Among the single-pass rank-dependent methods, MemFLoRA reaches an average of 0.8270.827, compared with 0.7980.798 for LoRA-C and 0.7770.777 for LoRA-Edge. Thus, its substantial memory reduction does not come at the cost of adaptation quality. MemFLoRA-SG provides an accuracy-oriented operating point by increasing the average macro-F1 to 0.8390.839, while retaining a substantially smaller memory footprint than conventional CNN LoRA methods. We next examine convergence across adaptation budgets. Figure 3 shows the rank-2 adaptation trajectories, which are qualitatively consistent across datasets and backbones. MemFLoRA narrows the gap to Full FT as adaptation proceeds while remaining competitive at convergence. MemFLoRA-SG generally provides the strongest performance through projection-gradient recovery. Figure 3. Adaptation performance at rank=2rank=2 across adaptation steps for T-ResNet (left) and MobileNetV2 (right).Adaptation trajectories across datasets and backbones Six line charts show the macro-averaged F1 score as the number of adaptation steps increases from fifty to two hundred fifty. The left column presents T-ResNet and the right column presents MobileNetV2. The rows present Opportunity, RealWorld, and RealDisp, respectively. Each panel compares Zero Shot, Full Fine-Tuning, Bias-Tuning, Batch-Normalization Tuning, LoRA-C, LoRA-Edge, MemFLoRA, and MemFLoRA-SG. Zero Shot appears as a constant dashed baseline, while the adapted methods generally improve during the first adaptation checkpoints and then level off. Full Fine-Tuning is highest or close to highest across the panels. MemFLoRA-SG is generally the strongest low-rank method and is closely followed by MemFLoRA. The RealWorld T-ResNet panel shows the widest separation among methods, whereas the MobileNetV2 trajectories are more tightly clustered, particularly on RealDisp. 4.3. Design Ablations and Training Cost Table 4 summarizes three design choices. First, Gaussian initialization consistently outperforms the orthogonalized projection across the three evaluated ranks. We therefore use Gaussian initialization in the remaining experiments. Second, placing the spatial kernel in the trainable factor improves macro-F1 while preserving the same rank-bounded saved-state. Finally, the scale-normalization ablation shows that adding an unnormalized adapter to the normalized backbone output causes performance to collapse. Either bottleneck normalization or the frozen source-BN scale restores a usable adapter scale, but the source-BN scale alone gives the best average result. Following prior on-device adaptation protocols, we calibrate AdaBN using one batch of target data, showing that it has a negligible effect. Entries in the table report mean macro-F1 over all target subjects from an independent ablation sweep; differences from Table 1 are within seed-level variation. Table 4. Ablations on Opportunity with T-ResNet at 50 adaptation steps. Factor Variant r=2r=2 r=4r=4 r=8r=8 Init. Normal P 0.759 0.777 0.802 Orthogonal P 0.673 0.709 0.751 Geom. 1×11×1 down, k×k×k up 0.760 0.777 0.802 k×k×k down, 1×11×1 up 0.731 0.755 0.773 Scale None 0.200 0.197 0.187 BNrBN_r 0.728 0.772 0.805 asa_s 0.764 0.792 0.808 AdaBN Off 0.753 0.779 0.804 On 0.762 0.783 0.801 Table 5 summarizes the training-side cost at rank 2 on Opportunity. MemFLoRA closely matches the computational cost of LoRA-Edge: 222.5222.5 versus 218.6218.6 million MACs for T-ResNet and 286.0286.0 versus 283.2283.2 million MACs for MobileNetV2. These correspond to only 70% and 71% of the Full Fine-Tuning cost, respectively. MemFLoRA uses more trainable parameters than LoRA-Edge on T-ResNet (10.610.6K versus 2.32.3K) and the same number on MobileNetV2 (19.819.8K). This is a deliberate geometry-level trade-off: the trainable parameter footprint remains small, while the adapter directly reduces the activations saved for backward, which dominate CNN training memory. MemFLoRA therefore provides comparable or better adaptation quality and nearly identical computational cost to LoRA-Edge, while substantially reducing peak training-state memory. MemFLoRA-SG retains the same number of trainable parameters but requires 104% and 106% of the Full Fine-Tuning MACs because of its additional forward replay. It therefore offers higher adaptation accuracy in exchange for additional computation, while remaining in the same low-memory regime as MemFLoRA. To isolate the convergence behavior during adaptation, we measure the adaptation steps until each method first reaches 85% of the final Full FT macro-F1 at the adaptation checkpoints shown in Figure 3. MemFLoRA and MemFLoRA-SG reach the threshold fastest on T-ResNet and are comparable to LoRA-Edge on MobileNetV2. Table 5. Training-performance summary at rank 2. Total adaptation MACs are reported as AdaBN calibration MACs plus 50 adaptation steps. Backbone Method Par. [103][10^3] MACs [106][10^6] Relative cost [%][\%] Steps to 85%85\% [n][n] T-ResNet Full FT 560.9 320.0 – – LoRA-C 12.8 320.2 100 22 LoRA-Edge 2.3 218.6 68 24 MemFLoRA 10.6 222.5 70 19 MemFLoRA-SG 10.6 331.7 104 15 MobileNetV2 Full FT 2250 403.0 – – LoRA-C 37.1 390.9 97 32 LoRA-Edge 19.8 283.2 70 24 MemFLoRA 19.8 286.0 71 24 MemFLoRA-SG 19.8 426.0 106 23 MemFLoRA also outperforms non-LoRA activation-minimal approaches, such as TinyTL (Cai et al., 2020). Since TinyTL is not a plug-and-play LoRA adapter, its implementation requires architecture-specific design choices; we therefore evaluate both an activation-minimal frozen-block variant with eval-mode BN and a train-BN variant that preserves BN adaptation at higher memory cost. On Opportunity, these variants reach 0.682 and 0.737 macro-F1 with 150.47 MB and 518.83 MB peak training-state memory, respectively. In contrast, rank-2 MemFLoRA reaches 0.747 macro-F1 with only 17.83 MB, showing that our low-rank memory-floor design provides a simpler adapter-style alternative with substantially lower training memory. 4.4. Jetson Orin Deployment Validation We validate training-state memory on an NVIDIA Jetson Orin Nano (8 GB) using both backbones and all methods at r=2r=2 and B=64B=64. We measure one complete update after two warm-up updates using Opportunity-shaped inputs. The device profiler tracks allocations associated with model parameters and buffers, materialized optimizer state, gradients, saved-backward state, and total training state. Peak memory denotes the maximum concurrent storage of these tracked training-state components, whereas saved-backward state is measured separately at the end of the forward pass. Pre-existing inputs and labels, checkpoint copies, general intermediates and workspaces, and allocator cache are outside this metric. Table 6. Training-state memory measured at rank r=2r=2 and batch size B=64B=64. Entries report peak concurrent training-state storage / saved-backward state [MB]. Method T-ResNet MobileNetV2 Full FT 54.52 / 47.77 666.72 / 639.55 LoRA-C 52.37 / 49.96 592.61 / 583.04 LoRA-Edge 35.12 / 32.79 491.21 / 481.78 MemFLoRA 3.22 / 0.83 17.83 / 8.39 At B=64B=64, MemFLoRA reduces saved-backward storage relative to Full FT by 98.3% and 98.7% on T-ResNet and MobileNetV2, respectively, with corresponding peak training-state reductions of 94.1% and 97.3%. These measurements closely follow the memory reduction trend estimated in Table 3, where the corresponding reductions were 98.5%/98.7% for saved state and 94.9%/97.3% for peak training state. Nevertheless, the deployment measurements preserve the large training-state memory advantage of MemFLoRA. Table 7 reports the peak requested CUDA memory from the same runs. This footprint includes training state, input batches, transient tensors, and workspaces, but excludes unused allocator cache and allocation rounding. These additional runtime allocations lie outside the training-state memory floor of Section 3.1. Table 7. Peak requested CUDA memory at r=2r=2, B=64B=64 [MB], from the same runs as Table 6. Values include inputs, temporaries, and workspaces in addition to training state. Method T-ResNet MobileNetV2 Full FT 66.20 673.39 LoRA-C 64.08 601.45 LoRA-Edge 44.80 503.32 MemFLoRA 19.43 95.43 Figure 4 traces the requested memory through the forward and backward passes of one MobileNetV2 update at B=64B=64, recorded in a separate Full FT and MemFLoRA run; the optimizer step adds no peak and is omitted. Full FT peaks at the start of the backward pass, when 639.2 MB of saved activations are still held. MemFLoRA instead peaks early in the forward pass. The first, high-resolution layers produce the largest feature maps, and their transient intermediates occupy 82.7 of MemFLoRA’s 95.4 MB peak, while only 1.5 MB is saved for backward. These buffers are released after each layer instead of accumulating. A second, nearly equal spike (95.2 MB) occurs at the end of the backward pass, when backpropagation reaches the same early layers and their full-width activation gradients are formed. Full FT shows the same late spike in working buffers, but by then most of its saved activations have been released. MemFLoRA’s peak is therefore set by the transient working set of the largest layers, not by retained state. Two stacked area plots of requested GPU memory over one training update. Full FT rises during the forward pass to a peak of 673.4 MB dominated by saved activations, then declines during the backward pass. MemFLoRA stays low, with a narrow 95.4 MB spike of working buffers early in the forward pass and a second 95.2 MB spike at the end of the backward pass. Figure 4. Requested GPU memory during one MobileNetV2 update at r=2r=2, B=64B=64 on Jetson Orin Nano.Two stacked area plots of requested GPU memory over one training update. Full FT rises during the forward pass to a peak of 673.4 MB dominated by saved activations, then declines during the backward pass. MemFLoRA stays low, with a narrow 95.4 MB spike of working buffers early in the forward pass and a second 95.2 MB spike at the end of the backward pass. 5. Conclusion MemFLoRA shows that efficient edge CNN adaptation depends not only on trainable parameters but also on activations retained for backpropagation. It reduces saved-activation memory by 98.5–98.7% and peak training-state memory by 94.9–97.3% relative to full fine-tuning. By defining an activation-memory-floor criterion, deriving the design constraints it imposes, and showing that adaptation quality can be preserved while satisfying it, this work reframes LoRA for on-device CNN adaptation as a saved-state design problem rather than merely a parameter-count problem, providing a foundation for future edge adaptation research. Acknowledgements. This work has been developed in the project DI-EDAI funded by the German Federal Ministry of Research, Technology and Space (BMFTR) under contract No. 16ME0991. References Aleem et al. (2024) S. Aleem, J. Dietlmeier, E. Arazo, and S. Little ConvLoRA and adabn based domain adaptation via self-training. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), Vol. , p. 1–5. External Links: Document Cited by: §2.1. Banos et al. (2014) O. Banos, M. A. Toth, M. Damas, H. Pomares, and I. Rojas Dealing with the effects of sensor displacement in wearable activity recognition. Sensors 14 (6), p. 9995–10023. External Links: ISSN 1424-8220, Document Cited by: Table 2. Ben Zaken et al. (2022) E. Ben Zaken, Y. Goldberg, and S. Ravfogel BitFit: simple parameter-efficient fine-tuning for transformer-based masked language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), S. Muresan, P. Nakov, and A. Villavicencio (Eds.), Dublin, Ireland, p. 1–9. External Links: Document Cited by: §4. Cai et al. (2020) H. Cai, C. Gan, L. Zhu, and S. Han TinyTL: reduce memory, not parameters for efficient on-device learning. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 11285–11297. Cited by: §1, §2.2, §4.3. Chavarriaga et al. (2013) R. Chavarriaga, H. Sagha, A. Calatroni, S. T. Digumarti, G. Tröster, J. del R. Millán, and D. Roggen The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34 (15), p. 2033–2042. Note: Smart Approaches for Human Action Recognition External Links: ISSN 0167-8655, Document Cited by: Table 2. Chen et al. (2021) J. Chen, L. Zheng, Z. Yao, D. Wang, I. Stoica, M. Mahoney, and J. Gonzalez ActNN: reducing training memory footprint via 2-bit activation compressed training. In Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang (Eds.), Proceedings of Machine Learning Research, Vol. 139, p. 1803–1813. Cited by: §2.2. Chen et al. (2016) T. Chen, B. Xu, C. Zhang, and C. Guestrin Training deep nets with sublinear memory cost. ArXiv abs/1604.06174. Cited by: §2.2, §3.5. Ding et al. (2024) C. Ding, X. Cao, J. Xie, L. Fan, S. Wang, and Z. Lu LoRA-c: parameter-efficient fine-tuning of robust cnn for iot devices. ArXiv abs/2410.16954. Cited by: §2.1, §4. Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §1, §2.1. Kingma and Ba (2014) D. P. Kingma and J. Ba Adam: a method for stochastic optimization. CoRR abs/1412.6980. Cited by: §4. Kwak et al. (2026) H. Kwak, K. Lee, J. Lee, and W. Lee LoRA-edge: tensor-train–assisted lora for practical cnn fine-tuning on edge devices. In 2026 Design, Automation & Test in Europe Conference (DATE), Vol. , p. 1–7. External Links: Document Cited by: §2.1, §4, §4, §4. Li et al. (2018) Y. Li, N. Wang, J. Shi, X. Hou, and J. Liu Adaptive batch normalization for practical domain adaptation. Pattern Recognition 80, p. 109–117. External Links: ISSN 0031-3203, Document Cited by: §2.1. Lin et al. (2022) J. Lin, L. Zhu, W. Chen, W. Wang, C. Gan, and S. Han On-device training under 256kb memory. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 22941–22954. Cited by: §2.2. Liu et al. (2022) X. Liu, L. Zheng, D. Wang, Y. Cen, W. Chen, X. Han, J. Chen, Z. Liu, J. Tang, J. Gonzalez, M. W. Mahoney, and A. Cheung GACT: activation compressed training for generic network architectures. In International Conference on Machine Learning, Cited by: §2.2. Matsutani et al. (2025) H. Matsutani, M. Kondo, K. Sunaga, and R. Marculescu Skip2-lora: a lightweight on-device dnn fine-tuning method for low-cost edge devices. In Proceedings of the 30th Asia and South Pacific Design Automation Conference, ASPDAC ’25, New York, NY, USA, p. 51–57. External Links: ISBN 9798400706356, Document Cited by: §2.1. Nadalini et al. (2023) D. Nadalini, M. Rusci, L. Benini, and F. Conti Reduced precision floating-point optimization for deep neural network on-device learning on microcontrollers. Future Generation Computer Systems 149, p. 212–226. External Links: ISSN 0167-739X, Document Cited by: §1. Novikov et al. (2023) G. S. Novikov, D. Bershatsky, J. Gusak, A. Shonenkov, D. V. Dimitrov, and I. Oseledets Few-bit backward: quantized gradients of activation functions for memory footprint reduction. In Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, p. 26363–26381. Cited by: §2.2. Patil et al. (2022) S. G. Patil, P. Jain, P. Dutta, I. Stoica, and J. Gonzalez POET: training neural networks on tiny devices with integrated rematerialization and paging. In Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, p. 17573–17583. Cited by: §2.2. Ridnik et al. (2021) T. Ridnik, H. Lawen, A. Noy, E. Ben Baruch, G. Sharir, and I. Friedman TResNet: high performance gpu-dedicated architecture. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 1400–1409. Cited by: §4. Sandler et al. (2018) M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen MobileNetV2: inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §4. Sunaga et al. (2023) K. Sunaga, M. Kondo, and H. Matsutani Addressing the gap between training data and deployed environment by on-device learning. IEEE Micro 43 (6), p. 66–73. External Links: Document Cited by: §1. Sztyler and Stuckenschmidt (2016) T. Sztyler and H. Stuckenschmidt On-body localization of wearable devices: an investigation of position-aware activity recognition. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Vol. , p. 1–9. External Links: Document Cited by: Table 2. Zhang et al. (2023) L. Zhang, L. Zhang, S. Shi, X. Chu, and B. Li LoRA-fa: memory-efficient low-rank adaptation for large language models fine-tuning. arXiv preprint arXiv:2308.03303. Cited by: §2.1. Zhong et al. (2024) Z. Zhong, Z. Tang, T. He, H. Fang, and C. Yuan Convolution meets lora: parameter efficient finetuning for segment anything model. ArXiv abs/2401.17868. Cited by: §2.1. Zhu et al. (2024) J. Zhu, K. Greenewald, K. Nadjahi, H. S. De Ocáriz Borde, R. B. Gabrielsson, L. Choshen, M. Ghassemi, M. Yurochkin, and J. Solomon Asymmetry in low-rank adapters of foundation models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. Cited by: §2.1. Zhu et al. (2023) L. Zhu, L. Hu, J. Lin, W. Chen, W. Wang, C. Gan, and S. Han PockEngine: sparse and efficient fine-tuning in a pocket. In Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO ’23, New York, NY, USA, p. 1381–1394. External Links: ISBN 9798400703294, Document Cited by: §1.