Paper deep dive
Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning
Douwe den Blanken, Martin Lefebvre, Charlotte Frenkel
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/3/2026, 2:56:25 AM
Summary
This paper introduces Embedder-Centric Learning (ECL), a framework that unifies few-shot learning (FSL), continual learning (CL), zero-shot learning (ZSL), and in-context learning (ICL) for on-device adaptation at the edge. By splitting learning scenarios into a shared temporal embedder (using TCNs) and scenario-specific heads, ECL enables versatile, energy-efficient learning on resource-constrained hardware (Chameleon SoC) without cloud reliance, achieving state-of-the-art results across multiple benchmarks.
Entities (12)
Relation Signals (10)
Chameleon SoC â implements â Embedder-Centric Learning
confidence 95% · We validate the performance and low cost of the ECL framework ... using our recently introduced Chameleon system-on-chip (SoC)
Embedder-Centric Learning â unifies â Continual Learning
confidence 95% · ECL unifies four different online learning scenarios: ... CL...
Embedder-Centric Learning â unifies â Zero-shot Learning
confidence 95% · ECL unifies four different online learning scenarios: ... ZSL...
Embedder-Centric Learning â unifies â In-context Learning
confidence 95% · ECL unifies four different online learning scenarios: ... ICL
Embedder-Centric Learning â unifies â Few-shot Learning
confidence 95% · ECL unifies four different online learning scenarios: FSL... CL... ZSL... and ICL
Embedder-Centric Learning â uses â Temporal Convolutional Networks
confidence 92% · We therefore propose to use temporal convolutional networks (TCNs) as the temporal embedder for ECL.
Embedder-Centric Learning â achievesbaselineon â NeuroBench
confidence 90% · first hardware baseline for CL in keyword spotting (NeuroBench keyword FSCIL: 71.8% for 200-way 5-shot)
Embedder-Centric Learning â achievesstateofthearton â
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:With the ever-increasing pervasiveness of smart edge devices, the demand is growing for applications that can be tailored to users (e.g., custom keyword spotting) or patients (e.g., adaptive health monitoring). Yet, most edge devices rely on fixed inference algorithms and thus cannot learn on-device to personalize predictions. When they can, devices typically support only a specific learning scenario, such as few-shot learning (FSL): going beyond this requires resorting either to another specialized device or to cloud-based retraining, which implies significant energy and latency overheads, a lack of real-time capabilities, and privacy concerns. In this work, we introduce embedder-centric learning (ECL), a framework that unifies four different online learning scenarios: FSL for on-the-fly customization, continual learning (CL) for knowledge accumulation, zero-shot learning (ZSL) for leveraging semantic data, and in-context learning (ICL) for adapting beyond classification. We demonstrate in silicon that ECL can be deployed on resource-constrained devices across four real-world use cases representative of the aforementioned learning scenarios. Our approach establishes a new state-of-the-art performance for FSL character recognition (Omniglot: 96.8% for 5-way 1-shot, 83.3% for 32-way 1-shot), and the first hardware baseline for CL in keyword spotting (NeuroBench keyword FSCIL: 71.8% for 200-way 5-shot). Moreover, we present the first hardware demonstrations of ZSL with semantic data (60.6% for 5-way spoken sentence classification) and ICL (46.2% at the 500th token of RegBench) operating at micro-to-milliwatt power budgets. Therefore, by unifying multiple learning scenarios, we pave the way for smart and versatile devices that can adapt right at the edge, without reliance on the cloud.
Tags
Links
- Source: https://arxiv.org/abs/2607.29353v1
- Canonical: https://arxiv.org/abs/2607.29353v1
Trouble viewing inline? Open PDF directly â
Full Text
69,709 characters extracted from source content.
Expand or collapse full text
1 Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning Douwe den Blanken, Graduate Student Member, IEEE, Martin Lefebvre, Member, IEEE and Charlotte Frenkel, Member, IEEE AbstractâWith the ever-increasing pervasiveness of smart edge devices, the demand is growing for applications that can be tailored to users (e.g., custom keyword spotting) or patients (e.g., adaptive health monitoring). Yet, most edge devices rely on fixed inference algorithms and thus cannot learn on-device to personalize predictions. When they can, devices typically support only a specific learning scenario, such as few-shot learning (FSL): going beyond this requires resorting either to another specialized device or to cloud-based retraining, which implies significant energy and latency overheads, a lack of real-time capabilities, and privacy concerns. In this work, we introduce embedder-centric learning (ECL), a framework that unifies four different online learning scenarios: FSL for on-the-fly customization, continual learning (CL) for knowledge accumulation, zero-shot learning (ZSL) for leveraging semantic data, and in-context learning (ICL) for adapting beyond classification. We demonstrate in silicon that ECL can be deployed on resource-constrained devices across four real-world use cases representative of the aforementioned learning scenarios. Our approach establishes a new state-of- the-art performance for FSL character recognition (Omniglot: 96.8% for 5-way 1-shot, 83.3% for 32-way 1-shot), and the first hardware baseline for CL in keyword spotting (NeuroBench keyword FSCIL: 71.8% for 200-way 5-shot). Moreover, we present the first hardware demonstrations of ZSL with semantic data (60.6% for 5-way spoken sentence classification) and ICL (46.2% at the 500th token of RegBench) operating at micro-to-milliwatt power budgets. Therefore, by unifying multiple learning scenarios, we pave the way for smart and versatile devices that can adapt right at the edge, without reliance on the cloud. I. INTRODUCTION I N the last decade, the integration of neural networks (NNs) on resource-constrained devices at the edge has become in- creasingly commonplace. Example use cases include navigation for drones [1], keyword spotting (KWS) on smart speakers [2], [3], [4], [5], and health monitoring on wearables [6], [7], [8], [9], [10]. However, most of these devices rely on fixed, pre-trained NNs that cannot be adapted post-deployment. Consequently, these devices cannot locally handle shifting data distributions, emerging features, specific or additional users, or evolving task requirements [11], such as the sim-to-real gap for drones, new keywords for KWS, and user- or patient-specific tailoring for wearables. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. This publication was funded by the Dutch Research Council (NWO) as part of the projects AdaptEdge (file number 20267 in the NWO Talent Programme â Veni) and Transforming the Adaptability of Decentralized AI (file number NGF.1609.242.038 in the NGF - AiNed AiNed XS Europe programme). Douwe den Blanken, Martin Lefebvre and Charlotte Frenkel are with the Microelectronics Department (EEMCS Faculty), Delft University of Technology, 2628 CD Delft, Netherlands (e-mail: d.m.j.denblanken@tudelft.nl; m.lefebvre@tudelft.nl; c.frenkel@tudelft.nl; Corresponding author: Douwe den Blanken). Temporal embedder Query sample Support samples Query Support CL FSL ZSL ICL Flexible learning scenario selection Wide range of modalities âș âč Lack of exibility across multiple learning scenarios Learning scenarios Sensory modalities CL FSL ICL ImagesAudio ZSL Tokens âș O-device adaptation âč Breadth = Fundamental challenge Strength Limitations Energy-ecientHigh accuracyMemory-ecient No on-device learning High latency, power, mem. overhead Limited accuracy & scalability Constrained inference perf. High energy, latency and privacy costs Approach Adapt on-chip Scenario- specic head Low dim. embeddings Fig. 1. (a) Overview of current approaches to learning at the edge and their limitations. (b) Learning at the edge requires supporting different learning scenarios, such as few-shot learning (FSL), continual learning (CL), zero- shot learning (ZSL), and in-context learning (ICL), while accommodating for different sensory modalities, forming a fundamental challenge. (c) Outline of the proposed embedder-centric learning (ECL) framework, which splits each learning scenario into a shared embedder and a scenario-specific head. This embedder-centric framing lets ECL support a wide range of sensory modalities while unifying four learning scenarios. Addressing these needs is challenging, due to the various limitations of existing strategies to adapt (Fig. 1(a)). We group these strategies into three categories. The first category, comprising inference accelerators, relies on the cloud for off-deviceadaptation, which comes at the expense of the energy cost for the cloud link [12], [13], latency penalties that preclude online learning [13], [14], and risks of exposing user private data [13], [14]. The second trains a model from scratch with backpropagation (BP) directly on the device [15], [16], [17] but requires storing all intermediate activations, which is especially prohibitive for long temporal signals [11]. The third aims to alleviate this overhead of BP by implementing specialized learning algorithms in hardware [18], [19], [20], [21], or by exploring algorithms based on local learning rules [11], [22]. However, while multi-layer variants start emerging [23], [24], arXiv:2607.29353v1 [cs.LG] 31 Jul 2026 2 their hardware implementation remains an open challenge to the best of our knowledge. Furthermore, custom learning- optimized hardware typically trades inference efficiency for learning accuracy [25]. Therefore, enabling on-chip learning at minimal hardware cost remains a challenge. Beyond efficiently supporting a single learning scenario on-chip, real-world deployment must cope with diverse data dynamics, label availability, and task structures. This demands support for additional learning scenarios, also across sensory modalities, from images to long-timescale audio (Fig. 1(b)). This breadth forms a fundamental challenge: current accelera- tors cannot accommodate more than one learning scenario, yet designing a specialized chip for every combination of scenario and sensory modality is impractical. Therefore, we aim to answer the following question: How can we optimally support different learning scenarios for versatility and efficiency across use cases at the edge? In this work, we propose the embedder-centric learning (ECL) framework, which unifies few-shot learning (FSL), continual learning (CL), zero-shot learning (ZSL), and in- context learning (ICL) with support for a wide range of sensory modalities (Fig. 1(c)). Our core idea is to frame each learning scenario in an embedder-centric way, splitting it into two components: a strong embedder N, and a scenario-specific head of one or more fully-connected (FC) layers that processes the resulting embeddings into a prediction. This embedder- centric design is effective for two reasons. First, embeddings represent data in a reduced-dimensional space, keeping the knowledge memory small enough to maintain and reuse on- device across FSL, CL, ZSL, and ICL. Second, an embedder that excels on temporal data, especially across long temporal dependencies, lets ECL support different sensory modalities efficiently: audio (provided raw or sequentially as Mel- frequency cepstral coefficients (MFCCs) [26] frames), images (provided sequentially as pixels), or, more generally, tokens. We validate the performance and low cost of the ECL frame- work for each of the learning scenarios both in software and hardware using our recently introduced Chameleon system- on-chip (SoC) [25]. We surpass state-of-the-art (SotA) FSL accuracies on the Omniglot dataset [27] (96.8% for 5-way 1- shot, 83.3% for 32-way 1-shot), and we demonstrate for the first time on-device CL on the NeuroBench keyword few-shot class- incremental learning (FSCIL) dataset to classify 200 classes at a9.5 ÎŒWreal-time power. We also perform ZSL using semantic data for the first time on-device, using only3.1 ÎŒJto learn five new classes. In addition, we present the first demonstration of on-device ICL on a formal language, requiring only16.8 ÎŒJ per token. These results establish new baselines across learning scenarios and sensory modalities, and underline our embedder- centric approach as an enabler for learning at the edge. The remainder of this article is structured as follows. First, Section I introduces the four learning use cases at the edge and the corresponding learning scenarios considered in this work. Then, Section I presents our proposed ECL framework and details its implementations of the FSL, CL, ZSL, and ICL learning scenarios. Then, Section IV outlines our test setup and presents hardware and software results for each learning scenario, followed by a synopsis of the overall results. Corresp. learning scenarios Accumulate knowledge over time âMorningâ âWake up!â âHeyâ Customize on the y Adapt using semantic information only Adapt beyond classication Apple â Pomme Book â ???? Dierent learning use cases Cat â Chat Car â Voiture shell beak fin Fig. 2. (a) Comparison of four different learning use cases, from customizing to new tasks on the fly to adapting beyond classification tasks. (b) To enable each of these use cases, a learning scenario can be adopted: FSL, CL, ZSL, or ICL, each corresponding to a use case. Finally, we offer concluding remarks in Section V. To promote reproducibility, reuse, and improvement, all code for this paper is open-source, including the training frameworks, accelerator source code, and the test/simulation setup. 1 I. BACKGROUND: ON-DEVICE LEARNING USE CASES AT THE EDGE Figs. 2(a) and (b) respectively depict four different use cases for learning at the edge and the learning scenarios that are used to tackle them. The first highlighted use case is to customize to new data or tasks on the fly. Examples of this include learning with only a few examples to detect new keywords [30], to classify novel image classes [31], and to detect custom gestures [32]. Few-shot learning (FSL) addresses this problem of recognizing new classes that were not seen during pre-training using only a few labeled samples per class. The new classes are referred to as ways while the samples per class are referred to as shots [28], [33], [34], [35], [36]. As FSL only requires limited data, it is a particularly good fit for the edge, where the online appearance of new data is often scarce (e.g., examples given by users or rare events not accommodated for during training). FSL, however, requires every new class to be available at once. When classes instead emerge incrementally, the model must accumulate knowledge over time rather than acquire it in a single step. Examples include incrementally learning additional image classes [37], such as a growing alphabet of handwritten characters, or learning to detect an expanding set of keywords [38]. To handle this second set of use cases, continual learning (CL) can be applied. CL allows an N to gain knowledge of new tasks or data distributions over time, without forgetting the previously acquired knowledge [39]. However, both FSL and CL still depend on labeled data, which the edge cannot always provide. A third category of use cases instead offers only semantic information from different sensory modalities, such as high- 1 https://github.com/cogsys-tudelft/ecl 3 Temporal embedder Query sample Support samples Query Support Embeddings Scenario- specic head Prediction Semantic embedder Embedder Embedder Concat Embedder MLP layer Concat MLP layer Support Query α ÎČ Îł ÎŽ 3 ways Semantic properties 2 shots 2 ways Query sample Support samples FC layer T = 0T = 1T = 2 Avg. embeds. Convert to w+b Query sample Embedder FC layer Δ Îł ÎŽ T = 0 T = 1 T = 2 FCFC âRelation headâ âPrototypeâ L2 distance calc. = Logit Classes Compressed accu- mulated knowledge Di. sensory modalities L2 distance calc. = Support samples Query sample Supp. Query Supp. Query Support Query Fig. 3. Overview of the ECL framework. (a) ECL unifies four online learning scenarios through an embedder-centric formulation: support and query samples are embedded before a scenario-specific head, consisting of one or more FC layers, produces a prediction on the query sample. (b) FSL in ECL uses prototypical networks (PNs) [28]: support embeddings are averaged class-wise into prototypes, and the query sample is assigned to its nearest prototype. Prototypes are transformed into an equivalent representation in terms of weights and biases (w + b) of an FC layer, to compute the distance between the query sample and the prototypes efficiently. (c) CL in ECL reuses the same PN-based approach, but appends the parameters corresponding to a new class (learned at timeTâ„ 1) to existing FC layersâ parameters (learned during an initial training phase atT = 0), instead of overwriting them. (d) ZSL in ECL uses a relation network (RN) [29] with two embedders, one for the semantic support data and one for the query sample; their embeddings are concatenated and passed to a relation head, here an multi-layer perceptron (MLP), for the final prediction. (e) ICL in ECL concatenates the query and support samples before embedding, and the scenario-specific head feeds the joint embedding to an MLP to perform the prediction. level attributes, audio, or textual descriptions. Examples of this include animal classification from high-level properties only [40] or spoken-sentence detection from transcriptions alone. Zero-shot learning (ZSL) enables learning from such semantic information alone, removing the need for labeled test-domain data for each new task [28], [40]. In effect, ZSL learns new tasks using a sensory modality different from that of the test data. Each scenario so far stops at classification. Yet many edge use cases call for more, for example, forecasting temporal data or few-shot language modeling, forming the fourth and final group of use cases. Using in-context learning (ICL), use cases such as regression or next-token prediction tasks [41], [42], [43], where the next symbol should be predicted given a vocabulary and labeled input-output pairs, can also be dealt with. ICL relies purely on the forward pass of a model: learning happens by building memory associations between a sequence of labeled examples combined with a test input inside a context window, without modifying any model parameters [43]. ICL was first observed as an emergent property of the transformer architecture [44] in large-scale language modeling [43]. However, it has recently been shown that it is not exclusive to these models [45]. Supporting these four learning scenarios at the edge con- ventionally requires a separate specialized system for each. However, this is impractical under the tight compute and memory budgets of edge devices. Hence, how to efficiently support all four within one device remains an open problem. I. THE ECL FRAMEWORK FOR VERSATILITY AT THE EDGE In this work, we propose a framework called ECL, that unifies the above four learning scenarios by framing each of them in an embedder-centric way. ECL thus allows splitting each learning scenario into two key components illustrated in Fig. 3(a): (i) an embedder N and (i) a scenario-specific head. First, the N maps support samples, the labeled data for learning, and a query sample, the input to process after learning, to an embedding, i.e., a feature vector of substantially lower dimensionality than the input. Together, the embedded support samples represent a compressed form of accumulated knowledge. Second, the scenario-specific head processes em- bedded knowledge together with the query embedding using one or several fully-connected (FC) layers, in a way that 4 is dependent on the learning scenarios. Through this shared structure, ECL efficiently supports versatile adaptation. While our framework removes the need for designing a specialized implementation per scenario, supporting adaptation across sensory modalities, however, would still require a different N type for each. To avoid this, ECL requires an N that can embed samples containing long temporal dependencies, such as audio consisting of MFCC frames or even raw samples, images converted into pixel streams, or effectively any data type provided as tokens. We therefore propose to use temporal convolutional networks (TCNs) [46] as the temporal embedder for ECL. TCNs are NNs that use stacked causal 1D convolutions combined with residuals to capture long-range relationships in sequential data [46]. These NNs can extract high-quality embeddings from long sequences while their memory cost scales only logarithmically with sequence length [25], allowing them to efficiently support a wide range of sensory modalities. While a new embedder still has to be pre-trained off-chip for each new sensory modality, the target deployment platform for ECL only needs to support one N type, simplifying the hardware requirements. Independent of the chosen embedder, by construction, the embedder-centric stance of ECL itself yields three further advantages for hardware implementation. First, the target hardware only needs to support regular N inference, since generating an embedding is equivalent to a forward pass. Second, the knowledge memory that stores embeddings can stay small (a fewkB), because embeddings have far lower dimensionality than the input samples. Third, formulating the scenario-specific head as one or more FC layers lets ECL execute within the same N inference pipeline, requiring only minor control logic to construct the layer. Overall, ECL avoids learning-specific memories and introduces only minor control overhead while relying on standard inference hardware, bringing efficient learning across FSL, CL, ZSL, and ICL to extreme-edge devices. Furthermore, let us now explain how the ECL framework implements each learning scenario (Fig. 3), by detailing the operation of the embedders and scenario-specific heads. 1) FSL Implementation in ECL: For FSL deployment, we use a technique called prototypical networks (PNs) [28]. In PNs, an embedder N is used to embed the available support samples. A prototype for a new class is then formed by averaging the embeddings from the support samples for that class (Fig. 3(b)). After embedding, an unknown query sample is then classified as the class of the prototype with the lowest L2 distance to its embedding [28]. To efficiently support PNs in ECL, we employ the equivalent transformation of PNs into weights and biases that parametrize a single FC layer [28]: exact L2- based nearest neighbor classification can then be performed in a single matrix-vector multiplication instead of requiring additional hardware for distance computation, a technique also used in Chameleon [25]. This equivalent FC layer constitutes the scenario-specific head for FSL. 2) CL Implementation in ECL: For CL, we largely follow the same procedure as for FSL. In FSL, however, the equivalent FC layer can be computed instantaneously, as all support data is present at adaptation time. In CL, on the other hand, we assume that new classes appear over time, denoted asTâ„ 1, after an initial adaptation phase, denoted asT = 0(Fig. 3(c)). Therefore, instead of calculating the FC parameters once and overwriting the previous ones, we append new entries to the PNâs FC parameters over time for each additional class while keeping the previous FC parameters unchanged, thereby alleviating catastrophic forgetting. 3) ZSL Implementation in ECL: To perform ZSL in ECL, we use a relation network (RN) [29]. Although PNs also support ZSL [28], we choose an RN since it learns a distance function rather than assuming one (for example, L2 in PNs), which can improve performance for ZSL [29]. Since ZSL uses two data modalities, we require two embedder NNs for the RN (Fig. 3(d)). The first N produces support embeddings on a per-class basis to learn from the provided semantic information. This N can either be a TCN or a multi-layer perceptron (MLP), depending on whether the semantic information is temporal or not. The second N then produces the embedding for the to- be-classified sample in the target domain. All embeddings are then concatenated and fed into an MLP, the so-called relation head, for prediction. These last two operations constitute the scenario-specific head for ZSL. In this way, the two embedder NNs specialize in producing high-quality embeddings in their input domain while the relation head learns to combine this information into a prediction. 4) ICL Implementation in ECL: For ICL, the dataflow is slightly different from the previous learning scenarios. Instead of concatenating support and query samples after the embed- ding step like in ZSL, we concatenate all samples before pro- cessing with the embedder network (Fig. 3(e)). For ZSL, this is not possible as the scenario incorporates two different data types and hence requires two different embedders. For ICL, however, this order reversal has two key implications. First, it gives the embedder network the ability to directly model relationships between all inputs, instead of the relation head in ZSL. Second, it allows the TCN to produce one output prediction for each in- put, which is necessary for tasks such as regression or language modeling. After this joint-embedding step, a single next-output prediction can be calculated using one or more FC layers in an MLP, which forms the scenario-specific head for ICL. IV. RESULTS This section presents the experimental results from using the ECL framework. First, SectionIV-Aintroduces our hardware test platform. Then, for each learning scenario (SectionsIV-B toIV-E), we describe the representative benchmark or dataset used in this work, followed by software results before presenting ECLâs performance when deployed in hardware. SectionIV-Ffinally outlines a synopsis of the obtained results. A. Test Platform To validate ECL in silicon, we use the Chameleon SoC [25] as our test platform. Chameleon is an N accelerator optimized for generating and processing dense embeddings from temporal data using TCNs. Fig. 4(a) outlines Chameleonâs architecture. The SoCâs matrix-vector processing element (PE) array uses 5 Learning controller FSL CL ZSL ICL Extract W + b? # inner steps shots shots ways 1 ways ways 1 1 # outer steps Append to mems? Load ... ? FC FC Train embds. - 4-bit log 2 Mat.-vec. PE array x << w Mul-free PE Weight mem. (65 kB) Act. mem. (2 kB) Bias mem. (3.5 kB) Proto. to FC params 4-bit Infer. contr- oller Learn. contr- oller Controller used Store embeddings ECL component Active module Inference Learning (1 way) Embedder Embedder EmbedderEmbedderEmbedder FC PE arrayPE arrayPE arrayPE arrayPE array 1 shot Inference Learning (2 ways; T = 0) Embedder Embedder PE array 1 shot PE array FC Embedder Embedder PE array PE array FC Embedder PE array 1 shot Inference Learning (1 way; T = 1) Time Time Sem. emb. PE array Sem. emb. PE array Sem. emb. PE array Sem. emb. PE array Embedder MLP PE array Time PE array Time PE arrayPE array Embedder MLP PE array Inference Sem. emb. PE array Learning (5 ways) Learn + infer Embedder MLP Embedder MLP Embedder MLP Load embeddings Replace embedder Controller used ECL component Active module Controller used ECL component Active module Controller used ECL component Active module 1 shot Learning (1 way) Learn + infer Learn + infer Learn + infer Outer loop step Inner loop step Load train embeds. 2 ways learned 3 ways learned Prot. FC Use PE array for averaging embeddings W b Fig. 4. (a) Simplified architecture overview of Chameleon. The SoC includes a matrix-vector processing element (PE) array which uses 4-bitlog 2 weights for multiplier-free PEs and 4-bit activations. Together with the inference control logic, it forms the SoCâs N inference data path. (b) Comparison of Chameleonâs configurations per learning scenario, which are implemented by the SoCâs learning controller. (c)-(f) Qualitative illustration of ECLâs execution schedules on Chameleon for FSL, CL, ZSL and ICL respectively. We show per scenario which main module and controller are used over time for the two ECL components. Overall, ECLâs mapping to the SoC can be partitioned in three phases: (i) embedding all support samples for a single way (inner loop), (i) optionally converting them to prototypical parameters (outer loop), and (i) using the support embeddings, FC or MLP with the embedded query sample to predict. Dashed brackets indicate inner loop steps, while regular black brackets indicate outer loop steps. Blue brackets indicate the inference phase. Colors in (c)-(f) match the blocks in (a) and components in Fig. 3. inference-optimized quantization with 4-bit activations and 4- bitlog 2 weights, to shrink the memory footprint and reduce the multipliers to bit shifters. The N weights and biases reside in 65 kB and 3.5 kB memories respectively while activations and embeddings share a 2 kB memory. An inference controller manages inference and embedding generation while a separate learning controller handles embedding processing for learning. Fig. 4(b) compares how the latter controller is configured to support each of ECLâs four learning scenarios. These configurations result in distinct execution schedules. Fig. 4(c)-(f) show how ECLâs two key components map onto Chameleon over time for FSL, CL, ZSL, and ICL respectively. Additionally, they indicate which main module and controller are used over time. All schedules execute ECL in three phases, as annotated in Fig. 4(c) for FSL. First, the TCN embeds a support sample, and the SoC stores the resulting embedding in its activation memory. We call this an inner-loop step, which repeats a number of times dependent on the scenario (see Fig. 4(b)). Second, if the scenario requires it, the stored support embeddings are converted into equivalent prototypical parameters and written to the same weight and bias memories that hold the embedderâs parameters. Together, the first two phases form one outer-loop step, which also repeats a a number of times dependent on the scenario. Third, the query sample is embedded and passed to the scenario-specific head, i.e., one or a few FC layers, to make a prediction. Fig. 4(d) shows these three phases for CL under ECL in a 1-shot use case. AtT = 0, a single-step inner loop and a two-step outer loop learn two ways, enabling inference over those two ways. A single inner and outer loop then learn an additional class, whose weights and bias are appended to the weight and bias memories, enabling inference over three ways. In contrast, for ZSL (Fig. 4(e)), no parameter conversion is needed, reducing the outer loop to a single step but changing the inner loop step count to the number of ways. After embedding the semantic samples, the semantic embedder is replaced by the query- data embedder. Then, the SoC is ready for inference. For ICL (Fig. 4(f)), all samples are concatenated before processing by the embedder, collapsing both the inner and outer loops to a single step. These schedules verify a key advantage of our embedder-centric formulation: extensive reuse of the inference data path for both TCN embedding generation and FC-layer computation. As a result, ECLâs hardware efficiency hinges primarily on how efficiently inference is performed. Since Chameleon is optimized for processing TCNs, it underpins the learning efficiency results that follow. B. ECL Supports On-the-fly Customization with FSL We first demonstrate FSL under ECL, where an N can learn to recognize new classes from only a few labeled examples. 1) Representative Benchmark: To evaluate the performance of our framework on FSL for classification tasks, we use the popular Omniglot dataset [27]. The dataset consists of a total of 1623 different handwritten characters across a large variety of alphabets, containing 20 sample images per character of 28Ă28pixels. Since our framework is designed for processing sequential data, we flatten each input image to shape it into a 1D sequence. We test on Omniglot across the standard evaluation settings of 1, 5-shot and 5, 20-way, as well as 32-way 1-shot. We choose this dataset as it is widely used across both software and hardware works to measure FSL performance [18], [20], [28], [33]. Hence, Omniglot enables a direct comparison to these previous works. 2) Software Results: Table I shows the FSL accuracies across the standard FSL evaluation settings for both the TCN embedder that ECL employs and a commonly used convolutional neural network (CNN) alternative [28], [33]. Compared to the similarly-sized CNN, the TCN incurs a drop of 0.2 to 1.6 accuracy points across all settings. However, this is acceptable and expected, as neighboring pixels from the original image can be tens of timesteps apart when flattened for the TCN. Hence, by using a sequential embedder in ECL, we maintain support for learning from image data while we extend support 6 TABLE I. Accuracy comparison between similarly sized FP32 TCN and CNN across standard evaluation settings for FSL on Omniglot using PNs. Model5-way20-way 1-shot5-shot1-shot5-shot CNN [28]98.8%99.7%96.0%98.9% TCN97.9%99.5%94.4%98.5% 1-shot5-shot 70 80 90 100 Accuracy (%) +3.4 +0.5 5-way 1-shot5-shot 70 80 90 100 +16.6 20-way 1-shot 70 80 90 100 +11.3 32-way Kim et al.* [21]SAPIENS* [20]FSL-HDnn [18]This work Fig. 5. FSL test accuracy comparison on the Omniglot dataset between FSL accelerators that have reported Omniglot results in silicon. We set new accuracy records across all shot-way scenarios, with improvement ranging from 0.5 to 16.6 accuracy points. Missing bars originate from non-reported accuracies. Error bars indicate 95%-confidence intervals for our work. Green annotations indicate accuracy point improvements over the SotA. â indicates the use of an off-chip FP32 embedder. for learning from variable-length sequences across modalities, which we will demonstrate in the remainder of this work. 3) Hardware Demonstration: Fig. 5 compares the FSL accuracies of the ECL framework on Chameleon to those of other silicon designs that perform FSL on Omniglot. These results for FSL were previously reported in [25] and are included here to show the performance of ECL on a representative use case for FSL. It can be seen that through our framework, we achieve new accuracy records across all shot-way scenarios versus prior works. When comparing in detail with the FSL-HDnn chip [18] (Table I), which is, to the best of our knowledge, the only other end-to-end work with silicon results on Omniglot, we additionally demonstrate major gains in hardware efficiency. We find that, at iso-clock- frequency, our framework reduces power by2.3Ă, latency by90Ă, and model size by93Ă. Furthermore, while FSL- HDnn [18] requires external memory accesses for the N, we eliminate these in our framework by specifically training a quantized N that can be stored fully in on-chip SRAM. In addition, the total silicon area overhead of the hardware required for learning with our framework is only 0.5%, compared to 25% for FSL-HDnn [18]. This stems from FSL-HDnnâs reliance on hyperdimensional computing (HDC) [47], which is significantly more complex than our equivalent FC-layer forward pass and requires dedicated hardware. By setting new accuracy records while significantly reducing the power, latency, and area, we validate ECLâs ability to efficiently support on- the-fly customization at the edge using FSL. C. ECL Supports Knowledge Accumulation with CL Second, we demonstrate CL under ECL, where an N can learn multiple tasks over time without overwriting knowledge of previously learned tasks, unlike with FSL. 1) Representative Benchmark: To validate the CL capabil- ities of our framework, we test it with the keyword FSCIL TABLE I. Comparison to FSL-HDnn [18] on power, latency, model size and FSL area overhead. At iso-frequency, ECL is significantly more efficient, while relaxing latency constraints yields micro-watt level power. Bold indicates best. FSL-HDnn [18]This work Power27 mW11.6 mW12.9 ÎŒW Latency53 ms0.59 ms0.54 s Frequency100 MHz100 MHz100 kHz Model size5.5 MB59 kB All weights on-chipââ FSL logic overhead25%0.5% Total core area11.3 m 2 0.83 m 2 100110120130140150160170180190200 Number of classes 40 50 60 70 80 90 100 Accuracy (%) +4.0 -8.4 (a) M5 CNN [38] Rec. SNN [38] ECL (SW, FP32) ECL (HW, 4-bit log 2 ) M5 CNN [38]Rec. SNN [38]ECL (FP32) ECL (4-bit log 2 ) Parameters1.40M2.12M115k115k Bit width3232324 Model size5.61 MB8.50 MB459 kB58.2 kB (b) Fig. 6. (a) CL accuracy comparison between ECL in software (SW) and in hardware (HW) on Chameleon and the NeuroBench baselines on the keyword FSCIL dataset [38]. While ECL in FP outperforms both baselines after learning 20 new classes, our framework incurs a 4.4 accuracy-point drop compared to the ⌠96Ălarger M5 baseline after continually learning the full dataset under tight quantization. Shaded areas indicate 95%-confidence intervals. (b) Parameter count, bit width, and total model size comparison between the four models. benchmark from the NeuroBench initiative [38]. The dataset consists of 100 spoken keyword classes for pre-training and 100 extra keyword classes for class-incremental CL, all with a duration of 1 s. This high number of classes combined with the temporal sensory modality of the samples makes the keyword FSCIL benchmark an ideal fit for measuring the CL performance of ECL. During CL, we follow the original setting and learn the 100 keywords over ten 10-way 5-shot sessions. To ensure a fair comparison with the baseline models from the original NeuroBench work [38], each48 kHzaudio sample is pre-processed off-chip into MFCCs. By requiring a pre- trained model to learn new classes while retaining knowledge of previously learned ones, this benchmark aims to emulate a realistic use case at the edge. 2) Software Results: Fig. 6 shows how the test accuracy varies with an increasing number of classes learned continually compared to two baseline models published in NeuroBench [38]: the M5 CNN [48] and a two-layer recurrent leaky integrate-and-fire (LIF) spiking neural network (SNN). Compared to the CNN, ECLâs floating-point (FP) N has a slightly lower accuracy from pre-training but starts to outper- 7 TABLE I. CL performance summary of ECL deployed on Chameleon for the NeuroBench keyword FSCIL dataset. @ 0.73 V, 14.4 kHz Final accuracy (200 classes)71.8% Latency per sample1.0 s (real-time) Average power9.5 ÎŒW Energy per sample9.5 ÎŒJ Embedder network size58.2 kB Total embeddings size (200 classes)6.4 kB form it after learning only twenty classes with a final increase of four accuracy points, even though ECLâs N is⌠12Ă smaller. We postulate that our smaller embedder is forced by its capacity limit to learn more generic, reusable features, leading to a higher accuracy after the novel classes are introduced. Furthermore, we find that our framework improves significantly over the SNN, even though ECLâs FP N is⌠18.4Ăsmaller. Overall, our framework sets a new standard for CL on the keyword FSCIL dataset at a significant reduction in model size. 3) Hardware Demonstration: Fig. 6 also displays the CL accuracies of the ECL framework when the model is quantized and executed on the Chameleon chip. The⌠8Ăreduced model size of only 58.2 kB, compared to the FP ECL model, results in a drop of 8.4 accuracy points after learning 200 classes. Compared to the96Ălarger M5 model, our quantized model only incurs a loss of 4.4 accuracy points. Table I reports the corresponding hardware performance of ECL for CL on this benchmark. To the best of our knowledge, this is the first work to demonstrate end-to-end fully on-chip CL on the keyword FSCIL dataset from NeuroBench. The only other end-to-end CL work with silicon results is Clo-HDnn [19]. However, it only supports learning up to 128 classes due to high dimensionality of the used embeddings (1024-8192). Additionally, similar to FSL-HDnn [18], it requires reloading the on-chip weights dur- ing inference as it cannot store all weights on-chip, introducing significant additional latency and energy penalties. ECL ad- dresses both limitations. First, ECLâs much lower embedding dimension for CL (64) enables4â32Ăsmaller embedding data sizes, so that 200 classes can be stored on-chip using only 6.4 kB. Second, by compressing ECLâs embedding N, we can store all model parameters fully on-chip. Due to being fully end-to-end on-chip, ECL uses only9.5 ÎŒWon Chameleon for learning and inference while processing the keywords in real time. By demonstrating the first end-to-end fully on-chip deployment on the NeuroBench keyword FSCIL dataset, we show our frameworkâs ability to accurately learn 100 new classes with only a few samples and validate how ECL can be used to accumulate knowledge over time at the edge with CL. D. ECL Leverages Semantic Data with ZSL Third, we demonstrate ZSL under ECL, where an N learns to recognize new classes from semantic information alone, i.e., without labeled data in the query domain, unlike FSL and CL. 1) Representative Benchmark: To demonstrate ZSL with our framework, we use the Fluent Speech Commands (FSC) dataset [49]. The FSC dataset consists of 248 unique spoken phrases up to 13 s long, each with a corresponding transcription. TABLE IV. ZSL performance of ECL deployed on Chameleon for the FSC dataset. On this dataset, the latency and energy for inference are significantly higher than those for learning. This is due to the large size of the audio sample used for inference compared to the size of the semantic data used for learning. @ 0.73V, 5.8 kHz Uses semantic data?â 5-way accuracy60.6% Learning power8.2 ÎŒW Total learning latency (5 ways)383 ms Total learning energy (5 ways)3.1 ÎŒJ Inference power8.0 ÎŒW Inference latency (per sample)13 s (real-time) 1 Inference energy (per sample)104 ÎŒJ 1 Processing time matches maximum audio length (13 s); computation averages 16.4 ms/frame, match- ing one MFCC frameâs duration. Every sentence concerns a single action, object, and location. For example: âBathroom (location) heat (object) down (action)â. We use 75% of these phrases for training, 10% for validation, and 15% for testing, with no class overlap across splits. To enable ZSL on FSC, we use an MLP to create the semantic transcription embeddings, while a TCN embeds the test audio sample, which is converted to MFCCs beforehand. These embeddings are then combined and passed into the relation head to predict which never-heard-before transcription was spoken. While FSC was originally proposed as a benchmark for spoken language understanding through supervised classification, we adopt it under a zero-shot setting. In this setup, the semantic zero-shot data are the transcriptions of the to-be-classified sentences. We opt for this benchmark as there are currently no datasets based on sequential data suitable for zero-shot learning in extreme-edge deployments. 2) Software Results: Under a 5-way ZSL scenario on FSC, ECL achieves a test accuracy of 80.8% with FP precision. Since ZSL on FSC is introduced in this paper as a small-scale alternative to existing benchmarks, no direct comparisons currently exist: Relation Network [29], for instance, demonstrated ZSL using embedding networks ranging from approximately 10 to 50 million parameters to achieve 84.5% test accuracy on a 10-way task. By contrast, ECL uses a semantic embedder with only 120k parameters, a test embedder with 96.7k parameters, and a relation head with 28.6k parameters, making the total network approximately 40 to 200Ă smaller than those in reference works. 3) Hardware Demonstration: Table IV reports the perfor- mance of ECL for ZSL on the FSC dataset when deployed on Chameleon. To the best of our knowledge, this is the first demonstration of ZSL using semantic data at aÎŒWpower budget suitable for the edge. The only other work that performs ZSL in silico with a similar power budget is the work by Liu et al. [50]. In that work, EEG data are used to adapt a seizure-prediction model to unseen patients, enabling learning without requiring seizure recordings from the new patient. However, their approach has three key limitations. First, it does not support semantic data, so it cannot generalize to truly unseen classes: all learned classes were seen during training. Second, it requires replay of data from previous patients, incurring energy-expensive off-chip memory accesses. 8 (sharing) -19.6 (quant + sharing) Fig. 7. Effect of weight and activation quantization on ZSL accuracy for FSC. (a) Training accuracy at 2k steps. (b) Test accuracy at the checkpoint with the highest validation accuracy. The FP32 bars show the baseline model that initializes all quantized runs. (a) As Chameleon does not support per- channel scaling, both embedder NNs must share a single embedding quantizer. A unique quantizer per N recovers the FP32 training accuracy at 16 bit, whereas sharing one costs 4.7 accuracy points, although the confidence intervals overlap. (b) Test accuracy increases slightly from 16 to 6 bit but collapses at 4 bit. The 4-bitlog 2 weights supported by Chameleon perform better than standard 4-bit weights, but trail FP32 by 19.6 accuracy points, against 5.8 points for 6-bit quantization. 4-bitlog 2 accuracy here is higher than in Table IV because this ablation uses a larger bias bit width than Chameleon supports. Hyperparameters are identical across runs. Bars show the mean acrossnruns, points individual runs, and error bars 95% confidence intervals. Third, it relies on backpropagation for learning, requiring 16-bit data widths and incurring a 7% area overhead. Our proposed ECL framework addresses all of these shortcomings. By leveraging semantic data, ECL enables genuine zero-shot generalization to five previously unseen classes without any replay requirement. In addition, by using an RN as part of ECL for ZSL, we can use inference-level bit widths during learning. Combined, this yields a learning latency of 383 ms to learn five new classes for a total learning energy of3.1 ÎŒJ. During inference, we process each sample within its duration to match the audio data rate, by setting the clock frequency to 5.8 kHz, which leads to a real-time inference power of8.0 ÎŒW. While advantageous for tight energy or power constraints, our low-bit-width RN does impact ZSL accuracy, going from 80.8% to 60.6% when quantized. Two limitations of Chameleon explain this gap: (i) the lack of per-channel scaling support requires both embedder NNs to share a single embedding quantizer (Fig. 7(a)), which leads to a 4.7-point training accuracy drop at 16-bit quantization, and (i) while 6-bit quantization only loses 5.8 points compared to FP32, the 4-bit log 2 weight format incurs a 19.6-point drop (Fig. 7(b)). By demonstrating end-to-end ZSL under ECL using semantic data at aÎŒWbudget for the first time, we enable learning even when labeled target-domain data are unavailable and add a level of versatility beyond what FSL and CL alone can provide. E. ECL Supports Adaptation Beyond Classification with ICL Fourth, we demonstrate ICL under ECL, where an N can learn to perform regression or predict the next token in a sequence, going beyond the previous learning scenarios focused only on classification. 050100150200250300350400450500 Sequence length 30 35 40 45 50 55 Cumulative accuracy (%) Chance level = 10% -1.0 -5.3 TF (FP16) ECL (SW, FP32) ECL (HW, 4-bit log 2 ) Fig. 8. Cumulative ICL test accuracy comparison between ECL in software (SW) and in hardware (HW) on Chameleon and a parameter-matched TF on the RegBench dataset. Accuracy is measured over increasing sequence length for 5k query examples. While our FP model nears the transformer (TF) to within one accuracy point, when quantized, the final accuracy drops by 5.3 points. 1) Representative Benchmark: To demonstrate ICL under ECL, we use RegBench [51], a dataset designed to assess in-context language learning ability. The RegBench dataset con- sists of a collection of problem instances, each represented as a sequence of examples. All examples within a given instance are sampled from the same probabilistic language [51]. The goal in each example is to predict the next token based on the previous input. Since, with increasing context length, more examples have been shown, prediction performance should increase. In this work, we follow the standard setting of the benchmark, with a vocabulary size of twenty tokens and between ten and twenty examples of at most fifty tokens. This benchmark dataset aims to be simple enough for analysis in small models while still capturing the core characteristics of ICL in large language models. This makes RegBench a good fit for this work, as we aim to demonstrate our frameworkâs capabilities at the edge, constrained by limited memory and compute. 2) Software Results: Fig. 8 shows how the cumulative test accuracy varies with increasing sequence length for both ECL at 133k parameters and a parameter-matched transformer (TF) baseline. Despite ECLâs TCN model not having direct connections from all to all input tokens, unlike a TF [44], the TCNâs FP accuracy approaches that of the TF within one accuracy point at the 500th token. This result shows that, at a small scale, ECL achieves near-TF ICL performance while requiring 31Ăless activation memory storage (9.2 vs. 288 kB) by exploiting the TCNâs dilation-induced sparsity [25]. 3) Hardware Demonstration: Fig. 8 shows that the quantized model deployed on Chameleon reduces the cumulative accuracy by 5.3 points compared to ECL in FP at the 500th token. The quantized network occupies only 66.4 kB, making it suitable for execution on severely memory-constrained edge devices. We achieve a latency of 2.15 ms per token and an average power consumption of 7.83 mW to yield an energy cost of 16.8ÎŒJper token at a clock frequency of 100 MHz. To the best of our knowledge, this is the first demonstration of ICL at a milliwatt power budget. By showing how ICL can tackle this type of task, we demonstrate how ECL extends beyond plain classification, increasing the frameworkâs versatility. 9 F. Discussion Putting together the results in SectionsIV-BtoIV-E, we identify four common threads. First, while FSL is the only sce- nario that explicitly requires few labeled samples by definition, we also demonstrate low-sample regimes for CL (5 samples per class), ZSL (0 samples in the target domain), and ICL (20-sample sequences). Thus, across all four learning scenarios, ECL remains suitable for edge deployment, where labeled data are inherently scarce [25]. Second, by choosing the TCN as the embedder rather than modality-specific N architectures, ECL operates across images (FSL on Omniglot), audio (CL and ZSL), and tokens (ICL on RegBench). To the best of our knowledge, no prior work has demonstrated learning across all three modalities on a single device at a micro-to-milliwatt power budget. Third, since all learning scenarios share the same embedder N type, and since all scenario-specific embedding processing logic can be incorporated into a single hardware block with only 0.5% area overhead, ECL significantly expands accelerator versatility at minimal added cost. Fourth, by supporting learning scenarios ranging from FSL to ICL, we support a progression of relaxing assumptions about available data at test time. Namely, FSL and CL assume labeled samples in the target modality while ZSL removes that assumption by substituting these with semantic data. ICL then removes the class structure entirely. This property allows ECL to be matched to the data available in a given use case. Although above we presented the four learning scenarios as a progression, they are not necessarily mutually exclusive. For example, combining FSL with CL yields few-shot class- incremental learning (FSCIL) [37], where a model incremen- tally learns new classes with only a few shots per class. Likewise, combining ZSL with CL gives continual ZSL [52], [53], [54] or lifelong ZSL [55]. More generally, ICL learns next-token prediction tasks, so it is compatible with FSL, CL, and ZSL once they are framed in this format. One way to do this is to predict a class token given a context of the support embeddings and a query embedding. However, each of these learning scenarios still requires an embedder network to embed all samples: adding an ICL mechanism on top increases the operation count, latency, and weight storage requirements, an overhead to be considered in light of the target deployment scenario specifications. Taking a step back, we find that most prior works already rely on a fixed N embedder for on-chip learning, yet none recognized this commonality explicitly or exploited it for hardware unification. Instead, prior works differentiated themselves by how the embeddings are used and which learning scenario is targeted. For example, SAPIENS [20] and Kim et al. [21] apply L1 distance on top of embeddings for FSL, while FSL-HDnn [18] and Clo-HDnn [19] use HDC [47] on embeddings for FSL and CL respectively. HDC expands embedding dimensionality to encode information, but this requires specialized hardware blocks [18], [19]; by contrast, ECL uses dense embeddings that map directly onto standard matrix-vector PE arrays, removing this overhead. Liu et al. [50] take a different approach and apply BP to update the last two FC layers of a fixed CNN embedder for ZSL. While each of these choices suit the target scenario well, they prevent the support of additional learning scenarios. Alternatively, Kwon et al. [56] selectively update parts of the full N with BP, relaxing the fixed-embedder assumption entirely. Yet, in this work too, this greater flexibility was not leveraged beyond FSL. The key insight of ECL is therefore not in any individual scenario implementation, but in recognizing this shared embedder-centric structure and exploiting it for hardware unification across four learning scenarios. With regard to future work, we propose four key avenues. First, one limitation of our evaluation is that we demonstrated ECL on relatively small datasets and NNs, reflecting the constraints of Chameleon as our extreme-edge demonstration platform. However, the compute capabilities of edge devices are still increasing: they can then also deal with larger and more complex datasets. Hence, future research should investigate how our framework scales, not only in terms of model and data size but also in task complexity. Second, we postulate that ECL can be expanded to support simple few-shot reinforcement learning (RL) tasks by repurposing ECLâs ICL mechanism, since RL problems can be viewed as sequence modeling problems that can also be learned in- context [57], [58]. Third, ECL is currently formulated to be completely gradient-free for maximum hardware efficiency through simplicity. Hence, a natural next step would be to explore whether allowing gradient descent steps, restricted to the scenario-specific heads, could further enhance performance while retaining hardware efficiency. Fourth, weight-transport- free [59] or forward-learning [60] methods could also be considered for the weight updates without incurring the cost of full backpropagation. How such mechanisms can be efficiently incorporated into our unified embedder-centric framework is a promising avenue for future work. V. CONCLUSION In this work, we presented ECL, a framework that unifies the FSL, CL, ZSL, and ICL online learning scenarios across sensory modalities. We enabled this unification by framing every scenario in an embedder-centric way, which allowed close compatibility with existing inference hardware while keeping the embeddings to a few kB in memory. To learn across sensory modalities, we used TCNs to support samples with long temporal dependencies. To validate our frameworkâs performance on a resource- constrained edge device, we used the Chameleon SoC. In total, we considered four real-world use cases, one per learning scenario. First, when performing FSL on the Omniglot dataset, we set new accuracy records across all shot-way scenarios, improving on the SotA by 0.5 to 16.6 accuracy points at2.3Ă lower power. Second, using CL under ECL, we provided the first end-to-end, fully on-chip hardware results on the keyword FSCIL task from NeuroBench. Compared to NeuroBenchâs original FP baseline, our model is96Ăsmaller, incurring a loss of only 4.4 accuracy points after 200 classes at a real-time power of9.5 ÎŒW. Third, we demonstrated the first end-to- end in-silico implementation of ZSL using semantic data. In particular, we showed how ECL learned to classify five spoken 10 sentences from the FSC dataset using only their transcription at an accuracy of 60.6%. Learning consumes8.2 ÎŒW, while real- time inference afterward consumes only8.0 ÎŒW. Fourth, we also demonstrated for the first time end-to-end, fully on-chip ICL at a milliwatt power budget. On the RegBench dataset, ICL under ECL requires 16.8ÎŒJper token at a clock frequency of 100 MHz, showing ECLâs ability to adapt beyond classification. Together, these results establish ECL as a unified framework for versatile learning across scenarios at the edge, thereby enabling privacy-friendly, energy-efficient, and low-latency on- device adaptation. ACKNOWLEDGEMENT The authors thank Prof. Makinwa for his feedback, Dr. Marco P. Apolinario for his detailed input and for our fruitful discussions, and Dr. Johannes von Oswald for his RegBench implementation. REFERENCES [1] P. McEnroe, S. Wang, and M. Liyanage, âA survey on the convergence of edge computing and ai for uavs: Opportunities and challenges,â IEEE Internet Things J., vol. 9, no. 17, p. 15 435â15 459, 2022. [2]J. S. P. Giraldo, S. Lauwereins, K. Badami, and M. Verhelst, âVocell: A 65-nm speech-triggered wake-up soc for 10-ÎŒw keyword spotting and speaker verification,â IEEE J. Solid-State Circuits, vol. 55, no. 4, p. 868â878, 2020. [3]K. Kim, C. Gao, R. Grac ̧a, I. Kiselev, H.-J. Yoo, T. Delbruck, and S.-C. Liu, âA 23-uw keyword spotting ic with ring-oscillator-based time-domain feature extraction,â IEEE J. Solid-State Circuits, vol. 57, no. 11, p. 3298â3311, 2022. [4]F. Tan, W.-H. Yu, J. Lin, K.-F. Un, R. P. Martins, and P.-I. Mak, âA 1.8% far, 2 ms decision latency, 1.73 nj/decision keywords-spotting (kws) chip incorporating transfer-computing speaker verification, hybrid-if-domain computing and scalable 5t-sram,â IEEE J. Solid-State Circuits, vol. 60, no. 3, p. 1103â1112, 2025. [5]S. Park, K. Shin, D. Lee, M. Kang, S. Lee, Y. Park, M. Seok, and D. Jeon, âA 5.6ÎŒw 10-keyword end-to-end keyword spotting system using passive-averaging sar adc and sign-exponent-only layer fusion with 92.7% accuracy,â in 2024 IEEE Symp. VLSI Technol. Circuits (VLSI Technol. Circuits), 2024, p. 1â2. [6]J. Liu, J. Fan, Z. Zhong, H. Qiu, J. Xiao, Y. Zhou, Z. Zhu, G. Dai, N. Wang, Q. Liu, et al., âAn ultra-low power reconfigurable biomedical ai processor with adaptive learning for versatile wearable intelligent health monitoring,â IEEE Trans. Biomed. circuits Syst., vol. 17, no. 5, p. 952â967, 2023. [7]L. Yan, J. Bae, S. Lee, T. Roh, K. Song, and H.-J. Yoo, âA 3.9 mw 25- electrode reconfigured sensor for wearable cardiac monitoring system,â IEEE J. Solid-State Circuits, vol. 46, no. 1, p. 353â364, 2011. [8] S. Yin, M. Kim, D. Kadetotad, Y. Liu, C. Bae, S. J. Kim, Y. Cao, and J.-S. Seo, âA 1.06-ÎŒw smart ecg processor in 65-nm cmos for real-time biometric authentication and personal cardiac monitoring,â IEEE J. Solid-State Circuits, vol. 54, no. 8, p. 2316â2326, 2019. [9]X. Zhang, Z. Zhang, Y. Li, C. Liu, Y. X. Guo, and Y. Lian, âA 2.89 ÎŒw dry-electrode enabled clockless wireless ecg soc for wearable applications,â IEEE J. Solid-State Circuits, vol. 51, no. 10, p. 2287â 2298, 2016. [10]J. Liu, J. Fan, Z. Zhong, H. Qiu, J. Xiao, Y. Zhou, Z. Zhu, G. Dai, N. Wang, Q. Liu, Y. Xie, H. Liu, L. Chang, and J. Zhou, âAn Ultra- Low Power Reconfigurable Biomedical AI Processor With Adaptive Learning for Versatile Wearable Intelligent Health Monitoring,â IEEE Trans. Biomed. Circuits Syst., vol. 17, no. 5, p. 952â967, Oct. 2023, Conference Name: IEEE Transactions on Biomedical Circuits and Systems, ISSN: 1940-9990. Accessed: Nov. 1, 2024. [11]C. Frenkel and G. Indiveri, âReckon: A 28nm sub-m2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales,â in 2022 IEEE Int. Solid-State Circuits Conf. (ISSCC), vol. 65, 2022, p. 1â3. [12]V. Jain, S. Giraldo, J. De Roose, L. Mei, B. Boons, and M. Verhelst, âTinyvers: A tiny versatile system-on-chip with state-retentive emram for ml inference at the extreme edge,â IEEE J. Solid-State Circuits, vol. 58, no. 8, p. 2360â2371, 2023. [13]M. Verhelst and B. Moons, âEmbedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices,â IEEE Solid-State Circuits Mag., vol. 9, no. 4, p. 55â65, 2017. [14]V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, âEfficient processing of deep neural networks: A tutorial and survey,â Proc. IEEE, vol. 105, no. 12, p. 2295â2329, 2017. [15]J. K. Kim, P. Knag, T. Chen, and Z. Zhang, âA 640m pixel/s 3.65 mw sparse event-driven neuromorphic object recognition processor with on-chip learning,â in 2015 Symp. VLSI Circuits (VLSI Circuits), IEEE, 2015, p. C50âC51. [16]S. K. Gonugondla, M. Kang, and N. Shanbhag, âA 42pj/decision 3.12 tops/w robust in-memory machine learning classifier with on-chip training,â in 2018 IEEE Int. Solid-State Circuits Conference-(ISSCC), IEEE, 2018, p. 490â492. [17]A. Amravati, S. B. Nasir, S. Thangadurai, I. Yoon, and A. Raychowd- hury, âA 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots,â in 2018 IEEE Int. Solid-State Circuits Conference-(ISSCC), IEEE, 2018, p. 124â126. [18]H. Yang, C. E. Song, W. Xu, B. Khaleghi, U. Mallappa, M. Shah, K. Fan, M. Kang, and T. Rosing, âFsl-hdnn: A 5.7 tops/w end-to- end few-shot learning classifier accelerator with feature extraction and hyperdimensional computing,â in 2024 IEEE Eur. Solid-State Electron. Res. Conf. (ESSERC), 2024, p. 33â36. [19] C. E. Song, W. Xu, K. Fan, S. Jain, G. Hota, H. Yang, L. Liu, K. Akarvardar, M.-F. Chang, C. H. Diaz, G. Cauwenberghs, T. Rosing, and M. Kang, âClo-hdnn: A 4.66 tflops/w and 3.78 tops/w continual on-device learning accelerator with energy-efficient hyperdimensional computing via progressive search,â in 2025 Symp. VLSI Technol. Circuits (VLSI Technol. Circuits), 2025, p. 1â3. [20]H. Li, W.-C. Chen, A. Levy, C.-H. Wang, H. Wang, P.-H. Chen, W. Wan, H.-S. P. Wong, and P. Raina, âOne-shot learning with memory- augmented neural networks using a 64-kbit, 118 gops/w rram-based non-volatile associative memory,â in 2021 Symp. VLSI Technol., 2021, p. 1â2. [21]S. Kim, W. Lee, S. Kim, S. Park, and D. Jeon, âAn in-memory computing sram macro for memory-augmented neural network,â TCAS-I, vol. 69, no. 3, 2022. [22]G. K. Chen, R. Kumar, H. E. Sumbul, P. C. Knag, and R. K. Krishnamurthy, âA 4096-neuron 1m-synapse 3.8-pj/sop spiking neural network with on-chip stdp learning and sparse weights in 10-nm finfet cmos,â IEEE J. Solid-State Circuits, vol. 54, no. 4, p. 992â1002, 2018. [23]M. P. E. Apolinario, K. Roy, and C. Frenkel, âTess: A scalable temporally and spatially local learning rule for spiking neural networks,â in 2025 Int. Joint Conf. Neural Netw. (IJCNN), 2025, p. 1â9. [24]T. Bohnstingl, S. Wo Ì zniak, A. Pantazi, and E. Eleftheriou, âOnline spatio-temporal learning in deep neural networks,â IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 11, p. 8894â8908, 2023. [25]D. d. Blanken and C. Frenkel, âChameleon: A multiplier-free temporal convolutional network accelerator for end-to-end few-shot and continual learning from sequential data,â IEEE J. Solid-State Circuits, p. 1â16, 2026. [26]S. Davis and P. Mermelstein, âComparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,â IEEE Trans. Acoust., speech, signal Process., vol. 28, no. 4, p. 357â366, 1980. [27] B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, âHuman-level concept learning through probabilistic program induction,â Science, vol. 350, no. 6266, 2015. [28] J. Snell, K. Swersky, and R. Zemel, âPrototypical networks for few-shot learning,â in Advances Neural Inf. Process. Syst., vol. 30, 2017. [29]F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, âLearning to compare: Relation network for few-shot learning,â in Proc. IEEE Conf. Comput. Vis. pattern Recognit., 2018, p. 1199â1208. [30] M. Mazumder, C. Banbury, J. Meyer, P. Warden, and V. J. Reddi, âFew-shot keyword spotting in any language,â arXiv preprint arXiv:2104.01454, 2021. [31]O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al., âMatching networks for one shot learning,â Advances neural Inf. Process. Syst., vol. 29, 2016. [32] T. Pfister, J. Charles, and A. Zisserman, âDomain-adaptive discriminative one-shot learning of gestures,â in Eur. Conf. Comput. Vis., Springer, 2014, p. 814â829. [33] C. Finn, P. Abbeel, and S. Levine, âModel-agnostic meta-learning for fast adaptation of deep networks,â in Int. Conf. Mach. Learn., PMLR, 2017, p. 1126â1135. 11 [34]B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum, âOne shot learning of simple visual concepts,â in Proc. Annu. meeting Cogn. Sci. Soc., vol. 33, 2011. [35]G. Koch, R. Zemel, R. Salakhutdinov, et al., âSiamese neural networks for one-shot image recognition,â in ICML deep Learn. workshop, Lille, vol. 2, 2015, p. 1â30. [36]L. Fei-Fei, R. Fergus, and P. Perona, âOne-shot learning of object categories,â IEEE Trans. pattern analysis Mach. Intell., vol. 28, no. 4, p. 594â611, 2006. [37]X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y. Gong, âFew- shot class-incremental learning,â in Proc. IEEE/CVF Conf. Comput. Vis. pattern Recognit., 2020, p. 12 183â12 192. [38]J. Yik, K. Van den Berghe, D. den Blanken, Y. Bouhadjar, M. Fabre, P. Hueber, W. Ke, M. A. Khoei, D. Kleyko, N. Pacik-Nelson, et al., âThe neurobench framework for benchmarking neuromorphic computing algorithms and systems,â Nature Commun., vol. 16, no. 1, p. 1545, 2025. [39]J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., âOvercoming catastrophic forgetting in neural networks,â Proc. Nat. Acad. Sci., vol. 114, no. 13, p. 3521â3526, 2017. [40]C. H. Lampert, H. Nickisch, and S. Harmeling, âAttribute-based classification for zero-shot visual object categorization,â IEEE Trans. pattern analysis Mach. Intell., vol. 36, no. 3, p. 453â465, 2013. [41]R. Zhang, S. Frei, and P. L. Bartlett, âTrained transformers learn linear models in-context,â J. Mach. Learn. Res., vol. 25, no. 49, p. 1â55, 2024. [42]S. Garg, D. Tsipras, P. S. Liang, and G. Valiant, âWhat can transformers learn in-context? a case study of simple function classes,â Advances neural Inf. Process. Syst., vol. 35, p. 30 583â30 598, 2022. [43]T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., âLanguage models are few-shot learners,â Advances neural Inf. Process. Syst., vol. 33, p. 1877â1901, 2020. [44]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ć. Kaiser, and I. Polosukhin, âAttention is all you need,â Advances neural Inf. Process. Syst., vol. 30, 2017. [45]W. L. Tong and C. Pehlevan, âMlps learn in-context on regression and classification tasks,â arXiv preprint arXiv:2405.15618, 2024. [46]S. Bai, J. Z. Kolter, and V. Koltun, âAn empirical evaluation of generic convolutional and recurrent networks for sequence modelingâ, 2018. arXiv:1803.01271. [47]P. Kanerva, âHyperdimensional computing: An introduction to comput- ing in distributed representation with high-dimensional random vectors,â Cogn. computation, vol. 1, no. 2, p. 139â159, 2009. [48]W. Dai, C. Dai, S. Qu, J. Li, and S. Das, âVery deep convolutional neural networks for raw waveforms,â in 2017 IEEE Int. Conf. Acoust., speech signal Process. (ICASSP), IEEE, 2017, p. 421â425. [49]L. Lugosch, M. Ravanelli, P. Ignoto, V. S. Tomar, and Y. Bengio, âSpeech Model Pre-training for End-to-End Spoken Language Understandingâ, arXiv:1904.03670 [eess], Jul. 2019. Accessed: Nov. 10, 2025. [50]J. Liu, X. Liu, X. Wang, Z. Xie, C. Guo, Z. Zhong, J. Fan, H. Qiu, Y. Xu, H. Qin, et al., âA high accuracy and ultra-energy-efficient zero- shot-retraining seizure detection processor,â IEEE J. Solid-State Circuits, vol. 59, no. 11, p. 3549â3565, 2024. [51]E. Aky Ì urek, B. Wang, Y. Kim, and J. Andreas, âIn-context language learning: Architectures and algorithms,â arXiv preprint arXiv:2401.12973, 2024. [52]A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny, âEfficient lifelong learning with a-gem,â arXiv preprint arXiv:1812.00420, 2018. [53]I. Skorokhodov and M. Elhoseiny, âNormalization matters in zero-shot learning,â 2020. [54]C. Gautam, S. Parameswaran, A. Mishra, and S. Sundaram, âTf- gczsl: Task-free generalized continual zero-shot learning,â Neural Netw., vol. 155, p. 487â497, 2022. [55] K. Wei, C. Deng, X. Yang, et al., âLifelong zero-shot learning.,â in IJCAI, 2020, p. 551â557. [56] Y. D. Kwon, R. Li, S. I. Venieris, J. Chauhan, N. D. Lane, and C. Mascolo, âTinytrain: Resource-aware task-adaptive sparse training of dnns at the data-scarce edge,â arXiv preprint arXiv:2307.09988, 2023. [57]L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, âDecision transformer: Reinforcement learning via sequence modeling,â Advances neural Inf. Process. Syst., vol. 34, p. 15 084â15 097, 2021. [58] M. Janner, Q. Li, and S. Levine, âOffline reinforcement learning as one big sequence modeling problem,â Advances neural Inf. Process. Syst., vol. 34, p. 1273â1286, 2021. [59]C. Frenkel, M. Lefebvre, and D. Bol, âLearning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks,â Frontiers Neuroscience, vol. Volume 15 - 2021, 2021, ISSN: 1662-453X. [60]G. Hinton, âThe forward-forward algorithm: Some preliminary investi- gationsâ, 2022. arXiv:2212.13345. Douwe den Blanken (Graduate Student Member, IEEE) received the M.Sc. degree (with honors) in embedded systems from Delft University of Tech- nology (TU Delft), Delft, The Netherlands, in 2023, where he is currently pursuing the Ph.D. degree, under the supervision of Prof. C. Frenkel. His current research interests include efficient learning algorithms and their implementation in silicon, as well as the quantization and acceleration of modern DNNs. Martin Lefebvre (Member, IEEE) received the M.Sc. and Ph.D. degrees in engineering sciences from the Universit Ì e catholique de Louvain (UCLouvain), Belgium, in 2017 and 2024. His research interests include hardware-aware machine learning algorithms, mixed-signal vision chips for embedded image processing, and low-power current reference architectures. He currently is a post- doctoral researcher in the cognitive sensor nodes and systems (CogSys) laboratory led by Prof. Frenkel at TU Delft, The Netherlands, working on neuromorphic hardware/software co-design for efficient on-chip learning. Dr. Lefebvre serves as a reviewer for various IEEE journals and conferences including IEEE Journal of Solid-State Circuits and IEEE Transactions on Circuits and Systems I and I. Charlotte Frenkel (Member, IEEE) received the M.Sc. degree (summa cum laude) in Electromechan- ical Engineering and the Ph.D. degree in Engineer- ing Science from Universit Ì e catholique de Louvain (UCLouvain), Louvain-la-Neuve, Belgium in 2015 and 2020, respectively. In February 2020, she joined the Institute of Neuroinformatics, UZH and ETH Zurich, Switzerland, as a postdoctoral researcher. She is an Assistant Professor at Delft University of Technology, Delft, The Netherlands, since July 2022, and a Research Scientist at Google since February 2026. Her research aims at bridging the bottom-up (bio-inspired) and top-down (engineering-driven) design approaches toward neuromorphic intelligence, with a focus on hardware-algorithm co-design for (Neuro)AI, digital hardware accelerators, and brain-inspired on-device learning. Dr. Frenkel received a best paper award at the IEEE International Symposium on Circuits and Systems (ISCAS) 2020 conference in the Neural Networks track, and her Ph.D. thesis was awarded the FNRS-FWO / Nokia Bell Scientific Award 2021 and the FNRS-FWO / IBM Innovation Award 2021. In 2023, she was awarded prestigious Veni and AiNed Fellowship grants from the Dutch Research Council (NWO). She presented several invited talks, including keynotes at the tinyML EMEA technical forum 2021 and at the Neuro-Inspired Computational Elements (NICE) neuromorphic conference 2021. She serves or has served as a program co-chair of NICE 2023-2024 and of the tinyML Research Symposium 2024, as a TPC member of IEEE ISSCC for 2027 and IEEE ESSERC for 2022-2024, and as an associate editor for the IEEE Transactions on Biomedical Circuits and Systems for 2022-2025.