Paper deep dive

BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator

Yuhao Liu, Salim Ullah, Akash Kumar

Year: 2026Venue: arXiv preprintArea: cs.ARType: PreprintEmbeddings: 42

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 8:41:22 AM

Summary

The paper introduces BiKA (Binarized Kolmogorov-Arnold Network), a lightweight neural network architecture inspired by KAN. BiKA replaces expensive learnable nonlinear functions with binary learnable thresholds, enabling a multiply-free architecture using only comparators and accumulators. FPGA implementation on Ultra96-V2 demonstrates significant resource reduction compared to BNN and QNN accelerators while maintaining competitive accuracy.

Entities (9)

BiKA · architecture · 98%CIFAR-10 · dataset · 95%Kolmogorov-Arnold Network · model · 95%MNIST · dataset · 95%Binarized Neural Network · model · 90%PyTorch · framework · 90%Quantized Neural Network · model · 90%Ultra96-V2 · hardware · 90%FINN · framework · 85%

Relation Signals (7)

BiKA → evaluatedon → MNIST

confidence 95% · evaluate both MLP and CNN versions of the model on MNIST

BiKA → evaluatedon → CIFAR-10

confidence 95% · evaluate both MLP and CNN versions of the model on MNIST and CIFAR-10.

BiKA → inspiredby → Kolmogorov-Arnold Network

confidence 95% · Inspired by KAN, we propose BiKA

BiKA → implementson → Ultra96-V2

confidence 90% · Our FPGA prototype on Ultra96-V2 shows that BiKA reduces hardware resource usage

BiKA → outperforms → Quantized Neural Network

confidence 90% · BiKA reduces hardware resource usage by ... 51.54% compared with ... quantized neural network systolic array accelerators

BiKA → outperforms → Binarized Neural Network

confidence 90% · BiKA reduces hardware resource usage by 27.73% ... compared with binarized ... accelerators

BiKA → usesframework → PyTorch

confidence 90% · We develop a PyTorch [20] and CUDA training framework for BiKA

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Lightweight neural network accelerators are essential for edge devices with limited resources and power constraints. While quantization and binarization can efficiently reduce hardware cost, they still rely on the conventional Artificial Neural Network (ANN) computation pattern. The recently proposed Kolmogorov-Arnold Network (KAN) presents a novel network paradigm built on learnable nonlinear functions. However, it is computationally expensive for hardware deployment. Inspired by KAN, we propose BiKA, a multiply-free architecture that replaces nonlinear functions with binary, learnable thresholds, introducing an extremely lightweight computational pattern that requires only comparators and accumulators. Our FPGA prototype on Ultra96-V2 shows that BiKA reduces hardware resource usage by 27.73% and 51.54% compared with binarized and quantized neural network systolic array accelerators, while maintaining competitive accuracy. BiKA provides a promising direction for hardware-friendly neural network design on edge devices.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

40,773 characters extracted from source content.

Expand or collapse full text

BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator Yuhao Liu 1,2,3 , Student Member, IEEE, Salim Ullah 1 , Akash Kumar 1 , Senior Member, IEEE 1 Ruhr University Bochum, Germany 2 Dresden University of Technology, Germany 3 Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany Email: yuhao.liu, salim.ullah, akash.kumar@rub.de Abstract—Lightweight neural network accelerators are essen- tial for edge devices with limited resources and power constraints. While quantization and binarization can efficiently reduce hardware cost, they still rely on the conventional Artificial Neural Network (ANN) computation pattern. The recently proposed Kolmogorov-Arnold Network (KAN) presents a novel network paradigm built on learnable nonlinear functions. However, it is computationally expensive for hardware deployment. Inspired by KAN, we propose BiKA, a multiply-free architecture that replaces nonlinear functions with binary, learnable thresholds, introducing an extremely lightweight computational pattern that requires only comparators and accumulators. Our FPGA prototype on Ultra96-V2 shows that BiKA reduces hardware resource usage by27.73%and51.54%compared with binarized and quantized neural network systolic array accelerators, while maintaining competitive accuracy. BiKA provides a promising direction for hardware-friendly neural network design on edge devices. I. INTRODUCTION With the rapidly increasing complexity of modern Neural Networks (NNs), reducing the hardware cost of N accelerators has become a key research topic. Related researchers have thoroughly explored a wide range of techniques to simplify computation and reduce memory consumption, including quantization, binarization, and approximate arithmetic. For instance, FINN [1, 2], HLS4ML [3, 4], and LogicNets [5] proposed the efficiency of Binarized Neural Network (BNNs) and Quantized Neural Network (QNNs) on FPGAs. Other works have explored approximate multipliers, such as Ullah et al. [6, 7], or alternative neural paradigms, including Spiking Neural Networks (SNNs), which are implemented in systems like SpiNNaker2 [8] and S2N2 [9]. However, most existing accelerator research is focusing on conventional Artificial Neural Networks (ANNs). This motivates the exploration of emerging network models that provide new paradigms of trade-offs in accuracy, resource consumption, and hardware friendliness. The Kolmogorov-Arnold Network (KAN) [10] is a recently introduced model that replaces the multiplication and activation operations in conventional ANNs with learnable nonlinear functions, offering a fundamentally different computational paradigm and new opportunities for hardware-efficient design. Despite its promising potential, recent research still lacks the exploration of KAN-based hardware accelerators, especially for resource-limited edge devices. MUL � � � � MUL � � � � MUL � �−� � �−� . . . ACCUReLU Output MLP 푶풖풕풖풕 =푹풆( �=� �−� � � � � +푩�) Learnable Activation 0 � � Learnable Activation 1 � � Learnable Activation n � �−� . . . ACCU Output KAN 푶풖풕풖풕 = �=� �−� � � (� � ) Fig. 1: Difference between the computation of single neuron in MLP (up) and KAN (down) A. Background 1) What is Kolmogorov-Arnold Network: From the 1960s, the Kolmogorov-Arnold representation theorem and related works [11, 12, 13] proved that every continuous function can be represented as a sum of functions of fewer variables. Building upon this principle, Liu et al. [10] introduced the Kolmogorov–Arnold Network (KAN) in 2024 as a new neural architecture aimed at improving interpretability and reducing model complexity. As shown in Figure 1, KAN replaces the Multiplication-and-Activation (MAC) computations in a Multilayer Perceptron (MLP) with a set of learnable nonlinear functionsLA i (x)applied to each input activationA i . The outputs of these functions are summed as the neuron output. Previous work [14] evaluated the accuracy of KAN against MLP on MNIST [15]. KAN shows a competitive accuracy (98.52%) compared to MLP (98.02%). Despite its potential, KAN introduces several challenges for both training and hardware acceleration: •Difficulties in Training: Compared to the weight and bias in ANN, learnable nonlinear functions in KAN are more complex and consume more computational resources in training. For example, based on our experiment, training even a small three-layer KAN model (64/32/10 neurons) using the native pykan library [10] can exceed 48 GB of memory on an NVIDIA A40 GPU. •Difficulties in Accelerator Design: Learnable nonlinear arXiv:2602.23455v1 [cs.AR] 26 Feb 2026 TABLE I: Resource Consumption of MLP and KAN on FPGA Platform Based on Post-implementation in Previous Work DatasetModel TypeModel SizeFrequencyLUTFFDSPBRAM Wine MLP13,32,8,369749936172 KAN13,4,314684374741950132 Dry Bean MLP16,20,15,10,7889411328170 KAN16,2,716775587345449111781 Mashroom MLP8,64,64,21093218903174 KAN8,24,231122751337291162991347 function design is a novel computational pattern intro- duced by KAN. However, nonlinear functions complicate hardware implementation and cannot be directly applied to common simplification techniques, such as quantiza- tion, because KAN is both multiply-free and weight- free. Therefore, it needs a novel design paradigm for lightweight hardware accelerators on the edge. 2) Previous Accelerators Design for KAN: As an emerging network structure, several recent works explored the design of a hardware accelerator for KAN. For instance, Tran et al. [16] first explored the hardware accelerator design of KAN on an FPGA based on a handmade High-Level Synthesis (HLS) implementation, comparing it with MLP. Huang et al. [17] first presented the KAN accelerator based on Analog-digital-mixed hardware produced with TSMC 22 nm node. Yin et al. [14] introduced their quantized KAN accelerator based on Look-Up Table (LUT) optimization for FPGA platforms. As shown in Table I, Tran et al. [16] implemented four small KAN-based MLP classifiers using Vitis HLS. Their results show that directly mapping native KAN to hardware leads to extremely high resource usage (e.g., 3.1M LUTs for only 34 fully connected kernels), even for toy-scale models. These findings highlight a fundamental challenge: the nonlinear- function-based computation in KAN is prohibitively expensive for FPGA-based or edge-oriented accelerator designs. Huang et al. [17] explored the hardware-software co-design framework for the lightweight edge accelerator based on N:1 Time Modulation Dynamic Voltage input generator for RRAM- ACIM. However, the proposed analog-digital signal mixed hardware in this work is complicated and not easily scalable in design and manufacturing, and cannot be simply integrated into conventional FPGA/ASIC design flows for general edge devices. Yin et al. [14] applied global and fine-grained post-training quantization to KAN, converting each learned nonlinear function into a low-bit lookup table. This enables KAN infer- ence using integer arithmetic and reduces computational cost. However, their method generates a model-specific hardware design: each trained network requires regenerating all LUTs, rebuilding the hardware architecture, and performing full synthesis and bitstream generation, which currently takes 6–10 hours. Moreover, because their accelerator implements the entire network structure on the FPGA, resource consumption grows accordingly with the model size, because every nonlinear function requires one LUT. For example, a three-layer KAN with 64/32/10 neurons for MNIST, as presented in their ANN BNN KANBiKA Remove Non-linear Activation Function Replaced Multiplication as Learnable Non-linear Function Remove Sign Threshold Activation Function Replaced Multiplication as Learnable Threshold Function Replace Non-linear Activation Function as Sign Threshold Fucntion Replace Leanrable Non- linear Function as Learnable Threshold Fucntion Fig. 2: Difference between the ANN, BNN, KAN, and BiKA work, requires a maximum of 65,680 LUT-6s. Applying their approach to the VGG-like convolutional network used in our BiKA evaluation for the CIFAR-10 [18] dataset would require approximately 4.3 million LUT-6s, which exceeds the capacity of typical edge FPGAs. B. Motivation The prior works discussed above suggest that the fundamen- tal challenge of deploying KAN on edge-oriented hardware accelerators lies in implementing its learnable nonlinear activation unit. Efficiently simplifying or replacing this operator is essential for enabling a KAN-based accelerator on FPGA/A- SIC realization. Moreover, maintaining compatibility with conventional systolic-array architectures would significantly improve scalability to larger network models. Therefore, we thoughtfully rethink the conversion between ANN, KAN, and QNN, especially the 1-bit quantized BNN. As shown in Figure 2, typical BNNs replace the fixed nonlinear activation in ANN with a fixed Sign threshold function. As a mirror design, we consider replacing the learnable nonlinear functions in KAN with learnable threshold functions to propose a Binarized KAN (BiKA). From another perspective, since removing the fixed nonlinear activation function and replacing the multiplication with learnable nonlinear functions can convert the ANN to KAN, correspondingly, removing the fixed threshold function and replacing the multiplication with learnable threshold functions should suggest the existence of a potential binarized KAN structure, BiKA, converting from the binarized ANN. Therefore, based on this BiKA network, we can expect a simplified, ultra-lightweight model structure con- sisting of only comparators and integer adders, supporting the systolic-array structure, without any multiplication or nonlinear operations. Furthermore, considering BiKA is a fully integer- based network, it does not aim to preserve the interpretability properties of KAN. Instead, it follows a different design goal: achieving extremely low-cost computation for efficient hardware acceleration. C. Contributions •Extending on our prior abstract in [19], we propose BiKA, a new KAN-inspired neural network that replaces KAN’s expensive learnable nonlinear functions with binary learnable thresholds. This design removes all multiplications and nonlinear processing, resulting in an extremely lightweight computation model based only on comparators and integer adders. • We provide a mathematical explanation showing that a KAN nonlinear function can be approximated using a learnable threshold. This gives a clear and direct theoretical basis for the BiKA architecture. • We develop a PyTorch [20] and CUDA training framework for BiKA, and evaluate both MLP and CNN versions of the model on MNIST and CIFAR-10. We further analyze BiKA’s training behavior and explain the source of its accuracy loss. 1 •We design a systolic-array accelerator for BiKA and compare it with BNN and QNN systolic-array accelerators. BiKA fits well into this structure because its operations are fully integer and multiplication-free. • FPGA implementations on an Ultra96-V2 show that the BiKA accelerator reduces hardware resource usage by 27.73%compared with BNN and51.54%compared with QNN, while keeping< 1%accuracy loss on MNIST and about10%on CIFAR-10. These results show that BiKA is a potential and highly efficient option for ultra-lightweight neural network acceleration on edge devices. D. Organization This manuscript is structured as follows: Section I discusses the background of KAN and previous KAN accelerator designs, presenting our motivation and contribution introduced in this work. Section I introduces the mathematical principle of BiKA and the design of BiKA accelerators. Section I presents the training results and hardware resource consumption of BiKA and its accelerator in comparison with QNN and BNN. Section IV discusses the further potential improvement and concludes the contents of this paper. I. IMPLEMENTATION A. Approximate the Nonlinear Function as the Form of Thresholds Considering the complexity of a nonlinear function, if we want to approximate it to a single threshold function in BiKA, we need to prove that it can be converted into a format constructed with a threshold-based representation first, for further approximation. Because the modern digital system cannot process continuous functions in principle, we can use a discrete functionf(x)as shown in Equation 1 to approximate the continuous nonlinear function in a certain input range [s 0 , s t ), which can also be considered as one piecewise constant function withtinput slots,s i ⩽ x < s i+1 (0⩽ i⩽ t− 1). 1 An open-source reference implementation and training library of BiKA is available at https://github.com/liuyh-Horizon/BiKA. A mirrored repository for archival purposes is maintained at https://git.noc.ruhr-uni-bochum.de/liuyuhyc/ bika. . Whentis large enough, which means that each input slot, s i ⩽ x < s i+1 , is small enough,f(x)can approximately represent any continuous nonlinear function. f(x) =          O 0 if s 0 ⩽ x < s 1 O 1 if s 1 ⩽ x < s 2 · O t−1 if s t−1 ⩽ x < s t (1) Therefore, as shown in Equation 2, we definetweighted threshold activations,α i Thres i (x) (0⩽ i⩽ t− 1), based on input slots,s i ⩽ x < s i+1 (0⩽ i⩽ t− 1), whereα i is the weight for each threshold and the left ends i of input slot is the threshold value for eachThres i (x). When inputxis smaller than threshold values i ,Thres i (x)outputs−1. Else, Thres i (x) outputs 1. α i Thres i (x) = ( α i s i ⩽ x −α i x < s i (2) Assume f(x)≈ f ′ (x) = t−1 X i=0 α i Thres i (x) (3) Based on the sum of all weighted threshold activations, we construct one function,f ′ (x), as shown in Equation 3, attempting to approximate the original discrete nonlinear function as f(x)≈ f ′ (x). When s i ⩽ x < s i+1 ,                      α 0 Thres 0 (x) = α 0 α 1 Thres 1 (x) = α 1 · α i Thres i (x) = α i · α t−2 Thres t−2 (x) =−α t−2 α t−1 Thres t−1 (x) =−α t−1 Therefore , f(x) = O i ≈ f ′ (x) = t−1 X i=0 α i Thres i (x) = i X l=0 α l − t−1 X r=i+1 α r (4) As shown in Equation 4, for our constructed functionf ′ (x), when input iss i ⩽ x < s i+1 , we can compute the result of each weighted threshold function. Therefore, for the weighted threshold functions,α 0 Thres 0 (x)toα i Thres i (x), because their threshold values ares 0 tos i and the input isx⩾ s i , based on Figure 3, the output for these weighted thresholds are α 0 toα i . According to the same principle, for the weighted threshold functions,α i+1 Thres i+1 (x)toα t−1 Thres t−1 (x), because their threshold values ares i+1 tos t−1 and the input is x < s i+1 , the output for these weighted thresholds are−α i+1 to−α t−1 . Therefore, the sum of all weighted thresholds isf ′ (x) = P t−1 i=0 α i Thres i (x) = P i l=0 α l − P t−1 r=i+1 α r . Because as shown in Equation 1, whens i ⩽ x < s i+1 , f(x) = O i , we assume here theO i can be approximated as O i = P i l=0 α l − P t−1 r=i+1 α r .                  O 0 = α 0 − P t−1 j=1 α j s 0 ⩽ x < s 1 O 1 = α 0 + α 1 − P t−1 j=2 α j s 1 ⩽ x < s 2 O 2 = α 0 + α 1 + α 2 − P t−1 j=3 α j s 2 ⩽ x < s 3 · O t−2 = P t−2 j=0 α j − α t−1 s t−2 ⩽ x < s t−1 O t−1 = P t−1 j=0 α j s t−1 ⩽ x < s t (5) Therefore, as shown in Equation 5, we traverse inputxwith all input slots i ⩽ x < s i+1 (0⩽ i⩽ t− 1)to compute the conversion betweenO i andα i based on the formula of O i = P i l=0 α l − P t−1 r=i+1 α r .                          O 0 = α 0 − P t−1 j=0 α j + α 0 O 1 = α 0 + α 1 − P t−1 j=1 α j + α 1 · O t−2 = t−3 X j=0 α j + α t−2 − α t−2 − α t−1 + α t−2 O t−1 = t−2 X j=0 α j + α t−1 − α t−1 + α t−1 ⇒              O 0 = 2α 0 − O t−1 O 1 = 2α 1 + O 0 · O t−2 = 2α t−2 + O t−3 O t−1 = 2α t−1 + O t−2 ⇒              α 0 = (O 0 + O t−1 )/2 α 1 = (O 1 − O 0 )/2 · α t−2 = (O t−2 − O t−3 )/2 α t−1 = (O t−1 − O t−2 )/2 (6) Then, by substituting the expressions ofO t−1 in Equation 5 down toO 0 in Equation 6, andO i in Equation 5 intoO i+1 in Equation 6 for0⩽ i⩽ t− 2, we obtain a closed-form solution for each threshold weight: α 0 = O 0 + O t−1 2 , α i = O i − O i−1 2 , 1⩽ i⩽ t− 1.(7) Since allO i values are fixed constants determined by the original functionf(x), we can compute everyα i directly. Therefore, the discrete nonlinear functionf(x)can be ap- proximated by the weighted-threshold-based form as shown in Equation 3. B. Simplification of Approximate Nonlinear Functions for BiKA Considering the implementation of BNN in the FINN framework [1, 2], we can convert a learnable threshold Thres i (x)as a combination of weightw, biasβ, and Sign activation as shown in Equation 8.− β w is learned threshold value. Sign(wx + β) = 1 wx + β⩾ 0 −1 wx + β < 0 ⇒ Sign(x) = 1 x⩾− β w −1 x <− β w (8) � � . . . MUL � � SUM MUL+ADD � � Sign � � Thresholds MUL � � MUL+ADD � � Sign � � MUL � �−� MUL+ADD � �−� Sign � �−� Fig. 3: Converting one learnable nonlinear function to a series of weighted learnable thresholds � � . . . SUM MUL+ADD � � Sign � � MUL+ADD � � Sign � � MUL+ADD � �−� Sign � �−� . . . MUL+ADD � � Sign � � MUL+ADD � � Sign � � MUL+ADD � �−� Sign � �−� . . . . . . � � � � � �−� Fig. 4: Duplicated learnable thresholds when allα i are approximated as integers Therefore, based on the above-discussed mathematics con- version, Figure 3 shows that we can convert the training of one learnable nonlinear functionf(x)as a series of parameters of weightsw, biasesβ, Sign activations, and threshold weights α with two multipliers and one adder. However, this conversion of one nonlinear function remains too complex and hardware-expensive due to the dense number of parameters and computation. To simplify it, if we approx- imate theO 0 + O t−1 andO i − O i−1 in Equation 7 as even integers, theα i values of O 0 +O n−1 2 and O i −O i−1 2 are integers too. Therefore, as shown in Figure 4, multiplying the output of the Sign activation by threshold weightsα i can be simplified by duplicatingα i times of the input to remove one multiplier for the threshold weight. As a result, comparing Figure 3 and Figure 4, one weighted learnable threshold is converted into α i times of learnable thresholds. Considering the order of thresholds will not influence the final output, we can define an integer numbermas the sum of all quantizedα i to mix all thresholds converted from different weighted thresholds in one nonlinear function together, as shown in Figure 5. Therefore, a highermcan represent the nonlinear function more accurately. Then, for � � SUM MUL+ADD � � Sign � � MUL+ADD � � Sign � � MUL+ADD � �−� Sign � �−� . . . � � = �=� �−� � � � � � � Fig. 5: Mixed all learnable thresholds with the sum of α � � . . . MUL+ADD � � � Sign � � � MUL+ADD � �−� � Sign � �−� � � � � . . . SUM MUL+ADD � � � Sign � � � MUL+ADD � �−� � Sign � �−� � � � �−� . . . MUL+ADD � � �−� Sign � � �−� MUL+ADD � �−� �−� Sign � �−� �−� � � � � � � �−� � . . . Fig. 6: Settingmas a quantized parameter to approximate and simplify learnable thresholds further approximation, we setmas a unified parameter for all nonlinear functions in our approximated KAN as shown in Figure 5. As a result, the output range of eachO i in Figure 5 is[−m,m]. Therefore, the inputs, outputs, and thresholds of every layer and neuron in this approximated KAN model are quantized as integers, adjusted by the quantization factor m. This approximation method has an extreme situation: when mis set to one, each learnable nonlinear activation function will be approximated as a single learnable threshold. We named this situation Binarized KAN (BiKA) as shown in Figure 7. In the traditional ANN models, the computation in one neuron or kernel isReLU( P n−1 i=0 W i a i +b) , whereW i is the weight,a i is the input activation, andbis the bias. In our BiKA, the computation in one neuron or kernel is P n−1 i=0 Sign(W i a i + b i ), whereW i is the weight,a i is the input activation, andb i is the individual bias for each input activation.Sign(W i a i + b i ) can be converted into one threshold. Therefore, this architecture can be customized for training based on PyTorch and CUDA. We implemented a customized training library for BiKA by introducing BiKALinear and BiKAConv2d, which modify the standard Linear and Conv2d layers to match BiKA’s threshold- based computation pattern. Because BiKA relies heavily on Sign threshold operations, the true gradient is not available. To enable stable backpropagation, we replace the backward � � � � MUL+ADD � �−� . . . Output BiKA MUL+ADD MUL+ADD 푶풖�풖� = �=� �−� 푻풉� � � � SUM Sign Sign Sign 푻풉� � � = �푰�>− � � � � −�푰�≤− � � � � Learnable Threshold 0 � � � � Fig. 7: Computation in BiKA neurons when set quantization parameter, m, as one. MULACCU Threshold A W Out 8-bit QNN accelerator: XNORPopcount ACCU A W Out BNN accelerator: Threshold Thres x255 Thres x1 ThresholdACCU A Out BiKA accelerator: Thres x1 Binary MUL Fig. 8: Hardware difference in accelerator designs between BNN, QNN, and BiKA networks pass of the Sign function with the derivative of the hard-tanh function in our CUDA implementation. This straight-through estimator greatly improves the training stability and accuracy of the BiKA models. C. Processing Element and Systolic Array Design of BiKA Considering the above-discussed designs of BiKA networks, we present and compare the different hardware designs between QNN, BNN, and BiKA in this manuscript. We implemented three systolic array accelerators for QNN, BNN, and BiKA. Each systolic array consists of8 × 8processing elements as shown in Figure 9. Both BNN and QNN accelerators implemented in this work are inspired by FINN [1, 2]. The original FINN framework is designed for the data-streaming architecture of FPGA-based N accelerators. We expand its design as systolic arrays. Compared to BNN, QNN, and our BiKA processing elements shown in Figure 8, the major differences are: •Both BNN and QNN processing elements need threshold- based activation modules. The BNN processing element only has one threshold. Then-bit QNN processing element requires2 n thresholds for activation and quan- tization in output according to FINN-R [2]. To reduce the hardware resource consumption, we implement only PEPEPEPE PE PEPEPEPE PE PEPEPE PEPEPE Activ Activ Activ Activ T h r e s T h r e s T h r e s T h r e s PEPEPEPE PE PEPEPEPE PE PEPEPE PEPEPE Activ Activ Activ Activ T h r e s W e i g h t T h r e s W e i g h t T h r e s W e i g h t T h r e s W e i g h t > ACCU Activ Out Thres MULACCU Activ Out Weight > CNT Thres 1) Systolic Array for BiKA and QuKA 2) Processing Element in Systolic Array for BiKA and QuKA 3) Systolic Array for BNN and QNN 4) Processing Element in Systolic Array for BNN and QNN Fig. 9: Hardware difference of systolic array accelerators between BNN, QNN, and BiKA one comparator in the QNN accelerator and process all threshold comparisons in serial. • Because BNN represents−1as 1-bit ’0’ and1as 1-bit ’1’, the computation in BNN is XNOR. Moreover, because the BNN accelerator computes multi-bits in parallel to speed up the inference, one PopCount module is used to compute the sum of XNOR output inspired by FINN [1]. •Compared with the processing element design of BNN and QNN, the BiKA processing element only needs to implement one comparator to replace the multiplier and XNOR+PopCount unit without the additional threshold activations as in BNN and QNN, which can highly simplify the design of the systolic array engine and the control state machine in the BiKA accelerators. Based on the processing element shown in Figure 8, we implement systolic-array-based approaches in this work to achieve higher scalability for larger network models. Compared to BNN/QNN and our BiKA systolic array engine designs shown in Figure 9, the major differences of our BiKA accelerator are: • In our BiKA accelerator, thresholds replaced the weights to execute the systolic array processing. Therefore, Multiply-Accumulators (MACs) have been replaced with Comparison-Accumulators (CACs). • In our BNN/QNN accelerator, we implement one ad- ditional pipeline in the systolic array to reuse the loaded threshold for processing elements and improve the throughput. After finishing the multiply-accumulation computing in all processing elements, our BNN/QNN systolic array will idle the MACs and load the thresh- olds for nonlinear activation and output quantization processing. This implementation is unnecessary in our BiKA accelerator, since BiKA doesn’t need the additional nonlinear activation after CAC computation. I. EVALUATION A. Training Experiment of BiKA As shown in Table I, we trained 14 different models of BiKA, KAN, BNN, and 8-bit QNN networks for comparison based on four network structures: • Tiny Fully Connected Network (TFC), Small Fully Con- nected Network (SFC), and Large Fully Connected Network (LFC) are three MLP structures. TFC is a three- layer MLP structure that consists of 64/32/10 neurons, the same as the evaluated MLP network in work [14]. SFC and LFC are two four-layer MLP structures that consist of 256/256/256/10 neurons and 1024/1024/1024/10 neurons. All of these three MLP structures are used to evaluate the MNIST dataset [15] with 200 epochs of training. •Convolution Neural Network (CNV) is one tiny VGG-like CNN structure that consists of six convolutional layers and three fully connected layers. Two convolutional layers and one MaxPool layer are combined as a block. This structure is used to evaluate the CIFAR-10 [18] dataset with 200 epochs of training. As shown in the second row of Table I, F64 means this layer is a fully connected layer with 64 neurons, and C64 and P2 are one convolutional layer with 643× 3kernels with the padding of 1 and the stride of 1, and one 2x2 MaxPool layer with the padding of 1 and the stride of 1. The third and fourth rows of Table I list the reported training results of KAN and quantized KAN from Yin et al. [14], showing98.02%for KAN and98.52%for their LUT- based quantized KAN on the MNIST [15] dataset using the TFC structure. As KAN training is known to be sensitive to initialization and hyperparameter choices, we additionally reproduced the TFC KAN model in our own environment for a consistent comparison against BNN, QNN, and BiKA. Since Yin et al.’s quantized KAN is directly derived from their trained KAN model, we expect its accuracy in our environment to be close to that of our reproduced KAN. Therefore, the training results listed in Table I show that: • Due to the high memory usage of the native pykan library during training, we only trained the TFC and SFC models for KAN. In our experiments, KAN achieves higher accuracy than BNN in the TFC structure, but becomes lower than BNN in the larger SFC structure. • In the TFC model, KAN and 8-bit QNN outperform BNN by2.8%and4.58%, respectively, while BiKA shows the lowest accuracy,1.4%below BNN. However, as the network size increases, BiKA starts to outperform KAN from the SFC model onward. The accuracy gaps between BNN and BiKA on MNIST also shrink to0.93%and 0.19%in the SFC and LFC models. Since the quantized KAN of Yin et al. [14] is derived from their trained KAN model, we expect its accuracy in our environment to be close to that of our reproduced KAN. Under TABLE I: Inference Accuracy of BNN, QNN, KAN, and BiKA on MLP and tiny CNN with MNIST Classification DatasetMNIST[15]CIFAR-10 [18] Network TFCSFCLFCCNV F64/F32/F10F256/F256/F256/F10F1024/F1024/F1024/F10 C64/C64/P2/C128/C128/P2/C256/C256/P2/ F512/F512/F10 Q-KAN [14]98.52%--- KAN [14]98.02%--- BNN93.34%97.39%98.05%65.20% QNN97.92%98.59%98.68%72.76% KAN96.14%95.83%-- BiKA91.94%96.46%97.86%55.80% AB C DEF G H Batch 1024 Batch 512 Batch 256 96.0596.0597.4997.4997.8697.8697.5797.5797.1197.1197.7797.7797.2397.2397.1697.16 91.5591.5596.596.596.6696.6696.8496.8497.1397.1397.7797.7797.2197.2197.297.2 80.3280.3292.6292.6292.3492.3493.7693.7695.0395.0397.5197.5197.4497.4497.1997.19 92 94 96 AB C DEF G H Batch 1024 Batch 512 Batch 256 45.6345.6354.1554.1554.2154.2153.7253.7248.4448.4449.3149.3152.6352.6351.7951.79 35.6435.6446.3346.3345.2945.2943.0543.0547.3847.3853.5653.5653.4353.4352.8352.83 30.2530.2539.9739.9739.9739.9738.9738.9747.9347.9354.5554.5555.855.853.9553.95 30 40 50 Learning Rate Combinations: A: LR 0 =0.0010, LR 1 =0.0010, LR 2 =0.0010B: LR 0 =0.0010, LR 1 =0.0005, LR 2 =0.0002C: LR 0 =0.0010, LR 1 =0.0005, LR 2 =0.0001D: LR 0 =0.0010, LR 1 =0.0002, LR 2 =0.0001 E: LR 0 =0.0005, LR 1 =0.0005, LR 2 =0.0005F: LR 0 =0.0005, LR 1 =0.0002, LR 2 =0.0001G: LR 0 =0.0002, LR 1 =0.0002, LR 2 =0.0002H: LR 0 =0.0001, LR 1 =0.0001, LR 2 =0.0001 Fig. 10: Accuracy in MNIST (left) and CIFAR-10 (right) datasets based on LFC and CNV models with different hyperparameters Fig. 11: Accuracy and loss in training and validation based on LFC and CNV models with MNIST (left) and CIFAR-10 (right) this assumption, BiKA achieves competitive accuracy compared with both native and quantized KAN. •To evaluate BiKA on a more complex dataset, we additionally tested CIFAR-10. The last column of Table I compares BNN, QNN, and BiKA, where BiKA shows a 9.4%accuracy drop compared with BNN. To investigate this gap, we trained BiKA under various initial hyperpa- rameter settings. As shown in Figure 10, BiKA is highly sensitive to the choice of batch size and learning rate, with accuracy changes up to17.45%on MNIST and25.55% on CIFAR-10 [18]. The heat map also indicates that larger batch sizes and smaller learning rates generally yield better results. Therefore, a more refined training strategy is expected to improve BiKA’s CIFAR-10 accuracy. •Figure 11 shows the training and validation curves for MNIST and CIFAR-10. BiKA achieves approximately 90%training accuracy on CIFAR-10 but only about55% validation accuracy, while such divergence does not appear on MNIST. These observations suggest that BiKA has sufficient expressivity to fit the CIFAR-10 training data. However, the capacity of the lightweight CNV model is not enough for good generalization, leading to overfitting. Increasing the network size or applying regularization techniques, such as weight decay, is likely to further improve BiKA’s performance on CIFAR-10 [18]. B. Hardware Accelerator Evaluation of BiKA As shown in Table I, we implemented four systolic array accelerators for BiKA, BNN, and QNN on Ultra96v2 FPGA platform with8× 8processing elements based on the design shown in Figure 9. The QNN accelerator in the work is designed to infer the network models with 8-bit quantized activation and weights. The 8-bit BNN accelerator loads the 8-bit binarized inputs and weights in one shot to execute TABLE I: Resource Consumption of BNN, QNN, and BiKA Accelerators on Ultra96V2 FPGA Platform Accelerator Hardware ResourceFrequencyTotal DelayADPDynamic PowerPDPLatency (μs) LUTFFBRAM(MHz)(ns)(ns)(W)(W*ns)TFCSFCLFC QKAN-max [14] >65680---------- QKAN-min [14] >28371--100--0.612-4.74-- BiKA8900923219.53002.74424421.6001.7784.87811.20171.421611.890 BNN12315996224.53003.01337105.0951.8605.6041.64610.66384.753 QNN183661317923.52503.61066301.2601.8036,50934.915236.0281327.980 XNOR-Popcount computation eight times in parallel for higher throughput. For the BiKA accelerator, we implemented one 8-bit instance in this evaluation. As shown in Figure 7, when outputs of the processing element are set as 8-bit, the output range of the accumulator is[−128, 127], since the outputs of all threshold activations are−1or1. Therefore, in principle, the 8-bit output of the accumulator can only support at most 127 inputs. However, in our experiments, we found that in most cases, the sum values in the accumulator of networks are not out of the range of[−128, 127]. Therefore, we implemented an 8-bit BiKA accelerator with a sum limitation design in the accumulator to prevent the value from being out of bounds. The hardware implementation results listed in Table I show that: • Hardware Resource Consumption: The 8-bit BiKA accelerator significantly reduces LUT consumption com- pared to 8-bit BNN and QNN accelerators, by27.73% and51.54%, respectively. This suggests our BiKA has the potential to implement the ultra-lightweight accelerator with scalability to support larger network inference based on the systolic-array structure. •Clock Frequency: All BiKA and BNN accelerators support a 300MHzclock, higher than QNN accelerators. However, the total delay of BiKA accelerators reported from the implementation in Vivado is lower than the BNN and QNN accelerators. •Aera-Delay-Products and Power-Delay-Products: Fur- thermore, with the power, total delay, and LUT consump- tion reported from Vivado implementation, our BiKA has the lowest Aera-Delay-Products (ADP) and Power-Delay- Products (PDP) compared to BNN and QNN accelerators, which shows the design and power efficiency of BiKA accelerators. • Inference Latency: We evaluated three network models, TFC, SFC, and LFC, on these four accelerators. Results show that our BiKA is2.17×-3.30×faster than 8-bit QNN. However, the 8-bit Single Instruction/Multiple Data (SIMD) design in the BNN systolic array engine has a significant speedup compared with other accelerators, which makes it the fastest accelerator implementation in our evaluation. However, the ultra-low hardware resource consumption of BiKA can be more suitable than the BNN accelerator in some extreme resource-limited, non-speed- sensitive scenarios. Moreover, we also compared our BiKA accelerator with the quantized KAN accelerator of Yin et al [14]. While their previous work presents a low power figure, it does not include post-synthesis implementation results such as LUT utilization or timing reports. The LUT consumption in this accelerator is determined by software compilation, based on the number of nonlinear functions in KAN models, excluding the hardware consumption of quantization units, adder trees, state machines, and FIFO controls, among others. Therefore, the reported latency and power appear to be based on modeled estimations rather than measured hardware execution. As a result, the practicality of these numbers cannot be fully validated, and the comparison should be interpreted with caution. In summary, our BiKA accelerator offers the simplest hardware architecture design, the lowest resource consumption, the highest supported clock frequency, and the highest power and hardware efficiency compared to the BNN and QNN accelerators. However, as a trade-off, our BiKA achieves a competitive accuracy on the MNIST dataset, while incurring a higher accuracy loss on CIFAR-10 [18]. However, according to our experiment, the refined train strategies and extension of the network model can be expected to improve the accuracy of BiKA on CIFAR-10 [18]. Moreover, compared with the extremely high hardware resource consumption shown in [16] or the complex hardware fabrication required for the previous KAN accelerator designs of Huang et al. [17], our BiKA implementation demonstrates the feasibility of implementing KAN-like networks as hardware accelerators for ultra-lightweight edge devices. IV. CONCLUSION AND FURTHER WORKS In this manuscript, we propose a novel multiply-free ultra- lightweight neural network, BiKA, inspired by KAN and BNN. This network highly simplifies the hardware design of accelerators by replacing the multiplier and activation functions with learnable thresholds. Our experiments and implementation results on Ultra96-V2 show that the BiKA network can reduce the hardware resource consumption by27.73%and51.54% compared to BNN and QNN accelerators. In our future works, we plan to complete and optimize our BiKA training strategy and explore the potential of BiKA training with larger datasets and complex network models, such as VGG-16 [21], ResNet-50 [22], ImageNet [23], etc. Furthermore, to address the major shortcomings in our current design, such as the accuracy loss in the CIFAR-10 datasets, we plan to continually explore the influence of different quantization factors m with larger models. ACKNOWLEDGMENTS This research was supported in part by the Deutsche Forschungsgemeinschaft (DFG) under the X-ReAp project (Project number 380524764) and by the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dres- den/Leipzig), Germany. REFERENCES [1]Yaman Umuroglu et al. “Finn: A framework for fast, scalable binarized neural network inference”. In: Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2017, p. 65–74. [2] Michaela Blott et al. “FINN-R: An end-to-end deep-learning frame- work for fast exploration of quantized neural networks”. In: ACM Transactions on Reconfigurable Technology and Systems (TRETS) 11.3 (2018), p. 1–23. [3]Farah Fahim et al. “hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices”. In: CoRR abs/2103.05579 (2021). arXiv: 2103.05579. [4]Giuseppe Di Guglielmo et al. “Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML”. In: CoRR abs/2003.06308 (2020). arXiv: 2003.06308. [5] Yaman Umuroglu et al. “LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications”. In: 2020 30th International Conference on Field-Programmable Logic and Applica- tions (FPL). IEEE. 2020, p. 291–297. [6]Salim Ullah et al. “Area-Optimized Low-Latency Approximate Multipliers for FPGA-based Hardware Accelerators”. In: 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). 2018, p. 1– 6. [7] Salim Ullah et al. “High-Performance Accurate and Approximate Multipliers for FPGA-based Hardware Accelerators”. In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2021), p. 1–1. [8] Hector A. Gonzalez et al. SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning. 2024. arXiv: 2401.04491 [cs.ET]. [9] Alireza Khodamoradi, Kristof Denolf, and Ryan Kastner. “S2N2: A FPGA Accelerator for Streaming Spiking Neural Networks”. In: The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. FPGA ’21. Association for Computing Machinery, 2021, 194–205. [10] Ziming Liu et al. KAN: Kolmogorov-Arnold Networks. 2024. arXiv: 2404.19756 [cs.LG]. [11] Andrei Nikolaevich Kolmogorov. “On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition”. In: Doklady Akademii Nauk. Vol. 114. 5. Russian Academy of Sciences. 1957, p. 953–956. [12]Vladimir I Arnold. “On the representation of functions of several variables as a superposition of functions of a smaller number of variables”. In: Collected works: Representations of functions, celestial mechanics and KAM theory, 1957–1965 (2009), p. 25–46. [13] J ̈ urgen Braun and Michael Griebel. “On a constructive proof of Kolmogorov’s superposition theorem”. In: Constructive approximation 30 (2009), p. 653–675. [14] Mengyuan Yin et al. Optimizing Neural Networks with Learnable Non- Linear Activation Functions via Lookup-Based FPGA Acceleration. 2025. arXiv: 2508.17069 [cs.AR]. [15] Li Deng. “The mnist database of handwritten digit images for machine learning research”. In: IEEE Signal Processing Magazine 29.6 (2012), p. 141–142. [16]Van Duy Tran et al. Exploring the Limitations of Kolmogorov- Arnold Networks in Classification: Insights to Software Training and Hardware Implementation. 2024. arXiv: 2407.17790[cs.LG]. [17]Wei-Hsing Huang et al. “Hardware Acceleration of Kolmogorov- Arnold Network (KAN) for Lightweight Edge Inference”. In: Pro- ceedings of the 30th Asia and South Pacific Design Automation Conference. ASPDAC ’25. Tokyo, Japan: Association for Computing Machinery, 2025, 693–699. [18] Alex Krizhevsky, Geoffrey Hinton, et al. “Learning multiple layers of features from tiny images”. In: (2009). [19]Yuhao Liu, Salim Ullah, and Akash Kumar. “BiKA: Binarized KAN-inspired Neural Network for Efficient Hardware Accelerator Designs”. In: 2025 IEEE 33rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 2025, p. 276–276. [20]Adam Paszke et al. “Pytorch: An imperative style, high-performance deep learning library”. In: Advances in neural information processing systems 32 (2019), p. 8026–8037. [21]Karen Simonyan and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition”. In: arXiv preprint arXiv:1409.1556 (2014). [22]Kaiming He et al. Deep Residual Learning for Image Recognition. 2015. arXiv: 1512.03385 [cs.CV]. [23]Jia Deng et al. “ImageNet: A large-scale hierarchical image database”. In: 2009 IEEE Conference on Computer Vision and Pattern Recogni- tion. 2009, p. 248–255.

Instant research discovery