Paper deep dive
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
Yuhao Liu, Salim Ullah, Akash Kumar
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/20/2026, 8:48:30 AM
Summary
This paper proposes BitSys, a runtime-reconfigurable multi-precision multi-channel bitwise systolic array architecture for Quantized Neural Network (QNN) accelerators. It addresses the accuracy-resource trade-off of mixed-precision quantization by enabling dynamic precision reconfiguration (1/2/4/8-bit) in hardware, achieving significant speedups over fixed-precision designs on FPGA platforms.
Entities (8)
Relation Signals (6)
BitSys → implementson → Ultra96 FPGA
confidence 95% · We have implemented and evaluated our work on the Ultra96 FPGA platform.
BitSys → supports → Mixed-Precision Quantization
confidence 95% · BitSys supports runtime reconfiguration for signed/unsigned 8/4/2/1-channel 1/2/4/8-bit multiplication.
BitSys → uses → Bitwise Systolic Array
confidence 92% · we propose a runtime reconfigurable multi-precision multi-channel bitwise systolic array design
MNIST → usedfortraining → BitSys
confidence 88% · The TFC models... are trained with the MNIST dataset... evaluated our multipliers... on the Ultra96-V2 FPGA platform
BitSys → outperforms → BitFusion
confidence 85% · Results show that our work can achieve 1.3185 to 3.5671 times speedup... compared with previous works.
BitSys → outperforms → Bitshifter
confidence 85% · Results show that our work can achieve 1.3185 to 3.5671 times speedup... compared with previous works.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Neural network accelerators have been widely applied to edge devices for complex tasks like object tracking, image recognition, etc. Previous works have explored the quantization technologies in related lightweight accelerator designs to reduce hardware resource consumption. However, low precision leads to high accuracy loss in inference. Therefore, mixed-precision quantization becomes an alternative solution by applying different precision in different layers to trade off resource consumption and accuracy. Because regular designs for multiplication on hardware cannot support the precision reconfiguration for a multi-precision Quantized Neural Network (QNN) model in runtime, we propose a runtime reconfigurable multi-precision multi-channel bitwise systolic array design for QNN accelerators. We have implemented and evaluated our work on the Ultra96 FPGA platform. Results show that our work can achieve 1.3185 to 3.5671 times speedup in inferring mixed-precision models and has less critical path delay, supporting a higher clock frequency (250MHz).
Tags
Links
- Source: https://arxiv.org/abs/2602.23334v2
- Canonical: https://arxiv.org/abs/2602.23334v2
Trouble viewing inline? Open PDF directly →
Full Text
51,209 characters extracted from source content.
Expand or collapse full text
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators Yuhao Liu 1,3 , Student Member, IEEE, Salim Ullah 2 , Akash Kumar 2,3 , Senior Member, IEEE 1 Dresden University of Technology, Germany 2 Ruhr University Bochum, Germany 3 Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany Email: yuhao.liu1@tu-dresden.de, salim.ullah, akash.kumar@rub.de Abstract—Neural network accelerators have been widely applied to edge devices for complex tasks like object tracking, image recognition, etc. Previous works have explored the quantization technologies in related lightweight accelerator designs to reduce hardware resource consumption. However, low precision leads to high accuracy loss in inference. Therefore, mixed-precision quantization becomes an alternative solution by applying different precision in different layers to trade off resource consumption and accuracy. Because regular designs for multiplication on hardware cannot support the precision reconfiguration for a multi-precision Quantized Neural Network (QNN) model in runtime, we propose a runtime reconfigurable multi-precision multi-channel bitwise systolic array design for QNN accelerators. We have implemented and evaluated our work on the Ultra96 FPGA platform. Results show that our work can achieve1.3185×to3.5671×speedup in inferring mixed-precision models and has less critical path delay, supporting higher clock frequency (250M Hz). I. INTRODUCTION Recent research of edge hardware devices widely applied Neu- ral Networks (N) on state-of-the-art applications, such as autonomous driving, the Internet of Things, wearable devices, voice and image recognition, etc. Considering the conflict between limited resources on the edge device and continually extending sizes of neural network models, related works explored the Quantized Neural Network (QNN) to reduce storage and hardware resource consumption by applying lower precision. For instance, NVDLA [1] and Vitis DPU [2] support the INT8 8-bit quantization in their deep learning processor designs. FINN [3, 4], HLS4ML [5], LogicNets [6], etc. pro- posed different frameworks to generate specialized inference accelerator designs on FPGA for the given low-precision trained (< 8bits) QNN models to reduce the on-chip resource consumption. However, various prior works have presented a higher accuracy loss in lower-precision quantized network models. For instance, the 1-bit quantized Multilayer Perceptron (MLP) model shown in work [7] of Su et al. has an8× higher memory saving rate than the 8-bit quantized model applying the same network structure. However, the error of 1-bit models is about32.7%higher than the 8-bit model. Therefore, to trade off the low resource consumption and high accuracy loss in QNN hardware accelerator designs, the works from HAQ [8], Chen et al. [9], Tang et al. [10], etc. explored mixed-precision quantization by using different precision in different layers. Compared to uniform quantization schemes TABLE I: Inference Accuracy of Quantized Network Models Applying Unified-Precision and Mixed-Precision Schemes Network Type Precision Settings in Four Layers of TFC and TCV models 1/1/1/12/2/2/21/2/4/84/1/2/84/4/4/48/8/8/8Float TFC Accuracy/%92.2996.3795.91-97.5597.3697.89 Weights/Byte7376147529984-2950459008236032 TCV Accuracy/%96.2698.96-98.7999.1099.1499.14 Weights/Byte2984859696-55712119392238784955136 TABLE I: Comparison of Unified- and Mixed-Precision Quan- tized MLPs Inferred on FPGA-based N Accelerator DesignPrecisionLUTFFBRAMFrequencyLatencyAccuracy Vivado IP 8/8/8/8 2409022175135150MHz 137.654us97.74% 1/2/4/8131.059us95.96% across all layers (either high or low precision), mixed-precision quantized networks have middle-level inference accuracy and memory consumption for weight storage. We trained six tiny MLPs and six tiny Convolution Neural Networks (CNNs) models based on the Brevitas [11] with different precision to evaluate the accuracy loss and memory saving. Tiny MLP (TFC) and Tiny CNN (TCV) models are trained with the MNIST dataset [12, 13]. The TFC models comprise four layers with 64, 64, 64, and 10 neurons, respectively. TCV models have two convolution layers, each followed by a2×2max pooling layer. Following the final pooling layer are two fully connected layers. Each convolution layers have 643× 3kernels, while two fully connected layers have 64 and 10 neurons, respectively. To achieve maximum compression of network weights, we apply lower precision to layers with a higher number of weights. Therefore, as shown in Table I, TFC applies 1/2/4/8-bit quantization, and TCV applies 4/1/2/8-bit quantization, respectively, as their mixed-precision schemes. Results show that 8-bit quantized models have the highest accuracy, similar to 32-bit floating-point-based networks. The two 1-bit models have the lowest accuracy with the least memory storage for weights. Meanwhile, two mixed-precision have balanced accuracy and memory requirements. A. Motivation Mixed-precision QNNs show the potential to achieve a better and more flexible trade-off between resource consumption and accuracy loss. Prior works explored the design of related accelerators better to support the inference of mixed-precision arXiv:2602.23334v2 [cs.AR] 22 Jun 2026 B B B B I n p u t I n p u t Partial Sum Partial Sum Weight F-PEF-PEF-PEF-PE F-PEF-PEF-PEF-PE F-PEF-PEF-PEF-PE F-PEF-PEF-PEF-PE I n p u t I n p u t Partial Sum Partial Sum Weight F - P E F - P E F - P E F - P E I n p u t I n p u t Partial Sum Partial Sum Weight F-PE I n p u t I n p u t Partial Sum Partial Sum Weight 1.a) Fusion Unit Created with 16 BitBricks (BBs) 1.b) 16 2x2-bit Fusion Processing Elements 1.c) 4 2x8-bit or 4x4-bit Fusion Processing Elements 1.d) 1 8x8-bit Fusion Processing Element MULMUL MULMUL <<2 <<4 <<0 <<2 + 2) Sub-Multiplier Architecture Matrix Buffer Matrix Buffer AND Popcount Shift Neg. Accu. 3) Bit-Serial Architecture in BISMO A N D A r r a y M a s k A r r a y 2 - b i t S h i f t i n g 2 - b i t O u t p u t 4 - b i t S h i f t i n g 4 - b i t O u t p u t 8 - b i t S h i f t i n g 8 - b i t O u t p u t O u t p u t M U X 4) Bitshifter Architecture I n p u t L o a d e r Input Loader P P P P P P P P <<<< << << << << <<<< << << << << << << << 5) BitSys Architecture Fig. 1: Architectures of Prior Works and BitSys networks on hardware. Results report that utilizing fixed- precision multipliers diminishes the performance advantages of mixed-precision accelerators. As shown in Table I, one previous work of Liu et al. [14] implemented one single-layer N accelerator on Ultra96-V2 FPGA platform with 64 8-bit integer Vivado multiplier IPs to infer one 8-bit quantized MLP and one 1/2/4/8-bit mixed-precision quantized MLP trained by Brevitas [11] with MNIST dataset [12, 13]. Both MLPs have four layers with 64, 64, 64, and 10 neurons, respectively. Table I listed the average inference latency of one MNIST input, computed by averaging the total latency of 1000 times inputs. The results indicate that the inference speed of the mixed-precision MLP has not significantly improved compared to the uniformly 8-bit quantized network. Because the input width of 8-bit Vivado multiplier IP cannot be reconfigured as 1/2/4-bit in runtime, all input data must be unified and extended to the largest precision, 8 bits. As a result, the inference acceleration of the mixed-precision model can only benefit from the transmission speedup between off-chip memory and FPGA based on low-precision data, not from the computation. Therefore, if multipliers can reconfigure the input precision and channel number in runtime, for instance, reset a single- channel 8-bit input as a dual-channel 4-bit input for signed 8/4- bit quantized layers, the inference of mixed-precision network models can be sped up on hardware. B. Contributions Prior works, such as PIR-DSP [15], BitFusion [16], Multiplier- Tree [14], Bitshifter [14], etc., explored the designs of multi- precision multipliers. Extending on our abstract in [17], we proposed a Bitwise Systolic Array Architecture (BitSys) in this manuscript supporting quantized multi-precision multi- channel runtime reconfigurable multiplication for neural network accelerator designs. The key features and contributions of this work are: 1 1 The RTL source code, Vivado projects, and packaged Vivado IPs for the BitSys multiplier and MAC are available at https://github.com/liuyh-Horizon/ BitSys to facilitate validation, reproduction, and reuse. •We implemented one systolic-array-based multiplier, BitSys, based on the bitwise (1-bit) processing element and optimized it with LUT primitive for FPGA. Our design sup- ports runtime reconfiguration for signed/unsigned 8/4/2/1- channel 1/2/4/8-bit multiplication. Moreover, this multiplier is specially designed to support the XNOR multiplication for the Binarized Neural Network (BNN) in FINN [3, 4]. • We extended our multiplier as a Multiply-Accumulator (MAC) to implement one single-layer accelerator and one systolic array accelerator and evaluate them for the mixed- precision model inference acceleration. We evaluated our multipliers, MAC, and accelerator imple- mentations on the Ultra96-V2 FPGA platform and compared them with previous works. The synthesis and implementation report in Vivado shows our designs have low critical path delay from1.357nsto1.719ns. The measurement result proves that our systolic array accelerator is1.3185×to3.5671×faster in the inference of mixed-precision networks than previous works. C. Organization This manuscript is structured as follows: Section I compares our BitSys design with related works. Section I introduces implementations of BitSys architecture. Section IV shows the evaluation results on Ultra96-V2 platform compared with related works. Section V concludes the contents of this paper. I. BACKGROUND A. Classification of Prior Multi-precision Multiplier Designs Previous work explored different schemes for multi-precision multiplier designs, which can be classified by bit-serial/bit- parallel architectures and fixed/variable input widths. Bit-serial multipliers execute the bitwise processing for multiplication in serial, such as BISMO [22], the work of Ienne et al. [23], the work of Shafer et al. [24], etc. For example, as shown in Figure 1.3, BISMO loads the inputs with the batch size ofk- bit to execute the pipelined processing in serial. Form-bit inputs, it takes m k clock cycles to complete the multiplication. As a result, low-precision multiplication consumes fewer clock cycles TABLE I: Differences between the BitSys Architecture and Previous Works WorkPlatform Accu. orNoSigned orAvailable Precision Approx.DSPUnsigned 1× 1 2× 2 4× 4 4× 16 8× 8 8× 16 9× 9 16× 16 18× 27 24× 24 32× 32 Guo et al. [18]FPGAApprox. √ Signed × × × √ × √ × × × × × Neda et al. [19]FPGAApprox. √ Signed × × × × √ × × √ × × × Shun et al. [20]FPGAAccu. √ Signed × × × × √ × × √ × √ √ Pf ̈ ander et al. [21]FPGAAccu. √ Both × × × × √ √ × √ × √ √ PIR-DSP [15]FPGAAccu. ×Both × √ √ × × × √ × √ × × Multiplier-Tree [14]FPGAAccu. √ Both √ √ √ × √ × × √ × × √ Bitshifter [14]FPGAAccu. √ Both √ √ √ × √ × × √ × × √ BitSys (Ours)FPGAAccu. √ Both √ √ √ × √ × × × × × × than high-precision. Therefore, this design scheme can support temporal reconfiguration for different precision in runtime by completing more multiplications for lower precision inmclock cycles. However, forntimes inputs, this scheme requiresn×m cycles in computation, which leads to a high inference latency in hardware accelerators. Therefore, most prior works are designed as bit-parallel archi- tectures based on sub-multiplier schemes as shown in Figure 1.2, which generate one output per clock cycle, such as the works of Neda et al. [19], Guo et al. [18], Liu et al. [14], Pf ̈ ander et al. [21], and PIR-DSP [15]. For2n× 2n-bit multiplication, A × B = A 0 B 0 × 2 2n + (A 1 B 0 + A 0 B 1 ) × 2 n + A 1 B 1 , if two inputs are split as fourn-bit data,A 0 ,A 1 ,B 0 , and B 1 , the multiplication result is computed by summing the products of fourn × n-bit sub-multiplier results by2 2n , 2 n , and1separately, which can be converted as2n/n/0-bit preset left-shifting. Therefore, if we bypass the outputs of two sub-multipliers withn-bit left shifting, the sum of four sub- multipliers is dual-channeln× n-bit multiplication. Otherwise, the result is single-channel 2n× 2n-bit multiplication. However, bypassing two sub-multipliers leads to low hardware efficiency. The works of Li et al. [25], Dai et al. [26], and BitFusion [16] explored another scheme to utilize all sub- multipliers in different precision. For instance, the BitFusion architecture shown in Figure 1.1 implemented sixteen 2-bit multipliers, BitBricks (BBs), as the basic processing elements, F-PE, to organize a systolic array. Based on the principle of sub-multiplier architecture designs, sixteen 2-bit multipliers in BitFusion can create four 4-bit multipliers and one 8-bit multiplier. The major difference is, as shown in Figure 1.1c, BitFusion applies the reconfigurable, not preset, left-shifters. Therefore, four 2-bit multipliers created a large F-PE to support both2× 8-bit and4× 4-bit multiplications to utilize all BBs with different input widths as 10 and 8 bits. The variable input width complicates the data streaming control designed as a series of multiplexers and registers. In principle, the BitFusion presents a multi-precision systolic array, not a multi-precision multiplier. Only the F-PE in in Figure 1.1c is the reconfigurable multiplier. For instance, as shown in Figure 1.1b, c, and d, BitFusion works as a4×4,1×4,1×1systolic array separately. This design limited the scenario of BitFusion architecture as the tensor processing unit. Differing from the designs mentioned above, Liu et al. [14] proposed a Bitshifter architecture inspired by the BISMO [22] converting the multiplication as the combination of bitwise AND and left-shifting. This is a bit-parallel multi-precision multiplier with a fixed input width. The result ofN-bit multiplication, A×B = P n−1 i=0 P n−1 j=0 2 i+j a i b j , is the sum of2 i+j a i b j .a i and b j are the bit values ofAandB,a i b j is the bitwise AND, and 2 i+j can be converted as the preset left-shifting. Therefore, as shown in Figure 1.5, Bitshifter architecture computes alla i b j with bitwise AND first, then filters the unnecessary results with the mask for different precision and applies the corresponding left-shifting to compute partial products. B. Comparison between the BitSys and Previous Works Considering the motivation in section IA, we target to explore a multi-precision multiplier design to speed up the computation of mixed-precision QNN models on hardware. To this end, we exclude the bit-serial multiplier scheme in our scope because of its long computation latency. To simplify the data steaming control and deploy our multiplier in variable scenarios of the existing hardware designs, such as the systolic array, single- layer accelerator, etc., we have not adopted the architecture similar to the BitFusion and works of Li et al. [25] and Dai et al. [26]. Therefore, our BitSys architecture presented a bit- parallel and input-width-fixed multi-precision multiplier design, inspired by BitShifter [14] and BitFusion [16] by converting the multiplication with bitwise operation with partial product mask and computing them with a systolic array. Table I compared it with related works. In this table, √ and×mean the selected features, like available precision, are applied in the corresponding works or not: •Both two inputs of BitSys support multi-channel reconfig- uration for variable precision. The work of Guo et al. [18] only supports 1/2-channel 2N/N × M -bit multiplication. •Our work supports accurate computing, not the approximate designs of Neda et al. [19] and Guo et al. [18]. • Shun et al. [20] proposed an accurate multi-precision multiplier based on Radix-4 Booth multiplier. However, it is designed for 8/16/24/32-bit multiplication, which is unsuitable for the 1/2/4/8-bit multiplication we targeted for low-precision QNN models. • Pf ̈ ander et al. [21] extended the work of Shun et al. [20] as serial processing to reduce resource consumption. In con- trast to this work, BitSys adopts the bit-parallel architecture to speed up computation in hardware accelerators. •PIR-DSP [15] focuses on designing multi-precision multi- pliers based on DSP slices of FPGA. However, the input widths of DSP48/DSP58 resources in Xilinx FPGA are wider than 1/2/4/8-bit QNN models. Meanwhile, DSP slices cannot process the XNOR multiplication in BNN. 8bitsx8bitsMulti-PrecisionMultiplication P i+j � � � � (�=0→7,�=0→7) leftbitshift 1bit2bit4bit8bitsum P 0 � 0 � 0 0000 0 P 1 � 1 � 0 � 0 � 1 1111 1 P 2 � 0 � 2 � 1 � 1 � 2 � 0 0+2222 2 P 3 � 3 � 0 � 2 � 1 � 1 � 2 � 0 � 3 1+2333 3 P 4 � 4 � 0 � 3 � 1 � 2 � 2 � 1 � 3 � 4 � 0 0+40+444 4 P 5 � 5 � 0 � 4 � 1 � 3 � 2 � 2 � 3 � 1 � 4 � 0 � 5 1+41+455 5 P 6 � 6 � 0 � 5 � 1 � 4 � 2 � 3 � 3 � 2 � 4 � 1 � 5 � 0 � 6 0+62+466 6 P 7 � 7 � 0 � 6 � 1 � 5 � 2 � 4 � 3 � 3 � 4 � 2 � 5 � 1 � 6 � 0 � 7 1+63+477 7 P 8 � 7 � 1 � 6 � 2 � 5 � 3 � 4 � 4 � 3 � 5 � 2 � 6 � 1 � 7 0+80+80+88 8 P 9 � 7 � 2 � 6 � 3 � 5 � 4 � 4 � 5 � 3 � 6 � 2 � 7 1+81+81+89 9 P 10 � 7 � 3 � 6 � 4 � 5 � 5 � 4 � 6 � 3 � 7 0+102+82+810 10 P 11 � 7 � 4 � 6 � 5 � 5 � 6 � 4 � 7 1+103+83+811 11 P 12 � 7 � 5 � 6 � 6 � 5 � 7 0+120+124+812 12 P 13 � 7 � 6 � 6 � 7 1+121+125+813 13 P 14 � 7 � 7 0+142+126+814 14 1-bitSub-PartialProductsMask2-bitSub-PartialProductsMask4-bitSub-PartialProductsMask � 7 � 6 � 5 � 4 � 3 � 2 � 1 � 0 � 7 � 6 � 5 � 4 � 3 � 2 � 1 � 0 � 7 � 6 � 5 � 4 � 3 � 2 � 1 � 0 � 7 10000000 � 7 11000000 � 7 11110000 � 6 01000000 � 6 11000000 � 6 11110000 � 5 00100000 � 5 00110000 � 5 11110000 � 4 00010000 � 4 00110000 � 4 11110000 � 3 00001000 � 3 00001100 � 3 00001111 � 2 00000100 � 2 00001100 � 2 00001111 � 1 00000010 � 1 00000011 � 1 00001111 � 0 00000001 � 0 00000011 � 0 00001111 Fig. 2: 1/2/4/8 Channels 8/4/2/1 bits Multiplication and Corre- sponding Partial Products Masks 0000000 � � 000000 � � � � 00000 � � � � � � 0000 � � � � � � 0 000 � � � � � � 00 00 � � � � � � 000 0 � � � � � � 0000 � � � � � � 00000 � � � � 000000 � � 0000000 Loader Output Loader Buffer B B B B B B B B a 0 a 1 a 2 a 3 a 4 a 5 a 6 a 7 b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 Data Loader D a t a L o a d e r Fig. 3: Bitwise Systolic Array (left) and Input Loader (right) Therefore, DSP slices are inefficient and unsuitable in designing our BitSys architecture. •As shown in Figure 1.4 and Figure 1.5, we fused the AND array and Mask array in Bitshifter [14] as a bitwise systolic array inspired by BitFusion [16] for higher throughputs. The left shifters and output generation stages for different precision are fused as output generation pipelines in BitSys. •The processing elements in BitSys execute 1-bit operations, supporting higher clock frequency with lower critical path delay. Moreover, the XNOR multiplication in Bitshifter [14] is computed in an individual module. We fused it in our 1-bit processing elements to save the hardware resources. •We implemented the single-layer accelerator and systolic array accelerator based on BitSys to show its potential to be applied in different designs. I. IMPLEMENTATION A. Mathematics Principle ForN-bit multiplication,A× B = P n−1 i=0 P n−1 j=0 2 i+j a i b j , (0⩽ i,j⩽ n − 1),a i andb j are the bit value ofAand B,2 i+j can be replaced as left shifting,≪ (i + j), anda i b j is the bitwiseAND. As shown in Equation 1, if we define a i b j as sub-partial products, forN-bit multiplication, we can reorganize all sub-partial products as2N − 1groups. Each group applies the same left-shifting valueM, (0⩽ M⩽ 2N− 2). Therefore, we can define the sum of one group as the partial products,P M = P M=i+j a i b j . Therefore, alla i b j are organized as the triangular-aligned structure shown in Figure 2. The bold parts in Equation 1 represent the sign bit with±in the multiplication. When it is negative, the multiplication is signed. Therefore, by switching to add or subtract the AND results of a k b n−1 anda n−1 b k (0⩽ k < n− 1) from partial products, the multiplication can be reconfigured as signed/unsigned computing. After applying the corresponding left-shifting value for each partial product, their sum is the product ofN-bit multiplication. Based on the basic mathematics principle mentioned above, as shown in Figure 2, we extend it for runtime reconfigurable multi- channel multi-precision multiplication: Using8× 8-bit multi- precision multiplication as an example,P i+j (0⩽ i⩽ 7,0⩽ j⩽ 7) are the partial products in this computation, which are the sum of corresponding sub-partial products,a i b j , shown in the same row ofP i+j in the second column of Figure 2. For different precision, three sub-partial product masks shown in Figure 2 select the desired sub-partial products,a i b j , for 8/4/2 channel 1/2/4-bit multiplications. For instance, for the dual-channel4×4- bit multiplication, two green squares in the 4-bit sub-partial products mask of Figure 2 select the desireda i b j in computation. The filtereda i b j are set as zero, and one green square in the 4-bit sub-partial products mask selects thea i b j for one channel. Based on the same principle, the four orange and eight blue squares in 2/1-bit masks select the desired sub-partial products for corresponding 4/8 channels. Alla i b j are used to compute single- channel8× 8-bit multiplication. Therefore, we can compute all sub-partial products first, reconfigure the mask in runtime to filter the undesireda i b j for different precision and channels, and then compute the sum of filtereda i b j as partial products,P i+j . Considering the lower hardware utilization efficiency when more sub-partial products are filed as zero in lower precision, compared with the disabled sub-multipliers in the previous works shown in the Figure 1.2 of section I, this is a common trade- off to achieve the multi-precision reconfiguration for related bit-parallel input-width-fixed multiplier designs. A× B =⟨a n−1 a n−2 ...a 1 a 0 ⟩ bin ×⟨b n−1 b n−2 ...b 1 b 0 ⟩ bin =(±2 n−1 a n−1 + 2 n−2 a n−2 + ... + 2 1 a 1 + 2 0 a 0 ) × (±2 n−1 b n−1 + 2 n−2 b n−2 + ... + 2 1 b 1 + 2 0 b 0 ) =[(a n−1 b n−1 )≪ 2n− 2] +[(±a n−1 b n−2 +±a n−2 b n−1 )≪ 2n− 3] +... +[(a 1 b 0 + a 0 b 1 )≪ 1] +[(a 0 b 0 )≪ 0] (1) After we get the value of partial products,P i+j , the multiplier needs to apply the corresponding left shifting toP i+j and sum them as the multi-channel results. Therefore, as shown in the left bitshift column of Figure 2, for example, when the multiplier executes 8-channel 1-bit multiplication, each channel needs two partial products and applies 0/1-bit left shifting separately. For instance, the result in the first channel of 1-bit multiplication is(P 0 ≪ 0) + (P 1 ≪ 1). Actually, the 1-bit operation only needs oneP i in each channel, such asP 0 for the first channel. However, to keep the output as 8-channel-2-bit, theP 1 is used as a placeholder, and itsa 1 b 0 anda 0 b 1 are filtered as 0 by 1-bit sub-partial product masks. Based on the same principle, for instance, we can infer thatP 3 is also a placeholder partial product for the 1st channel of 2-bit multiplication. Considering the total output width of this 8-bit multiplier is 16 bits, the output widths of one channel in 1/2/4-bit modes are 2/4/8 bits. Therefore, in the final output, each 2/4/8-bit output from the i-th channel in 1/2/4-bit multiplication needs a channel offset by left-shifting to(i− 1)× 2,(i− 1)× 4, and(i− 1)× 8bits to avoid conflict with the(i− 1)-th channel. As shown in the sum of left bitshift column in Figure 2, for each partial product, P k (0⩽ k⩽ 14), the sums of partial product left shifting (black numbers in left bitshift column) and channel offset left shifting (red numbers in left bitshift column) are alwayskin all 1/2/4/8-bit multiplication modes. Therefore, differing from the individual three left shifting stages in Bitshifter architecture [14] shown in Figure 1.4, our BitSys applied the same left shifting for each partial product in all 1/2/4/8-bit modes. In conclusion, the computation of the runtime reconfigurable multi-precision multiplication in our work can be converted into four steps: 1) Computing all a i b j (0⩽ i⩽ n− 1, 0⩽ j⩽ n− 1). 2)Filtering to get the desireda i b j with corresponding sub- partial products mask for different precision. 3)Computing the partial products,P k (0⩽ k⩽ 2n− 2), and applying k-bit left shifting. 4) Computing the sum of P k as the final output. B. Bitwise Systolic Array Architecture for Multi-precision Multiplier To execute the first two steps mentioned above, we imple- mented a bitwise systolic array as shown in Figure 3 (left) and the input loader as shown in Figure 3 (right). The input loader works to prepare the inputs for the bitwise systolic array. For instance, in an 8-bit multiplier, the input loader implements a tiny FIFO buffer with eight8-bit registers, loading one new input in the diagonal (blue bits) and pushing the data from the bottom to the top as the loader outputs (yellow part). One bitwise systolic array requires two input loaders. The bitwise systolic array we implemented in Figure 3 (left) consists of bitwise processing elements, which fused the sub-partial products mask and 1-bit arithmetic operations. Considering the multiplication in the BNN presented in FINN [3, 4] is the XNOR operation, which represents the -1 as ’0’ and +1 as ’1’, we need two kinds of bitwise processing elements as shown in Figure 5a and Figure 5b: Type.I element switches between 1-bit XNOR and AND operation for 1-bit or 2/4/8-bit multiplication. Type.I element switches between 1-bit AND operation and zero output according to the sub-partial product mask in variable precision. Figure 4 presents the location mapping of bitwise processing elements and when they are available for different precision according to the sub-partial product masks: Type.I elements are located in Region I and Type.I elements are located in Region I/I/IV. For instance, when the multiplier works in 1-bit mode, the processing elements in Region I output the results of 1-bit � � � � � � � � � � � � � � � � � � I I IV 1.RegionIisavaiablefor � � I 8-channel1-bitmultiplication � � I I 2.RegionI,IIareavaiablefor � � I 4-channel2-bitmultiplication � � IV I I 3.RegionI,I,IIareavaiablefor � � I 2-channel4-bitmultiplication � � I I 4.RegionI,I,I,IVareavaiablefor � � I 1-channel8-bitmultiplication Fig. 4: Bitwise Processing Element Location in Systolic Array in_0 in_1 pattern in_0_valid in_1_valid result result _valid in_0 in_1 pattern in_0_valid in_1_valid result AND AND AND result _valid AND AND M U X XNOR LUT5_1 1’b1 M U X result result _valid in_0 in_1 pattern in_0_valid in_1_valid LUT6_2 a) Bitwise Processing Element Type. Ib) Bitwise Processing Element Type. I c) Bitwise Processing Element based on Look-Up Table Resources (LUT) on FPGA LUT5_1 Fig. 5: Design of Bitwise Processing Element XNOR, and other elements output 0. When precision is 4-bit, the processing elements in Region I/I/I output the results of 1-bit AND, and other elements output 0. One pattern signal generated according to the current precision controls the output switching of one bitwise processing element. Furthermore, because when two inputs of the 1-bit XNOR are ’0’, the output is ’1’, the bitwise processing element needs input and output a valid signal for the following adder to avoid mistake accumulation when no input is loaded. Therefore, as shown in Figure 5c, we define one bitwise processing element as 6-bit input and 2-bit output module: 2-bit input, 2-bit input valid, 1-bit pattern switching, 1-bit input is always ’1’ to enable 2-bit output, 1-bit output, and 1-bit output valid signal. Therefore, one bitwise processing element for both types can be implemented as one LUT62 primitive in Xilinx FPGA. For the second two steps in the computation of our BitSys, the multiplier needs to compute the value of partial products, P k (0⩽ k⩽ 2n− 2), apply the left-shifting to them, and add allP k as the final output. As shown in Figure 6 (left), numbers in this figure are the left-shifting bits applied to the outputs of their located bitwise processing elements. Therefore, the sum of the bitwise processing element results with the same left-shifting bits, which are in the same diagonal, is a partial product. Considering the signed multiplication in Equation 1, the numbers in Figure 6 (right) represent that, in which precision, the outputs of bitwise processing elements they located need to be subtracted from partial products. For instance,a 7 b 6 needs Left Shifting Bits Number in Each Elements � � � � � � � � � � � � � � � � � � 1413121110987 � � 131211109876 � � 12111098765 � � 1110987654 � � 109876543 � � 98765432 � � 87654321 � � 76543210 Sign Bits Location for 1/2/4/8-bit � � � � � � � � � � � � � � � � � � 1 2 4 8 4 84 88888 � � 2 4 8 1 � � 4 81 2 � � 4 821 � � 81 2 444 � � 82 41 � � 841 2 � � 8421 Fig. 6: Left-Shifting of Diagonal & Signed Elements D0 <<0 D6 <<6 Add D5 <<5 Add D4 <<4 Add D3 <<3 Add D2 <<2 Add D11 <<11 Add D10 <<10 Add D9 <<9 Add D8 <<8 Add D7 <<7 Add D1 <<1 Add D12 <<12 Add D13 <<13 Add D14 <<14 Add O u t p u t CUTCUTCUTCUT CUTCUTCUT Fig. 7: Design of Output Generator Pipeline In0 In1 In2 In3 In4 In5 In6 In7 In8 In9 In10 In11 In12 In13 In14 In15 <<1 <<1 <<1 <<1 <<1 <<1 <<1 <<1 Neg. Neg. Neg. Neg. Neg. Neg. Neg. Neg. ADD ADD ADD ADD ADD ADD ADD ADD <<2 <<2 <<2 <<2 ADD ADD ADD ADD <<4 <<4 ADD ADD<<8 ADD ACCU Input Adder and Left Shifter Tree of Accumulation Input Converter Fig. 8: Multi-Precision Accumulator Input Converter to be subtracted in 2/4/8-bit multiplication becausea 7 is a sign bit in this precision.a 7 b 7 needs to be subtracted in 1- bit multiplication because the XNOR output is signed output, representing -1 as ’0’ and +1 as ’1’. Moreover, because botha 7 andb 7 are sign bits in 2/4/8-bit multiplication,a 7 b 7 does not need to be subtracted. After finishing the computation of partial products,P k , our BitSys multiplier loads them as the inputs,D k , of the output generator pipeline shown in Figure 7 to applyk-bit left-shifting and sum the left-shifted partial products as final output. Considering the sum of signed partial products generates the carry bits in computation and influences the result in the next channel, we insert the carry-cutter modules in the output generator pipeline to limit the output width. For instance, in 1-bit multiplication, all carry-cutters are enabled to limit the output width of 8 channels; in 2-bit multiplication, only the carry-cutters afterD 3,7,11 are enabled to limit the output width of 4 channels. Because the bitwise systolic array generates the partial product fromP 0 toP 14 sequentially and executes multiple computations simultaneously, our output generator pipeline is designed for pipelined parallel processing. For instance, in the 1st cycle, the bitwise systolic array outputs theD 0 ofMUL 0 , and the output generator pipeline left-shifts it to 0-bit. In the 2nd cycle, the bitwise systolic array outputs theD 0 ofMUL 1 andD 1 ofMUL 0 . The output generator pipeline applies the 0/1-bit left-shifting on them separately and adds theD 0 and 1-bit left-shifted D 1 of MUL 0 together for the next step. C. Single-Layer and Systolic Array Accelerator Implementation based on BitSys To evaluate the multiplier based on our BitSys architecture, we implemented one single-layer accelerator and one systolic array accelerator as shown in Figure 9 and Figure 10. Both accelerators consist of four components: 1) Input Loader (Orange), 2) BitSys Multiplier (Yellow), 3) Accumulator (Green), and 4) Activation Module (Gray). Both accelerators contain 64 multipliers. The single-layer accelerator implements these multipliers as 8 neurons. Each neuron consists of 8 multipliers and 16 input loaders. The systolic array accelerator implements these multipliers as an8×8systolic array with 16 input loaders. Both single-layer and systolic array accelerators implemented a state machine to control the inference of network models, which loads and stores the layer settings, like input length and precision, in a FIFO of FPGA. To reconfigure the multipliers for different layer precision, the state machine uses three clock cycles to load the precision data from FIFO and rewrite the registers for multiplier settings. Considering the output of BitSys multiplier is multi-channel, if we implement the corresponding accumulator and activation module for all channels, one multiplier needs to connect with eight accumulators and eight activation modules at maximum (for 1-bit mode). However, when the multiplier works on higher precision, the required accumulators and activation modules are less than 1-bit mode because of fewer output channels, leading to low hardware efficiency. Therefore, we connect each multiplier with one accumulator and activation module in both accelerators. To this end, we implemented a tree-structure-based pipelined input converter shown in Figure 8 for the accumulator to sum all channels of multiplier output: Multiplier outputs 16-bit data to this input converter asin 0−15 . The left-shifters (Orange) in Figure 8 apply the bit weight,2 i , toin i by passing through four shifting-and-adding layers in this tree structure. Because if Ais a signed value,A =−2 n−1 a n−1 + P n−2 i=0 2 i a i .a i is the bit value of A anda n−1 is the sign bit. Therefore, we insert one value inverter (Neg. Block) in the first shifting-and-adding layer to negate the left-shifted sign bit. For different precision, different numbers of value inverters are enabled. For instance, for 8-bit dual-channel input, only the inverters connected with in 7 andin 15 are enabled. Furthermore, we applied the Multi- Thresholds activation function from FINN [3, 4] to design our activation module, which fused the activation and output re-quantization as multi-thresholds. This activation function required 1/3/15/255 thresholds to generate 1/2/4/8-bit output. The number of thresholds smaller than the accumulator output is the final output. Therefore, to reduce resource consumption and improve hardware efficiency, we only implement one comparator M ... M A ... A Q ... Q Output FIFO Input FIFO S i n g l e L a y e r A c c e l e r a t o r C o n t r o l L0 ... L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 L0 L1 Fig. 9: BitSys-based Single-Layer Accelerator L M L L L L L L L L M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M M Multiplier Systolic Array A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A Accumulator Array Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Activation & Quantization Array Thresholds Control Input FIFOOutput FIFO BitSys Control Fig. 10: BitSys-based Systolic-Array Accelerator in each activation module, sequentially loading the thresholds to compare with the accumulator output. IV. EVALUATION A. Experiment Setup We evaluate the Multiplier (MUL), Multiply-Accumulator (MAC), and accelerator instances of our BitSys architecture on Ultra96-V2 FPGA platform (Zynq UltraScale+ ZU3EG). Considering the discussion in section I-B, we selected the works of Liu et al. [14] as the baseline. All accelerators are evaluated by the TFC models we trained as the same as the network used in [14] with the Brevitas and MNIST dataset, which have been introduced in section I. B. Multiplier and Multiply-Accumulator Comparison Table IV shows the implementation results from Vivado: we implemented six instances, including one pure-Verilog- designed and one LUT-primitive-optimized variant, BitSys-base and BitSys-LUT, for 1/2/4/8-bit signed/unsigned MULs and MACs of our BitSys. As the baseline, we implement the MUL and MAC instances of Multiplier-Tree and Bitshifter from Liu et al. [14] as MTee-base and Bitshifter-base, supporting 1/2/4/8-bit signed/unsigned reconfigurable multiplication, and insert the registers between the sub-multipliers of Multiplier-Tree and AND/Mask/Shifting stages of Bitshifter as shown in Figure 1.2 and Figure 1.4 to create their pipelined instances, MTee-pipe and Bitshifter-pipe, for higher clock frequency. For MUL comparison, two Bitshifter instances consume fewer LUTs with less total path delay than Multiplier-Tree instances. The pipelined Multiplier-Tree and Bitshifter instances, MTee- pipe and Bitshifter-pipe, consume more LUTs and FFs than their basic instances, MTee-base and Bitshifter-base, with significant decrease in total path delay. Compared with these baseline instances, both MULs of BitSys have the lower total path delay: the lowest total path delay belongs to BitSys-LUT, which is65.36%,44.97%,62.18%, and33.51%of Bitshifter- pipe, Bitshifter-base, MTee-pipe, and MTee-base. The LUT- primitive-optimization of BitSys-LUT instances decreased the resource consumption and total path delay compared with BitSys-base. The LUT consumption of BitSys-LUT shows no advantages with the same or higher numbers than MTee- base, Bitshifter-base, and Bitshifter-pipe as100.00%,101.45%, and103.86%.However, we discussed the Area Delay Products (ADPs) in Table IV, which are the products between LUT consumption and total path delay. The lowest ADP of BitSys instances implies that our work achieved an efficient design with a good balance between performance and resource utilization. Based on the post-implementation timing simulation in Vivado, we analyzed the power consumption of all MUL instances with 16000 times random multiplication under the highest available clock shown in Table IV. Our BitSys instances have the highest power consumption. However, their Power Delay Products (PDPs) are lower than the other four instances, which are the products between power and total path delay. This means that our design has better power efficiency and achieves a good balance between minimizing power usage and maximizing speed. Moreover, the Computation Cycles column in Table IV shows that BitSys architecture has a longer pipeline path in computation than other instances, which explains the low total path latency and high F consumption of our work. Differing from the MUL instances, MAC instances of Multiplier-Tree cost less LUT than Bitshifter because we fused the accumulator input converter design of Multiplier-Tree by summing the results of sub-multipliers and passing it to higher precision multipliers. The output of this multiplier is the sum of all channels. Following the same trend as MULs, The MAC instances of our BitSys consume more resources and power than other instances with less total path delay, lower ADP, and lower PDP. The low power consumption of MACs compared with MULs is caused by the different testbench and longer pipeline as shown in Computation Cycles column. We simulated the MACs with 4096 times random multiplication and accumulation for each precision. Before starting the next round of computation for another precision, MACs need to wait to finish the accumulation of the current precision. In summary, our BitSys architecture has a better design optimization between the balance of hardware consumption, power usage, and processing speed, supporting the highest clock frequency with the lowest total path delay. C. Neural Network Accelerator Comparison Table V is the implementation and real measurement re- sults of accelerators we implemented on Ultra96-V2 platform, including six single-layer accelerator based on Vivado IP, MTee- base, MTee-pipe, Bitshifter-base, Bitshifter-pipe, and BitSys- LUT, and one systolic array accelerator based on BitSys-LUT TABLE IV: Resource Consumption of Multipliers (MUL) and Multiply-Accumulators (MAC) Design Instance SettingResource Consumption Frequency Total DelayArea-DelayDynamic PowerPDPCompuation Cycles TypePrecisionSigned/UnsignedAccurate/ApproximateLUTFF(ns)Products(mW )(mW × ns)BIN2-bit4-bit8-bit MTree-base [14] MUL1/2/4/8BothAccurate 38342250MHz3.8201463.0687332.341111 MTree-pipe429282375MHz2.282978.98125285.255555 Bitshifter-base [14]34537300MHz3.1561088.82107337.691111 Bitshifter-pipe337339375MHz2.171731.627122264.861999 BitSys-base416463500MHz1.433596.128156223.5522222222 BitSys-LUT350525500MHz1.419496.65159225.6222222222 MTree-base [14] MAC1/2/4/8BothAccurate 398199250MHz3.3971352.0179268.366666 MTree-pipe495388250MHz2.8281399.86102288.4610101010 Bitshifter-base [14]505198300MHz3.0841425.27102314.576666 Bitshifter-pipe538506375MHz2.1641164.23109235.886141414 BitSys-base597633375MHz2.0721236.98103213.4227272727 BitSys-LUT541689500MHz1.716928.36134229.9427272727 TABLE V: Resource Consumption of Previous and BitSys Accelerators on Ultra96V2 FPGA Platform DesignTypePrecision LUTFFBRAM FrequencyLatency/μs NumberRateNumberRateNumberRate Vivado IP [19]Single-Layer 8/8/8/8 2409034.14%2217515.71%13562.50%150MHz 137.654 1/2/4/8131.059 MTree - base [19]Single-Layer1/2/4/83702052.47%2250015.94%13863.89%100MHz69.27 Bitshifiter - base [19]Single-Layer1/2/4/84295260.87%2248615.93%13863.89%125MHz56.658 MTree - pipeSingle-Layer1/2/4/84716366.84%4210029.83%13863.89%150MHz48.443 Bitshifiter - pipeSingle-Layer1/2/4/85021271.16%5039335.71%13863.89%150MHz48.799 BitSys - LUTSingle-Layer1/2/4/84657066.00%5435238.51%13863.89%150MHz49.057 BitSys - LUTSystolic Array1/2/4/84446863.02%6417645.48%139.564.58%250MHz36.741 to compare the difference between single-layer accelerator architecture and systolic array architecture. The column of Latency/μsis the single frame inference delay by averaging the total inference latency of 1000 MNIST inputs. Vivado IP-based single-layer accelerator consumes the least LUTs and FFs with the longest single-frame inference latency compared with other accelerators because it does not support multi-precision multiplication. Both Bitshifter accelerators cost more hardware resources than Multiplier-Tree. The pipelined accelerator of both Multiplier-Tree and Bitshifter support higher clock frequency than their basic accelerators. According to the to- tal path delay of MUL and MAC shown in Table IV, in principle, the single-layer accelerators of BitSys-LUT and Bitshifter-pipe should support higher clock frequency than MTee-base, MTee- pipe, and Bitshifter-base. However, because the single-layer accelerator contains one more complex state machine than the systolic array for data streaming control to load the activations and weights from DDR to neurons and schedule the computation with a limited number of neurons and multipliers, 150MHzis the highest frequency that can be supported in our current single- layer accelerator architecture. The systolic array accelerator implemented with BitSys-LUT consumes95.49%of LUTs but 118.07%of FFs with 250MHzcompared with its single-layer accelerator. Comparing the two structures shown in Figure 9 and Figure 10, systolic array accelerator of BitSys-LUT requires much fewer input loaders that single-layer accelerator, which causes the LUT consumption decreasing shown in Table V. For the average inference latency, all accelerators implemented with multi-channel multi-precision multiplier have a high speed-up compared with Vivado IP-based accelerator. The single-layer accelerators of Bitshifter-pipe and MTee-pipe are116.1%and 142.99%faster than Bitshifter-base and MTee-base because of higher clock frequency. Compared with the single-layer acceler- ator of Bitshifter-pipe and MTee-pipe, and BitSys-LUT, BitSys- LUT instance is0.53%and1.25%slower than Bitshifter-pipe and MTee-pipe with same frequency. Considering the Compu- tation Cycles shown in Table IV, BitSys-LUT instance infers slower because of its long pipeline path. For the same reason, the single-layer accelerator of Bitshifter-pipe is0.73%slower than MTee-pipe. However, because the systolic array structure simplified the data streaming control in a single-layer accelerator, the inference latency of BitSys-LUT benefits both from the higher clock frequency and denser computation, which can highly efficiently utilize the fully pipelined design in our BitSys architecture. Therefore, the systolic array accelerator of BitSys- LUT supports 250MHzand is356.71%,188.54%,148.77%, 131.85%,132.82%, and133.52%faster than the single-layer accelerators of Vivado IP, MTee-base, MTee-pipe, Bitshifter- base, Bitshifter-pipe, and BitSys-LUT with mixed-precision TFC network. V. CONCLUSION In this manuscript, we present one multiplier design based on fully pipelined bitwise systolic array architecture, BitSys, supporting the runtime reconfigurable multi-precision multi- channel multiplication. The evaluation shows that our BitSys architecture has a low critical path delay to support higher clock frequency compared with previous works. In the acceleration of the mixed-precision network model, our work is more than 131.85%faster than original Multiplier-Tree and Bitshifter architecture and about356.71%faster compared with Vivado- IP-based accelerator. For our future work, we plan to explore the ASIC implementation of our BitSys architecture with emerging memory technologies, such as Racetrack Memory (RTM). VI. ACKNOWLEDGEMENTS This work is supported by the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig and Deutsche Forschungsgemeinschaft (DFG) under the X-ReAp project (Project number 380524764). REFERENCES [1]Gaofeng Zhou, Jianyang Zhou, and Haijun Lin. “Research on NVIDIA Deep Learning Accelerator”. In: 2018 12th IEEE International Confer- ence on Anti-counterfeiting, Security, and Identification (ASID). 2018, p. 192–195. [2]Vinod Kathail. “Xilinx Vitis Unified Software Platform”. In: Pro- ceedings of the 2020 ACM/SIGDA International Symposium on Field- Programmable Gate Arrays. FPGA ’20. New York, NY, USA: Associ- ation for Computing Machinery, 2020, 173–174. [3]Yaman Umuroglu et al. “Finn: A framework for fast, scalable binarized neural network inference”. In: Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2017, p. 65–74. [4]Michaela Blott et al. “FINN-R: An end-to-end deep-learning framework for fast exploration of quantized neural networks”. In: ACM Transactions on Reconfigurable Technology and Systems (TRETS) 11.3 (2018), p. 1– 23. [5]Farah Fahim et al. “hls4ml: An Open-Source Co-Design Workflow to Empower Scientific Low-Power Machine Learning Devices”. In: Research Symposium on Tiny Machine Learning. 2021. [6]Yaman Umuroglu et al. “LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications”. In: 2020 30th In- ternational Conference on Field-Programmable Logic and Applications (FPL). IEEE. 2020, p. 291–297. [7] Jiang Su et al. “Accuracy to throughput trade-offs for reduced precision neural networks on reconfigurable logic”. In: International Symposium on Applied Reconfigurable Computing. Springer. 2018, p. 29–42. [8] Kuan Wang et al. “HAQ: Hardware-Aware Automated Quantization With Mixed Precision”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 2019. [9]Weihan Chen, Peisong Wang, and Jian Cheng. “Towards Mixed- Precision Quantization of Neural Networks via Constrained Optimiza- tion”. In: CoRR abs/2110.06554 (2021). arXiv: 2110.06554. [10] Chen Tang et al. “Mixed-Precision Neural Network Quantization via Learned Layer-Wise Importance”. In: European Conference on Computer Vision. Springer. 2022, p. 259–275. [11]Alessandro Pappalardo. Xilinx/brevitas. 2023. [12] Yann LeCun et al. “Gradient-based learning applied to document recognition”. In: Proceedings of the IEEE 86.11 (1998), p. 2278–2324. [13]Li Deng. “The mnist database of handwritten digit images for machine learning research”. In: IEEE Signal Processing Magazine 29.6 (2012), p. 141–142. [14]Yuhao Liu et al. “High Flexibility Designs of Quantized Runtime Reconfigurable Multi-Precision Multipliers”. In: IEEE Embedded Systems Letters (2023), p. 1–1. [15]SeyedRamin Rasoulinezhad et al. “PIR-DSP: An FPGA DSP Block Architecture for Multi-precision Deep Neural Networks”. In: 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 2019, p. 35–44. [16] Hardik Sharma et al. “Bit Fusion: Bit-Level Dynamically Com- posable Architecture for Accelerating Deep Neural Network”. In: 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA). 2018, p. 764–775. [17]Yuhao Liu, Salim Ullah, and Akash Kumar. “BitSys: Bitwise Systolic Array Architecture for Multi-precision Quantized Hardware Accel- erators”. In: 2024 IEEE 32nd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 2024, p. 220–220. [18]Chuliang Guo et al. “A Reconfigurable Approximate Multiplier for Quantized CNN Applications”. In: 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC). 2020, p. 235–240. [19]Negar Neda et al. “Multi-Precision Deep Neural Network Acceleration on FPGAs”. In: 2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE. 2022, p. 454–459. [20]Zhou Shun et al. “A VLSI architecture for a Run-time Multi-precision Reconfigurable Booth Multiplier”. In: 2007 14th IEEE International Conference on Electronics, Circuits and Systems. 2007, p. 975–978. [21]Oliver A. Pf ̈ ander et al. “Configurable Blocks for Multi-precision Multiplication”. In: 4th IEEE International Symposium on Electronic Design, Test and Applications (delta 2008). 2008, p. 478–481. [22]Yaman Umuroglu, Lahiru Rasnayake, and Magnus Sj ̈ alander. “Bismo: A scalable bit-serial matrix multiplication overlay for reconfigurable com- puting”. In: 2018 28th International Conference on Field Programmable Logic and Applications (FPL). IEEE. 2018, p. 307–3077. [23] P. Ienne and M.A. Viredaz. “Bit-serial multipliers and squarers”. In: IEEE Transactions on Computers 43.12 (1994), p. 1445–1450. [24] Andrew G. Shafer, Lyndsi R. Parker, and Earl E. Swartzlander. “The fully-serial pipelined multiplier”. In: 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR). 2011, p. 1817–1822. [25]Kai Li et al. “A Precision-Scalable Energy-Efficient Bit-Split-and- Combination Vector Systolic Accelerator for NAS-Optimized DNNs on Edge”. In: 2022 Design, Automation and Test in Europe Conference and Exhibition (DATE). 2022, p. 730–735. [26]Liuyao Dai et al. “An Energy-Efficient Bit-Split-and-Combination Systolic Accelerator for NAS-Based Multi-Precision Convolution Neural Networks”. In: 2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC). 2022, p. 448–453.