Paper deep dive

Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review

Soumita Chatterjee, Sudip Ghosh, Tamal Ghosh, Hafizur Rahaman

Year: 2026Venue: arXiv preprintArea: cs.ARType: PreprintEmbeddings: 140

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 7/20/2026, 11:27:25 AM

Summary

This paper provides a comprehensive review of architectural designs and performance analysis for AI accelerators, with a specific focus on Field Programmable Gate Arrays (FPGAs). It compares FPGAs against Graphics Processing Units (GPUs) and Application-Specific Integrated Circuits (ASICs), highlighting FPGAs' flexibility, reconfigurability, and energy efficiency for deep learning workloads. The review covers hardware-level optimizations such as loop pipelining, parallelism, quantization, and memory hierarchy enhancements, and analyzes state-of-the-art accelerators for various neural network types including CNNs, RNNs, SNNs, and GNNs.

Entities (16)

FPGA · hardware · 99%Deep Learning · technology · 98%ASIC · hardware · 95%GPU · hardware · 95%CNN · model-type · 92%GNN · model-type · 92%RNN · model-type · 92%SNN · model-type · 92%CUDA · software-framework · 90%NPU · hardware · 90%TPU · hardware · 90%Loop Pipelining · optimization-technique · 88%Parallelism · optimization-technique · 88%Quantization · optimization-technique · 88%AccUDNN · system · 85%DianNao · system · 85%

Relation Signals (15)

FPGA → offersadvantage → reconfigurability

confidence 95% · FPGAs provide a flexible and reconfigurable platform... This reconfigurability... makes FPGAs particularly valuable

FPGA → offersadvantage → Flexibility

confidence 95% · FPGAs provide a flexible and reconfigurable platform, allowing model specific customization while maintaining high efficiency.

GPU → haslimitation → high_power_consumption

confidence 92% · Although GPUs provide high throughput... they often suffer from high power consumption

ASIC → haslimitation → inflexibility

confidence 92% · ASIC accelerators... are expensive to develop, unflexible after fabrication, and slow to adapt

FPGA → comparedto → GPU

confidence 90% · The limitations of ASIC and GPU accelerators have led to FPGAs becoming one of the prominent solutions

FPGA → comparedto → ASIC

confidence 90% · The limitations of ASIC and GPU accelerators have led to FPGAs becoming one of the prominent solutions

AccUDNN → istypeof → GPU

confidence 90% · Guo et al. [45] proposes "AccUDNN" ... a GPU accelerator

DianNao → istypeof → ASIC

confidence 90% · Chen et al. [18] proposed an 65 nm technology ASIC-based accelerator... (DianNao)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Deep learning (DL) has emerged as a rapidly developing advanced technology, enabling the performance of complex tasks involving image recognition, natural language processing, and autonomous decision-making with high levels of accuracy. However, as these technologies evolve and strive to meet the growing demands of real-life applications, the complexity of DL models continues to increase. These models require processing of massive volumes of data, demanding substantial computational power and memory bandwidth. This gives rise to the critical need for hardware accelerators that can deliver both high performance and energy efficiency. Accelerator types include ASIC based solutions, GPU accelerators, and FPGA based implementations. The limitations of ASIC and GPU accelerators have led to FPGAs becoming one of the prominent solutions, offering distinct advantages for DL workloads. FPGAs provide a flexible and reconfigurable platform, allowing model specific customization while maintaining high efficiency. This article explores various hardware level optimizations for DL. These optimizations include techniques such as loop pipelining, parallelism, quantization, and various memory hierarchy enhancements. In addition, it provides an overview of state-of-the-art FPGA-based neural network accelerators. Through the study and analysis of these accelerators, several challenges have been identified, paving the way for future optimizations and innovations in the design of FPGA-based hardware accelerators.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

138,718 characters extracted from source content.

Expand or collapse full text

Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review SOUMITA CHATTERJEE, Indian Institute of Engineering Science and Technology, Shibpur, India SUDIP GHOSH, Indian Institute of Engineering Science and Technology, Shibpur, India TAMAL GHOSH, Indian Institute of Engineering Science and Technology, Shibpur, India HAFIZUR RAHAMAN, Indian Institute of Engineering Science and Technology, Shibpur, India Deep learning (DL) has emerged as a rapidly developing advanced technology, enabling the performance of complex tasks involving image recognition, natural language processing, and autonomous decision-making with high levels of accuracy. However, as these technologies evolve and strive to meet the growing demands of real-life applications, the complexity of DL models continues to increase. These models require processing of massive volumes of data, demanding substantial computational power and memory bandwidth. This gives rise to the critical need for hardware accelerators that can deliver both high performance and energy efficiency. Accelerator types include ASIC based solutions, GPU accelerators, and FPGA based implementations. The limitations of ASIC and GPU accelerators have led to FPGAs becoming one of the prominent solutions, offering distinct advantages for DL workloads. FPGAs provide a flexible and reconfigurable platform, allowing model specific customization while maintaining high efficiency. This article explores various hardware level optimizations for DL. These optimizations include techniques such as loop pipelining, parallelism, quantization, and various memory hierarchy enhancements. In addition, it provides an overview of state-of-the-art FPGA-based neural network accelerators. Through the study and analysis of these accelerators, several challenges have been identified, paving the way for future optimizations and innovations in the design of FPGA-based hardware accelerators. CCS Concepts:• Hardware→Hardware accelerators;• Computing methodologies→Supervised learning by classification. Additional Key Words and Phrases: ASIC, CNN, FPGA, GNN, in-memory computing, NPU, optimizations, pipelining, RNN, SNN, TPU ACM Reference Format: Soumita Chatterjee, Sudip Ghosh, Tamal Ghosh, and Hafizur Rahaman. 2018. Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review. 1, 1 (March 2018), 36 pages. https://doi.org/X.X 1 Introduction Deep learning (DL) models are advanced machine learning systems composed of multiple layers of interconnected nodes that progressively transform input data into higher level representations. These models are designed to automatically extract complex patterns and features from large volumes of data without manual intervention. This capability makes DL highly effective for solving complex problems in domains like computer vision, natural language processing, speech recognition and autonomous systems. Using vast datasets and substantial computational power, DL models achieve Authors’ Contact Information: Soumita Chatterjee, Indian Institute of Engineering Science and Technology, Shibpur, Howrah, India, csoumita45@gmail. com; Sudip Ghosh, Indian Institute of Engineering Science and Technology, Shibpur, Howrah, India, sudip.ghosh@vlsi.iiests.ac.in; Tamal Ghosh, Indian Institute of Engineering Science and Technology, Shibpur, Howrah, India, tamalghosh.vlsi@faculty.iiests.ac.in; Hafizur Rahaman, Indian Institute of Engineering Science and Technology, Shibpur, Howrah, India, hafizur@vlsi.iiests.ac.in. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. Manuscript submitted to ACM Manuscript submitted to ACM1 arXiv:2603.08740v1 [cs.AR] 25 Feb 2026 2Chatterjee et al. high performance across a wide range of applications while remaining adaptable to innovations across diverse domains such as agriculture, healthcare, Automation, robotics, etc. In recent years, deep learning has experienced significant growth in prominence and scalability across various domains. Its applications span image classification [31,132], image segmentation [153], speech recognition [94,113], text processing [116,130], object detection [4], disease prediction [26], medical diagnosis [27], fault prediction [126] and detection [8], climate modeling [137], and fraud detection [3]. As DL models scale, the need to process and manage large volumes of data grows accordingly. Spectacular hardware accelerators that offload computation from the Central Processing Unit (CPU) address performance bottlenecks by enabling more efficient and faster execution. The increasing complexity of neural networks has fuelled the demand for hardware based Artificial Intelligence (AI) accelerators, which provide high throughput, low latency, and concurrent multiprocessing capabilities. These characteristics allow them to process large datasets with greater speed than traditional computing platforms. Among the various accelerators proposed over the years, ranging from Graphics Processing Units (GPUs) to Application-Specific Integrated Circuits (ASICs), each offers distinct advantages and limitations. GPUs provide substan- tial parallel processing capability through standardized instruction sets, while ASICs offer high computational efficiency but lack reconfigurability and require long development cycles. As DL models continue to expand on a larger scale, GPUs and ASICs alone are no longer sufficient. Field Programmable Gate Arrays (FPGAs), known for their low latency, parallelism, and reconfigurable architecture, have emerged as promising candidates for accelerating large scale neural networks. However, the growing diversity of FPGA-based accelerators presents challenges in identifying appropriate optimization techniques and methodologies. Recent survey efforts [114] highlight these trends and limitations. A brief overview of the architecture, operation, merits and demerits of the different types of AI hardware accelerators if given in the Table 1. The diagram shown in Fig. 1 provides a view of the various performance metrics in the acceleration of the neural network, including average throughput, speedup, latency, power efficiency, and area. FPGA based AI accelerators offer a balanced middle ground between the flexibility of GPUs and the efficiency of ASICs. Although GPUs provide high throughput and benefit from mature programming ecosystems [90,115], they often suffer from high power consumption and limited architectural specialization for new or rapidly evolving AI models. ASIC accelerators, although capable of delivering exceptional performance and energy efficiency through custom dataflow designs [20,55], are expensive to develop, unflexible after fabrication, and slow to adapt to emerging neural architectures. In contrast, FPGAs combine reconfigurable logic with energy-efficient computation, enabling designers to tailor hardware precisely to model requirements and update implementations as algorithms evolve [124,148]. This reconfigurability, along with their support for low-latency execution and customizable precision, makes FPGAs particularly valuable for edge and domain-specific AI applications where adaptability, efficiency, and rapid prototyping are essential. In this article, we present a comprehensive overview of accelerator advancements along with the optimization method- ologies at both the network and hardware levels. We also discuss the challenges and future research directions in accelerator optimization. This study analyzes various approaches from multiple perspectives, as illustrated in Fig. 2. Section 2 reviews the structures of different types of existing hardware accelerators and the methodologies employed within them. Section 3 surveys network specific FPGA accelerators including Convolutional Neural Networks (CNNs), Spiking Neural Networks (SNNs), Recurrent Neural Networks (RNNs), and Graph Neural Networks (GNNs) and discusses their design considerations, acceleration techniques, and associated challenges. Section 4 examines hardware level optimization strategies designed to minimize bottlenecks. A performance analysis of the state-of-the art model specific FPGA accelerators is provided by Section 5 which gives an estimate of achieved throughput and resource utilization. Although FPGAs demonstrate strong potential as AI accelerators, they face limitations such as restricted hardware Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review3 Table 1. Comparison of GPU, ASIC, and FPGA-based AI Accelerators Accelerator Type ArchitectureOperationMeritsDemerits GPU [58], [134], [75], [136] Massively parallel SIMD/SIMT architecture with thousands of cores; optimized for throughput-based computing. Executes large numbers of parallel threads; ideal for matrix multiplications and convolution operations. • Mature and well-supported software ecosystem (CUDA, ROCm) •Highcomputational throughput •Flexible for both training and inference •Widely available and cost- effective • High power consumption • Lower energy efficiency compared to ASICs •Limited memory bandwidth for certain workloads •General-purpose, not fully optimized for specific net- works ASIC (NPUs, TPUs) [18], [19], [80], [128], [143] Custom dataflow architectures optimized for neural network operations; fixed-function hardware tailored for AI kernels. Executes highly optimized tensor and matrix operations using specialized systolic arrays or dataflow engines. • Highest performance and en- ergy efficiency • Predictable latency • Optimized for specific DNN workloads •Suitable for large-scale data- center deployments •Very high development cost and long design cycle •Inflexible—cannot be recon- figured after fabrication • Limitedadaptabilityto evolving AI models FPGA [41], [11], [1], [13] Reconfigurable logic fabric with LUTs, DSP blocks, BRAM, and interconnects; adaptable hardware architecture. Implements custom data paths and operators through reconfiguration; supports pipelined and parallel execution. • Highly flexible and repro- grammable •Low latency and energy- efficient for edge workloads • Customizable precision (e.g., INT8, binary networks) •Suitable for prototyping cus- tom accelerators • Lower peak performance than GPUs/ASICs •Programming requires hard- ware expertise (HDL, HLS) • Long compilation and bit- stream generation times • Limited on-chip memory Fig. 1. Performance Comparison Metrics Across CPU, GPU, ASIC and FPGA. resources. These challenges are discussed in Section 6. Section 7 outlines potential directions for further optimization, and Section 8 concludes the article. Manuscript submitted to ACM 4Chatterjee et al. Fig. 2. Key features, existing optimization techniques, limitations & need for further optimization of FPGA-based hardware accelerators. 2 Background Hardware accelerators have emerged as a preferred solution for speeding up deep learning workloads, offering significantly higher computational capability and efficiency than traditional CPUs. Among these, GPUs are typically the first choice due to their massive parallelism and support from mature, standardized software libraries. ASIC-based accelerators form the next category, providing custom designed architectures tailored to specific network requirements; these primarily include Neural Processing Units (NPUs) and Tensor Processing Units (TPUs). Finally, FPGA based accelerators are gaining prominence for their reconfigurable architectures, which allow flexible hardware customization. The following subsections provide a detailed architectural review of different types of hardware accelerators. 2.1 Graphics Processing Units (GPU) GeForce 256 accredited as the "the world’s first ’GPU’ was described as "a single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines that is capable of processing a minimum of 10 million polygons per second" [23]. GPUs have evolved and now have become one of the foremost AI accelerator due to its ability to speed up the training and inference processes. DG-RePlAce, a new age GPU accelerator based on OpenROAD infrastructure (Ajayi et al. [2]) is proposed by Kahng et al. [59]. The methodology starts with the conversion of a structural netlist to a clustered one with the help of open source platform Hier-RTLMP (Kahng et al. [58]). This is followed by inclusion of datapath to find instances for each cluster and its placement according to proximity rules. A parallel analytical placement framework built on OpenROAD system is employed to reduce memory overhead and enhance computational speed. Xie et al. [134] has exploited the vast network of streaming multiprocessors in a GPU to execute parallel thread computations for a SNN network. The design has a reported speedup of 31x compared to its CPU counterpart. Compute Unified Device Architecture (CUDA), an extension of C language uses serial and parallel coding to swiftly switch from CPU to GPU operations. Computation intensive processes such as training of the accelerator and matrix multiplications are carried out in the GPU. Due to parallel processing capabilities the processes are accelerated to a certain extent which also makes future prediction possible. Li. et al. [75] proposes design of a core feedforward module. in which module takes input from the parameter training module and adds a new feature to the feedforward network accordingly. This model also reduces the processing time of each image with the help of a sliding window sample module. Xie et al. Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review5 [136] proposes a GPU accelerator for Graph Convolution Network (GCN) for optimization and provision of multi-level memory efficiency and reduction of workload imbalance. It consists of a load aware balancing scheme that evenly distributes work across warps which is further localized with the help of an execution model. Block-warp mapping combines several warps into one to handle entire workload. This ensures memory continuity and optimizes memory bandwidth. Juang et al. [57] proposed parallelization of the learning phase of Interval Type-2 Neural Fuzzy Systems approach to speedup the process. With the help of the CUDA model, dataset parallelism is ensured with the distribution of datasets over the shared memory. This feature ensures high throughput while maintaining a low latency. Each thread of a single core is responsible for obtaining the inference on different data samples. This approach is used for derivation of system outputs which trumps the traditional iterative method and obtains a 30x training speed improvement. Fei et al. [36] have proposed an alternative method to accelerate the training phase in the form of the processing-in-memory-GPU architecture. The network consists of a multiple 3D stack structure made of streaming multiprocessors. Each stack stores a set of weights for each layer which then operates concurrently thus providing model parallelism. For parallelization of data transmission, gradient transmission is used to reduce the communication overhead. A tradeoff between these two optimizations is made according to the model requirements. Guo et al. [45] proposes "AccUDNN" (given by Fig. 3), a GPU accelerator to reduce the memory resource scarcity issue. The design employs a memory optimizer module which uses a dynamic technique to find the optimal tradeoff between trainability and enhancement of training efficiency. This is decided by the optimal swapping of the feature maps onto the host memory which ranges from the extreme of transferring all data to the host to zero. Once trainabilty with the limited memory resource is attained the hyperparameter tuner proceeds onto ensure optimum training efficiency. Though the model makes the training process highly optimized in terms of memory usage but it heavily relies on the Peripheral Component Interconnect Express (PCIe) Bandwidth which can become a bottleneck due to its transfer speed. Fig. 3. Architecture of AccUDNN showing the process flow between the memory optimizer and hyperparameter tuner modules. [45] 2.2 ASIC based Accelerator Chen et al. [18] proposed an 65 nm technology ASIC-based accelerator for very large-scale convolutional and deep learning models. Employment of staggering pipeline methodology streamlines the convolutional, maxpooling and the sigmoid layers. The authors exploited the locality properties of the layers to design dedicated buffer storage for handling of the input, intermediate and output results. To further optimize memory access, the accelerator (DianNao) employs local transposition of loops and circular rotation of buffers for temporal reuse. After exploration of various fixed point representations, the authors observed that a fixed 16 bit representation is optimal and provides comparable results with an affordable tradeoff between chip area and power. On analyzing the performance of the accelerator, it is Manuscript submitted to ACM 6Chatterjee et al. seen that it achieves a speedup of (117.87x) while consuming a minimal power of 485 mW. However at the time they had not considered the energy and power overhead and was also faced with a limitation of memory bandwidth usage. DaDianNao, proposed by Chen along with a collaborator, Luo [19] a 28 nm technology multi-chip system was an improvised version of the previously stated accelerator. The architecture employs a tiled based design to counter the internal bandwidth issues where the large number of neurons are distributed over multiple Neural Functional Units (NFUs). It also uses biased nodal footprint which not only prioritizes storage than computations but also are placed closely to reduce data access. Though being able to achieve a massive speedup of (450.65x) over a GPU, it is limited by the demerits of interconnects which limits scalability of the deep learning network. Over the next years several improvisations has been implemented over the originally proposed accelerator generating a family of ASIC based accelerators. PuDianNao and Shidiannao proposed by Liu et al. [80] and Du et al. [28] are a part of this family. The former uses single port Static RAM (SRAM) based buffers connect to the same Direct Memory Access (DMA) which significantly reduces footprint and power consumption. This accelerator is designed in such a way so that it can employ several ML techniques with the help of dedicated novel units. Though lesser than [19] this achieves a speedup of (1.20x) over a GPU. Unlike its predecessors, the latter stores all the weights in SRAM itself thus eliminating the need of Dynamic RAM (DRAM) accesses and unnecessary data movements within the layers itself. This makes it (4688.13x) times more energy efficient than a GPU but is not able to achieve a massive speedup. These designs aim to optimize memory accesses but are not adaptable to different types of network. Parallelism is introduced to efficiently use computational resources and make scalable architecture. [85] proposed FlexFlow estimates degree of parallelism for each convolution layer and accordingly selects the data flow mechanism to solve the mismatch issues associated with the workload. This makes the design adaptable to a wider range of neural models. UniPRE, a hybrid accelerator given by Zhang et al. [155] is one such example which eliminates redundant computations resulting from maxpooling operations. It achieves this by using a channel order queue to perform computations only at maximum channels as predicted from earlier ones. It also reduces energy layer wise achieving an efficiency of 19.32 TSOPS/W. An attempt to design a low power accelerator was taken by Chen et al. [12]. The design includes a time reconfigurable neuron architecture which parallelizes multi-time operations thus reducing power significantly. Gallo et al. [70] cites the usage of a specialized ASIC architecture- International Business Machines (IBM) analog accelerator. This is designed for Analog In-Memory Computing (AIMC). In this type of computing, the AIMC core (given by Fig. 4) consisting of resistive crossbar arrays directly perform matrix–vector multiplications. The problems of conductance drift and device variability are mitigated by the noise model. The cores are combined with Digital Signal Processor (DSP) units to perform auxiliary Deep Neural (DN) computations which accelerate the computation process. The cores have weight update schemes like in-memory Stochastic Gradient Descent (SGD) for overlapping pulses over rows and columns of the matrix, Tiki-Taka for smoothing of noisy gradients, to increase robustness that often suffers due to analog updates. To streamline the memory computations, tiling and partitioning strategies are introduced. A hybrid digital-analog mode is provided for performing time sensitive operations digitally with the help of affine transformations. To further improve the performance, algorithm-hardware co-optimization is carried out to match the low quantized pre-trained models with the hardware resources. In this way analog efficiency meets with digital precision. 2.3 Neural Processing Units (NPUs) A NPU (also known as AI chip or AI accelerator) is a special type of deep learning accelerator designed specially to speedup the neural network computations. The architecture is modeled in such a way to emulate the processing of a human brain [53]. Though computationally not very precise, these have specialized models which can accelerate Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review7 Fig. 4. Architecture of an analog in-memory computing AIMC core with integrated crossbar arrays & memory-based unit cells. [70] operations like dot product and matrix multiplication. Unlike GPUs the instruction set is concise and is able to accomplish several tasks just with few instructions at same computing power. An in-memory NPU given by Jeon et al. [51] addresses the problem of heavy memory computation. In order to speed up the computation capability of a neural model, the design includes a Single Instruction Multiple Data (SIMD) parallel vector processing unit and systolic array structure. The systolic array is composed of a Multiplier Accumulator Unit (MAC) which performs concurrent multiplications thus accelerating the convolution operation. Further optimization is achieved by using library functions which makes it adaptable to any neural network. According to the requirements of the neural network, the BLAS library optimizes operations like dot vector, matrix matrix multiplication, matrix vector multiplication element wise addition and so on. The Network-on-chip (NoC) architecture is introduced by Ouyang et al. [97] which comprises of a set of NPUs which handle the computation part whereas the communication functions are carried out by the router. The router is responsible for multicast communication where it sends the relevant weights as packets to the NPUs. The NPUs then handle the processing of the operations. Wang et al. [128] proposes a Kernel Convolution Network for efficient handling of tasks like segmentation and detection. Some channels of the network are allocated for storing the feature maps while the remaining capture fine spatial detail. This makes the design adaptable to large size kernels. Cheng et al. [21] presents a more powerful deep CNN accelerator, Eyeriss which is capable of processing large vision based tasks. The spatial architecture contains a 12× 14 array of 168 Processing Elements (PEs) which is coupled with a unique Row Stationary (RS) dataflow. This reconfigures the PE array to map the computation of various CNN shapes to maximize data reuse locally within a four-level memory hierarchy. The hierarchy is divided into spads, inter-PE communication, Global Buffer, and DRAM. To further enhance performance, it also integrates a NoC for single-cycle data delivery which exploits zero-value statistics and use data gating to save up to 75% of DRAM bandwidth. The NPUs offer a very low latency and also quite efficient in acceleration of standard deep and convolutional neural network. Being application specific, it has a very low flexibility and suffers from scalability issues. These are unfit for applications where precision is of utmost importance as accuracy may worsen if not properly quantized. 2.4 Tensor Processing Units (TPUs) Tensor Processing Unit a custom ASIC accelerator has been developed by Google which are used in datacenter to speedup the inference phase of the models. Jouppi et al. [56] explained the architecture (given by Figure 5) of the TPU in details. It processes every operation in a predictable manner and omits general purpose tasks making its architecture simpler. This was only used in datacentres and was not made accessible for public use. Joppi et al. [54] laid out the Manuscript submitted to ACM 8Chatterjee et al. implementation details of the first generation TPU. It consists of a single core having a systolic MAC unit which not only optimizes data flow by reusing operands. The architecture is equipped with a small set of operations which retains its simple nature. Like its predecessor, this design also offers high throughput and bandwidth memory for large deep neural networks. Mummoju et al. [92] provided a thorough history of TPU. Both TPU versions 2 and 3 have four chips equipped with two cores consisting of a scalar unit, a vector processing unit and a couple matrix multiply units. In addition to this, they proposed two methodologies of diving the datasets required for a k-means algorithm. For both the methods, the batch datasets and the CPU computed centroids are passed onto the TPU for assignment and further computations. Notably, the runtime is significantly lower on TPU as compared to a CPU. Fig. 5. Architecture of TPU v1 showing dataflow between the systolic array, unified buffer & high-bandwidth memory interface.[54] Yuan et al. [143] proposes a dimensionality reduction technique in the form of a tensor-ring decomposition method. It combines two decomposition methods- Tensor Ring Alternative Least Squares (TRALS) and Tensor Ring Singular Value Decomposition (TRSVD) to make the process of tensor decomposition fast and reliable. Soto-Chirinos et al. [117] explore three TPU based architectures to solve different types of scalability problems. The first model - TPU Virtual Machine (VM) is a non-scalable architecture that while having high performance is suited only for single-node experiments. Despite being a slower model, the second one is a scalable version suited for moderate scalability. Also, the production time was a bit increased which is corrected by the last model. The last architecture scales horizontally across various nodes while being resource efficient. Other optimizations include time-triggered memory access approach given by Ezekiel [30] that fetches data before request for loading data is made. This also consists of an offline phase where the time-triggered schedule is generated to address memory time accesses. For design of modern accelerators, authors often sought the help of automatic Register-Transfer Level (RTL) generation. Using affine-based transformation, Lin et al. [79], LEGO is proposed which synthesizes adjustable memory sizes and mixes different type of dataflow methodologies. The back end of the method- ology transforms hardware into a lower level graph for remaining optimizations. Latest advancements include replacement of traditional electronic components with light based processing units (Tang et al. [120]). It consists of a single modulator having an optical core which performs all parallel tensor convolutions thus making it suitable for large data handling applications. The convolution operations are simultaneously encoded into different channels which are further multiplied with the weights. This is done with the help of light interference, Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review9 which then passes onto the circuits for application of activation. As a result of this, the architecture consists of a single computing hardware which minimizes data replication and clock synchronization issues. These photonic TPUs are low power systems with high throughput and maximum efficiency (Schwartz et al. [111]). A high- speed TPU accelerator given by Fardoost et al. [34] combines wavelength-division multiplexing with mode- division multiplexing for multidimensional encoding of matrices of different sizes. This makes the scalability multiplica- tive and also solves the problem of orthogonal dimensions. An application of this methodology has been given by Tang et al. [119] that takes a 64 channel CT scan as input. The reported average extraction time taken by the convolutions and average pooling are (1.17푥10 −5 ) and (3.8푥10 −6 ) respectively. To make the architecture more energy efficient, Li et al. [77] cited a convolution compression method employing CP decomposition (Lebedev et al. [71]).The methodology employs kernel decomposition and fine-tuning to decompose layer-by-layer convolutions. Adam optimizer is used for fine-tuning to adjust the reductions in accuracy. However, these systems suffer from an issue of a memory bottleneck- Von Neumann bottleneck (Schwartz et al. [112]). This rises when the capacity of memory nodes are insufficient for the high computation power. It becomes a compounding problem for DRAM main memory when there are frequent updates from the output. This is due to transmission delay and issues in memory access. The authors also state prob- able solutions such as cache memory, in-memory computation and advanced packaging of the memory and the processor. 2.5 FPGA based Accelerator GPUs as hardware accelerators achieve high throughput and ASICs provide high energy efficiency with predictable latency for fixed workloads. As discussed in Table 1, these have certain demerits and from the statistics given in the Fig. 1, FPGA-based accelerators occupy a unique position by facilitating reconfigurable yet application-specific hardware execution. The next sections discuss the architecture of a FPGA with the structural details and optimizations made at different levels. Architecture of FPGA and Basic Structure of FPGA as Hardware Accelerators. - An FPGA is a semiconductor device which lets the user configure according to their application needs at a low cost. Earlier versions of FPGAs lacked heterogeneous structure in which it relies on an external CPU and off-chip DRAM. These FPGA units were restricted in both logic density as well as clock speed. With the advancement in technology, FPGAs have evolved into a System on chip (SoC) based structure in which the memory and the processing unit are integrated. In the current structure, SoC is primarily divided into two parts- Processing System (PS) which incorporates the CPU on chip and the second is the Programmable Logic (PL). The PL part has the reconfigurability architecture which hosts the core of the accelerator. Fig. 6 illustrates the design of the SoC-based accelerator, highlighting all of its components. The accelerator relies on the parallel processing capabilities and reconfigurability features. The SoC structure integrates the software programmability with hardware acceleration features. Inside the PS, there is an Application Processing Unit (APU) consisting of CPU cores, on chip memory, cache memory and memory management module. The APU houses CPU consisting typically of ARM cores or hard processors in Intel SoCs for handling of sequential operations and control of accelerators. Snoop Control Unit (SCU) maintains cache coherency across multiple CPU cores. The memory store the frequently stored data thus reducing the number of DRAM accesses. Two memory controllers are provided which manage the communication between PS and Double-Data Rate (DDR) memory. This is also equipped with various peripherals like Universal Asynchronous Receiver/Transmitter (UART), General Purpose Input/Output (GPIO) and so on. The PL forms the base level architecture of the accelerator. Configurable Logic Blocks (CLBs) provide Manuscript submitted to ACM 10Chatterjee et al. for the implementation of the parallelization of the computation heavy tasks. DSP slices provide for the fixed arithmetic operations of the network. Block Random Access Memory (BRAMs) provide for the buffering of the memory data and often used for storage of intermediate data. Both the units are connected with the help of a Advanced eXtensible Interface (AXI) interconnect which provides high bandwidth communication and also provides exchanges between the DDR memory. The basic architecture of the accelerator remains the same. Although different models require different routing networks, memory access, and functionality, the accelerator is designed accordingly. Fig. 6. Complete Architecture of SoC based Accelerator depicting the sub-modules- PS, PL & and shared DDR memory. Over the years, researchers have exploited the various features of FPGA to propose a wide range of FPGA based hardware accelerators. Growing complexity of the networks calls for workload management, network specific and hardware level optimizations. The need of real-time data processing has now become inevitable. For this, efficient usage of hardware resources with an acceptable degree of accuracy is required. Section 3 elaborates on the variety of deep learning accelerator models with their structural details and acceleration techniques. Apart from this, optimizations are made at numerous levels for acceleration of the tasks. A review of some of the general hardware level optimization techniques are discussed in Section 4. 3 Model-Specific Design Approaches for FPGA based AI Accelerators AI accelerators often adopt model-specific design strategies to efficiently map neural network computations. These approaches change according to the architecture, dataflow, and memory hierarchy to the characteristics of specific models—such as CNNs, SNNs, RNNs and GNNs. Fig. 7 provides the architectural details of each model along with its key advantages and disadvantages. 3.1 Convolutional Neural Network (CNN) One of the early designs of a FPGA based accelerator, Convolutional Network Processor (CNP) was given by Farabet et al. [33]. The main computation engine utilizes the parallel nature of the convolution layers to provide efficient hardware usage. A dedicated hardware unit is included to perform 8 concurrent data accesses by interconnecting with the off-chip Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review11 Fig. 7. Overview of CNN, SNN, RNN, and GNN with principles, key advantages & disadvantages. memory. The dataflow is maintained with the help of first in first out buffers between the computation unit and the off-chip memory. The computation unit also performs MAC operations concurrently synchronized with each clock cycle. Streaming vector PEs (Farabet et al. [32]), 2D clusters (Sankaradas et al. [109], Graf et al. [42]), and other modules are examples of different architectures. Xie et al. [135] uses a roofline model to fix the bandwidth and performance limits. To enforce the defined bandwidth limit, the architecture leverages on-chip memory for storage of feature maps and weights. The memory is configurable such that a partitioned Unified Random Access Memory (URAM) stores the feature maps and simultaneously rotates once computed whereas the BRAM stores the respective biases. Like its predecessors, this architecture also uses a 2D systolic array for performing parallel computations. Each systolic array has a PE which computes with the help of pipelined registers. These registers has flexible modes for computing convolutions, vector multiplication and distance calculation which further maximize the DSP efficiency. With a 90% utilization of hardware resources, the accelerators achieves a performance peak of 9.39 Trillions of Operations Per Second (TOPS). An older version of the roofline model was used by Zhang et al. [149]. For upholding the performance requirements, the accelerator used loop pipelining and loop tiling for maximizing parallel MAC operations. However, as an improvement the authors proposed the usage of a cross-layer unrolling factor so that it can fit a variety of CNN architectures. Zhang et al. [37] deliberately introduces memory stalls to address bandwidth limits. Fig. 8a shows the architecture which combines bit-serial LUTs with loop unrolling and tiling DSPs to process operands bit by bit with massive parallel processing capabilities. Also, a hybrid Design Space Exploration (DSE) is proposed for providing layer-wise quantization and scheduling to increase the above stated concurrency. Manuscript submitted to ACM 12Chatterjee et al. In recent years, some authors have cited optimizations at a RTL level addressing various clock issues directly related to the accelerator performance. Apart from the standard optimization techniques, Kim et al. [67] applies low-power techniques on the generated baseline RTL code. Clock-Gating (CG) is used for elimination of redundant clock cycles. To ensure this a multiplexer is designed with a Local Explicit Clock Enable (LECE) which is further connected with a series of flip-flops to only update the required outputs. The enable signal is further controlled by a Enhanced Clock Gating (ECG) based on the input/output signals. Venkataramanaiah et al. [69] proposes a custom RTL module shown in Fig. 8b which contains Verilog modules for CNN training phase. Based on the library an automatic RTL compiler generates general FPGA-based accelerator designs. Apart from having a 2D systolic array unit, a MAC load balancing unit is present for different kernel sizes. Unlike traditional buffers, this accelerator consists of transposable weight buffers made up of a circular matrix format which allows for both normal and transpose data accesses. These optimizations make a high throughput and highly flexible hardware accelerator. Using the same methodology [89] cites for the inference phase of the CNN accelerators with an emphasis on the scalability. The second difference is that it uses a parametric systolic array fitted with a layer-wise compilation method to adapt for different types of computations involved in the inference phase. The compiler also plays a role in deciding the quantization factor and resource allocation strategy for different CNN methodologies. Along with these optimizations, the accelerator uses the traditional memory level computational methods to achieve a peak throughput of 1046 GOPS. Varadarajulu et al. [123] presents an improvised RTL library- SentiNet RTL library consisting of RTL modules for convolution, pooling, activation and fully connected layer computations. All the optimization methods not only increase throughput but also make the power efficient to some extent. But the power efficiency comes as an added advantage. G et al. [37] proposes a series of methodologies that can offer a trade-off between power consumption and performance. One such is dynamic precision handling which calculates the bit width according to the needs of the operations. A conventional systolic array can be coupled with multi-mode operation to support various operations without needing specialized hardware units. As discussed clock and power not only reduces power switching but also reduces the usage of inactive components to eliminate leakage power. These methods result in near about 11% in power reduction. So, apart from pruning and quantization, Maksoud et al. [29] proposes an extreme model for compression and pruning by quantizing the architecture to 4-bit power-of-two values. This makes the model to fit entirely in the BRAM itself thus eliminating the need of power-hungry DRAMs. The model is designed in such a way that it does not use any DSP. The main operation of the DSPs-MAC is replaced by bit shift operations banking on the fact that the weights are a power of two. This reduces the dynamic power consumed by the accelerator. The custom memory architecture comprising of 256 BRAM banks and multi-level buffers provides data reuse and minimal data transfers which further increases the power efficiency. 3.2 Spiking Neural Network (SNN) [72] proposes FireFly-an SNN accelerator which enhances memory efficiency by utilizing optimized on-chip RAM. The 48-bit ALU is activated in SIMD mode which performs four parallel 12-bit additions. High computational density is achieved by eight cascaded DSPs with each handling a 2×4 synaptic crossbar computation. A specialized line-buffer is used for performing 3x3 Statistical Convolutional Neural Network (SCNN) convolutions which incorporates First In, First Out (FIFO) partial reuse and multi-stage stream buffers to maximize weight reuse and thus reduce latency. Memory efficiency is enhanced with the usage of a Finite State Machine (FSM) controlled partial-sum accumulation and voltage storage membrane. However, membrane voltage value are stored for long periods resulting in burdening of the memory. Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review13 (a) Architecture of FPGA Hybrid Accelerator with combined LUT & DSP units for streamlined computation. [37] (b) Framework showing unified dataflow control with buffers, & compute modules of Automatic Compiled Accelerator. [69] Fig. 8. Block Diagram for CNN Architectures Yin et al. [140] proposes an accelerator designed for implementation of both inference and training phases of backpropa- gation algorithm. The input sparsity is exploited through spikes thus avoiding weight passing and accumulation passes in absence of spikes. Backpropagation to inactive outputs during the training phase is eliminated by using cases where spikes are generated by threshold crossing neurons. These techniques allow only most useful computations to take place per systolic array clock cycle and also reduce unnecessary memory accesses leading to increase in throughput. Li et al. [76] employs a Bitmap-based sparse decoding logic (shown in Fig.9b) to locate non zero weights and input spikes with assignment of masks. This introduces a performance-limiting bottleneck. Neil et al. [95] focuses on the event driven nature of irregular sparse spikes. They propose a deep pipelined structure for simultaneous processing of the spikes as given by Fig. 9a. This is done with the help of a series of parallel PEs. The system also uses a multi-bank memory system comprised of partitioned synapse storage to reduce memory stalls during the process of simultaneous fetching of same weights. (a) Architecture of a scalable processor Minitaur with dedicated units for synaptic computation. [95] (b) Framework of FireFlyS with optimized modules for convolu- tion, pooling, & data management.[76] Fig. 9. Block Diagram for SNN Accelerator Architectures Manuscript submitted to ACM 14Chatterjee et al. Zhang et al. [152] aims to reduce data movement with the help of temporal parallelism. For n timestep operation spikes from all the steps are distributed over n PEs which reuse the same weights and perform parallel computations simultaneously. Parallelism is also provided within each PE by performing parallel channel computations. They also use a streaming dataflow mechanism to incorporate pipelining in passing the input feature maps and the weight data from buffers through linebuffers to spike generation pooling and residual unit. Each stage has concurrent operations in every clock cycle which prevents stalling and contributes to a increase in throughput values. This methodology still relies on dependent Membrane Potentials (MPs) which require extra logic to distribute intermediate MPs. Karakchi [62] introduces a novel approach to make the storage type user configurable. Each neuron of the network is encoded into one of the PEs of the 2D array network. The current membrane level of the neuron is encoded as a storage element (i.e., memory). Parameterized Verilog coding configures the memory type from the available URAM, BRAM and registers. The authors observed that the URAM based accelerator (SINK) is low-power as well as highly suitable for scaled up-operations. This observation is limited only to the ZCU104 UltraScale device. Although this methodology gives the user a choice but is limited only to static memory fixing. proposes a model in which the compilation process allocates the task of the processing models according to the required configurations. Like its predecessor models, parallelism is ensured within the module which performs parallel computations for every feature map. In addition to this, intra module parallelism is also provided by duplicating the kernels and hardware reuse for every layer. To increase computational efficiency and reduce memory burden, traditional loop hierarchy is modified by making the input channels as inner loops so that it results in partial sums which are not only very precise but also storage friendly. 3.3 Recurrent Neural Network (RNN) Architecture of three hardware accelerators- DeepStream, DeepStore and DeepRNN for RNN applications are presented by Chang et al. [11]. DeepStream continuously streams weights and vectors from off-chip memory to the MAC units thus providing maximum utilization. But this memory suffers from high memory bandwidth requirement which limits its scalability. This problem is alleviated to some extent by DeepStore which stores all the weights in on-chip memory. It then preloads them onto the DMA and replicates 128 PEs for achieving a massively high-level of parallel computation. Although, it uses lower memory bandwidth but still suffers from scalability issues. This is due to the dependence of the number of MAC units and internal memory on the height of the weights. DeepRNN approaches the problem in a hybrid manner by both storage and streaming of weights. The MAC units are replaced with a scalable grid of SIMD-style MACs. To further optimize the communication and data access, double buffered memories are used. These techniques eliminate both memory bandwidth and scalability issues. Pacini et al. [98] proposes FPG-AI RNN accelerator for faster implementation of LSTM and GRU models. The RNN model is first compressed using a quantization process followed by pruning and weight reduction which keeps recurrent weights on an on-chip BRAM. Gate sharing reduces the number of repeated computations by sharing the intermediate values across the platform. The heart of the accelerator is a layer wise computation engine which consists of a systolic array of PEs for computation of matrix vector operations. It then feeds the optimized network onto an automatic mapping network for mapping different configurations of a RNN network. Very - Long Short-Term Memory (V-LSTM), a LSTM accelerator cited by Kim et al. [66] uses an improved Fixed Nonzero-ratio Viterbi based Pruning methodology to efficiently optimize the memory. Fig. 10a shows the architecture segregated into front-end, PEs, and a back-end. The role of the front-end is to fetch the activations and compressed sparse weights (Viterbi indices + nonzero values) from DRAM. In order to reduce bandwidth the inputs/hidden states are stored in a on-chip SRAM. This is followed by sparse weights decompression and computation of MAC operations in the Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review15 PEs. The back-end handles element-wise operations like activations and additions. In addition to this, double-buffering scheme and sub-block computation are used to further reduce latency and SRAM usage. This pruning methodology uses a nonzero ratio per row or column of the weight matrix to create a regular sparsity pattern. Using this, the problem of low utilization of PEs is fixed by keeping the number of nonzero weights constant. The architecture is structurally divided into three parts- the front end, PE and the back end. It includes a double buffered scheme which streams weights continuously when the front end fetches it. This reduces memory stalls. The PE arrays are distributed in such a way that every part of the array structure provides for one type of parallelism. These arrays compute MAC operations in parallel with the reconstruction of non-zero weights which reduce latency. This methodology also uses ping pong buffers for efficient memory usage. The accelerator is further streamlined with its backend performing partial sum-reductions and reusing it in the next steps. The accelerator achieves an impressive latency of 46.30 휇s. Jang et al. [52] uses a sparsity pruning methodology- Shared Index Bank-Balanced Sparsity (SIBBS) co-processor for accelerating LSTM applications. SIBBS groups row memory banks into bank-clusters for sharing index masks and also balanced distribution across banks. A set of parallel Sparse-Matrix×Vector processing engine (SpMxV PEs) provide inter row parallelism. Each SpMxV PE is further split into bank-PEs which individually holds private vectors, fetches the weights from memory matrix and subsequently feeds an adder tree. The processes are pipelined in a manner that Similarity Check Unit (SMCU) parallely performs similarity checks with the current inputs. The Central Control Unit further increase the throughput by preloading compressed weights and vectors into the vector unit. These provide low latency single sample inference. In order to reduce memory access with optimum computation efficiency, Gao et al. [40] proposes implementation of an Independently Recurrent Neural Network (IndRNN) on an accelerator. As shown in Fig. 10b, the hardware architecture comprises of a Controller Unit (IndCtrlUnit), Multiplication Unit (MultUnit), Element-wise Product Unit (EleUnit) and Activation Unit (ActUnit). IndCtrlUnit manages various control signals and regulates the data flow between modules. MAC operations are performed by the multiplier using data from the previous FIFO, on-chip weight BRAM, and the product unit. Using the MultUnit output, EleUnit computes the Hadamard product required in IndRNN operations. Lastly, the ActUnit applies the Rectified Linear Unit (ReLU) activation function to the computed results for the current timestep, for computing the final activated output. The computation steps are significantly reduced by replacement of hidden layer matrix multiplication with vector multiplication. In addition to this, weights and repeating vectors are stored on a dual mode BRAM to reduce the memory accesses. The array of PEs consist of 128 MAC units. In addition to this, EleUnit and ActUnit operate in a pipelined manner thus significantly increasing the throughput. These techniques make the accelerator energy efficient as well as faster than its predecessor architectures. Guo et al. [44] exploits fine-grained parallelism in an LSTM cell and coarse-grained parallelism during training. During forward propagation, dense vector-matrix multiplications are merged together alongwith loop initiation intervals during HLS that promotes parallel execution and decreases latency. To minimize the data movement inside the LSTM cell, local buffering and array partitioning is provided. The errors in backward propagation are computed simultaneously with parallel generation of weight gradients. A coarse grained pipeline architecture for training is provided for complementing the fine grained parallelism. 3.4 Graph Neural Network (GNN) Zhou et al. [158] presents BlockGNN which enhances computational speed by performing compression of the GNN architecture. Block-Circulant -a compression technique where convolutions in frequency domain using Fast Fourier Transform (FFT) and Inverse Fourier Transform (IFFT) on partitioned blocks. The heart of the Block-Circulant- CirCore Manuscript submitted to ACM 16Chatterjee et al. (a) Architecture of V-LSTM Accelerator demonstrating the mem- ory hierarchy which enables efficient LSTM computation. [66] (b) Depiction of architecture of IndRNN accelerator with parallel computational units and on-chip BRAM. [40] Fig. 10. Block Diagram for RNN Accelerator Architectures consists of a vector processing unit which acts as a specialized datapath for performing parallel FFT transformations and complex multiplications. The channels are arranged parallely in a systolic array structure which compute the calculations and transform them back to the spatial domain. The global buffer stores the intermediate results, input vectors and compressed weights thus lowering data accesses. Sarkar et al. [110] applies multi-queue dataflow (given by Fig. 11a) to solve the problem faced during parallelizing message passing across several edges. Each message passing unit is provided with its dedicated partitioned memory bank where node embeddings are sent as soon as they are produced. This alleviates memory stalls and thus increases the efficiency. Following which, two types of edge based scattering operations take place. One is node transformation followed by scattering and the second is gathering then transformation. The optimized dataflow structure promotes parallelism for a generalized GNN structure. Geng et al. [41] proposes hardware auto tuning methodologies- dynamic distribution smoothing, remote switching, and row remapping to solve the irregularity and imbalance in graph-node structure. Dynamic distribution smoothing methods divides the higher degree nodes into smaller ones while merging the smaller nodes into one. This balances the workload and promotes parallelism of the tasks. These techniques not only improves the throughput and hardware utilization but also accelerate the Graph Convolutional Network (GCN) inference phase. Liang et al. [78] presents an automatic GNN acceleration network which determines the performance bottleneck and accordingly selects the optimum computation, memory and graph manipulation template from the predefined hardware templates. The computation template comprising of an array of systolic elements accelerate both dense and sparse operations which are further optimized with the help of different memory modes. The memory template has partitioned on-chip memory and DRAM specifically allocated for storage of intermediate results and node features. The graph manipulation technique reorders edges and remaps rows/columns to regularize sparsity patterns. Dynasparse, a GNN accelerator proposed by Zhang et al. [146] uses a compiler where each kernel provides partitioning and generate specific execution scheme. Fig. 11b shows the three dynamic switched proposed schemes which are General Matrix- Matrix Multiplication (GEMM), Sparse-Dense Matrix Multiplication (SpDMM), and Sparse Matrix-Matrix Multiplication (SPMM). When both the matrices are dense, there are no optimizations and the systolic arrays perform the convolution as usual. The second one is used for partially sparse activations. In this a sparse–dense intermediate is multiplied with a dense matrix which allows dense processing for the other operand while maintaining sparse activations. SPMM is chosen when one is dense and the other one is sparse. For sparse where there are zero computations, no hardware resources are allocated thus reducing memory bandwidth. FP-GNN is proposed by Tian et al. [122] which aims to eliminate sparsity problems and solve memory bottleneck issues. It combines node and feature level parallelism to accelerate the Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review17 (a) FlowGNN architecture showing parallel node transformations and message passing techniques [110]. (b) Structure of Dynasparse with modular computation and adaptive dataflow modules [146]. Fig. 11. Block diagram for GNN accelerator architectures. workloads. Parallel processing of multiple nodes take place across PEs to balance irregular graph structures which are further subjected to dynamic load balancing. A pipelined dataflow with memory banking not only promotes efficient data movement but also provides conflict free memory access. 4 Hardware-Level Optimization Strategies This section addresses recent hardware level optimizations for FPGA-based accelerators. AI applications are also computationally intensive, needing improvements for both efficient execution and speeding up the computational process. Furthermore, resource-constrained hardware makes it even more critical. As a result, adaptations must be performed at multiple levels to speed up computation, ensure efficient execution, and optimize hardware consumption. Fig. 12 summarizes the discussed hardware level optimization techniques for FPGA based accelerators. These opti- mizations improve the performance and efficiency of accelerators by maintaining a balance between computation and memory usage. At the computation level, precision and arithmetic optimizations such as fixed-point representation, custom arithmetic units and reduced bit-width reduce hardware cost and latency. In addition to this there are loop transformation methodologies like pipelining and unrolling which improve the execution throughput. Computation reuse techniques minimize redundant operations by reusing weights and intermediate results. The last type of compu- tation level technique is employment of different forms of parallelism at various levels. These include instruction-level, data-level, task-level, and feature-level parallelism which enable simultaneous processing to boost speed. At the multi- level, tiling and blocking divide large computations into smaller segments to improve data locality and make on-chip memory deployment easier. At the memory level, banking and partitioning allow concurrent memory accesses. The other subtypes include double buffering which overlaps data transfer with computation, memory hierarchy which Manuscript submitted to ACM 18Chatterjee et al. organizes storage efficiently across levels, prefetching and scheduling to reduce stalls, and dataflow with stream buffers for enabling continuous data movement. These collectively reduce latency with maximization of throughput values. Subsections 4.1, 4.2 and 4.3 explore the implementation of these optimization techniques in some of the state-of-the accelerators. Fig. 12. Summary of the different types of Hardware-Level Optimizations- computation, memory and multi-level optimizations. 4.1 Computation Level Optimizations Deep learning techniques have high arithmetic density which spans to billion of operations. Optimizations helps these operations to meet the throughput and latency requirements. These also aid efficient usage of FPGA resources and eliminate stalled or ideal operations. These optimizations also enhance the performance of the FPGA accelerators. CAESAR, a CNN given by Kim et al. [65] removes redundant computations and thus significantly reduces the computa- tional overhead. Overlapping convolution windows and weight repetition result in many redundant multiplications cause many redundant computations, making the convolution operation bulky. To address this issue, weight reordering is applied in which the weights are restructured in such a way that the identical weights can be easily identified. The accelerator eliminates the redundant computations by scattering over several accumulators and keeping the record in a mapping table and sparse activation group. Liu et al. [83] proposes a series of computation optimization strategies such as loop unrolling and pipelining, data reuse, precision scaling and quantization. Tiling first decomposes a large computational tasks into smaller blocks. It is then followed by blocking which uses loop transformations to be applied on the entire block. This facilitates data reuse and reduces off-chip memory operations. Precision scaling and quantization further reduces the computational load by replacing floating point operations to fixed-point and reduced-bit operations. Alongwith tiling, Rai et al. [106] proposes a dynamic reconfigurable framework- FPUGen which has customized mantissa and exponent widths which provides for a variety of representations such as- Single Precision-32 (SP-32), Tensor-float-32 (TF-32), Half Precision-16 (HP-16), and Brain-FP format-16 (BF-16). It is a five-stage pipelined architecture with a shared adder/subtractor and bit-partitioned multiplier units to support this multi-point representation.The dynamic precision configuration comes at a cost of low resource efficiency and increased design complexity. [46] proposes a six-stage pipeline architecture for execution of simultaneous operations. It also prevents memory stalls thus reducing idle operations. Additionally these computations are spread across multiple PEs for exploiting data level parallelism. Pipelining comes with certain disadvantages as well. Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review19 These include data control and structural hazards. The training process is quite resource and memory intensive, so researchers have often opted for offline training solutions. Yu et al. [142] proposes a 3D CNN accelerator which provides a sparsity aware methodology to solve memory bandwidth issues encountered during training phase. It leverages sparsity to opt out from unnecessary MAC operations. A monolothic 3D design employs parallelism by tightly binding computation units with the memory layers thus reducing data movement which enhances the efficiency. Like the predecessor architecture this also employs quantization to further enhance the computational capabilities. Zhang et al. [154] uses an improved 3D pulse array structure maximizes parallel Multiply-accumulate (MAC) operations across spatial, input and output channel dimensions. The outputs are computed in such a way that the partial results from different PEs are are computed parallely. The input channels are distributed in such a way so that the multiple channels are processed simultaneously. These techniques not only reduce the pipeline latency but also improve throughput and speed up the inference time. Model wise optimizations also lead to improvement in computational efficiency. One such example is given by Kang et al. [61] where the fully connected layers of LeNet-5 (Han et al. [48]) are removed thus lowering the computational demand. The architecture also have convolution windows with shift registers and parallel multiply add module for simultaneous computations. 4.2 Memory Level Optimizations In FPGA based accelerators, the type of memory and its corresponding optimizations are fixed according to the requirements of the implemented model. Memory is classified as two types- on-chip and off-chip memory. Both memory types support memory requests for data transfer and storage of intermediate values. Improper handling of memory access can result in bottleneck and have an effect on the overall system performance. Additionally, because FPGAs are memory constrained devices, optimizations are performed to maximize memory bandwidth and streamline memory access requests. Khan et al. [64] studies the effects of four types of memory accesses- Multiple Input Single Output (MISO), Single Input Multiple Output (SIMO), Multiple Input Multiple Output (MIMO) and All Input Multiple Output (AIMO). The modes are configured based on the number of inputs and generated outputs. MISO uses multiple inputs and filters to generate a single output tile, whereas MIMO generates several output features. In the second case, a single input is convolved with numerous filters to produce multiple outputs. The final one involves the creation of partial outcomes while computing all outputs in a single phase. But first, tiling is done to accommodate large sized feature maps and weights into on-chip buffers. This is followed by reuse and double buffering to reduce memory access requests. It was observed that out of all the memory access techniques, AIMO offered best tradeoff between memory efficiency and performance for acceleration of CNN operations. Cheng et al. [22] presents a framework for providing memory level parallelism configured during High Level Synthesis (HLS). A profile guided tool first analyzes the runtime memory traces and accordingly partitions the memory accesses into independent groups. During HLS, according to the configurations, loop pipelining, instruction rescheduling, and operation reordering is applied. A vulnerability window measures the maximum extent of reordering between dependent memory accesses. Violation is raised if the partitioned memory’s access falls outside this window. Following which the execution from a correct start-point. This makes it adaptable to sustain higher bandwidth and achieve upto 52% performance improvement. Diamantopoulos et al. [158] proposes implementation of a Long Short-Term Memory (LSTM) accelerator by reshaping on-chip memory during HLS. Like the previous one, partitioning and reshaping of the BRAMs are provided with the help of directives like#pragma HLS PARTITIONand#pragma HLS RESHAPE. This is done Manuscript submitted to ACM 20Chatterjee et al. in such a way that uniform bit-widths are stored in BRAM and higher precision bit-widths are stored in Look-Up Table Random Access Memory (LUTRAM). This memory hierarchy and storing the data in a partitioned manner not only increases bandwidth but also provide for maximum parallelism. Kokkinis et al. [68] also uses dynamic configurations for memory optimization. Fig. 13 provides the details about the operation of the proposed framework. It operates in two phases: offline analysis (top part) and online execution/defragmentation (bottom part). In the offline phase, the DAS extractor profiles accelerator memory allocations and generates allocation patterns. This is followed by evaluation through Monte-Carlo simulation to identify Pareto-optimal configurations that minimize memory allocation failures (MAF). In the online phase, a runtime controller monitors the fragmentation ratio for each heap. When it exceeds the predefined threshold (Q), the garbage collection mechanism is triggered. The lower-left block shows the Mark-Compact process where live objects are marked, fragmented regions are identified, and memory blocks are compacted toward lower heap addresses. During compaction, the offset table updates the relocated addresses, while a 1-bit compaction flag stalls to prevent invalid memory access. After completion of compaction, addresses are updated and accelerator execution resumes which ensures efficient dynamic memory utilization. Fig. 13. Design flow for a HLS-based dynamic memory optimized framework with offline pattern analysis & Monte-Carlo-guided heap optimization. [68] Near memory computing (NMC) is the process of moving computation blocks closer to the on-chip memory in order to reduce high cost memory transfers. Tao et al. [121], the NMC core is presented which consists of a shift-and-add multiplier block, adder tree and a quantization block. The weights are loaded in the BRAMs and inputs are passed through the shift registers thus reducing the memory access distance. The pooling layers are also accelerated with the help of the comparators near the memory. This significantly increases the memory usage. Dhar et al. [24] also includes in-storage computing which uses partial calculations to further optimize the memory bandwidth. 4.3 Multi-level Optimizations Single-level optimizations often fail to address all issues. It may also cause problems that must be addressed by optimizations at other levels. Thus, multi-level optimization improves performance on all fronts. FixyFPGA cited by Meng et al. [91] optimizes resource utilization as well as computational overhead. For optimization of resource utilization, fusion of batch normalization and weights is done. Also the structure is fully pipelined and is equipped with buffers and shift registers. The weights are arranged in a manner such that the important weights are identified and accordingly pruning can be done to reduce the computational load. A 4 bit quantization method is implemented as a part of this approach for reduction of computational load as well as memory usage. Baranwal et Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review21 al. [7] proposes ReLAccS- a multi-level optimization strategy to accelerate the reinforcement learning on FPGA. At computational level loop pipelining and unrolling is done to implement parallelism of multiple operations. Also for precision optimization, fixed point is used. On the other hand, memory is partitioned for simultaneous access to multiple banks. The memory computations and accesses are further accelerated with the help of buffering and prefetching. Also in order to reduce idle units dynamic allocation of hardware resources is provided. Wang et al. [125] proposes optimizations for robotic intelligence applications. They propose a hardware estimation model with an empirical calibration with nonlinear fitting (푅 2 ≈0.98) to find out the number of BRAM and Lookup Tables (LUTs). AutoML further guided for selection of hardware based configurations to enhance the performance. Further optimizations are made by designing a pipelined and clock cycle balanced architecture. Unlike its predecessor, Zhong et al. [157] proposes design space exploration is done to find the balance between memory banks and other hardware resources. On-chip memory tiling, buffering and partitioning with weight mapping reduces external memory accesses and addresses bandwidth issues. Like all the accelerator configurations, loop unrolling, pipelining and quantization is done to enhance the arithmetic efficiency and reduce the computational load to an extent. Sun et al. [118] proposes a hybrid CNN-RNN accelerator which comprises of a streaming architecture and mapping strategy. The design of a unified convolution engine is presented which maps matrix-vector onto same hardware resources thus promoting reuse and maximization of resource utilization. Furthermore, scheduling strategies are provided: layer-by-layer, subgraph-by-subgraph and subnetwork-by-subnetwork. The first one is used for maximum flexibility operations whereas the second one decreases intermediate memory accesses. The accelerator achieves an impressive throughput of 646 Giga Operations Per Second (GOPS). Fig. 14 summarizes the major numerical hardware optimization methodologies across CNN, SNN, RNN, and GNN accelerators. It also highlights the imapact on acceleration. In CNNs, convolution퐹 out = Í 퐹 in ·푊 +푏is parallelized via kernel partitioning and input broadcasting. Another important methodology is the Winograd transform푌= 퐴 푇 [(퐺푔퐺 푇 ) ⊙ (퐵 푇 푑퐵)]퐴which reduces multiplications (e.g., a 3×3 convolution from 9 to 4 multipliers) There are also tiling factors which satisfying푇 푘 푇 푚 푇 푛 ≤ 퐷푆푃 avail and thus maximizes DSP utilization. In SNNs, neuron dynamics푉(푡)=푉(푡 −1)+푤 푖푗 푠 푗 (푡)−휃are optimized by reducing spike events with maintaining a balanced temporal workloads. RNNs apply structured pruning푊 pruned = 푊 ⊙ 푀, sparsity scheduling, similarity-based skipping, and quantization푄(푋)= 2 푘 −1 max푋−min푋 푋to reduce computation and memory precision. For GNNs, layer propagation 퐻 (푙+1) = 휎(퐷 −1/2 퐴퐷 −1/2 퐻푊)involves sparse-dense multiplications with complexityO(|퐸|퐹). In addition to this, there are block-circulant decomposition which reduce weight multiplication toO(푁 log푁). Symmetric adjacency 퐴 sym = 1 2 (퐴+ 퐴 푇 )avoids redundant edge operations, and topology-aware normalization improves stability while exploiting sparsity, collectively reducing MAC operations, memory bandwidth, and hardware resource usage. 5 Performance Analysis of the Accelerators For FPGA based accelerators, performance is determined in terms of throughput, precision achieved, efficiency and resource utilization utilization. Additionally, methodology and operating frequency gives a better perspective of the performance. Tables 2, 3, 4 and 5 provide a comparative analysis of the models used and its implementation details. 6 Challenges encountered by FPGA based Accelerators Over the previous sections, a thorough examination of the state-of-the-art FPGA accelerators for different deep learning models has been presented. It is observed that different types of neural networks do not encounter the same problems and thus has to be tackled according to the network requirements. After an intensive study of all the acceleration techniques and optimizations, FPGA accelerators still face certain challenges. For better understanding, the problems can be categorized as follows: Manuscript submitted to ACM 22Chatterjee et al. Table 2. Performance Summary of FPGA-based CNN Accelerators ModelMethodologyPlatformThroughputPrecision FrequencyResources (Utilization %)SpeedupBaseline CNP (2009) [33] LeNex-5Virtex-4 SX35 5.25 GOPSfixed 16 bit 200 MHz90.00% 28.00%–90.00%– MAPLE (2010) [9] 4 CONV layersVirtex-5 SX240T 7 GOPS–125 MHz– 92.00%–0.5C870 GPU DLA (2017) [6] AlexNetArria 10 GX 1150 1,382 GOPSfixed 16 bit 214 MHz258.00%97.00% 92.00% 40.00%– OpenCL CNN Accelerator (2017) [150] VGG-16Arria 10 GX 1150 1790 GOPSfloating 16 bit 385 MHz– 87.00% 53.40%– OpenCL CNN Accelerator (2017) [150] VGG-16Arria 10 GX 1150 866 GOPSfloating 32 bit 370 MHz– 90.80% 46.10%– Automatic RTL Generator (2017) [87] NiNStratix- V GXA7 282.67 GOPSfixed 16 bit 150 MHz 96.00% 100.00%59.00%– Automatic RTL Generator (2017) [87] VGG-16Stratix- V GXA7 352.24 GOPSfixed 16 bit 150 MHz 90.00% 100.00%86.00%– Automatic RTL Generator (2017) [87] ResNet-50Arria 10 GX 1150 587.63 GOPSfixed 16 bit 200 MHz 37.00% 100.00%56.00%– Angel-Eye (2018) [84] VGG-16Zynq XC7Z045 84.3 GOPSfixed 16 bit 150 MHz184.00%87.000%89.00% 29.00%– Optimized CONV Accelerator (2018) [88] VGG-16Stratix- V GXA7 278.2 GOPSfixed 16 bit 150 MHz 97.00% 100.00%59%– Winograd GEMM Accelerator (2019) [60] AlexNet and VGG-16 Virtex-7 VX690T 433.63 GOPSfloating 16 bit 200 MHz58.00% 39.80%92%99.0%– Winograd Accelerator (2020) [86] YOLOv2Arria 10 GX 1150 278.2 GOPS–240 MHz 35.00% 100.00%–5.8Xeon CPU Gold 5115 Object Recognition Accelerator (2021) [73] VGG-16Xilinx VC709 230.1 GOPs–200 MHz 28.00% 18.40% 31.00% 18.50%– Scalable accelerator (2023) [138] MNISTXilinx XC7z020 63.67 GOP/s–100 MHz 56.00% 17.00% 11.00% 10.00%– 1D-CNN accelerator (2024) [13] 1D-CNNXilinx XCZU2CG 63.67 (GOP/s)fixed 16 bit 200 MHz–75.00% 77.33%–21.13Intel Core i5-8300 Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review23 Table 3. Performance Summary of FPGA-based SNN Accelerators ModelMethodologyPlatformThroughputPrecision FrequencyResources (Utilization %)SpeedupBaseline LUT DSP BRAMFF SyncNN (2021) [99] SCNNXilinx ZCU102 N/Afixed 4 bit 200 MHz37.45%N/AN/AN/AN/AN/A BP-STDP SNN Accelerator (2022) [151] SFNNXilinx XCVU9P 5980floating 16 bit 100 MHz15.64% 0.00%N/A12.77%49.42CPU with 3.2GHz CPU and 16GB memory Cerebron (2022) [15] LIFXilinx XC7Z100 43.07 GSOP/sN/A200 MHz30.97% 0.00% 37.48% 12.71%N/AN/A Spiker (2022)[10] SFNNArtix 7N/Afixed 16 bit 100 MHz 55.00%N/A32.00% 25.00%N/AN/A Skydiver (2022)[14] SCNNXilinx XA7Z020 22,600 FPSN/A200 MHz21.04% 0.00% 48.07% 4.70%N/AN/A CONVP (2023)[82] N/AXilinx XA7Z020 5980fixed 8 bit 100 MHz30.47%N/AN/A10.65%58 FireFly (2023) [72] SCNN(5,7,9,11)Xilinx XCZU3EG 1,382.4 GOPS/sinteger 8 bit 300 MHz 21.43%80%75%N/AN/AN/A FrameFire (2023)[16] N/AXilinx XC7Z035 23.2 GOPS/sinteger 8 bit 200 MHz24.39% 0.00% 25.60%N/AN/AN/A DeepFire2 (2023) [5] N/AXilinx VCU118 17550450 MHz 12.03% 68.73% 20.45%N/AN/AN/A Diff. Time Encoding Accelerator (2023) [131] N/AXilinx ZCU102 N/Afixed 6 bit 365 MHz 10.70% 0.00% 60.67%N/AN/AN/A ELIF-NHAS (2023) [139] SFNN and SCNN Xilinx XC7K355T 3.2 GSOP/sfixed 6 bit 200 MHz19.24% 0.00% 11.27%N/AN/AN/A STBP SNN Accelerator (2024 )[140] STBPXilinx VC707 N/A13 bit200 MHz28.15% 30.18%N/A45.06%N/AN/A Sparse Weight SNN Acclerator (2024 )[129] N/AXilinx XCKU115 5.1028 bit120 MHz13.90%N/A1.62% 1.86%N/AN/A Hardware Software Optimised Accelerator (2024) [96] 38.4Xilinx XC7Z020 5.102fixed 8 bit 100 MHz22.43% 7.67% 67.86% 7.67%N/AN/A Synapse Accelerator (2024) [74] IF neuron model Xilinx ZCU102 47.3 GSOPS/sfixed 11 bit 30 MHz 29.06%N/AN/A17.33%N/AN/A SINK (2024)[62] N/AXilinx ZCU104 1575 GSOPS/sN/A350 MHz29.06%N/AN/A2.07%N/AN/A FireFly S (2025)[76] SCNN(5,7,9)Xilinx XCZU5EV 5,352 FPS4 bit333 MHz48.33%N/A21.33%42%N/AN/A Manuscript submitted to ACM 24Chatterjee et al. Table 4. Performance Summary of FPGA-based RNN Accelerators ModelMethodologyPlatformThroughputPrecision FrequencyResources (Utilization %)Speedup Baseline LUT DSP BRAMFF ESE[47] (2017) Std. LSTMXilinx XCKU060 282 GOPSinteger 16 bit 200 MHz88.60% 54.50% 87.70% 68.30%43.00Corei 75930k CPU DeepRNN (2017) [11] 2 LSTM LayersXilinx ZC706 400.98 Mops/s/W integer 16 bit 142 MHz–45.67Exynos 54224 Cortex- A15 LSTM-RNN (2017) [43] Std. LSTMXilinx VC707 7.26 GFLOP/sfloating 32 bit 150 MHz 65.31% 42.00% 52.04% 30.08%20.18Intel Xeon CPU E5-2430 FINN-L (2018) [108] 2 layer Std. LSTM Xilinx ZCU104 746@8bit8 bit266 MHz 26.08%–28.84%– IndRNN Accelerator (2018) [40] 2 layers IndRNN Xilinx ZC706 3.15 GOPS/s/Wfixed 16 bit 125 MHz 2.54% 3.44% 10.27% 2.89%Intel CORE i5 CPU – Transprecision Accelerator (2018) [25] Std. LSTMXilinx XCKU060 515 GOPSfloating 32 bit 240 MHz– DeltaRNN (2018) [38] RNNXilinx XC7Z100 192 GOP/sinteger 16 bits 125 MHz94.22% 38.02% 60.60% 21.50%63.8Intel i7-8700K CPU Open-CL based Accelerator (2019) [118] Std. LSTMIntel Arria10 GX1150 515 GOPS16 bit242 MHz– EdgeDRNN (2020) [39] 2 layer RNNXilinx XC7Z007S 20.2 GOP/sinteger 16 bits 125 MHz30.80% 13.6% 32.00% 34.10%– IMDB LSTM Accelerator (2020) [156] 1 layer LSTMXilinx XC7VX485 10.9 GOPSinteger 16 bits 200 MHz7.73% 5.07% 16.07% 3.52%43.13Intel Core i5-6500 Bidirectional accelerator (2022) [127] 1024 sized LSTM Virtex- 7VC707 91.47 GOPS/s–200 MHz21.8% 18.29% 23.98% 6.95%13.4Intel(R) Core i7- 10700F SIBBS Accelerator (2022) [52] Std. LSTMXilinx XCKU115 712.6 GOPSfixed 8 bit 200 MHz81.26% 75.36% 28.33% 43.82%– 푆 2 RNN (2025) [63] 8 layer LSTMAltera 10 GX –9%7%–1.79– FSS Accelerator (2025) [147] 8 layer LSTMXilinx Alveo U280 600 TFLOPSint 8 bit225 MHz 54.00% 81.40% 81.90% 35.00%2.80NVIDIA RTX A6000 FSS Accelerator (2025) [147] 8 layer LSTMXilinx Alveo U250 600 TFLOPSint 8 bit225 MHz 46.00% 80.00% 84.00% 33.00%2.42NVIDIA RTX A6000 LSTM Accelerator (2025) [1] 8 layer LSTMXilinx KCU105 32.5 GOPSfixed 20 bit 166.66 MHz 6.14% 4.90% 21.33% 4.02%– Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review25 Table 5. Performance Summary of FPGA-based GNN Accelerators ModelMethodologyPlatformThroughputPrecision FrequencyResources (Utilization %)Speedup Baseline LUT DSP BRAMFF AWB-GCN (2020) [41] GCNStratix 10SX 873.1floating 32 bit 330 MHz25%71.11%N/A13.7%3255Intel Xeon E5-2680 DeepBurning- GL (2020) [78] GCN, GS-Pool, R-GCN, EdgeConv Xilinx ZC706 8.6 GOPS/Wfixed 16 bit 100 MHz 52.72% 85.57% 76.21% 18.36%7.43Intel Xeon E5-2680 DeepBurning- GL [78] (2020) GCN, GS-Pool, R-GCN, EdgeConv Xilinx Alveo U50 53.5 GOPS/Wfixed 16 bit 200 MHzN/A93.57% 45.14% 32.93%346.98Intel Xeon (Sky- lake) 6151 BoostGCN (2021) [145] GCNStratix 10SX 512 or 1280floating 32 bit 250 MHz 10.5% 33.33%N/AN/A45Intel Xeon Gold 5120 CPU FP-GNN (2022) [122] GCN, Graphsage, GAT Xilinx VCU128 922floating 32 bit 225 MHz25.16% 90.78% 88.88% 19.83%665.33Intel Xeon Plat- inum 8260 GCoD (2022) [141] GCN, Graphsage, GAT, GIN Xilinx VCU128 N/Afixed 32 bit 330 MHzN/A 57.62%N/AN/A15286Intel Xeon E5-2680 I-GCN (2022) [50] GCN, Graphsage, GAT, GIN Stratix 10SX 2700floating 32 bit 330 MHzN/A 45.39%N/AN/A5549Intel Xeon E5-2680 FlowGNN (2022) [110] GCN, GIN, GAT, PNA, DGN, VN Xilinx VCU128 N/Aint 8,16,32 bit 225 MHz 7.57% 17.69% 13.76% 9.78%3645Intel Xeon Gold 6226R QEGCN (2022) [144] GCNXilinx Alveo U50 N/AN/A300 MHz 8.05% 31.21% 2.18% 0.48%1009Intel Xeon Gold 5120 CPU Dynasparse (2023) [146] GCN, LightGCN, Graphsage, GIN, GAT Xilinx Alveo U250 512floating 32 bit 225 MHz 58.51% 58.49% 42.60%N/A306Ryzen 3990x Graph-OPU (2023) [17] GCN, Graphsage, GAT, GIN Xilinx Alveo U50 459.6fixed 32 bit 225 MHz 54.48% 46.33% 68.97% 16.38%1654Intel I7- 12700KF FTW-GAT (2024) [49] GCN, Graphsage, GAT Xilinx Alveo U250 1080floating 16 bit 600 MHz65.4% 59.3% 57.3%N/A500.13AMD Ryzen 3990x Manuscript submitted to ACM 26Chatterjee et al. Fig. 14. Summary of optimization methodologies and roles of acceleration of various neural models (i) Quantization Issues- Fixed bit representation and custom bit not only reduces utilization of hardware resources but also provides for efficient usage. However, using this type of representation for sensitive layers may result in saturation or underflow errors. Quantized models further need additional finetuning for recovery of accuracy. Also, low bit arithmetic may require specialized hardware units. Custom bit representation alleviates some of the above problems but adds lookup overhead which further leads to burdening of the hardware resources. (i) Trade-off between Power Consumption and Efficiency- Some computation techniques like loop unrolling not only increase computational throughput but also lead to a significant rise in power consumption. This happens because when loops are unrolled, multiple PEs perform simultaneous iterations. Also when pipelining is used there is simultaneous switching and routing complexity. This increases dynamic power. While these techniques often promote efficiency but suffer from increased power consumption. This issue has also been raised by Liu et al. [81]. (i)Lack of Efficient Co-Design with CPUs - There are a large variety of optimization techniques available for acceleration of the inference and training phases. However, there is absence of sufficient co-design techniques of FPGA with CPU. These face issues such as stalls in communication and scheduling mismatch of operations. When integrated as a SoC, issues are on a lower scale as the available datapaths can be optimized. But these issues remain unresolved during integration of heterogeneous devices. (iv) Communication Inefficiency and Memory Bottlenecks- As Wu et al. [133] states that there is still lack in efficient data access techniques between memory and other configurable resources. Though these problems are mitigated to a certain extent with the help of memory partitioning and NMC, still some bottlenecks exist. These techniques often suffer from memory stalls, computational, area and power overheads which limits the acceleration capability. Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review27 (v)Scalability Issues and Lack of an Ecosystem - There is an absence of standardized libraries as that of CUDA model. Although high level synthesis tools are present but they are often vendor specific thus leading researchers to search for generalized or customized techniques. On the other hand, in spite of proposed optimizations there is always a burden on resource usage. These issues are addressed by Fat et al. [35]. (vi)Security Issues - Although FPGA reconfigurability makes it attractive for hardware acceleration, it also exposes the platform to security threats such as adversarial attacks [93,102–104], configuration tampering, and bit-flip injections [105,107]. Since the configuration bitstream contains detailed structural and functional information about the implemented accelerator, it becomes a critical attack surface. Such attacks can significantly degrade accuracy, induce misclassification, or alter the intended functionality. The functionality and the analysis of the FPGA configuration bitstream consists of plethora of information about the acceleration techniques. The attacks can be launched in such a way that it causes significant reduction in accuracy or leads to misclassification. 7 Future Directions for Optimizations From early FPGA models to the latest System-on-Chip (SoC) platforms, several advanced features and opti- mizations have been incorporated to enable high performance and efficient computational capability. These developments position FPGAs as strong contenders for optimal hardware acceleration. Section 6 presents an overview of the key challenges faced by FPGA-based accelerators despite extensive optimizations. Addressing these challenges requires accelerator designs that are adaptable to evolving computational demands. In this context, several potential methodologies and future research directions for optimization are discussed in this section. Architectural designs for both generalized and customized FPGA-based accelerators have been explored exten- sively. General-purpose FPGA accelerators support a wide range of workload templates but often lack fine-grained optimizations, resulting in lower throughput compared to customized solutions. In contrast, custom FPGA-based hardware accelerators enable fine-grained optimizations that yield low latency and high peak performance, although they limit architectural reusability. Achieving an effective balance between generality and customization therefore remains an open challenge. Moreover, compression techniques such as pruning frequently trade model accuracy for improved hardware efficiency. To address this, a risk-analysis framework can be developed to evaluate and balance accuracy and efficiency according to the specific requirements of the neural network and its application. Recent advances in automatic compiler generation, which analyze network characteristics and derive suitable optimization strategies, can inspire the design of such risk-analysis schemes. As discussed previously, the lack of comprehensive hardware–software co-design strategies often results in mismatches between algorithmic computation and hardware mapping. While modern in-house processing systems employ optimized dataflow mechanisms, similar architectural principles must be incorporated into FPGA accelerator design. Additionally, improved memory-access mechanisms and optimized caching strategies can further enhance throughput while reducing latency. Analog and digital in-memory computing have recently emerged as promising approaches for neural-network deployment in analog and ASIC implementations. Analog in-memory computing (AIMC) can be emulated in FPGA-based systems, as demonstrated by Petropoulos et al. [101], who introduced a Noise Injection Unit (NIU) within conventional processing elements to inject device-level noise into fetched weights instead of performing Manuscript submitted to ACM 28Chatterjee et al. multiply–accumulate operations. The noisy-weight implementation occurs during inference, thereby eliminating substantial computational overhead encountered in many FPGA-based accelerators. Similarly, digital in-memory computing (DIMC) solutions for SoC-based architectures have been investigated in [100], presenting a memory architecture that integrates DMA-enabled system memory, an interconnection fabric, and an embedded accelerator unit. Since hardware-aware models have been proposed to mitigate the limitations of AIMC and DIMC, analogous architectural strategies can be designed to address these challenges while emulating AIMC within FPGA platforms. Future research should focus on APC-based [102,104] runtime monitoring to detect anomalies caused by adversarial manipulation, configuration tampering, or malicious bit-flips in FPGA accelerators. Learning-based profiling of trusted configurations combined with lightweight on-chip sensors can enable real-time anomaly detection with minimal overhead. Overall, future methodologies should aim to achieve an optimal balance among performance, efficiency, and latency. However, performance enhancements may also increase vulnerability to security threats. Consequently, the development of attack resilient FPGA based hardware accelerators remains an important complementary research direction. 8 Conclusion This article presents a comprehensive review of recent advances in state-of-the-art hardware accelerators, emphasizing the fundamental trade-off between high performance and power efficiency and the consequent need for optimization across multiple architectural levels, including computation, memory, and system integration. A wide range of acceleration techniques is analyzed, spanning systolic array architectures, customized parallel processing elements, and innovative memory organizations such as weight-transposable buffers, along with key design strategies including tiling, blocking, partitioning, and hardware–software co-design for both on-chip and off-chip resources. Although several optimization principles are broadly applicable across neural network models, network-specific adaptations remain essential for maximizing throughput and efficiency. The study further examines critical challenges in accelerator design, distinguishing between limitations imposed by inherent architectural constraints and those arising from current optimization methodologies. To overcome these barriers, future research must develop robust and scalable solutions that fully exploit accelerator capabilities while enhancing performance, scalability, and security. In this context, FPGAs emerge as highly flexible and effective platforms for modern AI acceleration, and continued refinement of their design and optimization frameworks is key to realizing their full potential as efficient hardware accelerators. References [1] Mahmoud H. AbdElbaky, Mohammed H. Yacoub, Wafaa S. Sayed, and Lobna A. Said. 2025. High-performance FPGA-accelerated LSTM neural network for chaotic time series prediction. AEU - International Journal of Electronics and Communications 199 (2025), 155845. doi:10.1016/j.aeue. 2025.155845 [2] Tutu Ajayi, Vidya A. Chhabria, Mateus Fogaça, Soheil Hashemi, Abdelrahman Hosny, Andrew B. Kahng, Minsoo Kim, Jeongsup Lee, Uday Mallappa, Marina Neseem, Geraldo Pradipta, Sherief Reda, Mehdi Saligane, Sachin S. Sapatnekar, Carl Sechen, Mohamed Shalan, William Swartz, Lutong Wang, Zhehong Wang, Mingyu Woo, and Bangqi Xu. 2019. Toward an Open-Source Digital Flow: First Learnings from the OpenROAD Project. In Proceedings of the 56th Annual Design Automation Conference 2019 (Las Vegas, NV, USA) (DAC ’19). Association for Computing Machinery, New York, NY, USA, Article 76, 4 pages. doi:10.1145/3316781.3326334 [3]Fawaz Khaled Alarfaj, Iqra Malik, Hikmat Ullah Khan, Naif Almusallam, Muhammad Ramzan, and Muzamil Ahmed. 2022. Credit Card Fraud Detection Using State-of-the-Art Machine Learning and Deep Learning Algorithms. IEEE Access 10 (2022), 39700–39715. doi:10.1109/ACCESS.2022. 3166891 Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review29 [4]Ayoub Benali Amjoud and Mustapha Amrouch. 2023. Object Detection Using Deep Learning, CNNs and Vision Transformers: A Review. IEEE Access 11 (2023), 35479–35516. doi:10.1109/ACCESS.2023.3266093 [5]Myat Thu Linn Aung, Daniel Gerlinghoff, Chuping Qu, Liwei Yang, Tian Huang, Rick Siow Mong Goh, Tao Luo, and Weng-Fai Wong. 2023. DeepFire2: A Convolutional Spiking Neural Network Accelerator on FPGAs. IEEE Trans. Comput. 72, 10 (2023), 2847–2857. doi:10.1109/TC.2023.3272284 [6]Utku Aydonat, Shane O’Connell, Davor Capalija, Andrew C. Ling, and Gordon R. Chiu. 2017. An OpenCL(TM) Deep Learning Accelerator on Arria 10. arXiv:1701.03534 [cs.DC] [7]Akhil Raj Baranwal, Salim Ullah, Siva Satyendra Sahoo, and Akash Kumar. 2021. ReLAccS: A Multilevel Approach to Accelerator Design for Reinforcement Learning on FPGA-Based Systems. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 40, 9 (2021), 1754–1767. doi:10.1109/TCAD.2020.3028350 [8]Iqra Batool and Tamim Ahmed Khan. 2022. Software fault prediction using data mining, machine learning and deep learning techniques: A systematic literature review. Computers and Electrical Engineering 100 (2022), 107886. doi:10.1016/j.compeleceng.2022.107886 [9]Srihari Cadambi, Abhinandan Majumdar, Michela Becchi, Srimat Chakradhar, and Hans Peter Graf. 2010. A programmable parallel accelerator for learning and classification. In 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT). IEEE, Piscataway, NJ, USA, 273–283. [10]Alessio Carpegna, Alessandro Savino, and Stefano Di Carlo. 2022. Spiker: an FPGA-optimized Hardware accelerator for Spiking Neural Networks. In 2022 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). IEEE, Piscataway, NJ, USA, 14–19. doi:10.1109/ISVLSI54635.2022.00016 [11]Andre Xian Ming Chang and Eugenio Culurciello. 2017. Hardware accelerators for recurrent neural networks on FPGA. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Piscataway, NJ, USA, 1–4. doi:10.1109/ISCAS.2017.8050816 [12]Bo-Yu Chen and Tian-Sheuan Chang. 2025. Hardware Efficient Accelerator for Spiking Transformer With Reconfigurable Parallel Time Step Computing. In 2025 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/ISCAS56072.2025. 11043330 [13]Defu Chen, Xiaohu Liu, Yijian Sang, Hao Zhou, Xiaoai Lu, and Jingyang Song. 2024. A FPGA-Based Accelerator Design for Speaker Verification using 1D-CNN. In 2024 4th International Conference on Electronic Information Engineering and Computer (EIECT). IEEE, Piscataway, NJ, USA, 80–84. doi:10.1109/EIECT64462.2024.10867080 [14]Qinyu Chen, Chang Gao, Xinyuan Fang, and Haitao Luan. 2022. Skydiver: A Spiking Neural Network Accelerator Exploiting Spatio-Temporal Workload Balance. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 41, 12 (2022), 5732–5736. doi:10.1109/TCAD.2022. 3158834 [15]Qinyu Chen, Chang Gao, and Yuxiang Fu. 2022. Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 30, 10 (2022), 1425–1437. doi:10.1109/TVLSI.2022.3196839 [16]Qinyu Chen, Congyi Sun, Chang Gao, Xinyuan Fang, and Haitao Luan. 2023. FrameFire: Enabling Efficient Spiking Neural Network Inference for Video Segmentation. In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/AICAS57966.2023.10168660 [17] Ruiqi Chen, Haoyang Zhang, Shun Li, Enhao Tang, Jun Yu, and Kun Wang. 2023. Graph-OPU: A Highly Integrated FPGA-Based Overlay Processor for Graph Neural Networks. In 2023 33rd International Conference on Field-Programmable Logic and Applications (FPL). IEEE, Piscataway, NJ, USA, 228–234. doi:10.1109/FPL60245.2023.00039 [18]Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam. 2014. DianNao: a small-footprint high- throughput accelerator for ubiquitous machine-learning. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (Salt Lake City, Utah, USA) (ASPLOS ’14). Association for Computing Machinery, New York, NY, USA, 269–284. doi:10.1145/2541940.2541967 [19]Yunji Chen, Tao Luo, Shaoli Liu, Shijin Zhang, Liqiang He, Jia Wang, Ling Li, Tianshi Chen, Zhiwei Xu, Ninghui Sun, and Olivier Temam. 2014. DaDianNao: A Machine-Learning Supercomputer. In 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE, Piscataway, NJ, USA, 609–622. doi:10.1109/MICRO.2014.58 [20]Yu-Hsin Chen, Joel Emer, and Vivienne Sze. 2016. Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks. ACM SIGARCH computer architecture news 44, 3 (2016), 367–379. [21] Yu-Hsin Chen, Tushar Krishna, Joel S. Emer, and Vivienne Sze. 2017. Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks. IEEE Journal of Solid-State Circuits 52, 1 (2017), 127–138. doi:10.1109/JSSC.2016.2616357 [22]Shaoyi Cheng, Mingjie Lin, Hao Jun Liu, Simon Scott, and John Wawrzynek. 2012. Exploiting Memory-Level Parallelism in Reconfigurable Accelerators. In 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines. IEEE, Piscataway, NJ, USA, 157–160. doi:10.1109/FCCM.2012.35 [23] CNNfn. 1999. nVidia unveils new computer graphics accelerator. http://money.cnn.com/1999/08/31/technology/nvidia/. Accessed: 2025-07-23. [24]Ashutosh Dhar, Sitao Huang, Jinjun Xiong, Damir Jamsek, Bruno Mesnet, Jian Huang, Nam Sung Kim, Wen-mei Hwu, and Deming Chen. 2019. Near-Memory and In-Storage FPGA Acceleration for Emerging Cognitive Computing Workloads. In 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). IEEE, Piscataway, NJ, USA, 68–75. doi:10.1109/ISVLSI.2019.00021 [25]Dionysios Diamantopoulos and Christoph Hagleitner. 2018. A System-Level Transprecision FPGA Accelerator for BLSTM Using On-chip Memory Reshaping. In 2018 International Conference on Field-Programmable Technology (FPT). IEEE, Piscataway, NJ, USA, 338–341. doi:10.1109/FPT.2018.00068 Manuscript submitted to ACM 30Chatterjee et al. [26]Riddhi Virendra Doshi, Sagarkumar S. Badhiye, and Latika Pinjarkar. 2025. Deep Learning Approach for Biomedical Image Classification. 2948-2933 pages. doi:10.1007/s10278-025-01590-8 Early access. [27]Haoran Dou, Davood Karimi, Caitlin K. Rollins, Cynthia M. Ortinau, Lana Vasung, Clemente Velasco-Annis, Abdelhakim Ouaalam, Xin Yang, Dong Ni, and Ali Gholipour. 2021. A Deep Attentive Convolutional Neural Network for Automatic Cortical Plate Segmentation in Fetal MRI. IEEE Transactions on Medical Imaging 40, 4 (2021), 1123–1133. doi:10.1109/TMI.2020.3046579 [28]Zidong Du, Robert Fasthuber, Tianshi Chen, Paolo Ienne, Ling Li, Tao Luo, Xiaobing Feng, Yunji Chen, and Olivier Temam. 2015. ShiDianNao: Shifting vision processing closer to the sensor. In 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA). IEEE, Piscataway, NJ, USA, 92–104. doi:10.1145/2749469.2750389 [29] Ahmed J. Abd El-Maksoud, Mohamed Ebbed, Ahmed H. Khalil, and Hassan Mostafa. 2021. Power Efficient Design of High-Performance Convolutional Neural Networks Hardware Accelerator on FPGA: A Case Study With GoogLeNet. IEEE Access 9 (2021), 151897–151911. doi:10. 1109/ACCESS.2021.3126838 [30]Aniebiet Micheal Ezekiel, Daniel Onwuchekwa, and Roman Obermaisser. 2023. Optimization of the Versatile Tensor Accelerator (VTA) Load Module in a Time-Triggered Memory Access. In 2023 26th Euromicro Conference on Digital System Design (DSD). IEEE, Piscataway, NJ, USA, 146–152. doi:10.1109/DSD60849.2023.00030 [31]Ching-Lung Fan. 2024. Deep neural networks for automated damage classification in image-based visual data of reinforced concrete structures. Heliyon 10, 19 (2024), e38104. doi:10.1016/j.heliyon.2024.e38104 [32]Clément Farabet, Berin Martini, Polina Akselrod, Selçuk Talay, Yann LeCun, and Eugenio Culurciello. 2010. Hardware accelerated convolutional neural networks for synthetic vision systems. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE, Piscataway, NJ, USA, 257–260. doi:10.1109/ISCAS.2010.5537908 [33] Clement Farabet, Cyril Poulet, Jefferson Y. Han, and Yann LeCun. 2009. CNP: An FPGA-based processor for Convolutional Networks. In 2009 International Conference on Field Programmable Logic and Applications. IEEE, Piscataway, NJ, USA, 32–37. doi:10.1109/FPL.2009.5272559 [34]Alireza Fardoost, Fatemeh Ghaedi Vanani, Zheyuan Zhu, Christopher Doerr, Shuo Pang, and Guifang Li. 2022. A High-Speed Photonic Tensor Accelerator. In 2022 IEEE Photonics Conference (IPC). IEEE, Piscataway, NJ, USA, 1–2. doi:10.1109/IPC53466.2022.9975540 [35]John Fata, Wafa Elmannai, and Khaled Elleithy. 2025. Balancing Performance and Cost—FPGA-Based CNN Accelerators for Edge Computing: Status Quo, Key Challenges, and Prospective Innovations. IEEE Access 13 (2025), 1–1. [36] Xiang Fei, Jianhui Han, Jianqiang Huang, Weimin Zheng, and Youhui Zhang. 2022. Accelerating Neural Network Training with Processing-in- Memory GPU. In 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid). IEEE, Piscataway, NJ, USA, 414–421. doi:10.1109/CCGrid54584.2022.00051 [37]Sakthi G and Abhishek N Tripathi. 2025. Systolic Array Design for Efficient FPGA Implementation of CNN Accelerators: Power and Area Optimizations. In 2025 International Conference on Computational, Communication and Information Technology (ICCCIT). IEEE, Piscataway, NJ, USA, 483–485. doi:10.1109/ICCCIT62592.2025.10927809 [38] Chang Gao, Daniel Neil, Enea Ceolini, Shih-Chii Liu, and Tobi Delbruck. 2018. DeltaRNN: A Power-efficient Recurrent Neural Network Accelerator. In Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Monterey, CALIFORNIA, USA) (FPGA ’18). Association for Computing Machinery, New York, NY, USA, 21–30. doi:10.1145/3174243.3174261 [39]Chang Gao, Antonio Rios-Navarro, Xi Chen, Shih-Chii Liu, and Tobi Delbruck. 2020. EdgeDRNN: Recurrent Neural Network Accelerator for Edge Inference. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 10, 4 (2020), 419–432. doi:10.1109/JETCAS.2020.3040300 [40] Chen Gao and Fan Zhang. 2018. FPGA-based Accelerator for Independently Recurrent Neural Network. In 2018 IEEE 4th International Conference on Computer and Communications (ICCC). IEEE, Piscataway, NJ, USA, 2075–2080. doi:10.1109/CompComm.2018.8780644 [41]Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, and Martin C. Herbordt. 2020. AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, Piscataway, NJ, USA, 922–936. doi:10.1109/MICRO50266.2020.00079 [42] Hans Graf, Srihari Cadambi, Venkata Jakkula, Murugan Sankaradass, Eric Cosatto, Srimat Chakradhar, and Igor Dourdanovic. 2008. A Massively Parallel Digital Learning Processor. In Advances in Neural Information Processing Systems, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.), Vol. 21. Curran Associates Inc., Red Hook, NY, USA, 529–536. [43] Yijin Guan, Zhihang Yuan, Guangyu Sun, and Jason Cong. 2017. FPGA-based accelerator for long short-term memory recurrent neural networks. In 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, Piscataway, NJ, USA, 629–634. doi:10.1109/ASPDAC.2017.7858394 [44]Chuliang Guo, Yufei Chen, and Yu Fu. 2025. FPGA-based component-wise LSTM training accelerator for neural granger causality analysis. Neurocomputing 615 (2025), 128871. doi:10.1016/j.neucom.2024.128871 [45] Jinrong Guo, Wantao Liu, Wang Wang, Chunrong Yao, Jizhong Han, Ruixuan Li, Yijun Lu, and Songlin Hu. 2019. AccUDNN: A GPU Memory Efficient Accelerator for Training Ultra-Deep Neural Networks. In 2019 IEEE 37th International Conference on Computer Design (ICCD). IEEE, Piscataway, NJ, USA, 65–72. doi:10.1109/ICCD46524.2019.00017 [46] Shaoshan Guo, Yunqian Huang, Kailan Mai, and Guopeng Mo. 2024. A Superscalar Six-Stage Pipeline Neural Network Accelerator: Design and Implementation Based on FPGA. In 2024 6th International Conference on Artificial Intelligence and Computer Applications (ICAICA). IEEE, Piscataway, NJ, USA, 126–133. doi:10.1109/ICAICA63239.2024.10822997 [47]Song Han, Junlong Kang, Huizi Mao, Yiming Hu, Xin Li, Yubin Li, Dongliang Xie, Hong Luo, Song Yao, Yu Wang, Huazhong Yang, and William J. Dally. 2017. ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA. arXiv:1612.00694 [cs.CL] Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review31 [48]Xushen Han, Dajiang Zhou, Shihao Wang, and Shinji Kimura. 2016. CNN-MERP: An FPGA-based memory-efficient reconfigurable processor for forward and backward propagation of convolutional neural networks. In 2016 IEEE 34th International Conference on Computer Design (ICCD). IEEE, Piscataway, NJ, USA, 320–327. doi:10.1109/ICCD.2016.7753296 [49] Zerong He, Teng Tian, Qizhe Wu, and Xi Jin. 2023. FTW-GAT: An FPGA-Based Accelerator for Graph Attention Networks With Ternary Weights. IEEE Transactions on Circuits and Systems I: Express Briefs 70, 11 (2023), 4211–4215. doi:10.1109/TCSII.2023.3280180 [50]Shaoyi Huang, Yun Liang, Jinyang Li, Yuze Chi, Jianlei Yang, Yuchen Ma, Weiwen Jiang, and Yiyu Shi. 2021. AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing. In Proceedings of the 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). ACM, New York, NY, USA, 832–845. doi:10.1145/3466752.3480113 [51] Won Jeon, Jiwon Lee, Dongseok Kang, Hongju Kal, and Won Woo Ro. 2021. PIMCaffe: Functional Evaluation of a Machine Learning Framework for In-Memory Neural Processing Unit. IEEE Access 9 (2021), 96629–96640. doi:10.1109/ACCESS.2021.3094043 [52] Jingfei Jiang, Tao Xiao, Jinwei Xu, Dong Wen, Lei Gao, and Yong Dou. 2022. A low-latency LSTM accelerator using balanced sparsity based on FPGA. Microprocessors and Microsystems 89 (2022), 104417. doi:10.1016/j.micpro.2021.104417 [53]Josh Schneider and Ian Smalley. 2024. What is a neural processing unit (NPU)? https://w.ibm.com/think/topics/neural-processing-unit. Published: 27 September 2024; Accessed: 2025-08-08. [54]Norman Jouppi, Cliff Young, Nishant Patil, and David Patterson. 2018. Motivation for and Evaluation of the First Tensor Processing Unit. IEEE Micro 38, 3 (2018), 10–19. doi:10.1109/M.2018.032271057 [55]Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al.2017. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international symposium on computer architecture. IEEE, Piscataway, NJ, USA, 1–12. [56] Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Daniel Killebrew, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun Yoon. 2017. In-datacenter performance analysis of a tensor processing unit. In 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA). IEEE, Piscataway, NJ, USA, 1–12. doi:10.1145/3079856.3080246 [57]Chia-Feng Juang, Wei-Yuan Chen, and Chung-Wei Liang. 2015. Speedup of Learning in Interval Type-2 Neural Fuzzy Systems Through Graphic Processing Units. IEEE Transactions on Fuzzy Systems 23, 4 (2015), 1286–1298. doi:10.1109/TFUZZ.2014.2353136 [58] Andrew B. Kahng, Ravi Varadarajan, and Zhiang Wang. 2024. Hier-RTLMP: A Hierarchical Automatic Macro Placer for Large-Scale Complex IP Blocks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 43, 5 (2024), 1552–1565. doi:10.1109/TCAD.2023.3346284 [59]Andrew B. Kahng and Zhiang Wang. 2025. DG-RePlAce: A Dataflow-Driven GPU-Accelerated Analytical Global Placement Framework for Machine Learning Accelerators. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 44, 2 (2025), 696–708. doi:10.1109/TCAD. 2024.3436521 [60] S. Kala, Babita R. Jose, Jimson Mathew, and S. Nalesh. 2019. High-Performance CNN Accelerator on FPGA Using Unified Winograd-GEMM Architecture. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 27, 12 (2019), 2816–2828. doi:10.1109/TVLSI.2019.2941250 [61]Lei Kang, Hui Li, Xin Li, and Haowei Zheng. 2020. Design of Convolution Operation Accelerator based on FPGA. In 2020 2nd International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI). IEEE, Piscataway, NJ, USA, 80–84. doi:10.1109/MLBDBI51377.2020.00021 [62]Rasha Karakchi. 2024. A Scratchpad Spiking Neural Network Accelerator. In 2024 IEEE 3rd International Conference on Computing and Machine Intelligence (ICMI). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/ICMI60790.2024.10586065 [63]Kasem Khalil, Bappaditya Dey, and Magdy Bayoumi. 2025. S2RNN: Self-Supervised Reconfigurable Neural Network Hardware Accelerator for Machine Learning Applications. IEEE Internet of Things Journal 12, 6 (2025), 6708–6720. doi:10.1109/JIOT.2024.3490893 [64] Fatima Hameed Khan, Muhammad Adeel Pasha, and Shahid Masud. 2024. Exploring Memory Access Techniques for Efficient FPGA based 3D CNN Accelerator Design. In 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS). IEEE, Piscataway, NJ, USA, 218–222. doi:10.1109/AICAS59952.2024.10595963 [65]Seongwook Kim, Yongjun Kim, Gwangeun Byeon, and Seokin Hong. 2023. CAESAR: A CNN Accelerator Exploiting Sparsity and Redundancy Pattern. In 2023 International Technical Conference on Circuits/Systems, Computers, and Communications (ITC-CSCC). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/ITC-CSCC58803.2023.10212679 [66]Taesu Kim, Daehyun Ahn, Dongsoo Lee, and Jae-Joon Kim. 2023. V-LSTM: An Efficient LSTM Accelerator Using Fixed Nonzero-Ratio Viterbi-Based Pruning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42, 10 (2023), 3327–3337. doi:10.1109/TCAD.2023.3243879 [67]Victoria Heekyung Kim and Kyuwon Ken Choi. 2023. A Reconfigurable CNN-Based Accelerator Design for Fast and Energy-Efficient Object Detection System on Mobile FPGA. IEEE Access 11 (2023), 59438–59445. doi:10.1109/ACCESS.2023.3285279 [68]Argyris Kokkinis, Dionysios Diamantopoulos, and Kostas Siozios. 2022. Dynamic Optimization of On-Chip Memories for HLS Targeting Many- Accelerator Platforms. IEEE Computer Architecture Letters 21, 2 (2022), 41–44. doi:10.1109/LCA.2022.3190048 Manuscript submitted to ACM 32Chatterjee et al. [69]Shreyas Kolala Venkataramanaiah, Yufei Ma, Shihui Yin, Eriko Nurvithadhi, Aravind Dasu, Yu Cao, and Jae-Sun Seo. 2019. Automatic Compiler Based FPGA Accelerator for CNN Training. In 2019 29th International Conference on Field Programmable Logic and Applications (FPL). IEEE, Piscataway, NJ, USA, 166–172. doi:10.1109/FPL.2019.00034 [70] Manuel Le Gallo, Corey Lammie, Julian Büchel, Fabio Carta, Omobayode Fagbohungbe, Charles Mackin, Hsinyu Tsai, Vijay Narayanan, Abu Sebastian, Kaoutar El Maghraoui, and Malte J. Rasch. 2023. Using the IBM analog in-memory hardware acceleration kit for neural network training and inference. APL Machine Learning 1, 4 (11 2023), 041102. doi:10.1063/5.0168089 [71]Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, Ivan Oseledets, and Victor Lempitsky. 2015. Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition. arXiv:1412.6553 [cs.CV] doi:10.48550/arXiv.1412.6553 [72] Jindong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, and Yi Zeng. 2023. FireFly: A High-Throughput Hardware Accelerator for Spiking Neural Networks With Efficient DSP and Memory Optimization. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 31, 8 (2023), 1178–1191. doi:10.1109/TVLSI.2023.3279349 [73]Jixuan Li, Ka-Fai Un, Wei-Han Yu, Pui-In Mak, and Rui P. Martins. 2021. An FPGA-Based Energy-Efficient Reconfigurable Convolutional Neural Network Accelerator for Object Recognition Applications. IEEE Transactions on Circuits and Systems I: Express Briefs 68, 9 (2021), 3143–3147. doi:10.1109/TCSII.2021.3095283 [74]Mingyang Li, Yirong Kan, Renyuan Zhang, and Yasuhiko Nakashima. 2024. A Fully-Parallel Reconfigurable Spiking Neural Network Accelerator with Structured Sparse Connections. In 2024 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/ISCAS58744.2024.10558156 [75] Shijie Li, Yong Dou, Qi Lv, Qiang Wang, Xin Niu, and Ke Yang. 2016. Optimized GPU Acceleration Algorithm of Convolutional Neural Networks for Target Detection. In 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS). IEEE, Piscataway, NJ, USA, 224–230. doi:10.1109/HPCC-SmartCity-DSS.2016.0041 [76]Tenglong Li, Jindong Li, Guobin Shen, Dongcheng Zhao, Qian Zhang, and Yi Zeng. 2025. FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration With Reconfigurable Spatial Architecture. IEEE Transactions on Circuits and Systems I: Regular Papers 72, 8 (2025), 4007–4020. doi:10.1109/TCSI.2024.3496554 [77]Yixuan Li, Benshan Wang, Tengji Xu, Shaojie Liu, Qiarong Xiao, and Chaoran Huang. 2024. Scalable and Energy-Efficient Photonic Neural Networks Through Convolution Compression. In 2024 Asia Communications and Photonics Conference (ACP) and International Conference on Information Photonics and Optical Communications (IPOC). IEEE, Piscataway, NJ, USA, 1–6. doi:10.1109/ACP/IPOC63121.2024.10809912 [78]Shengwen Liang, Cheng Liu, Ying Wang, Huawei Li, and Xiaowei Li. 2020. DeepBurning-GL: an Automated Framework for Generating Graph Neural Network Accelerators. In 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, Piscataway, NJ, USA, 1–9. doi:10.1109/VLSID64188.2025.00052 [79] Yujun Lin, Zhekai Zhang, and Song Han. 2025. LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, Piscataway, NJ, USA, 1335–1347. doi:10.1109/HPCA61900.2025. 00101 [80]Daofu Liu, Tianshi Chen, Shaoli Liu, Jinhong Zhou, Shengyuan Zhou, Olivier Teman, Xiaobing Feng, Xuehai Zhou, and Yunji Chen. 2015. PuDianNao: A Polyvalent Machine Learning Accelerator. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (Istanbul, Turkey) (ASPLOS ’15). Association for Computing Machinery, New York, NY, USA, 369–381. doi:10.1145/2694344.2694358 [81]Fang Liu, Heyuan Li, Wei Hu, and Yanxiang He. 2024. Review of neural network model acceleration techniques based on FPGA platforms. Neurocomputing 610 (2024), 128511. doi:10.1016/j.neucom.2024.128511 [82]Hanwen Liu, Yi Chen, Zihang Zeng, Malu Zhang, and Hong Qu. 2023. A Low Power and Low Latency FPGA-Based Spiking Neural Network Accelerator. In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, Piscataway, NJ, USA, 1–8. doi:10.1109/IJCNN54540.2023. 10191153 [83]Yuhao Liu, Yanhua Ma, Bowei Zhang, Lu Liu, Jie Wang, and Shibo Tang. 2024. Improving the computational efficiency and flexibility of FPGA-based CNN accelerator through loop optimization. Microelectronics Journal 147 (2024), 106197. doi:10.1016/j.mejo.2024.106197 [84] Liqiang Lu, Yun Liang, Qingcheng Xiao, and Shengen Yan. 2017. Evaluating Fast Algorithms for Convolutional Neural Networks on FPGAs. In 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, Piscataway, NJ, USA, 101–108. doi:10.1109/FCCM.2017.64 [85]Wenyan Lu, Guihai Yan, Jiajun Li, Shijun Gong, Yinhe Han, and Xiaowei Li. 2017. FlexFlow: A Flexible Dataflow Accelerator Architecture for Convolutional Neural Networks. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, Piscataway, NJ, USA, 553–564. doi:10.1109/HPCA.2017.29 [86]Peng Lv, Wei Liu, and Jinghui Li. 2020. A FPGA-based accelerator implementaion for YOLOv2 object detection using Winograd algorithm. In 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). IEEE, Piscataway, NJ, USA, 1894–1898. doi:10.1109/ ICMCCE51767.2020.00415 [87]Yufei Ma, Yu Cao, Sarma Vrudhula, and Jae-sun Seo. 2017. An automatic RTL compiler for high-throughput FPGA implementation of diverse deep convolutional neural networks. In 2017 27th International Conference on Field Programmable Logic and Applications (FPL). IEEE, Piscataway, NJ, USA, 1–8. doi:10.23919/FPL.2017.8056824 Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review33 [88]Yufei Ma, Yu Cao, Sarma Vrudhula, and Jae-sun Seo. 2018. Optimizing the Convolution Operation to Accelerate Deep Neural Networks on FPGA. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 26, 7 (2018), 1354–1367. doi:10.1109/TVLSI.2018.2815603 [89]Yufei Ma, Yu Cao, Sarma Vrudhula, and Jae-Sun Seo. 2020. Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 2 (2020), 424–437. doi:10.1109/TCAD.2018.2884972 [90]Stefano Markidis, Steven Wei Der Chien, Erwin Laure, Ivy Bo Peng, and Jeffrey S Vetter. 2018. Nvidia tensor core programmability, performance & precision. In 2018 IEEE international parallel and distributed processing symposium workshops (IPDPSW). IEEE, Vancouver, BC, Canada, 522–531. [91]Jian Meng, Shreyas Kolala Venkataramanaiah, Chuteng Zhou, Patrick Hansen, Paul Whatmough, and Jae-sun Seo. 2021. FixyFPGA: Efficient FPGA Accelerator for Deep Neural Networks with High Element-Wise Sparsity and without External Memory Access. In 2021 31st International Conference on Field-Programmable Logic and Applications (FPL). IEEE, Piscataway, NJ, USA, 9–16. doi:10.1109/FPL53798.2021.00010 [92]Pranava Mummoju, Anna Wolff, Martin Perdacher, Claudia Plant, and Christian Böhm. 2022. Enhancing k-Means Algorithm with Tensor Processing Unit. In 2022 IEEE International Conference on Big Data (Big Data). IEEE, Piscataway, NJ, USA, 194–200. doi:10.1109/BigData55660.2022.10020427 [93]Abdullah Al Nomaan Nafi, Habibur Rahaman, Zafaryab Haider, Tanzim Mahfuz, Fnu Suya, Swarup Bhunia, and Prabuddha Chakraborty. 2025. DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples. arXiv preprint arXiv:2508.13309 (2025). [94] Ali Bou Nassif, Ismail Shahin, Imtinan Attili, Mohammad Azzeh, and Khaled Shaalan. 2019. Speech Recognition Using Deep Neural Networks: A Systematic Review. IEEE Access 7 (2019), 19143–19165. doi:10.1109/ACCESS.2019.2896880 [95]Daniel Neil and Shih-Chii Liu. 2014. Minitaur, an Event-Driven FPGA-Based Spiking Network Accelerator. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 22, 12 (2014), 2621–2628. doi:10.1109/TVLSI.2013.2294916 [96] Anagha Nimbekar, Prabodh Katti, Chen Li, Bashir M. Al-Hashimi, Amit Acharyya, and Bipin Rajendran. 2024. Hardware-Software Co-optimised Fast and Accurate Deep Reconfigurable Spiking Inference Accelerator Architecture Design Methodology. In 2024 IEEE 37th International System-on-Chip Conference (SOCC). IEEE, Piscataway, NJ, USA, 1–6. doi:10.1109/VLSID64188.2025.00052 [97]Yiming Ouyang, Chengming An, Jianhua Li, and HuaGuo Liang. 2025. PRLM: A parallel loading mechanism for a deep neural network accelerator based on NoC. Microelectronics Journal 160 (2025), 106684. doi:10.1016/j.mejo.2025.106684 [98]Tommaso Pacini, Pietro Nannipieri, Silvia Moranti, and Luca Fanucci. 2025. FPG-AI RNN: A Technology-Agnostic Framework for the Automatic Acceleration of LSTM/GRU-Based Models on FPGAs. IEEE Access 13 (2025), 100353–100369. doi:10.1109/ACCESS.2025.3577908 [99]Sathish Panchapakesan, Zhenman Fang, and Jian Li. 2021. SyncNN: Evaluating and Accelerating Spiking Neural Networks on FPGAs. In 2021 31st International Conference on Field-Programmable Logic and Applications (FPL). IEEE, Piscataway, NJ, USA, 286–293. doi:10.1109/FPL53798.2021.00058 [100]Stefania Perri, Cristian Zambelli, Daniele Ielmini, and Cristina Silvano. 2024. Digital In-Memory Computing to Accelerate Deep Learning Inference on the Edge. In 2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, Piscataway, NJ, USA, 130–133. doi:10.1109/IPDPSW63119.2024.00037 [101]Anastasios Petropoulos and Theodore Antonakopoulos. 2025. A Scalable FPGA Architecture With Adaptive Memory Utilization for GEMM-Based Operations. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 33, 8 (2025), 2334–2338. doi:10.1109/TVLSI.2025.3571677 [102] Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. Samurai: A framework for safeguarding against malicious usage and resilience of ai. In 2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6. [103]Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. Secure ai systems: Emerging threats and defense mechanisms. In 2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6. [104]Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2025. Runtime Detection of Adversarial Attacks in AI Accelerators Using Performance Counters. arXiv preprint arXiv:2503.07568 (2025). [105]Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2025. Secure and Storage-Efficient Deep Learning Models for Edge AI Using Automatic Weight Generation. arXiv preprint arXiv:2507.06380 (2025). [106]Himanshu Rai, Aishwarya Sridhar, Wolfgang Ecker, and Nanditha Rao. 2025. FPUGen: A FrameWork to Generate Custom Floating Point FMA Accelerators on FPGAs. In 2025 38th International Conference on VLSI Design and 2024 23rd International Conference on Embedded Systems (VLSID). IEEE, Piscataway, NJ, USA, 225–230. doi:10.1109/VLSID64188.2025.00052 [107]Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. 2019. Bit-flip attack: Crushing neural network with progressive bit search. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1211–1220. [108] Vladimir Rybalkin, Alessandro Pappalardo, Muhammad Mohsin Ghaffar, Giulio Gambardella, Norbert Wehn, and Michaela Blott. 2018. FINN-L: Library Extensions and Design Trade-off Analysis for Variable Precision LSTM Networks on FPGAs. arXiv:1807.04093 [cs.CV] [109]Murugan Sankaradas, Venkata Jakkula, Srihari Cadambi, Srimat Chakradhar, Igor Durdanovic, Eric Cosatto, and Hans Peter Graf. 2009. A Massively Parallel Coprocessor for Convolutional Neural Networks. In 2009 20th IEEE International Conference on Application-specific Systems, Architectures and Processors. IEEE, Piscataway, NJ, USA, 53–60. doi:10.1109/ASAP.2009.25 [110]Rishov Sarkar, Stefan Abi-Karam, Yuqi He, Lakshmi Sathidevi, and Cong Hao. 2023. FlowGNN: A Dataflow Architecture for Real-Time Workload- Agnostic Graph Neural Network Inference. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, Piscataway, NJ, USA, 1099–1112. doi:10.1109/HPCA56546.2023.10071015 [111]Russell L. T. Schwartz, Belal Jahannia, Nicola Peserico, Hamed Dalir, and Volker J. Sorger. 2024. Data Throughput for Efficient Photonic Neural Network Accelerators. In 2024 IEEE Silicon Photonics Conference (SiPhotonics). IEEE, Piscataway, NJ, USA, 1–2. doi:10.1109/SiPhotonics60897.2024. 10543636 Manuscript submitted to ACM 34Chatterjee et al. [112]Russell L. T. Schwartz, Hangbo Yang, Nicola Peserico, and Volker J. Sorger. 2024. The Von Neumann Bottleneck in Photonic Tensor Core Systems. In 2024 IEEE Photonics Society Summer Topicals Meeting Series (SUM). IEEE, Piscataway, NJ, USA, 1–2. doi:10.1109/SUM60964.2024.10614519 [113]Yousef O. Sharrab, Hani Attar, Mohammad Ali H. Eljinini, Yasmin Al-Omary, and Wala’A E. Al-Momani. 2025. Advancements in Speech Recognition: A Systematic Review of Deep Learning Transformer Models, Trends, Innovations, and Future Directions. IEEE Access 13 (2025), 46925–46940. doi:10.1109/ACCESS.2025.3550855 [114]Ahmad Shawahna, Sadiq M. Sait, and Aiman El-Maleh. 2019. FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review. IEEE Access 7 (2019), 7823–7859. doi:10.1109/ACCESS.2018.2890150 [115]Xuanhua Shi, Zhigao Zheng, Yongluan Zhou, Hai Jin, Ligang He, Bo Liu, and Qiang-Sheng Hua. 2018. Graph processing on GPUs: A survey. ACM Computing Surveys (CSUR) 50, 6 (2018), 1–35. [116]M Sinthuja, Chirag Ganesh Padubidri, Gaddam Sai Jayachandra, Mudduluru Charan Teja, and Golthi Sai Pavan Kumar. 2024. Extraction of Text from Images Using Deep Learning. Procedia Computer Science 235 (2024), 789–798. doi:10.1016/j.procs.2024.04.075 International Conference on Machine Learning and Data Engineering (ICMLDE 2023). [117]Jean Pierre Soto-Chirinos, Henry Ivan Condori-Alejo, and Guina Sotomayor Alzamora. 2023. Neural Rendering in the Cloud with Tensor Processing Unit. In 2023 IEEE X International Conference on Electronics, Electrical Engineering and Computing (INTERCON). IEEE, Piscataway, NJ, USA, 1–7. doi:10.1109/INTERCON59652.2023.10326073 [118]Yunfei Sun, Brian Liu, and Xianchao Xu. 2019. An OpenCL-Based Hybrid CNN-RNN Inference Accelerator On FPGA. In 2019 International Conference on Field-Programmable Technology (ICFPT). IEEE, Piscataway, NJ, USA, 283–286. doi:10.1109/ICFPT47387.2019.00048 [119] Kaifei Tang, Jiantao Wang, Xiang Ji, Jiahui Liu, Yu Xin, Haijiang Cao, Zhaobang Zeng, Rulei Xiao, and Wei Jiang. 2023. Complete photonic tensor convolution driven by single dataflow. In 2023 Asia Communications and Photonics Conference/2023 International Photonics and Optoelectronics Meetings (ACP/POEM). IEEE, Piscataway, NJ, USA, 1–3. doi:10.1109/ACP/POEM59049.2023.10369087 [120]Kaifei Tang, Jiantao Wang, Wenqu Xu, Xiang Ji, Jiahui Liu, Xiaobin Huang, Yu Xin, Pan Dai, Guozhu Sun, Zhaobang Zeng, Rulei Xiao, Xiangfei Chen, and Wei Jiang. 2024. Photonic Tensor Processing Unit With Single Dataflow and Programmable High-Precision Weighting Control. Journal of Lightwave Technology 42, 2 (2024), 659–669. doi:10.1109/JLT.2023.3317090 [121]Guanchen Tao, Yonggen Li, Yanfeng Xu, Jicong Fan, Haibin Shen, and Kejie Huang. 2022. A Near Memory Computing FPGA Architecture for Neural Network Acceleration. In 2022 2nd International Conference on Frontiers of Electronics, Information and Computation Technologies (ICFEICT). IEEE, Piscataway, NJ, USA, 543–548. doi:10.1109/ICFEICT57213.2022.00100 [122]Teng Tian, Letian Zhao, Xiaotian Wang, Qizhe Wu, Wei Yuan, and Xi Jin. 2022. FP-GNN: Adaptive FPGA accelerator for Graph Neural Networks. Future Generation Computer Systems 136 (2022), 294–310. doi:10.1016/j.future.2022.06.010 [123]Swetha Varadarajulu and K. Mariammal. 2021. Design of SentiNet RTL Library for CNN based Hardware Accelerator. In 2021 5th International Conference on Computer, Communication and Signal Processing (ICCCSP). IEEE, Piscataway, NJ, USA, 100–108. doi:10.1109/ICCCSP52374.2021.9465504 [124]Stylianos I Venieris, Alexandros Kouris, and Christos-Savvas Bouganis. 2018. Toolflows for mapping convolutional neural networks on FPGAs: A survey and future directions. ACM Computing Surveys (CSUR) 51, 3 (2018), 1–39. [125] Bo Wang, Qing Han, and Xiupeng Shi. 2024. Multi-Objective Optimization of ML Algorithms and FPGA Accelerators for Robotic Intelligence. In 2024 International Conference on Intelligent Robotics and Automatic Control (IRAC). IEEE, Piscataway, NJ, USA, 130–137. doi:10.1109/IRAC63143. 2024.10871700 [126]Cong Wang and Tianrui Chen. 2011. Rapid Detection of Small Oscillation Faults via Deterministic Learning. IEEE Transactions on Neural Networks 22, 8 (2011), 1284–1296. doi:10.1109/TNN.2011.2159622 [127]Hao Wang, Danfeng Qiu, Fen Ge, and Ying Yang. 2022. Implementation of Bidirectional LSTM Accelerator Based on FPGA. In 2022 IEEE 22nd International Conference on Communication Technology (ICCT). IEEE, Piscataway, NJ, USA, 1512–1516. doi:10.1109/ICCT56141.2022.10072756 [128]Jiawen Wang, Chenfei Liao, Dewei Li, Zhongqi Zhao, Jingchuan Chen, and Kehu Yang. 2025. An Efficient Large Kernel Convolution Network Designed for Neural Processing Unit. Engineering Applications of Artificial Intelligence 142 (2025), 109887. doi:10.1016/j.engappai.2024.109887 [129] Zilin Wang, Yi Zhong, Youming Yang, Xiaoxin Cui, and Yuan Wang. 2023. An Efficient Spiking Neural Network Accelerator with Sparse Weight. In 2023 IEEE Biomedical Circuits and Systems Conference (BioCAS). IEEE, Piscataway, NJ, USA, 1–5. doi:10.1109/BioCAS58349.2023.10388553 [130]Jerod Weinman, Ziwen Chen, Ben Gafford, Nathan Gifford, Abyaya Lamsal, and Liam Niehus-Staab. 2019. Deep Neural Networks for Text Detection and Recognition in Historical Maps. In 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, Piscataway, NJ, USA, 902–909. doi:10.1109/ICDAR.2019.00149 [131]Daniel Windhager, Bernhard A. Moser, and Michael Lunglmayr. 2023. SNN Architecture for Differential Time Encoding Using Decoupled Processing Time. doi:10.48550/arXiv.2311.14447 [132] Meng Wu, Jin Zhou, Yibin Peng, Shuihua Wang, and Yudong Zhang. 2024. Deep Learning for Image Classification: A Review. In Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023), Ruidan Su, Yu-Dong Zhang, and Alejandro F. Frangi (Eds.). Springer Nature Singapore, Singapore, 352–362. doi:10.1007/978-981-97-1335-6_31 [133] Yuhao Wu. 2023. Review on FPGA-Based Accelerators in Deep learning. In 2023 IEEE 6th Information Technology,Networking,Electronic and Automation Control Conference (ITNEC), Vol. 6. IEEE, Piscataway, NJ, USA, 452–456. doi:10.1109/ITNEC56291.2023.10082175 [134]Ermai Xie, Martin McGinnity, QingXiang Wu, Jianyong Cai, and Rontai Cai. 2011. GPU implementation of spiking neural networks for color image segmentation. In 2011 4th International Congress on Image and Signal Processing, Vol. 3. IEEE, Piscataway, NJ, USA, 1246–1250. doi:10.1109/CISP. 2011.6100451 Manuscript submitted to ACM Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review35 [135]Liang Xie, Xitian Fan, Wei Cao, and Lingli Wang. 2018. High Throughput CNN Accelerator Design Based on FPGA. In 2018 International Conference on Field-Programmable Technology (FPT). IEEE, Piscataway, NJ, USA, 274–277. doi:10.1109/FPT.2018.00052 [136]Xi Xie, Hongwu Peng, Amit Hasan, Shaoyi Huang, Jiahui Zhao, Haowen Fang, Wei Zhang, Tong Geng, Omer Khan, and Caiwen Ding. 2023. Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, Piscataway, NJ, USA, 01–09. doi:10.1109/ICCAD57390.2023.10323722 [137]Ruyi Yang, Jingyu Hu, Zihao Li, Jianli Mu, Tingzhao Yu, Jiangjiang Xia, Xuhong Li, Aritra Dasgupta, and Haoyi Xiong. 2024. Interpretable machine learning for weather and climate prediction: A review. Atmospheric Environment 338 (2024), 120797. doi:10.1016/j.atmosenv.2024.120797 [138]Jinlin Ye and Wei Zhang. 2023. A Scalable ARM+FPGA-Based CNN Accelerator with Limited Hardware Resources. In 2023 42nd Chinese Control Conference (C). IEEE, Piscataway, NJ, USA, 2498–2503. doi:10.23919/C58697.2023.10241078 [139]Wujian Ye, Yuehai Chen, and Yijun Liu. 2023. The Implementation and Optimization of Neuromorphic Hardware for Supporting Spiking Neural Networks With MLP and CNN Topologies. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42, 2 (2023), 448–461. doi:10.1109/TCAD.2022.3179246 [140]Mingqi Yin, Xiaole Cui, Feng Wei, Hanqing Liu, Yuanyuan Jiang, and Xiaoxin Cui. 2024. A reconfigurable FPGA-based spiking neural network accelerator. Microelectronics Journal 152 (2024), 106377. doi:10.1016/j.mejo.2024.106377 [141]Haoran You, Tong Geng, Yongan Zhang, Ang Li, and Yingyan Lin. 2022. GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, Piscataway, NJ, USA, 460–474. doi:10.1109/HPCA53966.2022.00041 [142] Ye Yu and Niraj K. Jha. 2022. SPRING: A Sparsity-Aware Reduced-Precision Monolithic 3D CNN Accelerator Architecture for Training and Inference. IEEE Transactions on Emerging Topics in Computing 10, 1 (2022), 237–249. doi:10.1109/TETC.2020.3003328 [143] Longhao Yuan, Chao Li, Jianting Cao, and Qibin Zhao. 2019. Randomized Tensor Ring Decomposition and Its Application to Large-scale Data Reconstruction. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, Piscataway, NJ, USA, 2127–2131. doi:10.1109/ICASSP.2019.8682197 [144]Wei Yuan, Teng Tian, Qizhe Wu, and Xi Jin. 2022. QEGCN: An FPGA-based accelerator for quantized GCNs with edge-level parallelism. Journal of Systems Architecture 129 (2022), 102596. doi:10.1016/j.sysarc.2022.102596 [145]Bingyi Zhang, Rajgopal Kannan, and Viktor Prasanna. 2021. BoostGCN: A Framework for Optimizing GCN Inference on FPGA. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, Piscataway, NJ, USA, 29–39. doi:10.1109/FCCM51124.2021.00012 [146]Bingyi Zhang and Viktor Prasanna. 2023. Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation. In 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, Piscataway, NJ, USA, 233–244. doi:10.1109/IPDPS54959.2023.00032 [147]Chen Zhang, Shijie Cao, Guohao Dai, Chenbo Geng, Zhuliang Yao, Wencong Xiao, Yunxin Liu, Ming Wu, Lintao Zhang, Guangyu Sun, Zhigang Ji, Runsheng Wang, and Ru Huang. 2025. Fine-Grained Structured Sparse Computing for FPGA-Based AI Inference. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 44, 7 (2025), 2544–2557. doi:10.1109/TCAD.2024.3524356 [148] Chen Zhang, Peng Li, Guangyu Sun, Yijin Guan, Bingjun Xiao, and Jason Cong. 2015. Optimizing FPGA-based accelerator design for deep convolutional neural networks. In Proceedings of the 2015 ACM/SIGDA international symposium on field-programmable gate arrays. Curran Associates Inc., Red Hook, NY, USA, 161–170. [149]Chen Zhang, Peng Li, Guangyu Sun, Yijin Guan, Bingjun Xiao, and Jason Cong. 2015. Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks. In Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Monterey, California, USA) (FPGA ’15). Association for Computing Machinery, New York, NY, USA, 161–170. doi:10.1145/2684746.2689060 [150]Jialiang Zhang and Jing Li. 2017. Improving the Performance of OpenCL-based FPGA Accelerator for Convolutional Neural Network. In Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Monterey, California, USA) (FPGA ’17). Association for Computing Machinery, New York, NY, USA, 25–34. doi:10.1145/3020078.3021698 [151] Jian Zhang, Ran Wang, Xudong Pei, Dan Luo, Sajjad Hussain, and Guohe Zhang. 2022. A Fast Spiking Neural Network Accelerator based on BP-STDP Algorithm and Weighted Neuron Model. IEEE Transactions on Circuits and Systems I: Express Briefs 69, 4 (2022), 2271–2275. doi:10.1109/TCSII.2021.3137987 [152] Jian Zhang, Yong Wang, Yanlong Zhang, Bo Bi, Qiliang Chen, and Yimao Cai. 2025. A power-efficient spiking convolutional neural network accelerator based on temporal parallelism and streaming dataflow. Microelectronics Journal 158 (2025), 106616. doi:10.1016/j.mejo.2025.106616 [153]Kehui Zhang, Lingfeng Li, Hao Liu, Jing Yuan, and Xue-Cheng Tai. 2025. Deep convolutional neural networks meet variational shape compactness priors for image segmentation. Neurocomputing 623 (2025), 129395. doi:10.1016/j.neucom.2025.129395 [154] Qianqian Zhang and Xin Zhang. 2024. Design of A Low-Latency General-Purpose CNN Hardware Accelerator Based on Pulsed Arrays on FPGAs. In 2024 20th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). IEEE, Piscataway, NJ, USA, 1–8. doi:10.1109/ICNC-FSKD64080.2024.10702206 [155] Tao Zhang, Yi Zhong, Youming Yang, Zilin Wang, Zhaotong Zhang, and Yuan Wang. 2025. UniPRE: An SNN-ANN Accelerator With Unified Max-Pooling Prediction and Redundancy Elimination. IEEE Transactions on Circuits and Systems I: Express Briefs 72, 8 (2025), 1088–1092. doi:10.1109/TCSII.2025.3582265 [156]Weifeng Zhang, Fen Ge, Chenchen Cui, Ying Yang, Fang Zhou, and Ning Wu. 2020. Design and Implementation of LSTM Accelerator Based on FPGA. In 2020 IEEE 20th International Conference on Communication Technology (ICCT). IEEE, Piscataway, NJ, USA, 1675–1679. doi:10.1109/ Manuscript submitted to ACM 36Chatterjee et al. ICCT50939.2020.9295665 [157]Guanwen Zhong, Alok Prakash, Siqi Wang, Yun Liang, Tulika Mitra, and Smail Niar. 2017. Design Space exploration of FPGA-based accelerators with multi-level parallelism. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017. IEEE, Piscataway, NJ, USA, 1141–1146. doi:10.23919/DATE.2017.7927161 [158]Zhe Zhou, Bizhao Shi, Zhe Zhang, Yijin Guan, Guangyu Sun, and Guojie Luo. 2021. BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matrices. In 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, Piscataway, NJ, USA, 1009–1014. doi:10.1109/ DAC18074.2021.9586181 Manuscript submitted to ACM

Instant research discovery