Paper deep dive

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

Bowen Wang, Chi Zhang, Diyou Shen, Renzo Andri, Navaneeth Kunhi Purayil, Luca Benini

Year: 2026Venue: arXiv preprintArea: cs.ARType: PreprintEmbeddings: 29

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 8/1/2026, 11:57:04 AM

Summary

The paper introduces Ventaglio, a runtime-configurable sparse execution unit and RISC-V Vector (RVV) ISA extensions designed to accelerate sparse tensor contractions on vector processors. By implementing native indexed gather-accumulate-scatter operations, Ventaglio overcomes the limitations of existing RVV architectures that rely on software-based metadata decoding. Integrated into the open-source Spatz vector processing cluster, Ventaglio achieves significant speedups in sparse kernel performance and end-to-end Transformer inference (LLaMA-3-8B) with minimal area overhead.

Entities (8)

Ventaglio · hardware-architecture · 98%LLaMA-3-8B · machine-learning-model · 96%Spatz · hardware-architecture · 95%RVV · instruction-set-architecture · 92%12 nm FinFET · technology-node · 90%Gustavson's dataflow · algorithm · 90%DuoGPT · pruning-method · 88%GVSoC · simulation-framework · 85%

Relation Signals (7)

Ventaglio → implements → indexed gather-accumulate-scatter

confidence 95% · Ventaglio... drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support.

Ventaglio → integratedinto → Spatz

confidence 95% · Integrated into an open-source vector processing cluster [Spatz]... Ventaglio accelerates sparse tensor contraction kernels

Ventaglio → extends → RVV

confidence 92% · Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions

Ventaglio → accelerates → LLaMA-3-8B

confidence 90% · Ventaglio achieves 2.40–5.25× and 2.06–3.16× speedup over dense baselines during prefill and autoregressive decoding... Using a DuoGPT-pruned LLaMA-3-8B model

Ventaglio → implementedin → 12 nm FinFET

confidence 90% · implemented in 12 nm FinFET

Ventaglio → uses → Gustavson's dataflow

confidence 88% · Gustavson’s dataflow provides a natural execution model... Ventaglio... elevates metadata-driven indexed accumulation to a native RVV execution primitive.

LLaMA-3-8B → prunedby → DuoGPT

confidence 85% · DuoGPT-pruned LLaMA-3-8B model

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models. In the moderate-sparsity regime, Gustavson's dataflow provides a natural execution model for exploiting both activation and weight sparsity on vector processors through metadata-driven indexed accumulation. However, existing RVV architectures lack native support for this pattern, forcing kernels to rely on software index decoding and L1-backed indexed memory operations that keep sparse tensor contractions far below their roofline performance bound. We present Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions that drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support. Integrated into an open-source vector processing cluster and implemented in 12nm FinFET, Ventaglio accelerates sparse tensor contraction kernels by $6.9\text{--}7.4\times$ over optimized RVV baselines, with only $3.1\%$ area overhead for a cluster of tightly-L1 coupled vector processing elements. We build a performance-accurate instruction-level model of the Ventaglio extension, calibrate it against RTL implementation, and leverage it for scale-out performance analysis on a large $4\times4$ multi-cluster system. Using a DuoGPT-pruned LLaMA-3-8B model with practical $40\text{--}60\%$ dual sparsity, Ventaglio achieves $2.40\text{--}5.25\times$ and $2.06\text{--}3.16\times$ speedup over dense baselines during prefill and autoregressive decoding, respectively.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

27,261 characters extracted from source content.

Expand or collapse full text

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference Bowen Wang*, Chi Zhang*, Diyou Shen*, Renzo Andri‡, Navaneeth Kunhi Purayil*, Luca Benini*† This work is supported by the ETH Future Computing Laboratory (EFCL) and Huawei ZRC. Abstract Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models. In the moderate-sparsity regime, Gustavson’s dataflow provides a natural execution model for exploiting both activation and weight sparsity on vector processors through metadata-driven indexed accumulation. However, existing RISC-V Vector (RVV) architectures lack native support for this pattern, forcing kernels to rely on software index decoding and L1-backed indexed memory operations that keep sparse tensor contractions far below their roofline performance bound. We present Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions that drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support. Integrated into an open-source vector processing cluster and implemented in 12 nm FinFET, Ventaglio accelerates sparse tensor contraction kernels by 6.9–7.4×6.9--7.4× over optimized RVV baselines, with only 3.1%3.1\% area overhead for a cluster of tightly-L1 coupled vector processing elements. We build a performance-accurate instruction-level model of the Ventaglio extension, calibrate it against RTL implementation, and leverage it for scale-out performance analysis on a large 4×44× 4 multi-cluster system. Using a DuoGPT-pruned LLaMA-3-8B model with practical 40–6040--60 dual sparsity, Ventaglio achieves 2.40–5.25×2.40--5.25× and 2.06–3.16×2.06--3.16× speedup over dense baselines during prefill and autoregressive decoding, respectively. I Introduction The success of Transformer architectures is closely tied to scaling laws, motivating the development of models with billions to trillions of parameters[1]. However, the resulting growth in model size incurs substantial memory footprint and computational cost, limiting efficient deployment on edge platforms with tight memory and bandwidth budgets. Sparsity has been exploited to mitigate these costs. Fine-grained model pruning statically induces weight sparsity by removing redundant parameters [20, 12, 6], whereas activation sparsity arises dynamically during inference and is exploited via gating and zero-skipping [11, 2, 22]. In this work, we focus on the moderate-sparsity (MS) regime, corresponding to a range of zero-element ratios of 40–75%, which commonly arises in practical Transformer inference. Translating these sparsity into practical speedups requires tight hardware–software co-design. Existing hardware support remains largely matrix-centric. NVIDIA extends Tensor Core MMA pipelines with 2:4 sparse operand support [13, 14, 15], AMD provides sparse-matrix ISA support on Instinct accelerators [3], and Arm has introduced analogous support in SME2[4], while Neural Processing Units (NPUs) such as Ethos-U rely on dedicated weight-decoding hardware [5]. These approaches map well to the prefill stage of Transformer inference, where computation retains high matrix-level parallelism. During autoregressive decoding, however, execution shifts toward lower-intensity matrix-vector operations, under which such engines are more easily underutilized. Vector processors are a compelling substrate for such workloads, offering a favorable balance of performance, energy efficiency, and programmability[8]. In particular, Gustavson’s dataflow provides a natural execution model for combining fine-grained weight sparsity with dynamic activation sparsity: as depicted in Fig. 1(a), non-zero activations trigger computation, while weight metadata determines the indexed output accumulations to perform. These properties make vector processors attractive for scalable Machine Learning (ML) systems, where lightweight Vector Processing Elements (VPEs) are organized into tightly coupled clusters with high-throughput interconnects to shared L1 memory[16], enabling further hierarchical scale-out across multiple clusters. However, existing RISC-V Vector (RVV) architectures provide no native support for this metadata-driven gather/accumulate/scatter pattern, leaving optimized kernel libraries to rely on explicit software metadata decoding and L1-backed indexed memory operations. The recently proposed IndexMAC [19] and SCG [18] RVV ISA extensions partially reduce this overhead through indexed access to Vector Register File (VRF)-resident matrix tiles, but remain tied to matrix-level tiling and reuse, leaving vector-matrix execution and activation-driven sparsity insufficiently addressed. TABLE I: Comparison with fine-grained sparse acceleration designs. Design Arch. Dataflow Fmt. Mat-Vec Eff. Mat-Mat Eff. Act. Sparsity Open Source NVIDIA Blackwell[15] GPU IP 2:4 lightred✗ lightgreen✓ lightred✗ lightyellowSW AMD MI300[3] GPU IP 2:4 lightred✗ lightgreen✓ lightred✗ lightyellowSW Arm C1-Ultra (SME2)[4] CPU OP 2:4 lightyellow∼ lightgreen✓ lightred✗ lightyellowSW Arm Ethos-U85[5] NPU IP 2:4 lightred✗ lightgreen✓ lightred✗ lightred– IndexMAC[19] VPE Gust. N:M lightred✗ lightgreen✓ lightred✗ lightred– SCG[18] VPE OP Custom. lightyellow∼ lightgreen✓ lightred✗ lightred– Ventaglio (This Work) VPE Gust. N:M, Bitmap lightgreen✓ lightgreen✓ lightgreen✓ lightgreen HW, SW To address these limitations, we present Ventaglio, a runtime-configurable sparse execution unit for vector processors that elevates metadata-driven indexed accumulation to a native RVV execution primitive. Ventaglio couples the Vector Arithmetic Unit (VAU) with a multi-channel memory unit that maintains expanded data structures outside the VRF and materializes only the referenced lanes required by each sparse update, thereby sustaining full VAU throughput. Its modular scatter/gather datapath further enables a unified execution framework across different sparse encoding schemes. The contributions of this paper are: • We present the Ventaglio microarchitecture and the coupled RVV ISA extensions tailored to metadata-driven sparse execution, providing index loading, indexed multiply-accumulate, and address-post-increment semantics for scalar and vector memory operations to reduce control overhead in sparse kernels111Available at: https://github.com/pulp-platform/spatz. • We integrate Ventaglio into the open-source dual-core Spatz cluster[16] and demonstrate 6.9–7.4×6.9--7.4× speedup over the optimized RVV kernel baselines at 89.3−97.4%89.3-97.4\% VAU utilization on MS tensor contraction kernels. Implemented in 12 nm FinFET technology, the extension incurs only 3.1%3.1\% cluster-level area overhead. • We build a performance-accurate instruction-level model of the Ventaglio extension in the GVSoC simulation framework [7], calibrate it against RTL implementation, and use it to evaluate end-to-end scale-out inference on a 4×44× 4 multi-cluster system running DuoGPT–pruned [21] LLaMA-3-8B [9] model. With 40-60% dual–sparsity, Ventaglio achieves 2.06–5.25× speedup over dense inference across prefill and autoregressive decoding. I Background and Motivation Figure 1: (a) Gustavson’s algorithm for sparse tensor contraction. (b) Baseline realization with standard RVV instructions. (c) Ventaglio kernel implementation with fused indexed-multiply-accumulate extensions. (d) Ventaglio address post-increment extension for streamlined sparse updates. I-A Gustavson’s Dataflow for Sparse Tensor Contraction We consider the multiplication of a gated activation vector with a weight matrix stored in a compact sparse format. Let ~=g()∈ℝN x=g(x) ^N denote the activation vector after gating, ∈ℝN×PtileW ^N× P_tile denote the logical weight matrix, and ^∈ℝN×Pc W ^N× P_c denote its compact representation. The target computation is defined over the logical matrix as =~, -6.00006pty= xW, -3.00003pt (1) where ∈ℝPtiley ^P_tile is the dense output vector, while W together with its metadata defines the stored representation used by the hardware. This formulation generalizes naturally to matrix–matrix contraction by processing multiple activation rows in parallel, i.e., loop unrolling along the M dimension. As depicted in Fig. 1(a), for each nonzero activation xix_i, the corresponding compressed row i,:W_i,: is fetched and used to update a sparse subset of the output accumulator. Denoting by ℐiI_i the set of output coordinates referenced by the compact metadata of row i, the update can be written as yj←yj+xi⋅wi,j,∀j∈ℐi. -6.00006pty_j← y_j+x_i· w_i,j, ∀ j _i. -3.00003pt (2) Hence, activation sparsity determines when useful computation is triggered, while weight metadata determines where the corresponding partial sums must be accumulated. Fig. 1(b) illustrates the baseline realization of this dataflow using standard RVV instructions. Concretely, each sparse update consists of: A decoding the metadata into explicit indices compatible with RVV indexed operations; B loading the activation, weight values, and indices; C executing gather-accumulate-scatter through indexed load/store instructions; and D explicit pointer updates. While functionally correct, this realization exposes three fundamental inefficiencies. Standard RVV provides indexed memory operations for irregular accesses, but it does not natively interpret compressed sparse metadata. Therefore, software must first decode the sparse representation into explicit output indices before RVV can perform the corresponding gather and scatter operations, placing metadata interpretation on the software critical path. Insight 1. Sparse metadata interpretation should be supported as a native hardware mechanism rather than reconstructed repeatedly in software. Additionally, indexed accumulation is performed through generic indexed memory operations against the L1 memory system. Therefore, gather and scatter imply memory traffic rather than being handled more efficiently as core-local sparse update semantics. This inflates access overhead and increases pressure on the memory hierarchy. Insight 2. Indexed gather/scatter for sparse accumulation should be handled in-core. Moreover, the baseline flow materializes both the gathered accumulator elements and the updated sparse results as explicit vectors in VRF. This loosens the coupling between data movement and computation, limiting efficient vector chaining across sparse updates. Insight 3. Scatter/gather and arithmetic should be fused and streamlined across sparse updates. I-B Spatz Cluster Figure 2: Block diagram of the Spatz cluster with Ventaglio integration. The Ventaglio (VTL) sub-block is highlighted within the VPE, and the inset illustrates the internal microarchitecture. Spatz [16] is an open-source RVV processor designed for efficient vector computation. It is tightly coupled to a tiny single-stage RV32I Snitch core, forming a Spatz Core Complex (C). The VPE comprises a controller, a latch-based VRF, and parallel functional units: the VAU and the Vector Load-Store Unit (VLSU). The VRF contains 32 vector registers distributed across four banks, each providing three read ports and one write port (3R1W). With a vector length (VLEN) of 512 bits, each vector register spans two consecutive banks. The VAU executes arithmetic instructions using four parallel 64-bit Floating-Point Units (FPUs). Two Spatz CCs form a cluster (Fig. 2a), sharing a 128-KiB L1 Scratchpad Memory (SPM) through a logarithmic crossbar with single-cycle latency. In total, the cluster sustains 512 bits of FP computational throughput per cycle. I Ventaglio Architecture The key challenge in sustaining full VAU throughput in Gustavson’s dataflow is a bandwidth mismatch: the VAU operates on compact vectors, whereas accumulator state is accessed through a wider expanded window. If this state resides in VRF, indexed reads and writes can saturate access bandwidth, delay dependent operands or results, and disrupt vector chaining. Ventaglio addresses these limitations by decoupling the expanded sparse state from the VRF and storing it in a dedicated multi-channel memory unit with integrated gather/scatter support. Operating alongside the VAU, it provides the required asymmetric bandwidth without contending for VRF ports. I-A Memory Unit Organization and Datapaths The Ventaglio memory unit is implemented as a multi-channel latch-based standard-cell memory (SCM) with 1R1W ports and interleaved addressing. It comprises C configurable channels, each contributing one word per cycle, such that the aggregate bandwidth scales linearly to C×C×. Each channel is accessed independently, allowing a contiguous expanded access window to be assembled through parallel accesses across interleaved channels. The capacity of the Ventaglio memory unit reflects a trade-off between local buffering and hardware cost. A larger unit improves latency hiding and supports deeper loop unrolling, but increases area overhead. This trade-off is particularly tight in Gustavson-style dataflow, where unrolling along the M dimension requires multiple accumulation vectors to remain resident in expanded form, reducing the effective LMUL and the available unrolling budget. As a result, loop-control and address-generation overheads become harder to amortize, motivating the ISA support described in Sec. I-B. The gather datapath converts wide, multi-channel reads from the memory unit into the compact operand representation required by the VAU, while the scatter datapath performs the inverse mapping for write-back. Ventaglio implements gather and scatter as modular datapaths, with only one sparsity mode active at a time; inactive datapaths are gated to reduce switching activity and energy. A beat counter manages packed metadata consumption and initiates preload of the next metadata word on the current word’s final valid beat. I-B Spatz Integration and ISA Extensions Ventaglio is integrated into Spatz as a runtime-configurable sparse execution unit operating alongside the VAU. The active gather/scatter datapath is selected through Control and Status Registers (CSRs) in the Spatz controller. Ventaglio reuses the standard RVV vector-register namespace by dynamically mapping selected vector registers to its memory unit. A 32-bit CSR identifies the mapped registers, and a VRF bypass channel redirects their accesses to Ventaglio instead of the standard VRF. This preserves architectural visibility, avoids invasive scoreboard changes, and keeps indexed sparse operations transparent to the VAU. TABLE I: Operand mapping of the proposed Ventaglio extensions. Instruction Funct6 [31:26] Src2 [24:20] Src1 [19:15] Sub-op [14:12] Dst [11:7] vfxmacc.vf -, idx[4:3] wt act idx[2:0] acc vlx.v – – addr – vdst p.vleEEW.v – inc addr – vdst p.vlx.v – inc addr – vdst p.flb,h,w,d – inc addr – rdst Table I summarizes the proposed ISA extensions. The vlx.v instruction loads packed index metadata from the L1 SPM into a VRF, while vfxmacc.vf performs fused indexed multiply-accumulate and selects the corresponding index context through a 5-bit idx field split across funct6 and sub-op. Fig. 1(c) shows the resulting kernel implementation. Ventaglio further extends memory operations with address-post-increment semantics [17], removing explicit pointer-arithmetic instructions D from the critical path. The resulting streamlined kernel is shown in Fig. 1(d), and its performance is evaluated in Sec. IV-A. IV Evaluation Methodology and Results We synthesized, placed and routed the Ventaglio-integrated Spatz cluster in a 12 nm12\,nm FinFET technology. The reported implementation results refer to the configuration with C=4C=4, which provides a total Ventaglio capacity of 1 KiB1\,KiB and supports 1:4, 2:4, and bitmap sparse formats for moderate-sparsity Transformer inference. Targeting 1 GHz1\,GHz under typical conditions (T, 0.80 V0.80\,V, 25∘C), the cluster occupies 4.21 MGE4.21\,MGE, corresponding to a logic-area overhead of 3.1 %3.1\, 37 compared with the baseline Spatz cluster, while introducing no new critical path. Within the C, Ventaglio contributes 9.1 %9.1\, 37 of the total area, of which 83.6 kGE83.6\,kGE (74.3 %74.3\, 37 ) is attributed to the memory unit, 23.4 kGE23.4\,kGE (20.8 %20.8\, 37 ) to the scatter/gather datapaths, and 5.7 kGE5.7\,kGE (4.9 %4.9\, 37 ) to control logic. IV-A Kernel Benchmark Setup and Roofline Analysis We evaluate three sets of sparse tensor-contraction kernels for both matrix–matrix and matrix–vector workloads: 1 dense activations with 50 %50\, 37 sparse weights in 2:4 format; 2 dense activations with 75 %75\, 37 sparse weights in 1:4 format; and 3 50 %50\, 37 sparse activations with 60 %60\, 37 sparse weights in bitmap format. We compare three hardware configurations: the baseline Spatz cluster; the Ventaglio-integrated Spatz cluster with the proposed vfxmacc extension; and the same design further augmented with address-post-increment support. The representative logical kernel size is 256(1)×256×256256(1)× 256× 256. All kernel benchmarks are simulated using QuestaSim 2022.3. Figure 3: Roofline analysis of sparse tensor contraction kernels. Circled numbers denote workload configurations. Presented in Fig. 3, the baseline Spatz reaches near-roofline VPU utilization for dense workloads, but performs well below the roofline on sparse workloads due to software metadata translation and L1-backed indexed gather/scatter. Ventaglio removes these bottlenecks by supporting metadata-driven indexed accumulation in hardware, bringing most sparse kernels close to the roofline. The remaining gap appears in two cases. First, as the weight matrix becomes sparser, Gustavson-style execution must maintain more output accumulation states, increasing memory-unit pressure and forcing a smaller LMUL configuration. The resulting shorter vector operations make explicit pointer updates more costly. Second, sparse activations require additional control instructions to select nonzero activation elements. Because the memory unit is capacity-constrained, these overheads cannot be fully amortized by vector operations. The address-post-increment extension reduces this residual overhead, bringing all evaluated workloads to the roofline and improving performance by 6.96.9–7.4×7.4× over baseline Spatz. IV-B End-to-End Evaluation Setup and Results We build a performance-accurate instruction-level model of the Ventaglio extension in GVSoC[7], calibrate it against the RTL implementation with a maximum performance discrepancy of 2.3%, and scale the Ventaglio-integrated Spatz cluster to a 4×44× 4 multi-cluster system connected through a 2D NoC. Each cluster interface exposes a 1024-bit network link and supports collective communication primitives, including row-wise and column-wise multicast and sum reduction. Off-chip memory is modeled in DRAMSys[10] as 4×24× 2 HBM2 channels distributed along the west and south edges of the array, for an aggregate peak bandwidth of 512 GB/s. We perform end-to-end inference evaluation on the LLaMA-3-8B model in FP16 precision for both prefill (S=512, 1K, 2K) and autoregressive decoding (S=1; KV=512, 1K, 2K). Fig. 4 compares dense execution against a DuoGPT-pruned LLaMA-3-8B model with practical 40–60% dual-sparsity, i.e., zero-element ratios in both weights and activations. During prefill, matrix–matrix projections dominate execution, accounting for over 95% of the layer runtime and making the stage compute-bound. Activation and weight sparsity therefore reduce the FLOP cost directly, while Ventaglio sustains at-the-roofline efficiency on the resulting sparse kernels. This yields a 2.402.40–5.25×5.25× speedup over the dense baseline. During autoregressive decoding, matrix–vector projections are memory-bound because each cluster must load its corresponding weight tile, leaving no weight reuse across clusters. Weight sparsity reduces HBM traffic, while the activation-sparsity-aware SUMMA dataflow transfers only the weight vectors corresponding to nonzero activation indices. However, the resulting sparse access pattern creates short, disaggregated DMA bursts that reduce bandwidth utilization. Together, we achieve a 2.062.06–3.16×3.16× speedup, translating Ventaglio-enabled at-the-roofline sparse tensor contraction and reduced HBM traffic into a sizable end-to-end acceleration. Figure 4: End-to-end runtime breakdown for LLaMA-3-8B inference across prefill and autoregressive decode stages. Dense baseline is normalized to 1.0 per case. Stars indicate HBM BW utilization (right axis). V Conclusion Ventaglio enables efficient moderate-sparsity tensor contractions by elevating metadata-driven indexed accumulation to a native RVV execution primitive. Integrated into Spatz, it delivers 6.9–7.4× kernel speedup at low area cost, and achieves 2.06–5.25× end-to-end acceleration for sparse LLaMA-3 inference. Ventaglio shows that low-overhead hardware enhancements coupled with ISA support can deliver a significant performance and efficiency boost for sparse tensor computation. References [1] A. Aghajanyan, L. Yu, A. Conneau, W. Hsu, K. Hambardzumyan, S. Zhang, S. Roller, N. Goyal, O. Levy, and L. Zettlemoyer (2023) Scaling laws for generative mixed-modal language models. In International Conference on Machine Learning, p. 265–279. Cited by: §I. [2] Y. Akhauri, A. F. AbouElhamayed, J. Dotzel, Z. Zhang, A. M. Rush, S. Huda, and M. S. Abdelfattah (2024) ShadowLLM: Predictor-based contextual sparsity for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 19154–19167. Cited by: §I. [3] AMD (2025) AMD Instinct MI300 Instruction Set Architecture Reference Guide. Technical report Advanced Micro Devices. External Links: Link Cited by: TABLE I, §I. [4] Arm Ltd. (2025)C3.11.19 structured sparsity outer product(Website) Note: Arm Architecture Reference Manual for A-profile architecture, accessed 2026-04-10 External Links: Link Cited by: TABLE I, §I. [5] Arm (2024)Arm Ethos-U NPU Hardware Architecture(Website) External Links: Link Cited by: TABLE I, §I. [6] G. Bai, Y. Li, C. Ling, K. Kim, and L. Zhao (2024) SparseLLM: Towards global pruning of pre-trained language models. Advances in Neural Information Processing Systems 37, p. 46203–46225. Cited by: §I. [7] N. Bruschi, G. Haugou, G. Tagliavini, F. Conti, L. Benini, and D. Rossi (2021) GVSoC: a highly configurable, fast and accurate full-platform simulator for RISC-V based IoT processors. In 2021 IEEE 39th International Conference on Computer Design (ICCD), p. 409–416. Cited by: 3rd item, §IV-B. [8] D. Dabbelt, C. Schmidt, E. Love, H. Mao, S. Karandikar, and K. Asanovic (2016) Vector processors for energy-efficient embedded systems. In Proceedings of the Third ACM International Workshop on Many-Core Embedded Systems, p. 10–16. Cited by: §I. [9] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. (2024) The LLaMA 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: 3rd item. [10] M. Jung, C. Weis, and N. Wehn (2015) DRAMSys: A flexible DRAM subsystem design space exploration framework. IPSJ Transactions on System and LSI Design Methodology 8, p. 63–74. Cited by: §IV-B. [11] J. Liu, P. Ponnusamy, T. Cai, Y. Kim, B. Athiwaratkun, et al. (2025) Training-free activation sparsity in large language models. In International Conference on Learning Representations, Vol. 2025, p. 98302–98322. Cited by: §I. [12] J. Liu, Z. Kong, P. Zhao, C. Yang, X. Shen, H. Tang, G. Yuan, W. Niu, W. Zhang, X. Lin, et al. (2025) Toward adaptive large language models structured pruning via hybrid-grained weight importance assessment. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 18879–18887. Cited by: §I. [13] NVIDIA (2020) NVIDIA A100 Tensor Core GPU Architecture. Technical report NVIDIA. External Links: Link Cited by: §I. [14] NVIDIA (2022)NVIDIA Hopper Architecture In-Depth(Website) External Links: Link Cited by: §I. [15] NVIDIA (2025)Blackwell SM100 GEMMs(Website) External Links: Link Cited by: TABLE I, §I. [16] M. Perotti, S. Riedel, M. Cavalcante, and L. Benini (2025) Spatz: Clustering compact RISC-V-based vector units to maximize computing efficiency. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 44 (7), p. 2488–2502. Cited by: 2nd item, §I, §I-B. [17] F. Schuiki, F. Zaruba, T. Hoefler, and L. Benini (2020) Stream semantic registers: A lightweight RISC-V ISA extension achieving full compute utilization in single-issue cores. IEEE Transactions on Computers 70 (2), p. 212–227. Cited by: §I-B. [18] X. Tengfei, F. Zhihua, X. Jing, Q. Shantian, Y. Xiaochun, and L. Wenming (2026) RISC-V ISA Extensions for Vectorized Unstructured Sparse SpMM in LLM Inference. In 2026 Design, Automation & Test in Europe Conference & Exhibition (DATE), p. 1–6. Cited by: TABLE I, §I. [19] V. Titopoulos, K. Alexandridis, C. Peltekis, C. Nicopoulos, and G. Dimitrakopoulos (2025) Optimizing structured-sparse matrix multiplication in RISC-V vector processors. IEEE Transactions on Computers 74 (4), p. 1446–1460. Cited by: TABLE I, §I. [20] Y. Yang, K. Zhen, B. Ganesh, A. Galstyan, G. Huybrechts, M. Müller, J. M. Kübler, R. V. Swaminathan, A. Mouchtaris, S. B. Bodapati, et al. (2025) Wanda++: pruning large language models via regional gradients. In Findings of the Association for Computational Linguistics: ACL 2025, p. 4321–4333. Cited by: §I. [21] R. Yin, Y. Li, D. Lee, and P. Panda (2026) DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs. Advances in Neural Information Processing Systems 38, p. 146883–146912. Cited by: 3rd item. [22] Y. Zhang, L. Zhao, M. Lin, S. Yunyun, Y. Yao, X. Han, J. Tanner, S. Liu, and R. Ji (2024) Dynamic sparse no training: training-free fine-tuning for sparse llms. In International Conference on Learning Representations, Vol. 2024, p. 249–264. Cited by: §I.

Instant research discovery