Paper deep dive
CheckOne: Lightweight Fault Detection and Mitigation for Vision Transformers
Mohammad Hasan Ahmadilivani, Sven-Markus Loorits, Jaan Raik
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 8/9/2026, 2:21:17 AM
Summary
The paper introduces CheckOne, a lightweight fault detection and mitigation method for Vision Transformers (ViTs) that addresses the high computational overhead of conventional Algorithm-Based Fault Tolerance (ABFT). CheckOne reduces the critical Soft Error Rate (SDR) by up to 26x compared to unprotected execution and achieves a 3.8x performance improvement over traditional ABFT methods by using implicit checksum calculations via appended vectors.
Entities (10)
Relation Signals (7)
CheckOne → improvesperformanceover → Algorithm-Based Fault Tolerance
confidence 95% · CheckOne ... achieves an average 3.8x higher performance than ABFT in ViTs.
CheckOne → mitigatesfaultsin → Vision Transformers
confidence 95% · CheckOne ... method for fault detection and mitigation in ViTs
Vision Transformers → isvulnerableto → hardware faults
confidence 90% · Vision Transformers (ViTs) are highly vulnerable to hardware-induced faults.
CheckOne → reducescriticalfaultrateby → 26x
confidence 90% · CheckOne mitigates critical faults by up to 26x
CheckOne → isevaluatedon → ImageNet
confidence 85% · Resilience evaluation is performed using 5,000 images from the ImageNet validation dataset.
CheckOne → isexecutedon → NVIDIA A100 GPU
confidence 85% · All experiments are implemented in PyTorch and executed on an NVIDIA A100 GPU.
CheckOne → isimplementedin → PyTorch
confidence 85% · All experiments are implemented in PyTorch
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The wide adoption of Vision Transformers (ViTs) in safety-critical applications raises reliability concerns related to hardware faults. Algorithm-Based Fault Tolerance (ABFT) methods have emerged as lightweight and symmetric protection mechanisms for DNNs. However, they are particularly challenging for ViTs due to their significant computational requirements. This work comprehensively evaluates the reliability of ViTs, emphasizing the need for symmetric protection in their layers. Furthermore, we present CheckOne, a novel, cost-effective method for fault detection and mitigation in ViTs that significantly reduces the computational cost compared to conventional ABFT. Through extensive experiments with multiple ViTs, CheckOne mitigates critical faults by up to $26\times$ and achieves an average 3.8x higher performance than ABFT in ViTs.
Tags
Links
- Source: https://arxiv.org/abs/2608.04035v1
- Canonical: https://arxiv.org/abs/2608.04035v1
PDF not stored locally. Use the link above to view on the source site.
Full Text
42,539 characters extracted from source content.
Expand or collapse full text
CheckOne: Lightweight Fault Detection and Mitigation for Vision Transformers †thanks: This paper is supported in part by EU Grant Project 101160182 “TAICHIP”, and the EU Grant 101194287 “NexTArc”. Mohammad Hasan Ahmadilivani1, Sven-Markus Loorits1, and Jaan Raik1 Abstract The wide adoption of Vision Transformers (ViTs) in safety-critical applications raises reliability concerns related to hardware faults. Algorithm-Based Fault Tolerance (ABFT) methods have emerged as lightweight and symmetric protection mechanisms for DNNs. However, they are particularly challenging for ViTs due to their significant computational requirements. This work comprehensively evaluates the reliability of ViTs, emphasizing the need for symmetric protection in their layers. Furthermore, we present CheckOne, a novel, cost-effective method for fault detection and mitigation in ViTs that significantly reduces the computational cost compared to conventional ABFT. Through extensive experiments with multiple ViTs, CheckOne mitigates critical faults by up to 26×26× and achieves an average 3.8×3.8× higher performance than ABFT in ViTs. I Introduction The rapid evolution of Deep Neural Networks (DNNs) continuously introduces both new opportunities and challenges. Transformer architectures have emerged as a central focus recently [17]. Their superior performance is appealing for edge-based, safety-critical applications [12]; however, reliability and efficiency remain significant challenges. The substantial computational and memory demands of these models [15], combined with the complexity of their reliability assessment and overheads associated with redundancy-based fault tolerance [1], further complicate their practical edge deployment. Recent studies have demonstrated that Vision Transformers (ViTs) are highly vulnerable to hardware-induced faults. Beam experiments have revealed that soft errors can severely degrade the inference accuracy of ViT models [4, 14]. Simulation-based Fault Injection (FI) further confirms that transient faults affecting memories can propagate through transformer architectures, leading to substantial output corruption [8, 3, 2]. Although these studies collectively establish the susceptibility of ViTs to hardware faults, they provide limited insight into the relative criticality of individual ViT components and the practical implications of single bit-flips during deployment. To address this gap, this work presents a comprehensive investigation into the fault criticality of ViTs, through an extensive layer-wise analysis. Our observations reveal that vulnerability is relatively uniform across transformer layers and blocks, indicating that selective protection strategies may offer limited effectiveness and motivating the need for symmetric fault tolerance mechanisms for reliable ViT deployment. a) FLOPs overhead b) Execution time overhead Figure 1: Overhead of conventional ABFT in ViT models on GPU. a) Theoretical operations overhead, b) execution overhead on GPU normalized to baseline ViTs execution. Algorithm-Based Fault Tolerance (ABFT) techniques have emerged as lightweight and symmetric protection mechanisms for DNNs by augmenting matrix multiplication (GEMM) operations with checksum-based error detection and correction schemes [18, 13]. These approaches typically introduce redundant checksum vectors for input and weight matrices and verify the integrity of the output through checksum recomputation during inference. Building upon this principle, several recent studies have proposed ABFT-oriented reliability mechanisms tailored to ViTs, incorporating architectural optimizations and transformer-specific adaptations to improve fault coverage and resilience [9, 11, 7, 5, 10, 16]. Nevertheless, the exceptionally large matrix dimensions and intensive memory traffic characteristic of transformer architectures substantially limit the efficiency of existing ABFT implementations. In particular, checksum generation and verification for input, weight, and output matrices introduce significant memory-bound overheads, despite requiring relatively few additional arithmetic operations. As illustrated in Fig. 1, although the computational overhead remains below 1.5%1.5\%, the resulting inference latency can increase dramatically, reaching up to 2.07×2.07× longer execution time due to the dominance of memory-access costs during checksum processing. To tackle this drawback, this work, for the first time, presents CheckOne, an innovative method for on-the-fly calculation of checksums for efficient fault detection and removes the need for checksum computations for each matrix. It implicitly produces the summations through constant added vectors and compares the obtained summations with stored values in memory. The contributions of this work are as follows: • Presenting a comprehensive FI analysis of ViTs targeting both weights and activations to characterize the reliability of transformer blocks and linear layers. • Proposing CheckOne, a novel and cost-efficient fault detection and mitigation mechanism for ViTs that reduces the critical SDC rate by up to 26×26× compared to unprotected execution while achieving an average 3.8×3.8× performance improvement over conventional ABFT. I Methodology I-A Fault Model With transistor scaling, SRAM-based on-chip memories become highly vulnerable to radiation-induced transient faults that manifest as bit-flips [6]. In this work, we focus on single transient faults occurring in on-chip memories of DNN accelerators that affect parameter and activation matrices during inference. To model this behavior at the application level, a single bit-flip is randomly injected into either weights or input activations of linear layers. We assume that off-chip memory is protected by ECC mechanisms, correcting single-bit faults; consequently, pre-stored values remain error-free. I-B Reliability Evaluation We perform random Fault Injection (FI) campaigns targeting the input activations and weights of ViTs, for each individual layer, independently. During each inference, a single random bit is flipped in a random weight or input activation value represented in 32-bit floating-point. Resilience evaluation is performed using 5,0005,000 images from the ImageNet validation dataset. Each FI experiment is repeated 1,0001,000 times. To quantify resilience, we adopt two metrics: 1) average accuracy degradation, i.e., the difference between the baseline accuracy and the average accuracy across FI campaigns, and 2) critical SDC rate, i.e., the ratio of inference outputs whose predicted class differs from the corresponding fault-free classification. The experimental evaluation is conducted using three pre-trained ViT models on ImageNet: ViT-Tiny, DeiT-Tiny, and Swin-Tiny, all obtained from the timm library in PyTorch. Their baseline accuracies are 75.78%75.78\%, 72.16%72.16\%, and 81.48%81.48\%, respectively. All experiments are implemented in PyTorch and executed on an NVIDIA A100 GPU. I-C CheckOne: Lightweight Fault Detection and Mitigation The key idea in CheckOne is to eliminate the need for recomputation in the output matrices after a GEMM operation in transformers and detect errors with minimal computational overhead. CheckOne enables in-place computations of checksums by appending a vector of 11s to the input rows and weight columns before a matrix multiplication in linear layers of a ViT, resulting in obtaining their summations by the GEMM computation. Thereafter, the CheckOne method employs pre-computed weights and output activation range values to perform on-the-fly lightweight fault detection and localization, replacing errors with 0. CheckOne consists of two main phases: 1) Offline Phase, where the structure of a ViT is modified, and the detection values are obtained, and 2) Online Phase, where the fault detection and mitigation are conducted during inference. Note that CheckOne specifically targets single bit-flips in the inputs and weights of linear layers. In the Offline Phase, the golden values for fault detection in the Online Phase are obtained using validation data. For each linear layer, the weights and inputs are organized as a 2D matrix for a GEMM operation, and two golden sets are exported: 1) golden weight sums 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu Swgoldenl S^l_w_golden , i.e., the column-wise summation of the weights (w) for a linear layer l, 2) golden input range [Min(Sxgoldenl),Max(Sxgoldenl)][Min(S^l_x_golden),Max(S^l_x_golden)], i.e., the minimum and maximum input values (X) to a linear layer l. All values are stored and assumed to be fault-free during inference. In the Online Phase, the error detection and mitigation are conducted as shown in Fig. 2. A linear layer l performs a matrix multiplication between 2D arrays of input activations Xl∈ℝm×n # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ ^m× n and weights Wl∈ℝn×k # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ ^n× k, producing output matrix Ol∈ℝm×k # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ ^m× k, while m, n, and k represent the matrix dimensions. Ol=Xl×Wl # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ O^l $ = # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ X^l $ × # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ W^l $ (1) In CheckOne, a vector 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu 1 is appended to the rows of inputs and to the columns of weights, as shown in 2. The modified inputs Xl~→∈ℝ(m+1)×n X^l ^(m+1)× n and weights Wl~→∈ℝn×(k+1) W^l ^n×(k+1) are multiplied, resulting in an output matrix Ol~→ O^l, in which the last row contains the column-wise summation of weight matrix (Swl→ S^l_w), and the last column contains the row-wise summation of input activations (Sxl→ S^l_x). The bottom right element of the matrix (S_ 1) contains the multiplication and accumulation of 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu 1 , which is equal to the dimension n. Ol~→=Xl~→×Wl~→=[Xl→]×[Wl→,]=[Ol→,Sxl→Swl→,S] O^l= X^l× W^l= bmatrix X^l\\ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ bmatrix× bmatrix W^l, # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ # -0.7pt $ 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu $ -0.3pt $ 1 $ bmatrix= bmatrix O^l, S^l_x\\ S^l_w,S_ 1 bmatrix (2) To detect and mitigate faults in the input matrices of a GEMM operation at the output matrix Ol~→ O^l, first, the vector Swl→ S^l_w is compared element-wise with 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu Swgoldenl S^l_w_golden . In the case of a mismatch, the corresponding obtained column in Ol~→ O^l is erroneous; thus, set to 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu 0 to remove the effect of the faulty weight. Then, the vector Sxl→ S^l_x is compared with the range vector [Min(Sxgoldenl),Max(Sxgoldenl)][Min(S^l_x_golden),Max(S^l_x_golden)]. In the case that a value exceeds the range, the corresponding row in Ol~→ O^l is erroneous; thus, set to 2.0mu $ $ $ $ $ $ $ $ -1.5mu $ 0.0mu $ $ $ $ $ $ $ $ 0.0mu$ -1.5mu 0 to remove the effect of the faulty input activation. Figure 2: CheckOne method for each linear layer. I-D ABFT Implementation We implemented a conventional ABFT scheme for ViTs as a baseline for comparison. In this implementation, weight checksums are precomputed offline, stored in memory, and attached to the corresponding layers during inference, similarly to CheckOne. In contrast, input and output checksums are generated dynamically at runtime. During inference, the output checksums are recomputed and compared against the expected checksum values derived from the stored weights. In the case of a mismatch, the erroneous output values and replaces with zero. In this approach, faults occurring in input activations cannot be detected. Consequently, the reliability evaluation of the baseline ABFT method is restricted to FI campaigns targeting model weights and is compared with CheckOne. I Experiments I-A Reliability Evaluation a) ViT-tiny FI into weight b) ViT-tiny FI into input activations c) DeiT-tiny FI into weight d) DeiT-tiny FI into input activations e) Swin-tiny FI into weight f) Swin-tiny FI into input activations Figure 3: Accuracy degradation for ViT-tiny, DeiT-tiny, and Swin-tiny with layer-wise FI into weights and input activations. Fig. 3 presents the average accuracy degradation obtained from layer-wise FI experiments. The results reveal that despite containing millions of parameters, ViTs are highly sensitive to even a single bit-flip in either weights or activations, leading to accuracy degradation of up to 3.41%3.41\%. This observation emphasizes the necessity of effective fault protection mechanisms for transformer-based architectures. Across most transformer blocks, the QKV weights exhibit comparatively lower vulnerability, whereas the sensitivity of other layers varies significantly across models and blocks, preventing the identification of a consistent vulnerability pattern suitable for selective protection. In contrast, activation vulnerability remains relatively uniform across layers and blocks, although the final transformer blocks, particularly the FC2 layers in Swin-Tiny, demonstrate slightly higher resilience. Overall, considering both weight and activation FI, the vulnerability of ViT layers remains consistently high without a clearly dominant subset of critical layers. Moreover, layers exhibiting low sensitivity in activations often remain highly vulnerable in their weights, and vice versa. These findings indicate that reliability enhancement techniques for ViTs should provide uniform and symmetric protection across all transformer layers and blocks rather than relying on selective hardening strategies. Table I reports the average critical SDC rate for each linear layer within the transformer blocks, averaged across all blocks. The results show that, although QKV weights are generally the least vulnerable parameters, the corresponding activations exhibit the highest vulnerability. Moreover, the critical SDC rates across different layers and models remain relatively close, indicating a uniformly high sensitivity to faults throughout the transformer architecture. These observations further confirm that effective reliability enhancement mechanisms for ViTs must provide symmetric protection across all layers, simultaneously covering both weights and activations. TABLE I: Average Critical SDC (%) per linear layer, across all blocks of each unprotected ViT model. FI Model QKV Proj FC1 FC2 Weight ViT-tiny 1.83% 3.52% 2.77% 3.29% DeiT-tiny 1.65% 3.15% 3.04% 2.93% Swin-tiny 0.97% 3.25% 2.69% 2.68% Activation ViT-tiny 3.39% 2.90% 2.71% 2.84% DeiT-tiny 3.44% 2.88% 2.66% 2.86% Swin-tiny 3.24% 2.67% 2.60% 1.99% TABLE I: Comparative results for accuracy degradation and critical SDC under FI into weights and input activations across ViT models. Accuracy Degradation (%) Critical SDC (%) Weight FI Activation FI Weight FI Activation FI ViT-Tiny DeiT-Tiny Swin-Tiny ViT-Tiny DeiT-Tiny Swin-Tiny ViT-Tiny DeiT-Tiny Swin-Tiny ViT-Tiny DeiT-Tiny Swin-Tiny Unprotected 2.14%2.14\% 1.92%1.92\% 1.92%1.92\% 2.22%2.22\% 2.12%2.12\% 2.09%2.09\% 2.85%2.85\% 2.69%2.69\% 3.39%3.39\% 2.96%2.96\% 2.96%2.96\% 2.62%2.62\% ABFT <0.01%<0.01\% 0.05% 0.92%0.92\% - - - 0.47% 0.46% 1.3%1.3\% - - - CheckOne <0.01%<0.01\% 0.05% <0.01%<0.01\% 0.07% 0.06% 0.10% 0.70% 0.59% 0.13% 0.18% 0.17% 0.17% I-B Impact of CheckOne on Reliability and Performance Table I summarizes the experimental results for unprotected ViTs, conventional ABFT, and CheckOne under both weight and activation FI campaigns. The results demonstrate that both ABFT and CheckOne significantly improve the resilience of ViTs against bit-flips in weights compared to unprotected execution. Overall, CheckOne achieves lower accuracy degradation than conventional ABFT, particularly for Swin-Tiny, while maintaining accuracy degradation consistently below 0.05%0.05\% across all evaluated models. Although the critical SDC rate of CheckOne is slightly higher than that of ABFT in some cases, it remains competitive. These observations indicate that CheckOne achieves reliability comparable to conventional ABFT for protecting model weights. Unlike conventional ABFT, CheckOne can also effectively protect input activations against single bit-flips. Across all evaluated ViT models, the observed accuracy degradation under activation FI remains below 0.1%0.1\%, while the critical SDC rate is consistently lower than 0.18%0.18\%. Compared to unprotected ViTs, CheckOne reduces the critical SDC rate by up to 26×26× under weight FI and up to 17×17× under activation FI. Overall, these results confirm that CheckOne provides efficient and consistent protection for both weights and activations against soft errors in transformer-based architectures. Beyond its reliability advantages, CheckOne also delivers substantially higher performance efficiency compared to conventional ABFT, primarily by eliminating the costly recomputation of checksums during inference. Performance evaluations conducted on an NVIDIA A100 GPU over 100100 inference runs using ViT-Tiny, DeiT-Tiny, and Swin-Tiny demonstrate that CheckOne achieves execution speedups of 4.67×4.67×, 4.72×4.72×, and 2.02×2.02×, respectively, relative to the conventional ABFT implementation. Overall, CheckOne provides an average performance improvement of 3.8×3.8× while maintaining a comparable level of resilience against soft errors. Note that the additional memory overhead introduced by CheckOne is negligible for storing precomputed values, accounting for less than 0.3%0.3\% of the total memory footprint of the evaluated ViT models. The evaluation results indicate that CheckOne is a significantly more practical and efficient fault tolerance solution for transformer-based architectures. IV Conclusions This work presents CheckOne, an innovative, cost-effective method for protecting ViTs against faults in on-chip memories during inference. In CheckOne, the inputs to a matrix multiplication are protected by a single additional vector of 11, implicitly producing their summations at the output matrix. Utilizing the pre-stored detection values, CheckOne conducts on-the-fly error detection and mitigation. Results indicate that CheckOne effectively improves the reliability of ViTs by reducing critical faults by up to 26×26× while being 3.8×3.8× faster than conventional ABFT, on average. References [1] M. H. Ahmadilivani et al. (2024) A systematic literature review on hardware reliability assessment methods for deep neural networks. ACM Computing Surveys 56 (6), p. 1–39. Cited by: §I. [2] M. H. Ahmadilivani et al. (2026) Effective and memory-efficient alternatives to ecc for reliable large-scale dnns. In 2026 IEEE 32nd International Symposium on On-Line Testing and Robust System Design (IOLTS), Cited by: §I. [3] M. H. Ahmadilivani et al. (2026) Late breaking results: uncovering the limits of eccs in vision transformers and a zero-cost reliability enhancement. In 2026 DATE, p. 1–3. Cited by: §I. [4] J. M. Badia et al. (2025) Reliability of vision transformers and cnns on edge ai systems under neutron radiation. IEEE Transactions on Nuclear Science. Cited by: §I. [5] H. Dai et al. (2025) FT-transformer: resilient and reliable transformer with end-to-end fault tolerant attention. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, p. 1085–1098. Cited by: §I. [6] I. Hill et al. (2021) CMOS reliability from past to future: a survey of requirements, trends, and prediction methods. IEEE Transactions on Device and Materials Reliability 22 (1), p. 1–18. Cited by: §I-A. [7] Y. Liang et al. (2025) ATTNChecker: highly-optimized fault tolerant attention for large language model training. In Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, p. 252–266. Cited by: §I. [8] E. Liao and T. Wang (2025) Analyzing and enhancing the reliability of vision transformer models against soft errors. In 2025 IEEE International Symposium on Circuits and Systems, p. 1–5. Cited by: §I. [9] H. Liu et al. (2024) ALBERTA: algorithm-based error resilience in transformer architectures. IEEE Open Journal of the Computer Society 6, p. 85–96. Cited by: §I. [10] K. Ma et al. (2023) Error resilient transformers: a novel soft error vulnerability guided approach to error checking and suppression. In 2023 IEEE European Test Symposium (ETS), p. 1–6. Cited by: §I. [11] K. Ma et al. (2026) Error resilient transformers: a novel soft error vulnerability guided approach to error checking and suppression. Journal of Electronic Testing, p. 1–22. Cited by: §I. [12] T. Mondal and A. D. Jagtap (2026) In transformer we trust? a perspective on transformer architecture failure modes. arXiv preprint arXiv:2602.14318. Cited by: §I. [13] J. E. Rodriguez Condia et al. (2026) FT-sparse: algorithm-based fault tolerance for sparse cnns using structured sparsity in gpus. In IEEE VLSI Test Symposium (VTS), p. 1–7. Cited by: §I. [14] L. Roquet et al. (2024) Cross-layer reliability evaluation and efficient hardening of large vision transformers models. In Proceedings of the 61st ACM/IEEE DAC, p. 1–6. Cited by: §I. [15] Y. Tay et al. (2022) Efficient transformers: a survey. ACM Computing Surveys 55 (6), p. 1–28. Cited by: §I. [16] V. Titopoulos et al. (2025) Custom algorithm-based fault tolerance for attention layers in transformers. In 2025 IEEE 38th International System-on-Chip Conference (SOCC), p. 1–6. Cited by: §I. [17] A. Vaswani et al. (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: §I. [18] K. Zhao et al. (2020) FT-cnn: algorithm-based fault tolerance for convolutional neural networks. IEEE Transactions on Parallel and Distributed Systems 32 (7), p. 1677–1689. Cited by: §I.