Paper deep dive
From Foundation ECG Models to NISQ Learners: Distilling ECGFounder into a VQC Student
Giovanni dos Santos Franco, Felipe Mahlow, Ellison Fernando Cardoso, Felipe Fanchini
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/31/2026, 2:02:17 AM
Summary
This paper investigates knowledge distillation of the ECGFounder foundation model into compact student models, including classical (ResNet-1D, CNN-1D) and quantum-ready (convolutional autoencoder + 6-qubit VQC) architectures, for binary ECG classification on PTB-XL and MIT-BIH datasets. The study demonstrates that distilled students can achieve competitive performance with significant parameter reduction, highlighting accuracy-efficiency trade-offs in resource-constrained environments.
Entities (6)
Relation Signals (3)
ECGFounder → evaluatedon → PTB-XL
confidence 100% · fine-tune ECGFounder as a high-capacity teacher for binary ECG classification on PTB-XL
ECGFounder → teaches → ResNet-1D
confidence 90% · distill the resulting teacher into compact students
ECGFounder → teaches → Variational Quantum Circuit
confidence 90% · distill the resulting teacher into compact students... including a quantum-ready student
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Foundation models have recently improved electrocardiogram (ECG) representation learning, but their deployment can be limited by computational cost and latency constraints. In this work, we fine-tune ECGFounder as a high-capacity teacher for binary ECG classification on PTB-XL and the MIT-BIH Arrhythmia Database, and investigate whether knowledge distillation can transfer its predictive behavior to compact students. We evaluate two classical 1D students (ResNet-1D and a lightweight CNN-1D) and a quantum-ready pipeline that combines a convolutional autoencoder, which compresses 256-sample ECG windows into a low-dimensional latent representation, with a 6-qubit variational quantum circuit implemented in Qiskit and executed in a simulated backend. Across both datasets, the teacher provides the strongest overall performance, while distillation yields competitive students under a considerable reduction in trainable parameters. We further analyze the sensitivity of student performance to distillation settings, highlighting consistent accuracy--efficiency trade-offs when compressing a foundation ECG model into classical and quantum-ready learners under a unified evaluation protocol.
Tags
Links
- Source: https://arxiv.org/abs/2603.27269v1
- Canonical: https://arxiv.org/abs/2603.27269v1
Trouble viewing inline? Open PDF directly →
Full Text
37,868 characters extracted from source content.
Expand or collapse full text
From Foundation ECG Models to NISQ Learners: Distilling ECGFounder into a VQC Student Giovanni dos Santos Franco 1* , Felipe Mahlow 1 , Ellison Fernando Cardoso 2,3 , Felipe Fanchini 2,1* 1 Faculty of Sciences, S ̃ao Paulo State University (Unesp), Bauru, Brazil. 2 Hospital Israelita Albert Einstein, S ̃ao Paulo, Brazil. 3 Laboratory of Magnetic Resonance in Neuroradiology (LIM-44), University of S ̃ao Paulo Faculty of Medicine Clinics Hospital, S ̃ao Paulo, Brazil. *Corresponding author(s). E-mail(s): gs.franco@unesp.br; felipe.fanchini@einstein.br; Contributing authors: f.mahlow@unesp.br; Ellison.Cardoso@einstein.br; Abstract Foundation models have recently improved electrocardiogram (ECG) represen- tation learning, but their deployment can be limited by computational cost and latency constraints. In this work, we fine-tune ECGFounder as a high-capacity teacher for binary ECG classification on PTB-XL and the MIT-BIH Arrhyth- mia Database, and investigate whether knowledge distillation can transfer its predictive behavior to compact students. We evaluate two classical 1D students (ResNet-1D and a lightweight CNN-1D) and a quantum-ready pipeline that com- bines a convolutional autoencoder, which compresses 256-sample ECG windows into a low-dimensional latent representation, with a 6-qubit variational quantum circuit implemented in Qiskit and executed in a simulated backend. Across both datasets, the teacher provides the strongest overall performance, while distillation yields competitive students under a considerable reduction in trainable param- eters. We further analyze the sensitivity of student performance to distillation settings, highlighting consistent accuracy–efficiency trade-offs when compress- ing a foundation ECG model into classical and quantum-ready learners under a unified evaluation protocol. 1 arXiv:2603.27269v1 [quant-ph] 28 Mar 2026 1 Introduction Electrocardiography (ECG) remains a primary, non-invasive modality for assessing cardiac function, and the increasing availability of large-scale digital ECG archives has accelerated the development of data-driven methods for automated interpreta- tion. [1] Deep learning, particularly convolutional neural networks, has demonstrated strong performance for rhythm and arrhythmia recognition, supporting the feasibility of learning-based ECG decision systems in clinically relevant settings. [2] This progress has been further enabled by openly available benchmarks that permit transparent and reproducible evaluation across studies and cohorts. Among these, PTB-XL provides a large, clinically annotated dataset, while the MIT-BIH Arrhythmia Database remains a long-established reference for arrhythmia detection and classification. [3, 4] More recently, representation learning via foundation-model pretraining has emerged as a principled strategy to leverage massive corpora and obtain transfer- able features that can be adapted to downstream tasks through fine-tuning. [5] ECGFounder operationalizes this paradigm by pretraining on over ten million ECG recordings with broad label coverage and by emphasizing external evaluation across multiple clinical settings. [6] Despite consistent gains, the deployment of high-capacity models may be limited by memory footprint, computational cost, and latency require- ments, particularly in resource-constrained environments. [7] Knowledge distillation (KD) provides a practical route to address these constraints by transferring a teacher’s predictive behavior to smaller students, typically via softened targets that convey informative inter-class structure beyond one-hot supervision; distillation can also be strengthened through intermediate representation matching when student capacity is highly constrained. [7, 8] In parallel, KD has begun to be explored in quantum machine learning as a mechanism to transfer knowledge across heterogeneous model classes under explicit quantum resource constraints, including classical-to-quantum distilla- tion using frozen classical teachers and strategies that compress quantum models while preserving their behavior. [9, 10] In this study, we fine-tune ECGFounder as a high-capacity teacher for binary ECG classification on PTB-XL and the MIT-BIH Arrhythmia Database, and then distill the resulting teacher into compact students to quantify accuracy–efficiency trade-offs across learning paradigms. [6, 3, 4, 7] We consider two classical 1D students (a compact ResNet-style model and a lightweight CNN) and a quantum-ready student based on a convolutional autoencoder coupled to a shallow variational quantum circuit, enabling a controlled comparison under a unified evaluation protocol. [11, 12, 13] The main contributions of this work are: • To distill a state-of-the-art ECG foundation model (ECGFounder) into compact student models for binary ECG classification on PTB-XL and MIT-BIH, quantifying the resulting accuracy–efficiency trade-offs; • To assess the performance of a VQC-based student within this distillation frame- work, using a quantum-ready pipeline (convolutional autoencoder + 6-qubit VQC) under a controlled evaluation protocol. The remainder of this paper is organized as follows. Section 2 reviews the back- ground on ECGFounder, knowledge distillation, and variational quantum circuits. 2 Section 3 details the proposed methodology, including preprocessing, teacher fine- tuning, student architectures, and the training and evaluation protocol. Section 4 presents the quantitative results and analyzes the impact of distillation settings. Finally, Section 5 concludes the paper and outlines directions for future work. 2 Theoretical Background 2.1 ECGFounder ECGFounder is a foundation model for electrocardiography (ECG) designed to learn transferable representations from large-scale ECG corpora and adapt them to down- stream clinical tasks via fine-tuning. [6] This direction is motivated by the strong empirical success of deep learning for arrhythmia detection and ECG classification, together with the growing availability of public ECG benchmarks that enable repro- ducible evaluation across cohorts and acquisition protocols. [1, 3, 4] At a high level, ECGFounder follows a representation-learning pipeline in which an ECG encoder maps an input segment to a compact embedding, and lightweight task-specific heads can be attached for supervised objectives such as classification. [6] Pretraining is performed at scale with broad label coverage, encouraging the encoder to capture mor- phology, rhythm, and temporally meaningful features that can be reused across tasks, thereby reducing the need for extensive task-specific feature engineering in downstream settings. [6, 1] A key emphasis of ECGFounder is external, multi-domain evaluation: rather than optimizing for a single benchmark, the framework stresses transfer across datasets and clinical settings to probe robustness under dataset shift—an important concern in ECG due to differences in patient populations, acquisition devices, and annotation conventions. [6, 3, 4] Because foundation models typically incur substantial deployment costs (memory, latency, and compute), they are commonly paired with compression methods—most notably knowledge distillation—so that smaller students can preserve teacher behavior while meeting real-world constraints. [7] When desired, distillation can be further strengthened by supervising intermediate representations in addition to output targets, improving the transfer of internal feature structure under tight capacity budgets. [8] 2.2 Knowledge Distillation KD is a compression paradigm in which a compact student is trained to approximate the predictive behavior of a high-capacity teacher, using the teacher’s outputs as an additional supervision signal. [14, 7, 15] The central idea is that, beyond the correct class, the teacher distribution encodes informative structure about class similarities, which the student can exploit to improve learning under limited capacity. [7, 15] Let v ∈R C and z ∈R C denote teacher and student logits for C classes. KD typically introduces a temperature-scaled softmax to control the softness of these targets: p (T) i = exp(v i /T ) P C j=1 exp(v j /T ) , q (T) i = exp(z i /T ) P C j=1 exp(z j /T ) ,(1) 3 where T > 1 increases the entropy of the teacher distribution, amplifying the relative contribution of non-argmax classes and exposing information contained in logit mag- nitudes that is largely suppressed at T = 1. [7, 16, 15] During training, the student is optimized to match the teacher distribution at the same temperature T ; at inference time, predictions are computed with T = 1. [7, 15] When ground-truth labels y are available, KD is commonly implemented by mixing a soft-target term with the standard supervised objective: L soft (T ) =− C X i=1 p (T) i logq (T) i ,(2) L hard =− C X i=1 y i logq (1) i ,(3) L KD = (1− α)T 2 L soft (T ) + αL hard , α∈ [0, 1].(4) Here, α controls the trade-off between fitting labels (hard supervision) and imitat- ing the teacher (soft supervision), while T determines how much dark knowledge is revealed by the teacher distribution. [7, 15] The factor T 2 is conventionally used to offset the temperature-dependent scaling of gradients, helping keep the relative contribution of the soft term comparable across different T values. [7, 15] 2.3 Variational Quantum Circuits Variational quantum circuits (VQCs) are parameterized quantum models in which a quantum state is prepared by a sequence of gates that depend on trainable parameters, and the parameters are optimized by a classical routine to minimize a task-dependent loss. [17, 13] This hybrid quantum–classical loop makes VQCs a natural candidate for learning in the noisy intermediate-scale quantum (NISQ) regime, where circuit depth and noise levels constrain fully quantum training. [13] A typical VQC can be written as |ψ(θ;x)⟩ = U (θ)U enc (x) |0⟩ ⊗n ,(5) where U enc (x) encodes an input x into a quantum state and U (θ) is a trainable ansatz composed of parameterized single-qubit rotations and entangling gates. [12, 18, 17] Model outputs are obtained by measuring expectation values of observables O m , s m (θ;x) =⟨ψ(θ;x)|O m |ψ(θ;x)⟩,(6) and mapping them to class probabilities (e.g., via a sigmoid for binary classification or a softmax for multi-class settings). [12] In machine learning applications, VQCs can be interpreted as trainable feature maps into a high-dimensional Hilbert space, where measurement statistics define non- linear decision functions. [18, 19] Their practical appeal is that they can be very parameter-efficient and hardware-aligned, but their trainability is affected by circuit 4 design choices, noise, and optimization pathologies such as barren plateaus, motivating careful ansatz selection and depth control. [17] In the context of model compression and distillation, a VQC can serve as a compact student that learns to approximate the predictive behavior of a large classical teacher while operating under explicit quantum resource constraints (number of qubits, cir- cuit depth, and measurement budget). [7, 13] This makes VQC students particularly relevant when deployment targets include quantum co-processors or when one seeks to study accuracy–efficiency trade-offs across heterogeneous learning paradigms. 3 Methodology We compare distilled quantum and classical students under a single, standardized pipeline (Fig. 1). The procedure comprises: (i) dataset preparation for binary ECG classification, (i) ECGFounder fine-tuning to obtain a high-capacity teacher, (i) knowledge distillation into three student families (CNN, compact ResNet, and a VQC- based student), and (iv) hyperparameter selection and evaluation with a fixed protocol. [6, 3, 4, 7] Fig. 1 Overview of the proposed pipeline. Left: a foundation model (ECGFounder) is fine-tuned to obtain a task-specific teacher. Center: a distillation loss combines hard labels and teacher logits to produce gradients. Right: three students (CNN, ResNet, and a compact VQC-based student) are trained under the same supervision signal for a controlled accuracy–efficiency comparison. 3.1 Data and Task Definition We use PTB-XL and the MIT-BIH Arrhythmia Database and follow standard usage to define a binary ECG classification task. [3, 4] To enforce a consistent input setting across datasets and student architectures, we restrict the signals to a single channel (one lead) and standardize all recordings into fixed-format model inputs. 5 As part of preprocessing, we apply wavelet-based denoising to mitigate baseline wander and high-frequency artifacts prior to input construction. [20, 21] Specifi- cally, each recording is decomposed via a discrete wavelet transform (DWT) and the detail coefficients are soft-thresholded using Donoho–Johnstone shrinkage principles (universal/SURE-type thresholding), with the noise level estimated from the median absolute deviation (MAD). [22, 23, 24] Model selection and evaluation are performed with stratified 5-fold cross-validation. Dataset splits are defined to prevent leakage (e.g., subject-level separation when applicable), and metrics are reported as averages across folds. [3, 4] 3.2 Teacher Fine-Tuning ECGFounder is fine-tuned on each dataset to obtain a task-specialized teacher for binary ECG classification. [6] Starting from the pretrained ECGFounder encoder, we attach a binary classification head and optimize the model end-to-end with supervised cross-entropy (binary logistic loss) on the training splits. [6] Model selection is per- formed using validation performance within the evaluation protocol (Section 3.5), and we retain the checkpoint that maximizes the chosen validation metric. The resulting fine-tuned teacher produces sample-wise logits that are stored and subsequently used as soft targets for knowledge distillation of all student architectures. [7] 3.3 Classical Students We distill the teacher into two classical convolutional students: a compact ResNet- 1D and a lightweight CNN-1D, both taking a single-lead ECG input (1 × L) and producing a single binary logit. ResNet and CNN-based students are widely adopted in knowledge distillation as accuracy–efficiency baselines under constrained capacity, enabling controlled comparisons of how architectural inductive biases and parameter budgets affect the transfer of teacher behavior. [7, 8, 25, 26, 27, 28, 29] The ResNet-1D student follows the residual learning principle adapted to 1D tem- poral signals: a convolutional stem performs early feature extraction, followed by a small number of residual stages built from BasicBlocks (skip connections with k=3 Conv1D layers), with temporal downsampling introduced via strided convolutions at stage transitions. [11] The network ends with global average pooling and a linear layer that outputs the final logit. In contrast, the CNN-1D student is a lightweight feed- forward backbone composed of a short stack of strided Conv1D blocks (Conv1D + BN + ReLU) that progressively increases channel capacity while reducing temporal resolu- tion, followed by global average pooling and a small fully connected head with dropout for regularization to produce the binary logit. [7, 15] Together, these two students provide complementary baselines: a residual architecture that improves optimization and feature reuse under moderate compression, and an ultra-compact convolutional model that probes the limits of distillation under a stringent parameter budget. 3.4 Quantum Student The quantum student couples a convolutional autoencoder with a 6-qubit variational quantum circuit (VQC) in order to interface high-dimensional ECG segments with 6 a small qubit register. Given an ECG window of length 256 samples, the encoder compresses the input into a latent vector of dimension 6, which is then used as the VQC input. The encoder follows a lightweight 1D convolutional design (three Conv1D blocks with 16/32/64 filters, kernel size 5), followed by flattening and a dense layer that outputs the 6-dimensional latent representation; the decoder mirrors this pathway to reconstruct the original window, encouraging the latent space to preserve mor- phology relevant for downstream classification while enforcing a strict dimensionality compatible with the quantum layer. [30] The VQC operates on n = 6 qubits and is instantiated to match the Qiskit con- struction shown in Fig. 2. The latent vector x∈R 6 is embedded via a ZZFeatureMap, which implements a data re-uploading feature map combining input-dependent single- qubit phase rotations with pairwise Z interactions under a linear entanglement pattern, yielding a nonlinear feature space for the subsequent variational circuit. [18, 12] This is followed by a hardware-efficient EfficientSU2 ansatz with trainable R y and R z rotations and linear entanglement, repeated for a small number of layers to keep depth compatible with NISQ constraints while retaining expressive capacity. [31, 17, 13] Readout is performed by measuring all qubits (e.g., ⟨Z i ⟩ for i = 1,..., 6), producing a 6-dimensional output vector that is mapped to a binary prediction and also serves as the student output for distillation. [12] All quantum circuits are implemented in Qiskit and evaluated using a simulated backend to ensure controlled experiments under fixed circuit specifications and measurement settings. [32] 7 q 0 ZZFeatureMap EfficientSU2 q 1 q 2 q 3 q 4 q 5 Fig. 2 Quantum-student circuit used in this work. A 6-qubit ZZFeatureMap (green) is followed by a hardware-efficient EfficientSU2 ansatz (blue). All qubits are measured to form the output vector used for binary prediction/distillation. 3.5 Hyperparameter Search and Evaluation We perform a small grid search over the main distillation hyperparameters—the tem- perature T and the mixing coefficient α—and select the best configuration on the validation set for each student and dataset (Fig. 4). The temperature controls the entropy of the teacher distribution used for soft supervision: larger T softens the tar- gets and makes relative logit structure more informative, which can stabilize learning for compact students and improve calibration when the teacher is highly confident. [7, 33, 34, 35] We evaluate T ∈ 2, 4, spanning a moderately softened regime that is commonly used in practical KD pipelines and that often yields measurable gains without overly flattening the supervisory signal. [7] The mixing coefficient α controls the balance between matching the teacher out- puts (soft supervision) and fitting ground-truth labels (hard supervision), thereby tuning how strongly the student relies on teacher-provided class relations versus direct label information. [7, 33, 15] For each temperature, we test α ∈ 0.3, 0.5, 0.7, yield- ing six (α,T ) configurations per student. For each configuration, classical students are trained with Adam via standard backpropagation, while the quantum student is trained in a hybrid quantum–classical loop using SPSA, which is widely adopted for variational circuits due to its robustness in stochastic settings and low measurement overhead. [36, 37, 17] Model selection is performed within the stratified 5-fold cross- validation protocol using validation performance, and final metrics are reported as averages over folds under the same evaluation procedure across all student families, enabling a controlled comparison of accuracy–efficiency trade-offs. [3, 4] 8 4 Results 4.1 Model complexity Figure 3 contextualizes the empirical results from a deployability perspective by comparing trainable parameter budgets (log scale). ECGFounder operates in a high- capacity regime (76.3M parameters), whereas the students are substantially smaller: ResNet-1D has 3.8M parameters, and both CNN-1D and the Autoencoder+VQC pipeline fall in the 10 4 range (33.0k and 25.0k, respectively). This setting therefore shows a strong compression regime, ranging from an∼6× reduction (ResNet) to nearly three orders of magnitude (CNN-1D and Autoencoder+VQC), and shifts attention to how effectively distillation preserves the teacher’s decision structure under constrained capacity. These budgets correspond to distinct operational profiles. The teacher is well-suited to offline analysis or clinical backends where memory and latency constraints are less restrictive, but can be impractical for edge deployment. ResNet-1D represents an inter- mediate compromise, retaining sufficient representational capacity to approximate the teacher while enabling substantially cheaper inference, consistent with typical distilla- tion objectives. In contrast, CNN-1D and Autoencoder+VQC target an aggressively lightweight regime, where preserving subtle morphological cues under limited capacity becomes the primary challenge; in this regime, the distillation signal is particularly important for stabilizing the learned decision rule and its calibration. For the Autoencoder+VQC student, the parameter count is dominated by the con- volutional autoencoder, which has approximately 25k trainable parameters, whereas the VQC contributes only 36 trainable parameters from the EfficientSU2 ansatz. Consequently, the total budget reported for the pipeline is largely determined by the encoder–decoder weights, with only a marginal contribution from the variational circuit parameters. 4.2 Quantitative results Tables 1 and 2 report mean test-set metrics (accuracy, precision, recall, and F1 score) for each student at T = 2 and T = 4, together with the fine-tuned teacher (ECG- Founder) as a reference. Across both datasets, the teacher sets a clear upper bound, while KD produces compact students that retain a substantial fraction of the teacher’s F1 under large parameter reductions. A consistent trend is that students preserve high recall more readily than precision, indicating that the dominant degradation mode under compression is increased false positives rather than missed detections. 4.2.1 Results on MIT-BIH Arrhythmia On MIT-BIH (Table 1), the teacher achieves very high recall (0.9900) and the best overall balance with F1 Score (0.8400). Distilled students remain in a high-recall regime (recall ∼0.93–0.98), indicating that knowledge distillation preserves much of the teacher’s sensitivity on this dataset. However, all students exhibit a noticeable pre- cision reduction relative to the teacher, which contributes to the lower accuracies and 9 Fig. 3 Model complexity comparison (log scale). ECGFounder is the high-capacity teacher, while ResNet, CNN, and Autoencoder+VQC are distilled students with substantially reduced parameter budgets. highlights MIT-BIH as the more challenging setting for compressed learners, where the remaining gap to the teacher is reflected primarily in precision and F1 Score. Within this regime, differences between student families are modest. At T = 2, the Autoencoder+VQC pipeline attains the best student accuracy and F1 (accuracy 0.6879, F1 0.8078), while ResNet achieves the highest recall (0.9763). At T = 4, performance becomes tightly clustered across students, with near-identical F1 scores (∼0.795−0.797), suggesting that under a reasonable distillation configuration, student choice has a smaller impact than the dataset difficulty itself. Importantly, the Autoencoder+VQC pipeline remains competitive with the clas- sical students despite employing an extremely compact variational circuit: the VQC includes only 36 trainable parameters. While most trainable capacity in the pipeline resides in the convolutional autoencoder, the quantum layer acts as a highly parameter- efficient decision module and still supports student-level performance comparable to ResNet-1D and CNN-1D on MIT-BIH, particularly in terms of accuracy and F1 Score at T = 2. 4.2.2 Results on PTB-XL On PTB-XL (Table 2), the teacher again provides the strongest reference performance (precision 0.8626, recall 0.9512, F1 0.8179). Relative to MIT-BIH, students typically achieve higher absolute accuracies on PTB-XL and a smaller accuracy gap to the teacher under the same protocol. In contrast, student recall is closer to the teacher on MIT-BIH than on PTB-XL, indicating that the two datasets differ in how compression impacts sensitivity versus calibration. 10 Table 1 Mean results on MIT-BIH Arrhythmia for different temperatures (T ). Temp ModelAccuracy Precision Recall F1 Score TeacherECGFounder0.80000.79000.99000.8400 T = 2ResNet0.67570.67750.97630.8033 VQC0.68790.71830.9392 0.8078 CNN0.66930.67570.95020.7971 T = 4ResNet0.67960.68480.93970.7952 VQC0.68170.69500.93330.7965 CNN0.66930.68640.9502 0.7971 In this regime, capacity differences are reflected more clearly among the classical baselines: ResNet-1D emerges as the strongest classical student overall, whereas CNN- 1D trails, consistent with the expected accuracy–capacity trade-off for the smallest model. The comparison between ResNet-1D and Autoencoder+VQC reveals a temperature-dependent precision–recall shift. At T = 2, ResNet-1D leads accuracy and F1 (accuracy 0.7480, F1 0.7897), while Autoencoder+VQC achieves the highest recall (0.8310), corresponding to a more sensitive operating point. At T = 4, Autoen- coder+VQC improves accuracy and precision (accuracy 0.7503, precision 0.7766) but reduces recall (0.7975), indicating a shift toward a more conservative decision regime. ResNet-1D remains marginally best in recall and slightly best in F1, while CNN-1D stays below both across temperatures. Finally, the VQC-based student maintains competitive performance on PTB-XL despite operating with an extremely small number of trainable parameters in the variational circuit. Table 2 Mean results on PTB-XL for different temperatures (T ). Temp ModelAccuracy Precision Recall F1 Score TeacherECGFounder0.80630.86260.95120.8179 T = 2ResNet0.74800.76070.8223 0.7897 VQC0.72930.73660.83100.7798 CNN0.72070.73670.80320.7680 T = 4ResNet0.74170.74860.8316 0.7876 VQC0.75030.77660.79750.7861 CNN0.71230.72680.80270.7626 4.3 Distillation hyperparameters: temperature andα Figure 4 complements the table-level outcomes by isolating how precision varies with the KD hyperparameters. Precision is reported as a function of α∈0.3, 0.5, 0.7 for T ∈ 2, 4, with PTB-XL in solid lines and MIT-BIH in dashed lines. In line with 11 Tables 1–2, PTB-XL tends to yield higher precision than MIT-BIH across students and settings, which may indicate that MIT-BIH is a more demanding regime for calibration under compression. The temperature effect appears to be most pronounced for the Autoencoder+VQC student. For T = 4, the VQC-based student may remain the top-precision model relative to the other students across the tested α values, with precision tending to improve as α increases and peaking at α = 0.7 on PTB-XL. A similar advantage may persist at T = 2, except for α = 0.5 on PTB-XL, where the VQC student shows a noticeable descent followed by recovery at α = 0.7. This pattern may indicate that higher temperature provides more informative soft targets for the most constrained student, while the choice of α can influence training stability by anchoring learning more strongly to hard-label supervision. In contrast, ResNet-1D and CNN-1D appear to exhibit smoother trends across α and smaller differences between T = 2 and T = 4, consistent with the more stable behavior suggested by the quantitative results in Tables 1–2. Fig. 4 Precision score as a function of the distillation hyperparameters α and temperature T for PTB-XL and MIT-BIH, comparing VQC, CNN, and ResNet students. Solid lines denote PTB-XL and dashed lines denote MIT-BIH. 5 Conclusion In this work, we investigated whether a large ECG foundation model can be effectively compressed into highly compact students for binary ECG classification on PTB-XL and MIT-BIH. Fine-tuning ECGFounder provided a strong teacher baseline, while knowledge distillation enabled substantial reductions in trainable parameters—from 76.3M parameters down to 3.8M (ResNet-1D), 33.0k (CNN-1D), 12 and 25.0k (Autoencoder+VQC)—while retaining competitive performance relative to the teacher. Across both datasets, the distilled students preserved a high-recall operating regime, with performance depending on the distillation hyperparameters. Tempera- ture and mixing coefficient effects were more evident on precision, and the T = 4 setting tended to yield more stable precision gains. Within this setting, the Autoen- coder+VQC pipeline remained consistently competitive with the classical students, despite using an extremely compact variational circuit with only 36 trainable param- eters, which may indicate that the quantum layer can serve as a parameter-efficient decision module within the distillation pipeline. Overall, these results suggest that distillation can transfer a substantial fraction of a foundation teacher’s behavior not only into compact classical models, but also into a quantum-ready student that couples a low-dimensional convolutional autoen- coder with a shallow variational circuit. The competitive performance observed for the VQC-based student supports the view that quantum students can be viable alter- natives under strong compression, motivating further studies on hybrid architectures, robustness under dataset shift, and execution on noisy quantum hardware to quantify the gap between simulation and NISQ devices. Funding This study was funded by the Funda ̧c ̃ao de Amparo `a Pesquisa do Estado de S ̃ao Paulo (FAPESP) through Project Nos. 2025/19585-6 (G.S.F.), 2023/04987-6 and 2024/00998-6 (F.F.), and by the Conselho Nacional de Desenvolvimento Cient ́ıfico e Tecnol ́ogico (CNPq) through Project No. 408884/2024-0 (F.F.). References [1] A.Y. Hannun, P. Rajpurkar, M. Haghpanahi, G.H. Tison, C. Bourn, M.P. Turakhia, A.Y. Ng, Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature Medicine 25(1), 65–69 (2019). https://doi.org/10.1038/s41591-018-0268-3 [2] D. Kim, K.R. Lee, D.S. Lim, et al., A novel hybrid cnn-transformer model for arrhythmia detection without r-peak identification using stockwell transform. Scientific Reports 15, 7817 (2025). https://doi.org/10.1038/s41598-025-92582-9. URL https://doi.org/10.1038/s41598-025-92582-9 [3] P. Wagner, N. Strodthoff, R.D. Bousseljot, D. Kreiseler, F.I. Lunze, W. Samek, T. Schaeffter, Ptb-xl, a large publicly available electrocardiography dataset. Scientific Data 7(1), 154 (2020). https://doi.org/10.1038/s41597-020-0495-6 [4] G.B. Moody, R.G. Mark, The impact of the MIT-BIH arrhythmia database. IEEE Engineering in Medicine and Biology Magazine 20(3), 45–50 (2001). https://doi. org/10.1109/51.932724 [5] K. McKeen, S. Masood, A. Toma, B. Rubin, B. Wang, Ecg-fm: an open elec- trocardiogram foundation model. JAMIA Open 8(5), ooaf122 (2025). https: //doi.org/10.1093/jamiaopen/ooaf122 13 [6] J. Li, A. Aguirre, J. Moura, C. Liu, L. Zhong, C. Sun, G. Clifford, B. West- over, S. Hong. An electrocardiogram foundation model built on over 10 million recordings with external evaluation across multiple domains (2024).https: //doi.org/10.48550/arXiv.2410.04133. URL https://arxiv.org/abs/2410.04133 [7] G. Hinton, O. Vinyals, J. Dean. Distilling the knowledge in a neural network (2015). https://doi.org/10.48550/arXiv.1503.02531. URL https://arxiv.org/abs/ 1503.02531 [8] A. Romero, N. Ballas, S.E. Kahou, A. Chassang, C. Gatta, Y. Bengio, FitNets: Hints for Thin Deep Nets, in International Conference on Learning Represen- tations (ICLR) (2015). https://doi.org/10.48550/arXiv.1412.6550. URL https: //arxiv.org/abs/1412.6550 [9] M.J. Hasan, M.R.C. Mahdy. Bridging classical and quantum machine learning: Knowledge transfer from classical to quantum neural networks using knowledge distillation (2023). https://doi.org/10.48550/arXiv.2311.13810. URL https:// arxiv.org/abs/2311.13810 [10] M. Alam, S. Kundu, S. Ghosh. Knowledge distillation in quantum neural network using approximate synthesis (2022). https://doi.org/10.48550/arXiv.2207.01801. URL https://arxiv.org/abs/2207.01801 [11] K. He, X. Zhang, S. Ren, J. Sun, Deep Residual Learning for Image Recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR) (2016), p. 770–778. https://doi.org/10.1109/CVPR.2016.90. URL https://arxiv.org/abs/1512.03385 [12] M. Schuld, M. Fingerhuth, F. Petruccione. Circuit-centric quantum classifiers (2018). https://doi.org/10.48550/arXiv.1804.00633. URL https://arxiv.org/abs/ 1804.00633 [13] J. Preskill, Quantum Computing in the NISQ era and beyond. Quantum 2, 79 (2018). https://doi.org/10.22331/q-2018-08-06-79 [14] C. Bucilu ̆a, R. Caruana, A. Niculescu-Mizil, Model Compression, in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (2006), p. 535–541. https://doi.org/10.1145/1150402. 1150464 [15] J. Gou, B. Yu, S.J. Maybank, D. Tao, Knowledge distillation: A survey. Inter- national Journal of Computer Vision 129(6), 1789–1819 (2021). https://doi.org/ 10.1007/s11263-021-01448-8 [16] J. Ba, R. Caruana. Do deep nets really need to be deep? (2014). https://doi.org/ 10.48550/arXiv.1312.6184. URL https://arxiv.org/abs/1312.6184 [17] M. Cerezo, A. Arrasmith, R. Babbush, S.C. Benjamin, S. Endo, K. Fujii, J.R. McClean, K. Mitarai, X. Yuan, L. Cincio, P.J. Coles, Variational quantum algo- rithms. Nature Reviews Physics 3(9), 625–644 (2021). https://doi.org/10.1038/ s42254-021-00348-9 [18] V. Havl ́ıˇcek, A.D. C ́orcoles, K. Temme, A.W. Harrow, A. Kandala, J.M. Chow, J.M. Gambetta, Supervised learning with quantum-enhanced feature spaces. Nature 567(7747), 209–212 (2019). https://doi.org/10.1038/s41586-019-0980-2 [19] G.S. Franco, F. Mahlow, P.M. Prado, G.E.L. Pexe, L.A.M. Rattighieri, F.F. Fanchini. Quantum phases classification using quantum machine learning with 14 shap-driven feature selection (2025). https://doi.org/10.48550/arXiv.2504.10673. URL https://arxiv.org/abs/2504.10673 [20] P.S. Addison, Wavelet transforms and the ecg: a review. Physiological Mea- surement 26(5), R155–R199 (2005). https://doi.org/10.1088/0967-3334/26/5/ R01 [21] D. Zhang, Wavelet Approach for ECG Baseline Wander Correction and Noise Reduction, in Proceedings of the 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS) (2005), p. 1212– 1215. https://doi.org/10.1109/IEMBS.2005.1616642 [22] D.L. Donoho, De-noising by soft-thresholding. IEEE Transactions on Information Theory 41(3), 613–627 (1995). https://doi.org/10.1109/18.382009 [23] D.L. Donoho, I.M. Johnstone, Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association 90(432), 1200–1224 (1995). https://doi.org/10.1080/01621459.1995.10476626 [24] S. Poornachandra, Wavelet-based denoising using subband dependent threshold for ecg signals. Digital Signal Processing 18(1), 49–55 (2008). https://doi.org/ 10.1016/j.dsp.2007.09.006 [25] S. Zagoruyko, N. Komodakis, Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer, in International Conference on Learning Representations (ICLR) (2017). https: //doi.org/10.48550/arXiv.1612.03928. URL https://arxiv.org/abs/1612.03928 [26] J. Yim, D. Joo, J. Bae, J. Kim, A Gift from Knowledge Distillation: Fast Opti- mization, Network Minimization and Transfer Learning, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), p. 7130–7138. https://doi.org/10.1109/CVPR.2017.753 [27] W. Park, D. Kim, Y. Lu, M. Cho, Relational Knowledge Distillation, in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), p. 3967–3976.https://doi.org/10.1109/CVPR.2019.00409. URL https://arxiv.org/abs/1904.05068 [28] Y. Tian, D. Krishnan, P. Isola. Contrastive representation distillation (2019). https://doi.org/10.48550/arXiv.1910.10699.URL https://arxiv.org/abs/1910. 10699 [29] S.I. Mirzadeh, M. Farajtabar, A. Li, N. Levine, A. Matsukawa, H. Ghasemzadeh. Improved knowledge distillation via teacher assistant (2020). https://doi.org/10. 48550/arXiv.1902.03393. URL https://arxiv.org/abs/1902.03393 [30] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006). https://doi.org/10.1126/ science.1127647 [31] A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J.M. Chow, J.M. Gambetta, Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets. Nature 549(7671), 242–246 (2017). https://doi.org/10. 1038/nature23879 [32] H. Abraham, I. Akhalwaya, A. Alekseev, et al. Qiskit: An open-source framework for quantum computing (2019). https://doi.org/10.5281/zenodo.2562110 [33] A.K. Menon, A.S. Rawat, S. Reddi, S. Kim, S. Kumar, A Statistical Perspective 15 on Distillation, in Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 139, ed. by M. Meila, T. Zhang (PMLR, 2021), p. 7632–7642. URL https://proceedings.mlr.press/ v139/menon21a.html [34] T. Huang, S. You, F. Wang, C. Qian, C. Xu, Knowledge Distillation from A Stronger Teacher, in Advances in Neural Information Processing Systems (NeurIPS) (2022). URL https://openreview.net/forum?id=157Uspkbi [35] Z. Li, X. Li, L. Yang, B. Zhao, R. Song, L. Luo, J. Li, J. Yang. Curriculum temperature for knowledge distillation (2023). https://doi.org/10.48550/arXiv. 2211.16231. URL https://arxiv.org/abs/2211.16231. AAAI 2023 [36] D.P. Kingma, J. Ba. Adam: A method for stochastic optimization (2015). https: //doi.org/10.48550/arXiv.1412.6980. URL https://arxiv.org/abs/1412.6980 [37] J.C. Spall, An overview of the simultaneous perturbation method for efficient optimization. Johns Hopkins APL Technical Digest 19(4), 482–492 (1998) 16