Paper deep dive
Investigating Quantum-Embedded Transformers on Classical Datasets for Cross-Modality Classification
Hao-Yuan Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/10/2026, 3:24:55 AM
Summary
This paper investigates the efficacy of Parameterized Quantum Circuits (PQCs) within a hybrid quantum-classical architecture called Quantum-Embedded Attention (QEA). The authors hypothesize that PQCs improve accuracy or stability over interface-matched classical controls. Using a controlled 2x2 factorial design on the Breast Cancer Wisconsin dataset, they find no consistent performance advantage for the PQC over classical maps. A broader cross-modality evaluation on five datasets (AG News, BirdCLEF, CIFAR-10, BCW, SUSY) shows comparable performance on some tasks but a significant deficit on CIFAR-10. The study concludes that current PQCs do not establish a quantum advantage and emphasizes the need for rigorous component attribution in hybrid models.
Entities (10)
Relation Signals (10)
Quantum-Embedded Attention → evaluatedon → Breast Cancer Wisconsin
confidence 99% · We test this with an interface-matched 2×2 factorial on Breast Cancer Wisconsin...
Quantum-Embedded Attention → evaluatedon → CIFAR-10
confidence 99% · ...but a large deficit on CIFAR-10...
Parameterized Quantum Circuit → demonstrates → No Quantum Advantage
confidence 95% · The results do not establish a quantum advantage; they demonstrate why controlled component attribution is necessary...
Quantum-Embedded Attention → evaluatedon → AG News
confidence 95% · A five-dataset cross-modality grid shows comparable accuracy on AG News...
Quantum-Embedded Attention → evaluatedon → BirdCLEF
confidence 95% · A five-dataset cross-modality grid shows comparable accuracy on... BirdCLEF...
Quantum-Embedded Attention → uses → Attention Decoder
confidence 95% · ...and a classical attention decoder to produce class logits.
Quantum-Embedded Attention → uses → Parameterized Quantum Circuit
confidence 95% · Our architecture, Quantum-Embedded Attention (QEA), uses a learnable projector... a shallow PQC...
Parameterized Quantum Circuit → outputs → Pauli Expectation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We test whether a parameterized quantum circuit (PQC) improves a hybrid quantum-classical model's performance on classical datasets, using an interface-matched classical map as the control while holding all other components fixed. Our architecture, Quantum-Embedded Attention (QEA), uses a learnable projector to compress backbone features into an $n_q$-dimensional angle vector, a shallow PQC to map those angles to one- and two-qubit Pauli expectations, and a classical attention decoder to produce class logits. We hypothesized the PQC would improve accuracy or seed-to-seed stability over a classical map with matched input/output dimensions. We test this with an interface-matched $2\times2$ factorial on Breast Cancer Wisconsin at $n_q\in\{4,8\}$, independently swapping the PQC for a classical map and the attention decoder for a linear head, across five paired seeds per cell. Three of four paired quantum-minus-classical $95\%$ confidence intervals include zero; the fourth, a $+1.63$ percentage-point contrast for the attention decoder at $n_q=4$, reverses sign at $n_q=8$ and does not survive correction across the four contrasts. The experiment thus shows no consistent PQC contribution and cannot establish equivalence. A five-dataset cross-modality grid shows comparable accuracy on AG~News, Breast Cancer Wisconsin, and BirdCLEF but a large deficit on CIFAR-10; these cells are not interface-matched and are interpreted descriptively. We report all planned canonical runs, distinguish current Pauli-readout results from legacy probability-readout experiments, and analyze bottleneck, simulation, finite-shot, and noise limitations. The results do not establish a quantum advantage; they demonstrate why controlled component attribution is necessary before crediting a hybrid model's performance to its quantum layer.
Tags
Links
- Source: https://arxiv.org/abs/2608.06846v1
- Canonical: https://arxiv.org/abs/2608.06846v1
Trouble viewing inline? Open PDF directly →
Full Text
50,315 characters extracted from source content.
Expand or collapse full text
Investigating Quantum-Embedded Transformers on Classical Datasets for Cross-Modality Classification Hao-Yuan Chen Computer Science, University of London hc118@student.london.ac.uk Abstract We test whether a parameterized quantum circuit (PQC) improves a hybrid quantum-classical model’s performance on classical datasets, using an interface-matched classical map as the control while holding all other components fixed. Our architecture, Quantum-Embedded Attention (QEA), uses a learnable projector to compress backbone features into an nqn_q-dimensional angle vector, a shallow PQC to map those angles to one- and two-qubit Pauli expectations, and a classical attention decoder to produce class logits. We hypothesized the PQC would improve accuracy or seed-to-seed stability over a classical map with matched input/output dimensions. We test this with an interface-matched 2×22× 2 factorial on Breast Cancer Wisconsin at nq∈4,8n_q∈\4,8\, independently swapping the PQC for a classical map and the attention decoder for a linear head, across five paired seeds per cell. Three of four paired quantum-minus-classical 95%95\% confidence intervals include zero; the fourth, a +1.63+1.63 percentage-point contrast for the attention decoder at nq=4n_q=4, reverses sign at nq=8n_q=8 and does not survive correction across the four contrasts. The experiment thus shows no consistent PQC contribution and cannot establish equivalence. A five-dataset cross-modality grid shows comparable accuracy on AG News, Breast Cancer Wisconsin, and BirdCLEF but a large deficit on CIFAR-10; these cells are not interface-matched and are interpreted descriptively. We report all planned canonical runs, distinguish current Pauli-readout results from legacy probability-readout experiments, and analyze bottleneck, simulation, finite-shot, and noise limitations. The results do not establish a quantum advantage; they demonstrate why controlled component attribution is necessary before crediting a hybrid model’s performance to its quantum layer. Keywords: quantum machine learning, hybrid quantum-classical, parameterized quantum circuits, component attribution, interface-matched controls, negative results 1 Introduction Hybrid quantum–classical models are a practical way to study quantum machine learning (QML) on present hardware while retaining classical feature extractors and optimizers Preskill [2018], Benedetti et al. [2019]. Restricting the quantum component to a small parameterized circuit (PQC) after a classical projector avoids loading the raw high-dimensional input as amplitudes. Keeping the circuit shallow also reduces—but does not eliminate—the trainability risks associated with deep PQCs, including barren plateaus McClean et al. [2018]. Motivation and hypothesis. Two properties make a PQC an appealing module to test inside a modern network. First, its input interface can be modality-agnostic: after a classical backbone has mapped a spectrogram, image, tabular vector or sentence to a feature vector, the same fixed-width circuit design can act on the projected features. Second, a PQC supplies a trainable non-linear map whose parameter count can grow slowly with circuit width. Neither property implies that the map is more useful than a classical alternative, and the exponential Hilbert-space dimension alone does not establish useful capacity or computational advantage. The scientific question is therefore comparative. Our guiding hypothesis is (H): after the same learned projector, a PQC improves accuracy or seed-to-seed stability relative to a classical map with the same input and output dimensions. Its empirical prediction is a non-zero paired performance difference when only that module is exchanged. Where to insert the circuit. The interface between a backbone and its classifier is a natural test location: the input has already been compressed, and the inserted module can be exchanged without changing the data pipeline. We call the design pattern a quantum embedding layer and the architecture studied here Quantum-Embedded Attention (QEA). An attention decoder acts on the measured channels after the circuit; it does not implement quantum attention. This work extends a preliminary single-qubit, single-dataset study by the same authors Chen et al. [2024] (Section 2) with a multi-qubit Pauli readout, a broader modality study, and a controlled intervention on the quantum layer. The placement is thus motivated by testability as much as by hardware constraints: it exposes a single quantum module whose classical surrogate can occupy the same interface. The contributions of this paper are: 1. A self-contained QEA specification: backbone → learnable projector → data-dependent PQC → one- and two-body Pauli readout → classical attention decoder. The qubit count nqn_q is decoupled from raw input dimensionality, and every stage is defined mathematically (Section 3). 2. An interface-matched factorial that tests the quantum contribution. At fixed nqn_q we independently replace the PQC with a classical map of identical input/output dimensions and replace the attention decoder with a linear head. The projector, readout interface, seeds and training budget are held fixed. We report paired effect intervals and do not infer equivalence from overlapping marginal intervals. 3. A five-dataset cross-modality evaluation and complete accounting of the controlled runs. QEA shows no consistent advantage over classical controls and fails strongly on CIFAR-10. We report the incomplete chemistry cell, unsuccessful runs, scaling costs and the distinction between legacy probability readout and the current Pauli-readout protocol instead of filling missing evidence by extrapolation. 2 Related work Variational encodings. Variational quantum classifiers use data-dependent circuits and trainable gates to fit decision boundaries Schuld et al. [2020], Havlíček et al. [2019]. Their function class depends strongly on encoding. Data re-uploading interleaves repeated encodings with trainable blocks Pérez-Salinas et al. [2020], and the resulting models can be analyzed as truncated Fourier series whose accessible frequencies depend on the encoding Schuld et al. [2021]. Pérez-Salinas et al. Pérez-Salinas et al. [2020] compare the repeated processing of a single-qubit classifier with a one-hidden-layer neural network, in which hidden units likewise receive repeated copies of the input. Moreover, a single qubit is efficiently classically simulable. Strong single-qubit performance is therefore insufficient evidence of a quantum contribution and motivates the explicit surrogate used in Section 4.2. Relation to the authors’ prior work. The closest predecessor is a study by the same authors Chen et al. [2024] that inserted a single-qubit circuit after a vision transformer and evaluated one BirdCLEF task, reporting an approximately three-point median-F1F_1 improvement. That comparison changed the hybrid pipeline as a whole and did not isolate the projector, circuit and decoder. The present work differs by using multi-qubit Pauli readout, a classical attention decoder, five current-protocol datasets and, most importantly, an interface-matched 2×22× 2 factorial. The earlier paper asked whether a hybrid model can work; this paper asks whether a measured change persists when the quantum module alone is exchanged. The current evidence does not show a consistent effect across the tested BCW settings. Quantum kernels—and why this is not one. Quantum kernel methods embed inputs into quantum states and evaluate pairwise similarities before fitting a classical kernel learner Havlíček et al. [2019], Schuld and Killoran [2019]. A common example is the fidelity k(xi,xj)=|⟨ϕ(xi)|ϕ(xj)⟩|2.k(x_i,x_j)=| φ(x_i)|φ(x_j) |^2. (1) QEA computes no inter-example similarity or Gram matrix. Each input is mapped to a vector of local Pauli expectations and decoded independently. This distinction also locates the work relative to current evidence. Schnabel and Roth’s benchmark of more than 20,00020,000 fidelity and projected quantum-kernel models across 64 datasets found strong sensitivity to encoding, kernel and hyperparameter choices and no universal quantum advantage Schnabel and Roth [2025]. Our model is not one of those kernels, but their emphasis on classical controls and component-level analysis applies directly. Quantum transformers and hybrid feature extractors. The recent quantum-transformer survey distinguishes PQC-based hybrid designs from fault-tolerant quantum-linear-algebra proposals and highlights scalability, benchmarking and trainability gaps Zhang and Zhao [2025]. Quantum vision transformers place PQCs inside or around attention operations Cherrat et al. [2024], whereas quanvolutional networks use small circuits as local feature extractors Henderson et al. [2020]. QEA is simpler: its attention is fully classical, and the PQC occupies a post-backbone embedding interface. This placement is chosen because the circuit can be exchanged without redefining the task, not because it is expected to accelerate classical self-attention. Frameworks such as Qiskit Machine Learning Qiskit Machine Learning Developers [2024] and PennyLane Bergholm and others [2018] support differentiation through quantum layers. The reported experiments use the batched PyTorch statevector implementation specified in Section 3.3; framework selection is not a novelty claim. 3 Method 3.1 Architecture overview QEA maps a raw input x to class logits ℓ through four composed stages, →fθ∈ℝd→gϕ∈ℝnq→∈ℝM→aψℓ∈ℝC,x\; \;f_θ\;\;h ^d\; \;g_φ\;\; ξ ^n_q\; \;Q_ ω\;\;m ^M\; \;a_ψ\;\; ^C, (2) where fθf_θ is a modality-specific classical backbone, gϕg_φ a learnable projector, Q_ ω a quantum embedding that returns a vector m of measurement statistics, and aψa_ψ an attention decoder. Figure 1 shows the core pipeline together with the two ablation switches used in Section 4. All parameters θ,ϕ,,ψ\θ,φ, ω,ψ\ are trained jointly by minimizing the cross-entropy loss with Adam Kingma and Ba [2015]; gradients through Q_ ω are analytic (Section 3.3). xBackbonefθf_θProjectorgϕ:ℝd→ℝnqg_φ:R^d\!→\!R^n_qPQCQ_ ωMeasure∈ℝMm ^MAttentiondecoder aψa_ψℓ switch A: replace bytanh(W+b) (W ξ+b) (same M)switch B: replace bylinear head Figure 1: The core QEA pipeline of Eq. (2). A classical backbone and projector reduce the input to an nqn_q-dimensional angle vector, a parameterized quantum circuit produces a vector of measurement statistics, and an attention decoder maps those statistics to class logits. The two dashed switches define the interface-matched factorial of Section 4: switch A replaces the circuit with an equal-width classical map, and switch B replaces the attention decoder with a linear head. The broader exploratory grid uses the QEA-R variant, which additionally concatenates the projected angles as a residual token stream; that variant is reported separately and is not used for component attribution. 3.2 Classical backbone and projector The backbone fθf_θ is chosen per modality: ResNeXt-50 Xie et al. [2017] for audio spectrograms and images, a multilayer perceptron for tabular, chemistry and physics feature vectors, and a frozen MiniLM Wang et al. [2020] sentence encoder followed by a small MLP for text. It outputs a feature =fθ()∈ℝdh=f_θ(x) ^d. The projector is a two-layer network gϕ()=W2σ(W1+b1)+b2,σ=GELU,g_φ(h)=W_2\,σ\! (W_1h+b_1 )+b_2, σ=GELU, (3) with W2∈ℝnq×hW_2 ^n_q× h, that compresses h to one angle per qubit. Crucially, nqn_q is a design choice independent of d: a 224×224224× 224 image (d from the backbone) and a 3030-dimensional tabular vector both terminate in the same nqn_q-wide bottleneck. The angles are bounded before encoding, =πtanh(s⊙gϕ()) ξ=π \! (s g_φ(h) ), with an optional learnable per-qubit scale s. The controlled factorial fixes s=s=1; the exploratory QEA-R grid learns s. 3.3 Quantum embedding layer The quantum stage applies a data-dependent unitary to the nqn_q-qubit ground state and reads out a fixed set of expectation values. The unitary factorizes into a data-encoding feature map ℱF and a trainable ansatz A, U(;)=()ℱ(),|ψ(;)⟩=U(;)|0⟩⊗nq.U( ξ; ω)=A( ω)\,F( ξ), |ψ( ξ; ω) =U( ξ; ω)\,|0 n_q. (4) The feature map is a layer of single-qubit rotations optionally followed by entangling ZZ rotations, ℱ()=∏r=1R[∏(i,j)∈Ee−iξiξjZiZj∏i=1nqe−iξiZi∏i=1nqHi],F( ξ)= _r=1^R [ _(i,j)∈ Ee^-i\, _i _j\,Z_iZ_j _i=1^n_qe^-i\, _i\,Z_i _i=1^n_qH_i ], (5) where R is the number of encoding repetitions and E the entangling graph; setting E=∅E= recovers a product (Z) feature map. Repeating the same angles within the feature map is data re-uploading in the broad sense, and R changes the accessible Fourier frequencies Pérez-Salinas et al. [2020], Schuld et al. [2021]. The controlled factorial uses R=2R=2 followed by the ansatz. The exploratory QEA-R grid uses the same two-repetition feature-map block before each trainable ansatz layer. We therefore do not describe either protocol as “single encoding”; the only single interface is the location at which the classical projector supplies ξ to the circuit. The ansatz is a hardware-efficient block of parameterized Y-rotations and nearest-neighbour CNOTCNOTs, ()=∏r=1L[∏i=1nq−1CNOTi,i+1∏i=1nqRY(ωr,i)]∏i=1nqRY(ω0,i),A( ω)= _r=1^L [ _i=1^n_q-1CNOT_i,i+1 _i=1^n_qR_Y( _r,i) ] _i=1^n_qR_Y( _0,i), (6) with depth L (the RealAmplitudes family; EfficientSU2, used in one ablation, adds RZR_Z rotations). The number of trainable circuit parameters is nq(L+1)n_q(L+1), i.e. linear in the qubit count while the state space is 2nq2^n_q-dimensional. Readout. Rather than returning the full 2nq2^n_q-dimensional probability vector, whose dimension grows exponentially and which is expensive to estimate on hardware, we read out a polynomial-size vector of Pauli expectation values, =(⟨Zi⟩i,⟨ZiZj⟩(i,j)∈E),M=nq+|E|,m= ( Z_i _i\,,\ Z_iZ_j _(i,j)∈ E ), M=n_q+|E|, (7) so ∈ℝMm ^M with M=(nq2)M=O(n_q^2). This avoids an exponential classical decoder but does not, by itself, establish a quantum speedup; Section 6 separates readout, simulation and hardware costs. All reported expectation values are computed by a batched, exact statevector simulator, and gradients are obtained by automatic differentiation through that simulator. Hardware execution would require finite-shot expectation estimates and a hardware-compatible gradient estimator such as parameter shift; it is not evaluated here. 3.4 Attention decoder The decoder aψa_ψ treats the readout as a short sequence of tokens. Each measurement channel mkm_k is lifted to a token tk=ek+r(mk)∈ℝDat_k=e_k+r(m_k) ^D_a, where eke_k is a learned per-channel embedding and r a linear projection of the scalar mkm_k. A stack of standard multi-head self-attention and feed-forward blocks transforms the tokens tk\t_k\, and the sequence is pooled and passed to a small MLP classifier: =Pool(Attnψ(tkk=1M)),ℓ=MLPψ().z=Pool (Attn_ψ(\t_k\_k=1^M) ), =MLP_ψ(z). (8) The decoder is entirely classical; its role is to model interactions among measurement channels that a single linear layer cannot. Its contribution is isolated by switch B in Figure 1, which replaces aψa_ψ with a single linear map ℓ=W+b =Wm+b. Exploratory residual variant. The cross-modality grid predates the controlled factorial and uses a stronger variant denoted QEA-R. After an identity-initialized learned mixer of the Pauli channels, QEA-R concatenates the projected angle vector ξ to m before attention. This classical residual path can bypass the PQC; accordingly, QEA-R is useful as a descriptive hybrid baseline but cannot attribute its predictions to the circuit. All component-attribution statements in this paper come from the core, residual-free factorial in Section 4.2. 3.5 Qubit count and circuit depth Because nqn_q is decoupled from d, it is a free bottleneck that we sweep as an ablation rather than fix to the input size. Following McClean et al. McClean et al. [2018] we keep the ansatz shallow (L∈1,2L∈\1,2\) to limit barren-plateau risk; the consequences of this choice for expressivity are discussed in Section 6. To keep the comparison against the Hybrid baseline free of a qubit-count confound, both methods are evaluated at identical nqn_q in the interface-matched study (Section 4). 4 Experimental setup The evaluation separates two questions. The interface-matched factorial tests hypothesis (H) by changing one architectural component at a time; it is the only analysis used for component attribution. The cross-modality grid asks whether an exploratory residual variant can be trained on heterogeneous inputs. Because its historical baselines differ in readout width and training details, that grid is descriptive rather than causal. 4.1 Datasets and analysis status The study repository contains loaders for six source datasets (Table 1). Current Pauli-readout results are complete for five: BirdCLEF-2021 “nocall” audio detection Kahl et al. [2021], CIFAR-10 images Krizhevsky [2009], Breast Cancer Wisconsin (Diagnostic, BCW) Street et al. [1993], SUSY high-energy physics Baldi et al. [2014], and AG News text classification Zhang et al. [2015]. The available QM9 chemistry rows Ramakrishnan et al. [2014] use an older two-qubit, probability-readout implementation and are not pooled with the current protocol. This revision therefore reports a five-dataset grid, not a six-dataset result. Appendix B gives the actual sample counts and split construction. Table 1: Dataset scope. “Current” denotes inclusion in the five-dataset Pauli-readout grid; QM9 is retained as a documented source dataset but excluded from current aggregate claims because only legacy-protocol rows are available. Modality Dataset Task Backbone Analysis status Audio BirdCLEF Kahl et al. [2021] binary ResNeXt-50 Current Image CIFAR-10 Krizhevsky [2009] 10-class ResNeXt-50 Current Tabular BCW Street et al. [1993] binary MLP Current + factorial Chemistry QM9 Ramakrishnan et al. [2014] 5-bin MLP Legacy protocol only HEP SUSY Baldi et al. [2014] binary MLP Current NLP AG News Zhang et al. [2015] 4-class MiniLM + MLP Current 4.2 Interface-matched factorial We define every controlled model by the four stages of Eq. (2): backbone fθf_θ, projector gϕg_φ, embedding Q_ ω, and decoder aψa_ψ. Two switches act on the last two stages: Switch A (embedding). The PQC is replaced by ~()=tanh(W+b) Q( ξ)= (W ξ+b), which has the same nqn_q-dimensional input and M-dimensional output as Eq. (7). The two maps do not have identical parameter counts, so we call the control interface-matched, not parameter-matched; total trainable counts are reported in Appendix A. Switch B (decoder). The attention decoder of Section 3.4 is replaced by a single linear classifier on the same readout vector. The four combinations are QEA (PQC + attention), Hybrid QNN (PQC + linear), Classical + Attention, and Classical + Linear. At each nq∈4,8n_q∈\4,8\, all four use the same BCW split for a given seed, the same MLP backbone, projector, M=nq(nq+1)/2M=n_q(n_q+1)/2 readout interface, optimizer schedule and 40-epoch budget. Five seeds (42,123,31,2024,742,123,31,2024,7) are paired across cells. Thus a within-seed row comparison localizes the architectural intervention to switch A; it does not turn a non-significant result into proof of equivalence. 4.3 Exploratory cross-modality grid The broader grid reports four historical model families: a plain backbone + MLP classifier (Classical), a projector + classical readout + attention model (Classical + Attention), a two-qubit PQC + linear head (Hybrid QNN), and the eight-qubit residual quantum model QEA-R defined in Section 3.4. These columns are intentionally not presented as an ablation: the Hybrid width is two qubits, the classical-attention readout has 282^8 channels, and QEA-R has 36 Pauli channels plus an eight-angle residual. The table answers whether each configured pipeline trains, not which component caused a difference. 4.4 Training, uncertainty and run accounting All models use Adam Kingma and Ba [2015] with nominal learning rate 10−310^-3, weight decay 10−610^-6, gradient clipping at 55, a cosine warm-restart schedule, and selection of the best validation checkpoint before one held-out test evaluation. The factorial uses 40 epochs for every cell. Cross-grid CSVs record 30 epochs for most completed runs; four historical rows ended at 15–25 epochs (one AG News Hybrid, two BirdCLEF Classical + Attention, and one CIFAR-10 QEA-R). They are retained only in the descriptive grid and are not evidence for component attribution. Marginal table intervals are two-sided 95%95\% Student-t intervals over seeds. For the factorial, the primary uncertainty is the Student-t interval of the paired quantum-minus-classical differences. With only five pairs, these intervals are imprecise; no equivalence margin was prespecified. A canonical run is marked collapsed when validation accuracy equals the majority-class rate at three consecutive scheduled evaluations. Smoke tests are diagnostics rather than planned analysis units. A non-finite loss raises an error and produces no result row; the earlier SUSY errors were traced to a truncated non-finite input row, after which all canonical SUSY Classical + Attention cells were rerun with a loader that drops non-finite rows before splitting. Run counts, duplicate-resolution rules and incomplete budgets are reported in Section 5.2; no failed canonical run is silently replaced by a different seed. 4.5 Reproducibility Each canonical cell is specified by a versioned configuration file, and the run-level CSVs, dependency lockfile and scripts that regenerate the tables and figures accompany the revision. Operational commands and legacy-code notes remain in the repository documentation rather than the scientific narrative. 5 Results We report the controlled factorial first, followed by the run ledger and the exploratory cross-modality grid. This order reflects evidentiary weight: only the factorial changes one interface at a time. 5.1 Paired component attribution on BCW Table 2 reports held-out accuracy for the four combinations of embedding and decoder at nq∈4,8n_q∈\4,8\. Marginal means are accompanied by 95%95\% Student-t intervals; the final column is the within-seed quantum-minus-classical difference, which is the relevant estimate for switch A. Table 2: Interface-matched factorial on BCW (percentage-point test accuracy, mean ± 95%95\% Student-t CI, five paired seeds). The paired effect is quantum minus classical at fixed decoder and nqn_q. Three intervals include zero. The nq=4n_q=4 attention contrast is positive before multiplicity correction, but the direction does not replicate at nq=8n_q=8. nqn_q Decoder Quantum (PQC) Classical surrogate Paired ΔQ−C _Q-C 4 Attention 97.21 ± 2.62 95.58 ± 2.58 +1.63 ± 1.29 4 Linear 95.81 ± 4.16 96.05 ± 2.62 -0.23 ± 2.14 8 Attention 95.58 ± 5.53 96.28 ± 1.88 -0.70 ± 5.06 8 Linear 97.21 ± 1.65 96.74 ± 1.58 +0.47 ± 0.79 Figure 2: Paired quantum-minus-classical switch-A effects in the BCW factorial. Points denote the mean within-seed difference and error bars the 95%95\% Student-t interval over five paired seeds. The vertical line marks zero. The plot makes visible both the isolated positive nq=4n_q=4 attention contrast and its failure to replicate at nq=8n_q=8; numerical values are in Table 2. At nq=4n_q=4 with attention, QEA exceeds its classical surrogate by 1.631.63 points (paired 95%95\% CI [0.34,2.92][0.34,2.92], unadjusted paired t test p=0.025p=0.025). The other three switch-A effects are −0.23-0.23 points with a linear decoder at nq=4n_q=4 ([−2.37,1.91][-2.37,1.91]), −0.70-0.70 points with attention at nq=8n_q=8 ([−5.76,4.37][-5.76,4.37]), and +0.47+0.47 points with a linear decoder at nq=8n_q=8 ([−0.33,1.26][-0.33,1.26]). Applying Benjamini–Hochberg correction to these four embedding contrasts gives pBH=0.100p_BH=0.100 for the smallest p value. The positive nq=4n_q=4 contrast is therefore a result to replicate, not a stable quantum effect: it is small relative to the task’s ≈96%≈ 96\% ceiling, changes direction at nq=8n_q=8, and was selected from four contrasts with only five pairs each. Conversely, the data are too imprecise for an equivalence claim because no smallest effect of interest was prespecified. The decoder contrasts are also inconsistent across widths. On this saturated dataset, the factorial neither establishes an attention benefit nor supports the stronger claim that the PQC is interchangeable with a classical map on all tasks. Its defensible conclusion is narrower: the prespecified architecture did not produce a consistent switch-A effect across the two tested widths on BCW. 5.2 Run accounting All 40 planned factorial runs (four cells × two widths × five seeds) completed without a majority-class collapse. Some CSVs contain repeated executions of the same seed at shorter budgets; aggregation deterministically selects the row with the largest recorded epoch count, so duplicates are not treated as independent observations (Table 3). Table 3: Canonical factorial run accounting after resolving repeated rows by maximum recorded epoch count. “Coll.” is the number meeting the fixed collapse rule. The all-run and clean-run means coincide because no factorial run collapsed. Cell nqn_q Planned Coll. Acc. (all) Acc. (clean) Quantum+Attn (QEA) 4 5 0 97.21 97.21 Classical+Attn 4 5 0 95.58 95.58 Quantum+Linear (Hybrid) 4 5 0 95.81 95.81 Classical+Linear 4 5 0 96.05 96.05 Quantum+Attn (QEA) 8 5 0 95.58 95.58 Classical+Attn 8 5 0 96.28 96.28 Quantum+Linear (Hybrid) 8 5 0 97.21 97.21 Classical+Linear 8 5 0 96.74 96.74 The five-dataset grid contains 84 planned canonical configurations: 21 each for Classical, Classical + Attention, Hybrid QNN and QEA-R, reflecting three seeds for image/audio and five for the other datasets. Classical, Classical + Attention after the corrected SUSY reload, and QEA-R have 0/210/21 collapsed configurations. Hybrid QNN has 3/213/21: one BirdCLEF and two SUSY seeds. These three remain in the run ledger and are excluded from clean-cell means by the stated rule, leaving two BirdCLEF and three SUSY Hybrid estimates. Four additional clean rows have shorter historical budgets, as disclosed in Section 4.4. Smoke diagnostics and repeated infrastructure attempts are not additional seed-level observations. QM9 is outside this 84-run current-protocol ledger because its available rows use the legacy probability readout. 5.3 Exploratory cross-modality grid Table 4 and Figure 3 summarize the five current datasets. Because the columns are not interface-matched, the comparisons are descriptive. On AG News, BCW and BirdCLEF, QEA-R’s point estimate is close to the classical estimates. This cannot be credited to the PQC: QEA-R includes an angle residual that bypasses it, and the classical-attention column uses a different readout width. The two harder datasets argue against a general performance benefit. On CIFAR-10, QEA-R reaches 40.28%40.28\% versus 84.11%84.11\% for the plain Classical model and 78.14%78.14\% for Classical + Attention. On SUSY, Classical + Attention reaches 80.06%80.06\%, compared with 69.35%69.35\% for QEA-R, 71.29%71.29\% for Classical, and 54.30%54.30\% for the collapse-prone Hybrid QNN. Thus the broad grid contains no consistent empirical advantage for the quantum path; its clearest result is the large failure on CIFAR-10. Table 4: Exploratory held-out test accuracy (mean ± 95%95\% Student-t CI). The columns are historical model families, not a component ablation. AG News, BCW and SUSY use five planned seeds; BirdCLEF and CIFAR-10 use three. After the fixed collapse rule, BirdCLEF and SUSY Hybrid retain two and three seeds, respectively. QEA-R contains a classical angle residual; the controlled core QEA comparison is Table 2. Dataset Backbone Classical Class.+Attn Hybrid nq=2n_q=2 QEA-R nq=8n_q=8 AG News MiniLM+MLP 88.47 ± 1.74 88.60 ± 0.82 82.98 ± 8.12 88.26 ± 1.09 BCW MLP 95.58 ± 1.88 96.28 ± 3.13 96.51 ± 1.44 96.28 ± 3.29 BirdCLEF ResNeXt-50 90.42 ± 4.74 89.86 ± 2.61 88.75 ± 26.47 90.97 ± 5.31 CIFAR-10 ResNeXt-50 84.11 ± 7.23 78.14 ± 0.44 36.46 ± 9.39 40.28 ± 11.20 SUSY MLP 71.29 ± 8.89 80.06 ± 0.38 54.30 ± 32.39 69.35 ± 5.84 Figure 3: Exploratory cross-dataset comparison corresponding to Table 4. Error bars are 95%95\% Student-t intervals; fill patterns distinguish methods without relying on color. The figure shows similar point estimates on three datasets, a QEA-R deficit on SUSY, and a large failure of both quantum models on CIFAR-10. 5.4 What the seed failures do and do not show The 3/213/21 Hybrid collapse rate is an observed property of this implementation and schedule, not evidence that linear quantum heads are intrinsically unstable. The non-collapsed attention models are associated with lower variance on AG News and SUSY, but this association is confounded by decoder capacity, readout width and, for QEA-R, the residual bypass. No controlled hard-task factorial is available. We therefore report collapse frequency as a reproducibility outcome and do not label QEA-R “quantum-robust.” 6 Discussion 6.1 What was learned about hypothesis (H)? Hypothesis (H) predicted that exchanging the classical surrogate for the PQC at the same interface would improve accuracy or stability. The BCW factorial does not show a consistent effect across widths or decoders. One of four paired contrasts is positive before multiplicity correction; the effect changes sign at the other width and the adjusted test is not significant. This pattern is compatible with a small task-specific effect, optimization noise, or a false positive. It is not compatible with a broad claim that the quantum layer is the source of QEA’s performance. It also does not prove that the maps are equivalent: five seeds on one saturated dataset provide little power to exclude small effects. A hybrid model performing well is evidence about the full hybrid pipeline, not its PQC. The nq=1n_q=1 results make the same point from another direction. A single-qubit re-uploading classifier is classically simulable, and its processing can be compared with a one-hidden-layer neural network whose repeated uses of the input play a role analogous to re-uploads Pérez-Salinas et al. [2020]. High accuracy at nq=1n_q=1 is therefore a useful debugging result but not evidence of quantum advantage. 6.2 Attention and the compression bottleneck The current evidence cannot fully separate compression from the subsequent map. QEA-R and Classical + Attention both use an eight-dimensional projector, but their readouts differ: QEA-R supplies 36 Pauli channels plus an eight-angle residual, whereas the historical classical control supplies 256 softmax channels. The gap from plain Classical (84.11%84.11\%) to Classical + Attention (78.14%78.14\%) is consistent with a cost from the bottleneck, but it also changes the decoder and parameterization. The further drop to QEA-R (40.28%40.28\%) is likewise not an isolated circuit effect. A decisive image experiment would use switch A with identical 36-channel readouts, residual paths, seeds and budgets on both sides. The same qualification applies to the role of attention. On BCW, switch B has no consistent effect because all four cells are near the task ceiling. On SUSY, the historical Classical + Attention column is strongest, while on AG News and BirdCLEF its point estimate is close to plain Classical. Since those columns differ by more than attention alone, the results suggest where a controlled decoder study would be informative but do not demonstrate that attention causes the improvement. In particular, the lower collapse rate of attention models should not be generalized beyond the tested optimizer and implementation. 6.3 Why the model is not a quantum kernel method QEA computes an explicit feature vector for each input. It never evaluates k(xi,xj)k(x_i,x_j) between two examples, constructs a Gram matrix, or trains a kernel classifier. The Pauli vector could be used later to define a projected quantum kernel, but the attention decoder used here is not such a kernel. Renaming the method removes an ambiguity that otherwise obscures both novelty and comparison: the contribution is a controlled audit of a variational quantum embedding, not a new quantum kernel or a quantum implementation of self-attention. 6.4 Readout and scaling The original probability-token design would expose 2nq2^n_q basis-state probabilities to dense attention, producing (4nq)O(4^n_q) decoder cost. The current protocol instead reads all one-body ZiZ_i and two-body ZiZjZ_iZ_j expectations, so M=nq+(nq2)=nq(nq+1)2,dense-attention cost=(M2)=(nq4).M=n_q+ n_q2= n_q(n_q+1)2, -attention cost=O(M^2)=O(n_q^4). (9) Table 5: Measured forward cost of the core quantum stage on BCW (two-repetition Z feature map, one-repetition RealAmplitudes ansatz, batch size as configured). Times are workstation statevector measurements, not hardware latencies. Circuit depth and readout dimension grow with nqn_q; the observed time jump at eight qubits reflects this implementation and should not be extrapolated as a hardware speed model. nqn_q Readout dim M Circuit depth Decoder tokens Fwd/batch (ms) 2 3 13 3 26 4 10 36 10 30 6 21 56 21 29 8 36 76 36 218 Three resources must not be conflated. The classical decoder is polynomial in nqn_q. Exact statevector simulation still requires (2nq)O(2^n_q) memory and time up to circuit-depth factors. A quantum device avoids storing that statevector classically but introduces circuit executions, latency, sampling and gradient-estimation costs. A short classical output does not by itself rule out a quantum speedup—many quantum algorithms return few classical values—but this paper supplies neither a hardness argument for estimating these observables nor an end-to-end complexity advantage including state preparation and training. All reported observables commute and can be estimated from the same computational-basis shots. Each bounded expectation has standard error (S−1/2)O(S^-1/2) after S shots; simultaneous control of all M expectations adds a confidence dependence, while parameter-shift training multiplies circuit evaluations by the number of differentiated parameters. The paper uses exact expectations and therefore does not measure these costs. 6.5 Noise, hardware and the early fault-tolerant regime No hardware or finite-shot result is reported. Gate noise could degrade the deeper circuits, but it could also interact with optimization as an implicit regularizer; its direction cannot be inferred from the ideal null result. A hardware claim would require a specified device, transpiled depth, shot budget, error-mitigation protocol and repeated comparison against the same classical surrogate. Early fault-tolerant devices would relax fidelity and depth constraints, but they would not automatically validate this architecture. The QEA decoder is classical and consumes local expectations, so simply running the same shallow circuit with error correction does not create an algorithmic advantage. Fault-tolerant quantum linear-algebra transformer proposals use different access assumptions and block-encoded operations Zhang and Zhao [2025]; they are not scaled versions of QEA. A closer continuation would keep more processing on-device and measure class observables directly, for example ℓc=⟨ψ|Oc|ψ⟩ _c= ψ|O_c|ψ , but its trainability, data-loading cost and resource advantage would still have to be demonstrated rather than assumed. 6.6 Limitations The controlled evidence is limited to BCW, five paired seeds and two qubit counts; the near-ceiling accuracy makes it a weak environment for detecting component effects. The five-dataset grid is exploratory, contains heterogeneous readout widths and four shorter-budget rows, and uses QEA-R with a classical residual bypass. QM9 lacks current-protocol results. BirdCLEF uses a clip-level rather than recording-grouped split (Appendix B), so its estimates may be optimistic. The BCW and SUSY scalers were also fit before splitting; this shared, label-free leakage does not favor one paired architecture but can inflate absolute tabular performance (Appendix B). Finally, all quantum results are exact statevector simulations without finite shots or hardware noise. These limitations prohibit claims of quantum advantage, hardware readiness, modality-independent superiority, or equivalence between the PQC and its surrogate. 7 Conclusion Quantum-Embedded Attention provides a clear interface at which to test a PQC inside a hybrid classifier. The interface-matched BCW factorial does not show a consistent quantum-minus-classical effect across two circuit widths and two decoders. One small paired contrast is positive before correction, but it changes direction at the other width and does not survive correction across the four embedding contrasts. The experiment is too small and saturated to establish equivalence. The appropriate conclusion is therefore neither quantum advantage nor proof of no effect: under this protocol, the PQC contribution is not replicated across the tested settings. The exploratory five-dataset grid reinforces the need for that caution. QEA-R is competitive in point accuracy on three datasets but contains a classical residual bypass, is worse on SUSY, and fails on CIFAR-10. These observations characterize the full pipelines; they do not isolate their quantum layers. The most reusable result is consequently methodological: hybrid models should be evaluated with paired seeds and controls that preserve the projector, readout interface, decoder and training budget, and should report failed and incomplete runs explicitly. The next empirical priority is the same residual-free switch-A comparison on a non-saturated image or text task, with a prespecified effect size, more seeds, finite-shot execution and hardware noise. Longer-term, early fault-tolerant work may study deeper or on-device decoders, but it must include state-preparation, sampling and training resources in any advantage claim. Those architectures are a new hypothesis, not an extrapolation of the present results. Appendix A Backbone and baseline architectures This appendix specifies the classical components referenced in Section 4.1. The descriptions follow the configurations used for the reported rows rather than the defaults of the upstream model families. Vector backbone (tabular and text). BCW and SUSY use an MLP ℝdin→128→128R^d_in\!→\!128\!→\!128 with GELU after each affine layer, where din=30d_in=30 for BCW and 1818 for SUSY. There is no batch normalization. AG News is first encoded by frozen all-MiniLM-L6-v2 into 384-dimensional sentence vectors and then uses the same 384→128→128384\!→\!128\!→\!128 MLP. The plain Classical baseline appends a 128→C128\!→\!C linear classifier directly to this backbone, without an nqn_q bottleneck. Vision/audio backbone. BirdCLEF spectrograms and CIFAR-10 images use an ImageNet-pretrained ResNeXt-50 (32×4d) Xie et al. [2017]. The configured drop_last_blocks: 2 replaces stages 3 and 4 by identity maps; global average pooling therefore returns a 512-dimensional feature, not the 2048-dimensional output of the complete ResNeXt-50. Projector and decoder. In every Hybrid, Classical + Attention and QEA cell the projector gϕg_φ is d→32→nqd\!→\!32\!→\!n_q with GELU. The attention decoder uses embedding dimension 32, four heads, one self-attention block and a 64-unit classifier MLP. In the factorial, the classical switch-A surrogate is tanh(W+b) (W ξ+b) with W∈ℝM×nqW ^M× n_q, so the decoder sees exactly the same M-channel interface as for the PQC. It is an interface control rather than a parameter-count control. Table 6: Trainable parameter audit for the BCW factorial. “Map” counts the PQC parameters or the classical switch-A surrogate; “decoder” counts the attention stack or linear classifier. Circuit depth is reported only for the PQC rows. The small total-count differences are why we use interface-matched, not parameter-matched. nqn_q Cell Total Map Decoder Circuit depth 4 Quantum + Attention 36,014 8 11,266 36 4 Classical + Attention 36,056 50 11,266 – 4 Quantum + Linear 24,770 8 22 36 4 Classical + Linear 24,812 50 22 – 8 Quantum + Attention 36,986 16 12,098 76 8 Classical + Attention 37,294 324 12,098 – 8 Quantum + Linear 24,962 16 74 76 8 Classical + Linear 25,270 324 74 – Appendix B Dataset sizes, splits, and leakage caveats Table 7 records the analysis units represented by the current result files. Counts are stated separately for train, validation and test so that caps are not confused with source-dataset sizes. Table 7: Dataset statistics for the reported analyses. QM9 is shown for scope but its available rows use the legacy protocol and are excluded from current aggregate claims. Dataset Input to backbone Train/val/test used Classes Split construction BCW 30 features 397/86/86 2 stratified 70/15/15 SUSY 18 features 14,000/3,000/3,000 2 stratified 70/15/15 AG News MiniLM 384 20,000/2,000/2,000 4 official train/test CIFAR-10 224×224224×224 RGB 20,000/4,000/4,000 10 official train/test BirdCLEF 5-s mel spectrogram 1,920/240/240 2 clip-level fold QM9 14 descriptors legacy rows only 5 bins excluded here For AG News, 20,000 examples are selected from the official training split; the first 4,000 examples of the official test split are divided equally and stratified into validation and test. For CIFAR-10, 15% of the official 50,000 training images form the validation pool before caps are applied; the retained sets contain 20,000 training, 4,000 validation and 4,000 official-test images. The SUSY loader reads the first 20,000 finite rows of the local UCI file. Feature-scaling leakage. The BCW and SUSY loaders used for the archived experiments fit StandardScaler to the complete sampled feature matrix before the train/validation/test split. This does not use labels and is identical across paired model cells, so it does not explain a within-seed quantum–classical difference. It nevertheless transfers validation/test feature moments into training and can make absolute generalization estimates optimistic. The reported tabular scores must therefore be read as internal comparisons under this shared preprocessing pipeline, not as leakage-free benchmark estimates. A definitive rerun should split first, fit the scaler on training data only, and transform validation and test with those fixed statistics. BirdCLEF split (recording leakage caveat). The BirdCLEF task is built by cutting soundscape recordings into five-second clips. The train/evaluation split is stratified over clips (using the nocall label), not grouped by source recording. Clips from the same recording can therefore occur on both sides of the split, which may bias every BirdCLEF estimate upward. A recording-grouped split is required for a definitive audio comparison; the present BirdCLEF row is treated as exploratory. Reproducibility statement The source archive, configuration grid (including the interface-matched factorial), run-level CSV files, and analysis scripts that regenerate the reported tables and figures accompany the revision. Random seeds and dependency versions are recorded in the configurations and lockfile. Data availability All six source datasets are publicly available from the repositories cited in Section 4.1. The run-level measurements supporting the five-dataset cross-modality analysis and the BCW factorial accompany the revision as CSV files. Code availability The implementation and exact experiment configurations are available at https://github.com/MarkCodering/quantum-embedding-with-transformer and are also supplied with the revision so that review does not depend on repository availability. Competing interests This declaration must be completed and approved by all authors before resubmission. Acknowledgments Omitted for anonymous review. References P. Baldi, P. Sadowski, and D. Whiteson (2014) Searching for exotic particles in high-energy physics with deep learning. Nature Communications 5, p. 4308. Cited by: §4.1, Table 1. M. Benedetti, E. Lloyd, S. Sack, and M. Fiorentini (2019) Parameterized quantum circuits as machine learning models. Quantum Science and Technology 4 (4), p. 043001. Cited by: §1. V. Bergholm et al. (2018) PennyLane: automatic differentiation of hybrid quantum-classical computations. arXiv:1811.04968. Cited by: §2. H. Chen, Y. Chang, S. Liao, and C. Chang (2024) Quantum embedding with transformer for high-dimensional data. arXiv:2402.12704. Cited by: §1, §2. E. A. Cherrat, I. Kerenidis, N. Mathur, J. Landman, M. F. Strahm, and Y. Y. Li (2024) Quantum vision transformers. Quantum 8, p. 1265. Cited by: §2. V. Havlíček, A. D. Córcoles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow, and J. M. Gambetta (2019) Supervised learning with quantum-enhanced feature spaces. Nature 567, p. 209–212. Cited by: §2, §2. M. Henderson, S. Shakya, S. Pradhan, and T. Cook (2020) Quanvolutional neural networks: powering image recognition with quantum circuits. Quantum Machine Intelligence 2, p. 2. Cited by: §2. S. Kahl, T. Denton, H. Klinck, H. Glotin, H. Goëau, W. Vellinga, R. Planqué, and A. Joly (2021) Overview of BirdCLEF 2021: bird call identification in soundscape recordings. CLEF Working Notes. Cited by: §4.1, Table 1. D. P. Kingma and J. Ba (2015) Adam: a method for stochastic optimization. In International Conference on Learning Representations (ICLR), Cited by: §3.1, §4.4. A. Krizhevsky (2009) Learning multiple layers of features from tiny images. Technical report University of Toronto. Cited by: §4.1, Table 1. J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven (2018) Barren plateaus in quantum neural network training landscapes. Nature Communications 9, p. 4812. Cited by: §1, §3.5. A. Pérez-Salinas, A. Cervera-Lierta, E. Gil-Fuster, and J. I. Latorre (2020) Data re-uploading for a universal quantum classifier. Quantum 4, p. 226. Cited by: §2, §3.3, §6.1. J. Preskill (2018) Quantum computing in the NISQ era and beyond. Quantum 2, p. 79. External Links: Document Cited by: §1. Qiskit Machine Learning Developers (2024) Qiskit machine learning. External Links: Link Cited by: §2. R. Ramakrishnan, P. O. Dral, M. Rupp, and O. A. von Lilienfeld (2014) Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data 1, p. 140022. Cited by: §4.1, Table 1. J. Schnabel and M. Roth (2025) Quantum kernel methods under scrutiny: a benchmarking study. Quantum Machine Intelligence 7 (1), p. 58. External Links: Document Cited by: §2. M. Schuld, A. Bocharov, K. M. Svore, and N. Wiebe (2020) Circuit-centric quantum classifiers. Physical Review A 101, p. 032308. Cited by: §2. M. Schuld and N. Killoran (2019) Quantum machine learning in feature Hilbert spaces. Physical Review Letters 122, p. 040504. Cited by: §2. M. Schuld, R. Sweke, and J. J. Meyer (2021) Effect of data encoding on the expressive power of variational quantum-machine-learning models. Physical Review A 103, p. 032430. Cited by: §2, §3.3. W. N. Street, W. H. Wolberg, and O. L. Mangasarian (1993) Nuclear feature extraction for breast tumor diagnosis. SPIE Biomedical Image Processing and Biomedical Visualization 1905, p. 861–870. Cited by: §4.1, Table 1. W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou (2020) MiniLM: deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 33. Cited by: §3.2. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He (2017) Aggregated residual transformations for deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 1492–1500. Cited by: Appendix A, §3.2. H. Zhang and Q. Zhao (2025) A survey of quantum transformers: approaches, advantages, challenges, and future directions. arXiv:2504.03192. Cited by: §2, §6.5. X. Zhang, J. Zhao, and Y. LeCun (2015) Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 28. Cited by: §4.1, Table 1.