Paper deep dive
AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL
Daniele Lizzio Bosco, Jacopo Cossio, Carla Piazza, Giuseppe Serra
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/20/2026, 5:24:12 AM
Summary
The paper introduces AlphaClifford, a model-based Reinforcement Learning framework utilizing Monte Carlo Tree Search (MCTS) to efficiently synthesize and transpile Clifford circuits. By leveraging the algebraic properties of symplectic matrices to represent the state space, AlphaClifford minimizes circuit cost (gate counts) using a restricted gate set {H, S, CNOT}. The method outperforms standard heuristics like Aaronson-Gottesman and existing RL-based compilers in unconstrained optimization, hardware-constrained transpilation, and as a component in Clifford+T logical synthesis pipelines.
Entities (10)
Relation Signals (8)
AlphaClifford â synthesizes â Clifford circuits
confidence 95% · designed to efficiently synthesize Clifford circuits
AlphaClifford â uses â Symplectic matrices
confidence 95% · By modeling the state space through the algebraic properties of the symplectic group
AlphaClifford â uses â Monte Carlo Tree Search
confidence 95% · AlphaClifford, a model-based Reinforcement Learning framework powered by Monte Carlo Tree Search
AlphaClifford â usesgateset â H, S, CNOT
confidence 95% · synthesize Clifford circuits from the fundamental gate set composed of H, S, and CNOT
AlphaClifford â outperforms â existing RL-based compilers
confidence 90% · hardware-constrained Clifford transpilation, where we outperform existing RL-based compilers
AlphaClifford â outperforms â Aaronson-Gottesman algorithm
confidence 90% · achieves a consistent reduction in both total and two-qubit (CNOT) gate counts compared to state-of-the-art synthesis heuristics
AlphaTensor â inspired â AlphaClifford
confidence 85% · Inspired by recent successes on similar tasks from AlphaTensor
AlphaCNOT â inspired â AlphaClifford
confidence 85% · and AlphaCNOT
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Clifford circuits play a foundational role in quantum computing, particularly due to their importance in quantum error correction and fault-tolerant logical synthesis. While these circuits can be efficiently simulated and represented as symplectic matrices, standard synthesis methods-such as the Aaronson-Gottesman algorithm-often yield sub-optimal circuits with excessively high gate counts. In this work, we introduce AlphaClifford, a model-based Reinforcement Learning framework powered by Monte Carlo Tree Search, designed to efficiently synthesize Clifford circuits from the fundamental gate set composed of H, S, and CNOT. By modeling the state space through the algebraic properties of the symplectic group, AlphaClifford effectively explores this combinatorial space to minimize overall circuit cost. For unconstrained Clifford optimization, our approach achieves a consistent reduction in both total and two-qubit (CNOT) gate counts compared to state-of-the-art synthesis heuristics, despite operating with a strictly less expressive gate set. Furthermore, we demonstrate the broad applicability of our framework on two additional tasks: hardware-constrained Clifford transpilation, where we outperform existing RL-based compilers, and as a post-synthesis optimization component within a full Clifford+T logical synthesis pipeline. Our results underscore that model-based RL is highly effective at addressing the combinatorial complexities of quantum compilation, offering a scalable pathway to mitigate hardware constraints in both near-term and future fault-tolerant quantum devices.
Tags
Links
- Source: https://arxiv.org/abs/2608.18946v1
- Canonical: https://arxiv.org/abs/2608.18946v1
Trouble viewing inline? Open PDF directly â
Full Text
59,507 characters extracted from source content.
Expand or collapse full text
AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL Daniele Lizzio Bosco Affiliation: University of Udine Udine, Italy University of Naples âFederico Iâ Naples, Italy lizziobosco.daniele@spes.uniud.it Jacopo Cossio Affiliation: University of Udine Udine, Italy cossio.jacopo@spes.uniud.it Carla Piazza Affiliation: University of Udine Udine, Italy carla.piazza@uniud.it Giuseppe Serra Affiliation: University of Udine Udine, Italy giuseppe.serra@uniud.it Abstract Clifford circuits play a foundational role in quantum computing, particularly due to their importance in quantum error correction and fault-tolerant logical synthesis. While these circuits can be efficiently simulated and represented as symplectic matrices, standard synthesis methodsâsuch as the Aaronson-Gottesman algorithmâoften yield sub-optimal circuits with excessively high gate counts. In this work, we introduce AlphaClifford, a model-based Reinforcement Learning framework powered by Monte Carlo Tree Search, designed to efficiently synthesize Clifford circuits from the fundamental gate set composed of H, S, and CNOT. By modeling the state space through the algebraic properties of the symplectic group, AlphaClifford effectively explores this combinatorial space to minimize overall circuit cost. For unconstrained Clifford optimization, our approach achieves a consistent reduction in both total and two-qubit (CNOT) gate counts compared to state-of-the-art synthesis heuristics, despite operating with a strictly less expressive gate set. Furthermore, we demonstrate the broad applicability of our framework on two additional tasks: hardware-constrained Clifford transpilation, where we outperform existing RL-based compilers, and as a post-synthesis optimization component within a full Clifford+T logical synthesis pipeline. Our results underscore that model-based RL is highly effective at addressing the combinatorial complexities of quantum compilation, offering a scalable pathway to mitigate hardware constraints in both near-term and future fault-tolerant quantum devices. Index Terms: Reinforcement Learning, Clifford Circuits, Quantum Synthesis, Quantum Optimization, Circuit Compilation â footnotetext: ©2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Fig. 1: a) We train our model to reproduce a target symplectic matrix. Each symplectic matrix is modelled as the node of a tree, where nodes are connected if the corresponding matrices can be reached with an operation from H, S, and CNOT. We train a MCTS-based model as a pair of value network (for nodes, e.g. lighter blue corresponds to a higher value) and policy network (for edges, e.g. wider arrows corresponds to higher probabilities). b) Given a target Clifford circuit CtarC_tar, we utilize our model to synthesize a circuit C with the same symplectic matrix, i.e. Pâ CâĄCtarP· C⥠C_tar for a suitable Pauli P. We then compute the P from initial circuit, and we apply the corresponding gates to the obtained circuit. c) AlphaClifford can be applied to a variety of tasks, including general Clifford circuits optimization, Clifford transpilation to ensure compatibility with a given hardware connectivity map, and as a component in a pipeline for logical synthesis, from a general initial circuit, to the final approximated version in Clifford+T. I Introduction The field of quantum computing is rapidly approaching the era of âquantum utilityâ [23], marking a pivotal transition where quantum devices cease to be experimental research instruments and begin to execute practical, large-scale applications. These applications are expected to span both scientific domains, such as Quantum Chemistry [2] and High Energy Physics [14], and industrial use cases, including discrete optimization [5] and finance [20]. Despite significant recent advancements in quantum hardware capabilitiesâmost notably in error rate reduction and coherence time improvements [17, 38]âthe execution of quantum algorithms remains fundamentally constrained by circuit depth and the total number of physical gates. To mitigate these hardware limitations, quantum circuit optimization plays a critical role. As equivalent unitary transformations can be realized by different sequences of quantum gates, the objective of quantum circuit synthesis [24, 3, 25, 12, 19] is to find a sequence of gates that implements a target circuit (or its mathematical representation) while minimizing a specific cost metric, such as overall circuit depth or the number of two-qubit entangling gates. A closely related challenge is quantum circuit transpilation [9, 53], which involves mapping a logical circuit onto a specific physical architecture. This requires decomposing the target circuit into a native gate set and respecting strict hardware constraints, such as the limited connectivity map of the physical qubits [40]. The problems of quantum circuit synthesis and transpilation have been extensively explored in the literature, utilizing a wide array of heuristic, algebraic, and search-based techniques [3, 55, 33, 22]. Recently, Reinforcement Learning (RL) has emerged as a highly effective paradigm for tackling these computationally hard optimization problems [44, 48, 29, 52, 36, 32, 42, 30, 10]. By learning complex patterns rather than relying on heuristic rules, RL-based approaches have demonstrated the ability to discover novel, non-intuitive gate sequences that rule-based compilers might otherwise miss. In this work, we propose a RL-based technique to address both the synthesis and transpilation tasks specifically for the set of Clifford circuits. Generated strictly by the Hadamard (H), Phase (S), and Controlled-NOT (CNOT) gates, Clifford circuits are one of the most studied gate sets, and play a central role in the logical synthesis pipeline, particularly within the context of Clifford+T decomposition [29]. A key advantage of Clifford gates is that they admit an efficient classical representation via symplectic matrices [1]. We exploit this compact representation to develop a RL framework designed to learn highly efficient syntheses and transpilation strategies for Clifford circuits. Inspired by recent successes on similar tasks from AlphaTensor [44] and AlphaCNOT [10], our method formulates the circuit compilation problem as a tree-based search over the space of symplectic matrices, with the aim of finding the shortest path from an initial matrix to the identity, allowing for a compact reconstruction of the corresponding Clifford circuit from the set gate H,S,CNOT\H,S,CNOT\. To demonstrate the broad applicability of our proposed method, we evaluate it on three different tasks: unconstrained Clifford optimization, Clifford transpilation to a target hardware connectivity map, and as a refinement step in a full logical synthesis pipeline. We show that our model achieves comparable or higher performance with respect to a variety of different techniques, including up to 56%56\% total gate reduction compared to the Aaronson-Gottesman algorithm [1] for the general Clifford optimization task, and up to 20%20\% compared to the RL-based Clifford transpiler from [30] on the task of transpilation. Our results suggest that AlphaClifford can be applied efficiently in different settings. The paper is organized as follows. In Section I we provide a brief introduction to the symplectic representation of Clifford circuits. In Section I we discuss related works. In Section IV we present our method. Experimental design and results are discussed in Section V. Finally, in Section VI we provide some final considerations and suggest possible directions for further works. I Symplectic Representation of Clifford Circuits I-A Stabilizer Formalism and Tableau Representation The n-qubit Pauli group nP_n consists of all n-fold tensor products of the Pauli matrices I,X,Y,Z\I,X,Y,Z\ together with the multiplicative phase factors ±1,±i\± 1,± i\. The Clifford group nC_n is defined as the normalizer of the Pauli group within the unitary group UâĄ(2n)U(2^n). Specifically, for any Clifford operator CânC _n and any Pauli operator PânP _n, the conjugation CâPâCâ CPC yields another operator in nP_n. The group nC_n is finitely generated by the Hadamard (H), Phase (S), and Controlled-NOT (CNOT) gates. By the Gottesman-Knill theorem [18], Clifford circuits can be efficiently simulated on classical hardware. The standard data structure for this is the tableau representation introduced by Aaronson and Gottesman [1]. The state of an n-qubit Clifford circuit is completely characterized by tracing the evolution of n destabilizer generators and n stabilizer generators. In this formalism, the circuit is represented as a 2ânĂ(2ân+1)2nĂ(2n+1) binary matrix T over the Galois field 2F_2: =[],T= [ array[]c|cX&Z&r array ], (1) where X and Z are 2ânĂn2nĂ n Boolean matrices denoting the presence of X and Z Pauli operators, respectively. The 2ân2n-dimensional column vector r contains Boolean phase bits indicating whether the overall sign of each respective generator is +1+1 (ri=0r_i=0) or â1-1 (ri=1r_i=1). As a simple example, consider a single-qubit system. The initial tableau corresponding to the identity circuit is I=[100010], _I= [ array[]c|c1&0&0\\ 0&1&0 array ], where the first row represents the destabilizer generator X and the second row represents the stabilizer generator Z. Applying a Hadamard gate H exchanges X and Z, resulting in the tableau H=[010100]. _H= [ array[]c|c0&1&0\\ 1&0&0 array ]. This illustrates how Clifford operations act linearly on the (,)(X,Z) components of the tableau by transforming Pauli generators. If we further apply an X gate, the Pauli structure remains unchanged, but the phase of the Z generator is flipped: XâH=[010101]. _XH= [ array[]c|c0&1&0\\ 1&0&1 array ]. Thus, while the matrices X and Z encode the Pauli action of the circuit, the vector r captures the accumulated phase information. I-B Symplectic Matrix Representation While the phase vector r is required to track the exact quantum state (including Pauli phases), the fundamental entangling structure and basis transformations induced by a Clifford circuit are entirely captured by the 2ânĂ2ân2nĂ 2n binary matrix M=[â]M=[X\;\;Z]. To preserve the commutation relations of the Pauli group, the matrix M must satisfy the symplectic condition over 2F_2: MâΩâMT=Ω(mod2),M M^T= 2, (2) where Ω is the 2ânĂ2ân2nĂ 2n block matrix defined as: Ω=[0InIn0], = bmatrix0&I_n\\ I_n&0 bmatrix, (3) and InI_n is the nĂnĂ n identity matrix. Consequently, M is an element of the symplectic group Sâpâ(2ân,2)Sp(2n,F_2). Crucially, there exists a surjective group homomorphism Ï:nâSâpâ(2ân,2)Ï:C_nâ Sp(2n,F_2) that maps a Clifford unitary to its corresponding symplectic matrix. The kernel of this homomorphism is precisely the Pauli group nP_n (ignoring global phases). I-C Implications for Circuit Synthesis In our context of synthesizing Clifford circuits, the homomorphism ÏÏ provides a powerful mathematical abstraction. If an RL agent restricts its objective to finding a sequence of H,S,CNOT\H,S,CNOT\ gates that yields a symplectic matrix MsynM_syn matching a target symplectic matrix MtargetM_target, the synthesized circuit CsynC_syn and the target circuit CtargetC_target are guaranteed to belong to the same coset of the Pauli group. Mathematically, this establishes an equivalence up to a Pauli layer: CtargetâĄPâ Csyn,C_target⥠P· C_syn, (4) where PânP _n. By tracking the evolution of the phase vector r alongside the synthesis process, the residual Pauli correction P can be computed efficiently in âĄ(n2)O(n^2) time. Therefore, mapping the RL state space to Sâpâ(2ân,2)Sp(2n,F_2) rather than the full Clifford group nC_n reduces the search space by a factor of 22ân2^2n, simplifying the learning process. I Related Work The problem of quantum circuit synthesis and compilation is extremely important for mitigating hardware limitations of current devices, and the literature addressing this challenge is extensive [51]. Within the context of Clifford circuits, the historical baseline is the well-known Aaronson-Gottesman algorithm [1], which demonstrated that this class of circuits can be efficiently simulated and synthesized in polynomial time with gates from H,S,CNOT\H,S,CNOT\. However, standard approaches usually generate sub-optimal circuits with respect to the number of gates. To address this, various advanced heuristic methods have been proposed. For instance, the approaches described in [49] (such as A*, greedy algorithms, and Volanto [47]) leverage a richer generating set to reduce the gate count in the circuit. Another approach is the one proposed in [7], where Clifford-specific template matching is combined with symbolic peephole optimization, obtaining relevant reductions in CNOT count. It is also worth noting that the literature offers solutions for the optimal and exact syntheses of Clifford circuits [6, 46]. However, finding the absolute minimum sequence of gates generally requires exponential time, drastically limiting the scalability of these methods to a very small number of qubits and rendering them unsuitable for practical, large-scale applications. Recently, many Reinforcement Learning (RL) based solutions have been proposed to address tasks related to circuit synthesis, optimization, and compilation, including state preparation [32], quantum architecture search [31], and âpre-compilationâ in logical synthesis pipelines in Q-PreSyn [34]. The synthesis of Clifford circuits is particularly well-suited for RL applications: the algebraic properties of these circuits provide a compact and highly efficient state representation, usually as a matrix of polynomial size. Initially, many works were based on model-free algorithms, such as Proximal Policy Optimization [45] (PPO), which learn directly by interacting with the environment without building an internal model of the dynamics. Notable among these is the recent work [30], which leverages RL for the routing and synthesis of Clifford circuits, yielding transpilers capable of significantly outperforming traditional heuristics like SABRE [54]. Another impactful model-free application is provided in [52] in the context of quantum circuit discovery for fault-tolerant logical state preparation, where RL was used to discover entirely new protocols that surpass the efficiency of manually designed circuits. Additionally, in [42] the authors propose a PPO-based approach to optimize CNOT count in linear reversible circuits. Despite the significant successes of model-free methods, recent developments demonstrate that when the mathematical structure of the problem allows it, model-based algorithms, usually based on Monte Carlo Tree Search [11] (MCTS), offer vastly superior capabilities for exploring complex combinatorial spaces. In this regard, AlphaTensor-Quantum [44] paved the way by demonstrating how the CNOT+T component of a circuit can be optimized using search techniques originally designed for board games [35]. Similarly, AlphaCNOT [10] provided a model to address the problems of CNOT optimization in both unconstrained and constrained (i.e., depending on a connectivity map) settings. Finally, another relevant contribution is QuSynth [15], in which the task of stabilizer state preparation is reformulated by using graph-state representations, and addressed by combining RL with MCTS for structured look-ahead. Our work, AlphaClifford, builds on these developments by bringing the strengths of search-informed reinforcement learning to the synthesis of general Clifford circuits. In particular, we draw inspiration from the AlphaTensor and AlphaCNOT frameworks, adapting their core ideas to exploit the algebraic structure of Clifford operations and further advance the state of the art in this domain. IV AlphaClifford A popular class of Reinforcement Learning (RL) approaches is classified as model-free, i.e., they learn directly from interactions with the environment and do not leverage an explicit model of its dynamics. Model-free approaches, such as Proximal Policy Optimization [45] (PPO), rely on sequences of stateâactionâreward transitions to iteratively update the policy via gradient-based optimization. Conversely, model-based RL algorithms construct a representation of the environment to simulate future outcomes [26]. Within complex combinatorial spaces, such as the set of symplectic matrices, their capacity for structured and efficient exploration enables them to obtain improved performance over both classical heuristics and model-free algorithms. Following the latter principle, and inspired by recent applications such as AlphaTensor [44] and AlphaCNOT [10], we propose AlphaClifford, a novel model-based approach for exploring the combinatorial space of symplectic matrices associated with Clifford circuits. Our method, illustrated in Fig. 1, is organized into the following steps: âą First, we model the space of symplectic matrices as a search tree (Fig. 1a), where each node represents a specific symplectic matrix and directed edges correspond to the application of Clifford gates. In this representation, two nodes are connected if the corresponding matrices can be reached through a Clifford gate operation. âą Second, we train a model to learn the structure of this space by predicting sequences of gate operations that transform the initial symplectic matrix into the target one. The training procedure, structured after the Monte Carlo Tree Search pattern [11], is detailed in Section IV-C. âą Third, once trained, the model is used to synthesize Clifford circuits (Fig. 1b) by generating a new sequence of gates from a target circuit. The generated sequence of Clifford gates will have the same symplectic matrix as the one associated to the target circuit. Fig. 2: Overview of the Monte Carlo Tree Search (MCTS) phases for Clifford circuit synthesis. The search space is explored by iteratively executing four distinct stages: (a) Selection: The tree is traversed from the root (representing the target symplectic matrix C) to a leaf node. Actions, corresponding to Clifford gates H,S,H,S, and CNOT, are chosen by balancing exploration and exploitation using the UCT formula. (b) Expansion: When reaching a leaf node, the tree is expanded by the addition of one or more child nodes, each representing the state resulting from a possible Clifford operation. (c) Simulation: A rollout is performed from the newly expanded node based on the policy network. A sequence of gates is sampled until a terminal state is reached, at which point the evaluation of the value network is computed based on the proximity to the identity matrix. (d) Backpropagation: The reward obtained during simulation is propagated upward to the root, updating the value estimates and visit counts for all nodes along the selected path to refine future selection steps. IV-A Modeling the problem of symplectic matrices The search space of our problem, represented by the set of symplectic matrices, is explored through a tree-based approach. For a given problem dimension n (corresponding to the number of qubits), given a target Clifford circuit C represented by its symplectic matrix MCâSâpâ(2ân,2)M_Câ Sp(2n,F_2), and possibly a connectivity map âłM, our aim is to determine a sequence of Clifford gates equivalent to C. To this end, we construct a search tree as follows: âą The root node represents the identity matrix I2ânI_2n, corresponding to the empty circuit; âą Each node N is labeled by a symplectic matrix M and has one child for each admissible Clifford operation gâHi,Si,CNOTiâj:i,jâ1,âŠ,ngâ\H_i,S_i,CNOT_ij:i,jâ\1,âŠ,n\\ (subject to âłM). The child node is labeled by the matrix obtained by applying g to M, i.e., by the product of M with the matrix corresponding to g; âą The terminal nodes (i.e., the leaves) are labeled by the target symplectic matrix MCM_C. In the unconstrained setting, each node has ÎâĄ(n2) (n^2) children due to the possible CNOT operations between pairs of qubits, along with single-qubit gates. In the case of limited connectivity constraints, only a subset of 2-qubit operations is allowed. IV-B Inverting the representation: from the target to the identity Training an RL agent with the forward representation described in Section IV-A would require providing, at each step k, both the current symplectic matrix MkM_k (encoding the partial circuit g1,âŠ,gkg_1,âŠ,g_k) and the fixed target matrix MCM_C. This increases the input dimensionality and, moreover, the target matrix contributes little information during an episode, as it remains constant. To avoid this, we adopt an inverted formulation that embeds the target directly into the state representation. At step k, the observation is defined as MCâMkâ1,M_C\,M_k^-1, where Mkâ1M_k^-1 is the inverse of the current symplectic matrix over 2F_2, corresponding to the inverse circuit. Under this representation, synthesizing the target Clifford circuit is equivalent to reaching the identity: MCâMCâ1=I2ân.M_C\,M_C^-1=I_2n. Hence, each episode begins in the state MCM_C, and the agentâs task is to reach the identity matrix. This turns the problem into a goal-reaching task with a fixed terminal state and allows us to provide a single observation matrix to the model, rather than a pair (Mk,MC)(M_k,M_C). When a gate gtg_t is applied in the forward construction of the circuit, the corresponding update in the inverted representation must use the adjoint action. Since Mt=MgtâMtâ1âčMtâ1=Mtâ1â1âMgtâ ,M_t=M_g_t\,M_t-1 M_t^-1=M_t-1^-1M_g_t , the state transition in the inverted formulation becomes MCâMtâ1â1â¶MCâ(Mtâ1â1âMgtâ ).M_C\,M_t-1^-1\; \;M_C (M_t-1^-1M_g_t ). Thus, although the agent selects forward Clifford operations, the evolution of the internal representation proceeds by right-multiplication with the corresponding symplectic matrices, consistently reflecting the reversed dynamics. IV-C Monte Carlo Tree Search Framework We adopt a Monte Carlo Tree Search [11] (MCTS) strategy to efficiently explore the space of symplectic matrices associated with Clifford circuits (see Fig. 2). The procedure is structured into four main phases: 1. Selection: Starting from the root node corresponding to the target symplectic matrix MCM_C, the tree is traversed by iteratively selecting actions (i.e., Clifford gates in H,S,CNOT\H,S,CNOT\) according to a tree policy. In particular, we adopt the Upper Confidence Bound for Trees (UCT) criterion [26], which selects the action a maximizing UCTâĄ(a)=QâĄ(s,a)+câlogâĄNâĄ(s)NâĄ(s,a),UCT(a)=Q(s,a)+c N(s)N(s,a), (5) where QâĄ(s,a)Q(s,a) is the estimated value of taking action a in state s, NâĄ(s)N(s) is the visit count of node s, NâĄ(s,a)N(s,a) is the number of times action a has been selected from s, and c is an exploration constant. This criterion balances exploration of less-visited actions and exploitation of high-value ones; 2. Expansion: When a non-terminal node with unexplored actions is reached, the tree is expanded by applying a new admissible Clifford operation, thus generating a new child node. A node is terminal if it corresponds to the identity matrix or if other stopping criteria are met (e.g., maximum depth). In the presence of connectivity constraints, only allowed CNOT operations are considered; 3. Simulation: From the expanded node, a rollout is performed by sampling a sequence of Clifford gates according to the policy network until a terminal node is reached. The resulting trajectory is then processed by the value function to estimate the quality of the new node; 4. Backpropagation: The outcome of the simulation is propagated back along the selected path, updating visit counts and value estimates for each node involved in the traversal. IV-D Training Details The training consists of constructing a different random target matrix for each episode, and asking the model to âreachâ the identity matrix starting from the inverse of the target with the fewest amount of moves. Each âmoveâ, corresponding to a step in the environment, represents the application of a feasible gate to the current circuit. Feasible gates are H, S, and CNOTs, possibly with constraints based on topology connectivity. An episode finishes after reaching the identity, or when the agent perform nmaxn_max moves without reaching the identity. We adopt a naive reward function to guide the learning process. At each step a penalty of â1/n-1/ n is applied to limit long sequences and a terminal reward of +n+n is assigned when reaching the identity matrix. This ânon-informedâ strategy is fundamental in our approach: without relying on problem-specific heuristics, the model itself naturally learns to find shorter decompositions. In particular, the training objective implicitly minimizes the total number of applied gates, which also implies a reduction of CNOTs. Note that different rewards could be used to minimize different target functions: for example, we can select a higher penalty to reduce the CNOT count. We employ a linear curriculum learning strategy [4] based on the (expected) difficulty of the instances. In practice, we generate each target symplectic matrix by selecting random k Clifford gates, where k is parameter we can tune during training. More in detail, we divide our training in tranches. Starting from k=1k=1, we train our model on a tranche of difficulty k. At the end of the tranche, we increase k if the model reached a solving rate of at least 95%95\%, i.e., the model is able to correctly reproduce most matrices. We stop the training after the max difficulty of kmax=4ân2k_ =4n^2 is reached. IV-E Inference Once the model has been trained to synthesize symplectic matrices of size 2ânĂ2ân2nĂ 2n, optionally subject to hardware connectivity constraints, it can be deployed for inference. Specifically, given a target Clifford circuit CtarC_tar characterized by a symplectic matrix MtarM_tar, our model generates a sequence of gates forming a synthetic circuit CsynC_syn that yields the identical symplectic matrix. Consequently, Mtar=MsynM_tar=M_syn, which establishes that the target and synthetic circuits are equivalent up to a Pauli operator (i.e., CtarâĄPâCsynC_tar⥠PC_syn for a pauli P). The complete inference procedure for a given target circuit proceeds as follows: (i) efficiently compute its target symplectic matrix MtarM_tar; (i) query the trained model to synthesize a gate sequence corresponding to this matrix; (i) efficiently compute the phase discrepancyânamely, the signs of the stabilizers and destabilizersâbetween the target and synthetic circuits; and (iv) apply the appropriate X and Z gates to correct the phase, yielding a fully equivalent circuit. A visual overview of this pipeline is provided in Fig. 1b. IV-F Applications We identify three primary applications where our model can be efficiently deployed: Clifford optimization (Fig. 1c, top), Clifford transpilation (Fig. 1c, middle), and as a subroutine within a standard logical synthesis pipeline (Fig. 1c, bottom). These settings are described below and are experimentally demonstrated in Sections V-A, V-B, and V-C, respectively. IV-F1 Optimization A direct application of our framework is Clifford circuit optimization. Given a target Clifford circuit, the objective is to find an equivalent representation that minimizes the total number of two-qubit gates. More broadly, the model can be tailored during training to minimize alternative target metrics, such as overall circuit depth. Beyond full-circuit synthesis, AlphaClifford can also be employed as a peephole optimizer [39] for Clifford circuits involving a larger number of qubits. Starting, for example, from a circuit synthesized using the AaronsonâGottesman algorithm, one can extract contiguous subcircuits whose support contains at most n qubits, resynthesize each block using the corresponding n-qubit AlphaClifford model, and replace the original block whenever the selected cost is reduced. By iteratively applying this procedure, models trained on relatively small problem sizes can be used to optimize substantially larger circuits, although the resulting improvements are local and do not imply global optimality. IV-F2 Transpilation We further evaluate our approach on the task of quantum circuit transpilation. Given n qubits and a hardware connectivity mapârepresented as a graph where nodes correspond to physical qubits and edges denote allowable direct interactionsâthe goal of transpilation is to transform a logical circuit into an equivalent physical circuit that strictly adheres to these connectivity constraints. This is usually achieved by mapping logical qubits to physical qubits and inserting additional gates (e.g., SWAP routing) to ensure all two-qubit operations occur between adjacent nodes. Addressing this task is critical for near-term quantum execution, as many hardware platforms (such as superconducting architectures [28]) only permit multi-qubit operations between physically coupled qubits. Similar to circuit optimization, the objective is to find a valid transpilation that minimizes a specific cost function, typically the two-qubit gate count or the overall circuit depth. Furthermore, for realistic hardware deployments, the cost function can be expanded to incorporate device-specific characteristics, such as penalizing the use of qubits or couplers with high error rates. To accomplish this within our framework, the modelâs action space during training is constrained to the specific connectivity map of the target hardware connectivity map. IV-F3 Optimizing Logical Synthesis Pipeline A crucial task in fault-tolerant quantum computing is logical synthesis, which involves approximating a target unitary using a discrete, universal gate set such as Clifford+T [24]. Unlike pure Clifford synthesis, optimal exact synthesis over universal gate sets is a notoriously difficult problem, with worst-case computational complexity scaling exponentially with respect to the number of qubits [13, 37]. To address this limitation, compilers typically employ a âpeepholeâ synthesis strategy, decomposing the target circuit into much smaller, tractable sub-blocks (typically acting on two or three qubits) [7]. A standard logical synthesis pipeline generally operates in successive stages. First, a pre-processing step can applied to the logical circuit. This may involve decomposing large multi-qubit gates into smaller two- or three-qubit instances (e.g., via recursive Cartan decomposition [8, 50]) or performing structural pre-synthesis edits to make the circuit more amenable to compilation [34]. Next, a local synthesis algorithm compiles each of these individual sub-blocks into a Clifford+T representation. Our framework can be integrated into the final stage of this pipeline as a post-synthesis optimizer. Once the full circuit has been compiled into Clifford+T, we can extract the largest contiguous blocks of purely Clifford gates. Our model can then be applied directly to these extracted Clifford sub-circuits to find equivalent representations that minimize the total gate count, effectively compressing the circuit without altering the expensive T-gate layout. While our experimental evaluation in Section V-C demonstrates this capability on the relatively small Clifford blocks generated by standard peephole synthesis, our approach can be applied more in general in any kind of compilation task involving large blocks of consecutive Clifford gates. V Experimental Results In the following, we describe the experimental evaluations and the obtained results for the tasks of Clifford Optimization (Section V-A), Clifford Transpilation (Section V-B), and optimization in the logical synthesis pipeline (Section V-C). Both policy and value networks share the same architecture, i.e., a 12-layer fully connected network with 256 hidden units per layer. We structured the net using 4 residual blocks with skip connections every three layers. We trained a different model for each number of qubits n, and for each selected connectivity map for the transpilation task. The training was executed in tranches of different complexity (computed as the number of generating gates) as described in Section IV-D, and increasing once the solving threshold of 95% is achieved for the tranche, up to a complexity of 4ân24n^2. V-A Clifford Optimization As a first task, we evaluate the ability of our model to efficiently synthesize sufficiently complex Clifford circuits. This problem can be reformulated as a Clifford optimization task: we start from a random Clifford circuit generated by qiskit [21], and we synthesize an equivalent circuit with gates from H,S,CNOT\H,S,CNOT\. For n qubits with nâ3,4,5,6,7nâ\3,4,5,6,7\, we evaluate our model on 100100 Clifford circuits involving up to 4ân24n^2 gates. These experiments evaluate full synthesis, in which the complete target circuit is provided to a single model. Note that, as discussed in Section IV-F, the same models can be applied as local peephole optimizers to blocks of larger Clifford circuits. To evaluate our approach, we consider both the average total number of gates, and the number of two-qubit gates (in our case, CNOTs) required to synthesize a circuit. We evaluate our model both one shot, and after 1010 repetitions, i.e. we sample our model 1010 times, and select the circuit with lowest amount of gates. We denote the latter as AlphaClifford10. Our results are reported in Table I. As comparison, we consider first the well-known Aaronson-Gottesman algorithm [1], as implemented in qiskit [21]), and then three different heuristic models (AâA^*, greedy, and Volanto [47]), as provided in [49]. These three methods provide a decomposition in a block of SWAP gates, followed by a block of one and two qubit transvection gates [27]. Given a n-qubit Pauli operator P, the corresponding transvection can be defined as TPâexpâĄ(iâÏ4â(InâP)).T_P ( iÏ4(I_n-P) ). (6) Note that by allowing a richer gate set provides an unfair comparison versus our approach that is constrained to the less expressive set H,S,CNOT\H,S,CNOT\. Nevertheless, we include this evaluation, where the SWAP layer is first decomposed in three CNOTs. For completeness, we also consider the CNOT count obtained by the same three methods when the obtained circuits are decomposed in H,S,CNOT\H,S,CNOT\ by the default transpiler provided by qiskit. Finally, we report the same metrics obtained by the qiskit greedy method, and by the well-known stim software for stabilizer simulation [16]. Problem Size (n) Approach 3 4 5 6 7 Total Gates Aaronson-Gottesman 19.99 25.89 38.05 51.37 65.61 Qiskit greedy 13.57 23.03 34.75 47.77 62.18 Stim 21.27 36.28 54.70 72.82 96.39 A* (with transv.) 12.97 19.56 26.54 35.04 43.15 Greedy (with transv.) 13.03 19.40 27.42 36.08 45.61 Volanto (with transv.) 12.95 19.93 28.65 38.12 48.14 A* (H,S,CNOTH,S,CNOT) 19.49 33.47 49.90 68.33 89.47 Greedy (H,S,CNOTH,S,CNOT) 19.17 33.21 51.44 72.16 98.29 Volanto (H,S,CNOTH,S,CNOT) 20.26 37.30 59.25 87.19 117.57 AlphaClifford 8.85 13.90 20.66 29.56 40.05 AlphaClifford10 8.78 13.39 19.49 27.11 36.75 2-Qubit Gates Aaronson-Gottesman 3.65 10.41 16.17 22.43 28.59 Qiskit greedy 4.35 8.01 13.01 18.79 24.95 Stim 7.29 13.20 20.32 29.49 40.35 A* (with transv.) 6.34 10.63 15.72 21.39 27.32 Greedy (with transv.) 6.28 10.60 16.55 22.71 29.86 Volanto (with transv.) 6.38 11.16 17.36 24.38 32.62 A* (H,S,CNOTH,S,CNOT) 8.84 15.38 23.10 31.89 41.59 Greedy (H,S,CNOTH,S,CNOT) 8.78 15.39 24.39 34.19 46.28 Volanto (H,S,CNOTH,S,CNOT) 9.19 17.04 27.37 39.85 54.23 AlphaClifford 4.25 7.50 11.85 17.47 25.53 AlphaClifford10 4.06 6.96 10.98 15.82 23.06 TABLE I: Average number of total and 2-Qubit gates for different optimization methods over varying problem sizes. Best results for each n are highlighted in bold. We observe that our model achieves the lowest amount of total gates, and two qubit gates, in most cases. Note that all methods were able to synthesize correctly each instance, with the exception of Greedy method that was not able to synthesize a few instances of larger dimensions (in this case, the average reported in Table I is considered only on the solved instances). We stress that the advantage compared to the methods in [49] is obtained with a less expressive gate. We expect our method to obtain even better performance when allowing more expressive gate sets. V-B Transpilation L Maps T Maps 3L 4L 5L 4T 5T 6T Approach Tot. CNOT Tot. CNOT Tot. CNOT Tot. CNOT Tot. CNOT Tot. CNOT RL-Qiskit1 14.52 4.91 26.80 10.51 36.95 16.78 26.38 9.60 33.63 15.15 50.85 24.41 RL-Qiskit10 12.19 4.60 22.90 9.54 33.31 16.08 21.58 8.43 30.92 14.29 46.11 22.74 AlphaClifford1 9.83 5.15 17.49 10.66 29.07 18.45 16.16 9.51 26.27 16.82 42.35 27.49 AlphaClifford10 9.81 4.99 17.08 10.08 27.58 17.31 15.76 8.96 25.04 15.56 38.84 25.10 TABLE I: Performance of RL-Qiskit [41] and AlphaClifford for the transpilation task. Results report average total (Tot.) and CNOT gate counts for L and T connectivity maps. Best values for metric are reported in bold. To evaluate our model, for each number of qubits and connectivity map, we generate a random Clifford circuit without connectivity constraints. We then use a model trained on the specific connectivity map to generate an equivalent circuit by using the gate set H,S,CNOT\H,S,CNOT\, where a CNOT can be selected only between pairs of qubits allowed by the connectivity map. Following the procedure in [30], we select 66 connectivity maps from 33 to 66 qubits (see Fig. 3 for additional information). As before, we provide total gate count, and CNOT count. As a comparison, we evaluate the RL-based Clifford transpiler by qiskit, available at [41]. Both methods are sampled once and 1010 times, to take into account the stochasticity or RL-based approaches. Results are provided in Table I. We observe that in general, our model provides consistently a lower total gate counts than the other approach. However, it also obtains a slightly higher number of CNOTs, suggesting that a reward function tailored on the number of two-qubit gates instead of total gates could be beneficial in this specific setting. (a) 3L (b) 4L (c) 5L (d) 4T (e) 5T (f) 6T Fig. 3: Hardware connectivity maps utilized for benchmarking: Line graphs (a-c) and T-shape graphs (d-f). Vertices represent physical qubits and edges represent available coupling map connections. V-C Full Logical synthesis pipeline Finally, we evaluate our proposed method as a component of a logical synthesis pipeline. To do so, we first generate 1010 random circuits for each number of qubits nâ5,10,15,20,25nâ\5,10,15,20,25\. Following the approach in [34], we consider the pipeline: âą Starting from the representation of the initial circuit, we decompose each 2 qubit gates in a block of CNOTs and single qubit gates with the KAK decomposition [8]; âą Then, we decompose each single qubit unitary in up to three Rzâ(Ξ)R_z(Ξ) rotations (with additional X and H gates). Each rotation is then decomposed in H,S,T\H,S,T\ with the gridsynth algorithm [43] with a tolerance of Δ=0.01 =0.01; âą Finally, we search for blocks of adjacent Clifford circuits. For each block of at least 22 qubits, we use AlphaClifford to synthesize an optimized version. Initial Compiled Qiskit Transpile AlphaClifford n Clifford (k)(k) T-count (k)(k) Clifford (k)(k) %râeâd\%_red Clifford (k)(k) %râeâd\%_red Without Pre-compilation 5 1.24 0.76 1.23 0.36 1.21 2.44 10 5.17 3.14 5.16 0.13 5.05 2.39 15 10.27 6.23 10.26 0.09 10.01 2.53 20 20.26 12.28 20.24 0.14 19.76 2.50 25 29.59 17.92 29.55 0.14 28.88 2.43 With Pre-compilation 5 1.09 0.66 1.08 0.35 1.06 2.75 10 4.90 2.97 4.90 0.17 4.78 2.46 15 9.98 6.06 9.97 0.15 9.72 2.57 20 19.77 11.98 19.75 0.12 19.28 2.50 25 29.18 17.66 29.12 0.20 28.45 2.49 TABLE I: Clifford synthesis performance: comparison of Clifford gate counts (in thousands) and reduction percentages relative to the original synthesized circuit. Results are grouped by whether pre-synthesis [34] was applied. In addition, we evaluate the same pipeline with the use of Q-PreSyn [34] to optimize the number of T gates with a sequence of 22 qubit merges. In particular, we apply a greedy optimizer to evaluate the largest sequence of gates that can be merged together, with the aim of reducing the number of resulting T gates after applying the KAK decomposition followed by gridsynth. In Table I we present the Clifford gate reduction, with the default qiskit transpiler as a comparison. Note that in this specific setting, due to the particular structure of the Clifford circuits (that are already optimal in the number of CNOT gates), the algorithms Aaronson-Gottesman and the ones from [49] did not obtain any reduction in total number of gates. Finally, note that the considered pipeline applied to random circuits leads to mostly 2 and 3 qubit Clifford blocks, therefore allowing for a small reduction of gates. We expect different choices of logical synthesis algorithms and different circuit families to provide higher reductions. VI Conclusion In this work, we introduced AlphaClifford, a novel model-based reinforcement learning framework designed to address the complex combinatorial challenges of Clifford circuit synthesis and transpilation. By formulating the compilation process as a Monte Carlo Tree Search over the algebraic space of symplectic matrices, AlphaClifford consistently discovers highly optimized gate sequences. Our experimental evaluations demonstrate that AlphaClifford achieves superior or comparable results to state-of-the-art methods across a variety of settings. For unconstrained optimization, it successfully minimizes both total and two-qubit gate counts compared to standard heuristics, even when restricted to a less expressive gate set. Furthermore, we showed its adaptability to hardware-constrained transpilation and its practical utility as a post-synthesis optimizer within a logical synthesis pipeline. Notably, these results were obtained using a single base architecture and a naive reward function, underscoring the generality and robustness of our technique. Ultimately, these results highlight the potential of model-based RL to transcend the limitations of traditional heuristic-driven compilers, advancing the trajectory established by frameworks like AlphaTensor [44] and AlphaCNOT [10]. As quantum hardware continues to scale toward the era of quantum utility, computationally intelligent approaches like AlphaClifford will be critical for bridging the gap between high-level logical algorithms and the strict physical constraints of near-term and future fault-tolerant quantum devices. We highlight several promising directions for future work. First, extending the action space to include more expressive operations, such as transvections, could further enhance compilation efficiency. Second, while our generalized reward function successfully minimized total gate counts, designing tailored reward structuresâsuch as heavily penalizing two-qubit gates to improve specific transpilation connectivity maps or incorporating error-aware routingâcould yield even better hardware-specific solutions. Finally, a further direction is to investigate systematic block-selection and iterative peephole strategies for applying the models developed here to large-scale Clifford circuits. VII Acknowledgements DLB thanks Gianluca MacrĂŹ for his support in preparing the figures and diagrams of this work. JC is supported by FSE/FVG PhD Grant on âComputer Science and Artificial Intelligenceâ (CUP G23C25000620008). This work has been partially supported by INdAM-GNCS project Algebra lineare quantistica, state preparation e compilazione di circuiti quantistici (CUP E53C25002010001) and by the regional project QUASAR-FVG Calcolo e simulazione quantistica: sviluppo, applicazioni e ricerca in Friuli Venezia Giulia (CUP G23C25001510002). References [1] S. Aaronson and D. Gottesman (2004) Improved simulation of stabilizer circuits. Phys. Rev. A 70, p. 052328. Cited by: §I, §I, §I-A, §I, §V-A. [2] Y. Alexeev, V. S. Batista, N. Bauman, L. Bertels, D. Claudino, R. Dutta, L. Gagliardi, S. Godwin, N. Govind, M. Head-Gordon, M. R. Hermes, K. Kowalski, A. Li, C. Liu, J. Liu, P. Liu, J. M. GarcĂa-Lastra, D. Mejia-Rodriguez, K. Mueller, M. Otten, B. Peng, M. Raugas, M. Reiher, P. Rigor, W. J. Shaw, M. van Schilfgaarde, T. Vegge, Y. Zhang, M. Zheng, and L. Zhu (2025) A perspective on quantum computing applications in quantum chemistry using 25â100 logical qubits. J. Chem. Theory Comput. 21 (22), p. 11335â11357. Cited by: §I. [3] M. Amy, D. Maslov, M. Mosca, and M. Roetteler (2013) A Meet-in-the-Middle Algorithm for Fast Synthesis of Depth-Optimal Quantum Circuits. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 32 (6), p. 818â830. Cited by: §I, §I. [4] Y. Bengio, J. Louradour, R. Collobert, and J. Weston (2009) Curriculum learning. In Proc. 26th Int. Conf. Mach. Learn. (ICML), Cited by: §IV-D. [5] A. Bochkarev, R. Heese, S. JĂ€ger, P. Schiewe, and A. Schöbel (2026) Quantum computing for discrete optimization: a highlight of three technologies. Eur. J. Oper. Res. 329 (3), p. 747â766. Cited by: §I. [6] S. Bravyi, J. A. Latone, and D. Maslov (2022) 6-qubit optimal Clifford circuits. npj Quantum Information 8 (1), p. 79. Cited by: §I. [7] S. Bravyi, R. Shaydulin, S. Hu, and D. Maslov (2021) Clifford Circuit Optimization with Templates and Symbolic Pauli Gates. Quantum 5, p. 580. External Links: ISSN 2521-327X Cited by: §I, §IV-F3. [8] E. Cartan (1926) Sur une classe remarquable dâespaces de Riemann. Bull. Soc. Math. Fr. 2, p. 214â264. Cited by: §IV-F3, 1st item. [9] F. Chong, D. Franklin, and M. Martonosi (2017) Programming languages and compiler design for realistic quantum hardware. Nature 549, p. 180â187. Cited by: §I. [10] J. Cossio, D. Lizzio Bosco, R. Romanello, G. Serra, and C. Piazza (2026) AlphaCNOT: learning cnot minimization with model-based planning. External Links: 2604.13812 Cited by: §I, §I, §I, §IV, §VI. [11] R. Coulom (2007) Efficient selectivity and backup operators in monte-carlo tree search. In in Computers and Games, 5th Int. Conf., Turin, Italy, May 2006, revised papers, Cited by: §I, 2nd item, §IV-C. [12] M. G. Davis, E. Smith, A. Tudor, K. Sen, I. Siddiqi, and C. Iancu (2020) Towards Optimal Topology Aware Quantum Circuit Synthesis. In 2020 IEEE International Conference on Quantum Computing and Engineering (QCE), Denver, CO, USA, p. 223â234. Cited by: §I. [13] C. M. Dawson and M. A. Nielsen (2006) The solovay-kitaev algorithm. Quantum Info. Comput. 6 (1), p. 81â95. Cited by: §IV-F3. [14] A. Di Meglio, K. Jansen, I. Tavernelli, C. Alexandrou, S. Arunachalam, C. W. Bauer, K. Borras, S. Carrazza, A. Crippa, V. Croft, R. de Putter, A. Delgado, V. Dunjko, D. J. Egger, E. FernĂĄndez-Combarro, E. Fuchs, L. Funcke, D. GonzĂĄlez-Cuadra, M. Grossi, J. C. Halimeh, Z. Holmes, S. KĂŒhn, D. Lacroix, R. Lewis, D. Lucchesi, M. L. Martinez, F. Meloni, A. Mezzacapo, S. Montangero, L. Nagano, V. R. Pascuzzi, V. Radescu, E. R. Ortega, A. Roggero, J. Schuhmacher, J. Seixas, P. Silvi, P. Spentzouris, F. Tacchino, K. Temme, K. Terashi, J. Tura, C. TĂŒysĂŒz, S. Vallecorsa, U. Wiese, S. Yoo, and J. Zhang (2024) Quantum computing for high-energy physics: state of the art and challenges. PRX Quantum 5, p. 037001. Cited by: §I. [15] M. Doherty, M. Puviani, J. Brewer, G. Matos, D. Amaro, B. Criger, and D. T. Stephen (2026) Fast stabilizer state preparation via AI-optimized graph decimation. arXiv. Cited by: §I. [16] C. Gidney (2021) Stim: a fast stabilizer circuit simulator. Quantum 5, p. 497. External Links: Document, Link, ISSN 2521-327X Cited by: §V-A. [17] Google Quantum AI, R. Acharya, I. Aleiner, R. Allen, T. I. Andersen, M. Ansmann, F. Arute, K. Arya, A. Asfaw, J. Atalaya, R. Babbush, D. Bacon, J. C. Bardin, J. Basso, A. Bengtsson, S. Boixo, G. Bortoli, A. Bourassa, J. Bovaird, L. Brill, M. Broughton, B. B. Buckley, D. A. Buell, T. Burger, B. Burkett, N. Bushnell, Y. Chen, Z. Chen, B. Chiaro, J. Cogan, R. Collins, P. Conner, W. Courtney, A. L. Crook, B. Curtin, D. M. Debroy, A. Del Toro Barba, S. Demura, A. Dunsworth, D. Eppens, C. Erickson, L. Faoro, E. Farhi, R. Fatemi, L. Flores Burgos, E. Forati, A. G. Fowler, B. Foxen, W. Giang, C. Gidney, D. Gilboa, M. Giustina, A. Grajales Dau, J. A. Gross, S. Habegger, M. C. Hamilton, M. P. Harrigan, S. D. Harrington, O. Higgott, J. Hilton, M. Hoffmann, S. Hong, T. Huang, A. Huff, W. J. Huggins, L. B. Ioffe, S. V. Isakov, J. Iveland, E. Jeffrey, Z. Jiang, C. Jones, P. Juhas, D. Kafri, K. Kechedzhi, J. Kelly, T. Khattar, M. Khezri, M. KieferovĂĄ, S. Kim, A. Kitaev, P. V. Klimov, A. R. Klots, A. N. Korotkov, F. Kostritsa, J. M. Kreikebaum, D. Landhuis, P. Laptev, K. Lau, L. Laws, J. Lee, K. Lee, B. J. Lester, A. Lill, W. Liu, A. Locharla, E. Lucero, F. D. Malone, J. Marshall, O. Martin, J. R. McClean, T. McCourt, M. McEwen, A. Megrant, B. Meurer Costa, X. Mi, K. C. Miao, M. Mohseni, S. Montazeri, A. Morvan, E. Mount, W. Mruczkiewicz, O. Naaman, M. Neeley, C. Neill, A. Nersisyan, H. Neven, M. Newman, J. H. Ng, A. Nguyen, M. Nguyen, M. Y. Niu, T. E. OâBrien, A. Opremcak, J. Platt, A. Petukhov, R. Potter, L. P. Pryadko, C. Quintana, P. Roushan, N. C. Rubin, N. Saei, D. Sank, K. Sankaragomathi, K. J. Satzinger, H. F. Schurkus, C. Schuster, M. J. Shearn, A. Shorter, V. Shvarts, J. Skruzny, V. Smelyanskiy, W. C. Smith, G. Sterling, D. Strain, M. Szalay, A. Torres, G. Vidal, B. Villalonga, C. Vollgraff Heidweiller, T. White, C. Xing, Z. J. Yao, P. Yeh, J. Yoo, G. Young, A. Zalcman, Y. Zhang, and N. Zhu (2023) Suppressing quantum errors by scaling a surface code logical qubit. Nature 614 (7949), p. 676â681 (en). Cited by: §I. [18] D. Gottesman (1997) Stabilizer codes and quantum error correction. External Links: quant-ph/9705052 Cited by: §I-A. [19] Z. He, X. Zhang, and X. Chen (2023) Unitary Diagonalization of the Generalized Complementary Covariance Quaternion Matrices with Application in Signal Processing. Mathematics 11 (23), p. 4840 (en). Cited by: §I. [20] D. Herman, C. Googin, X. Liu, Y. Sun, A. Galda, I. Safro, M. Pistoia, and Y. Alexeev (2023) Quantum computing for finance. Nat. Rev. Phys. 5 (8), p. 450â465 (en). Cited by: §I. [21] A. Javadi-Abhari, M. Treinish, K. Krsulich, C. J. Wood, J. Lishman, J. Gacon, S. Martiel, P. D. Nation, L. S. Bishop, A. W. Cross, B. R. Johnson, and J. M. Gambetta (2024) Quantum computing with Qiskit. arXiv. Cited by: §V-A, §V-A. [22] A. A. Khan, A. Ahmad, M. Waseem, P. Liang, M. Fahmideh, T. Mikkonen, and P. Abrahamsson (2023) Software architecture for quantum computing systems â a systematic review. Journal of Systems and Software 201, p. 111682. Cited by: §I. [23] Y. Kim, A. Eddins, S. Anand, K. X. Wei, E. Van Den Berg, S. Rosenblatt, H. Nayfeh, Y. Wu, M. Zaletel, K. Temme, and A. Kandala (2023) Evidence for the utility of quantum computing before fault tolerance. Nature 618 (7965), p. 500â505 (en). External Links: ISSN 0028-0836, 1476-4687 Cited by: §I. [24] A. Y. Kitaev (1997) Quantum computations: algorithms and error correction. Russ. Math. Surveys 52 (6), p. 1191â1249. Cited by: §I, §IV-F3. [25] V. Kliuchnikov, D. Maslov, and M. Mosca (2013) Fast and efficient exact synthesis of single-qubit unitaries generated by Clifford and T gates. Quantum Inf. Comput. 13 (7&8), p. 607â630. Cited by: §I. [26] L. Kocsis and C. SzepesvĂĄri (2006) Bandit based monte-carlo planning. In Machine Learning: ECML 2006, 17th European Conference on Machine Learning, Cited by: item 1, §IV. [27] R. Koenig and J. A. Smolin (2014) How to efficiently select an arbitrary clifford group element. J. Math. Physics 55 (12), p. 122202. Cited by: §V-A. [28] P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver (2019) A quantum engineerâs guide to superconducting qubits. Appl. Phys. Reviews 6 (2), p. 021318. Cited by: §IV-F2. [29] D. Kremer, A. Javadi-Abhari, and P. Mukhopadhyay (2025) Optimizing the non-clifford-count in unitary synthesis using reinforcement learning. External Links: 2509.21709 Cited by: §I, §I. [30] D. Kremer, V. Villar, H. Paik, I. Duran, I. Faro, and J. Cruz-Benito (2024) Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning. External Links: 2405.13196 Cited by: §I, §I, §I, §V-B. [31] A. Kundu and S. Mangini (2025) TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search. arXiv. Cited by: §I. [32] A. Kundu (2025) Improving thermal state preparation of sachdevâyeâkitaev model with reinforcement learning on quantum hardware. Machine Learning: Science and Technology 6 (2), p. 025066. Cited by: §I, §I. [33] G. Li, Y. Ding, and Y. Xie (2019) Tackling the Qubit Mapping Problem for NISQ-Era Quantum Devices. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, Providence RI USA, p. 1001â1014 (en). Cited by: §I. [34] D. Lizzio Bosco, L. Cincio, G. Serra, and M. Cerezo (2026) Quantum circuit pre-synthesis: learning local edits to reduce T-count. External Links: 2601.19738 Cited by: §I, §IV-F3, §V-C, §V-C, TABLE I, TABLE I. [35] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. (2015) Human-level control through deep reinforcement learning. Nature 518 (7540), p. 529â533. Cited by: §I. [36] K. Nakaji, J. Wurtz, H. Huang, L. M. CalderĂłn, K. Panicker, E. Kyoseva, and A. Aspuru-Guzik (2025) Quantum circuits as a game: a reinforcement learning agent for quantum compilation and its application to reconfigurable neutral atom arrays. External Links: 2506.05536 Cited by: §I. [37] M. A. Nielsen and I. L. Chuang (2000) Quantum computation and quantum information. Cambridge University Press, Cambridge. Cited by: §IV-F3. [38] A. P. M. Place, L. V. H. Rodgers, P. Mundada, B. M. Smitham, M. Fitzpatrick, Z. Leng, A. Premkumar, J. Bryon, A. Vrajitoarea, S. Sussman, G. Cheng, T. Madhavan, H. K. Babla, X. H. Le, Y. Gang, B. JĂ€ck, A. Gyenis, N. Yao, R. J. Cava, N. P. De Leon, and A. A. Houck (2021) New material platform for superconducting transmon qubits with coherence times exceeding 0.3 milliseconds. Nat. Comm. 12 (1), p. 1779 (en). Cited by: §I. [39] A. K. Prasad, V. V. Shende, I. L. Markov, J. P. Hayes, and K. N. Patel (2006) Data structures and algorithms for simplifying reversible circuits. J. Emerg. Technol. Comput. Syst. 2 (4), p. 277â293. External Links: ISSN 1550-4832, Link, Document Cited by: §IV-F1. [40] J. Preskill (2018) Quantum Computing in the NISQ era and beyond. Quantum 2, p. 79. External Links: Document Cited by: §I. [41] Qiskit (2024) AI-transpiler cliffords. Note: Hugging Face model repository. Accessed: Apr. 21, 2026 Cited by: §V-B, TABLE I. [42] R. Romanello, D. Lizzio Bosco, J. Cossio, D. Sutulovic, G. Serra, C. Piazza, and P. Burelli (2025) CNOT minimal circuit synthesis: a reinforcement learning approach. In 2025 IEEE International Conference on Quantum Artificial Intelligence (QAI), Cited by: §I, §I. [43] N. J. Ross and P. Selinger (2016) Optimal ancilla-free clifford+t approximation of z-rotations.. Quantum Inf. Comput. 16 (11-12), p. 901â953. Cited by: 2nd item. [44] F. J. R. Ruiz, T. Laakkonen, J. Bausch, M. Balog, M. Barekatain, F. J. H. Heras, A. Novikov, N. Fitzpatrick, B. Romera-Paredes, J. van de Wetering, A. Fawzi, K. Meichanetzidis, and P. Kohli (2025) Quantum circuit optimization with AlphaTensor. Nature Machine Intelligence 7, p. 374â385. Cited by: §I, §I, §I, §IV, §VI. [45] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. External Links: 1707.06347 Cited by: §I, §IV. [46] I. Shaik and J. van de Pol (2025) Cnot-optimal clifford synthesis as sat. Cited by: §I. [47] K. Volanto (2023) Minimizing the number of two-qubit gates in clifford circuits. Masterâs thesis, Aalto University. Cited by: §I, §V-A. [48] Z. T. Wang, Q. Chen, Y. Du, Z. H. Yang, X. Cai, K. Huang, J. Zhang, K. Xu, J. Du, Y. Li, Y. Jiao, X. Wu, W. Liu, X. Lu, H. Xu, Y. Jin, R. Wang, H. Yu, and S. P. Zhao (2024) Quantum compiling with reinforcement learning on a superconducting processor. External Links: 2406.12195 Cited by: §I. [49] M. Webster, S. Koutsioumpas, and D. E. Browne (2025) Heuristic and optimal synthesis of cnot and clifford circuits. External Links: 2503.14660 Cited by: §I, §V-A, §V-A, §V-C. [50] D. Wierichs, M. West, R. T. Forestano, M. Cerezo, and N. Killoran (2025) Recursive Cartan decompositions for unitary synthesis. arXiv. Cited by: §IV-F3. [51] G. Yan, W. Wu, Y. Chen, K. Pan, X. Lu, Z. Zhou, Y. Wang, R. Wang, and J. Yan (2024) Quantum circuit synthesis and compilation optimization: overview and prospects. Cited by: §I. [52] R. Zen, J. Olle, L. Colmenarez, M. Puviani, M. MĂŒller, and F. Marquardt (2025) Quantum circuit discovery for fault-tolerant logical state preparation with reinforcement learning. Phys. Rev. X 15, p. 041012. Cited by: §I, §I. [53] C. Zhu, X. Wu, Z. Yang, J. Wang, A. Wu, S. Zheng, and X. Wang (2025) Quantum Compiler Design for Qubit Mapping and Routing: A Cross-Architectural Survey of Superconducting, Trapped-Ion, and Neutral Atom Systems. External Links: 2505.16891 Cited by: §I. [54] H. Zou, M. Treinish, K. Hartman, A. Ivrii, and J. Lishman (2024) LightSABRE: A Lightweight and Enhanced SABRE Algorithm. arXiv. Cited by: §I. [55] A. Zulehner, A. Paler, and R. Wille (2019) An Efficient Methodology for Mapping Quantum Circuits to the IBM QX Architectures. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 38 (7), p. 1226â1236. Cited by: §I.