Paper deep dive

DreamQAS: Learning a Decision-Useful World Model for VQE-Efficient Quantum Architecture Search

Jiayang Niu, Yan Wang, Jie Li, Ke Deng, Azadeh Alavi, Muhammad Usman, Yongli Ren

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 95

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 8/3/2026, 3:05:19 AM

Summary

The paper introduces DreamQAS, a model-based reinforcement learning framework for Quantum Architecture Search (QAS) that reduces the computational cost of Variational Quantum Eigensolver (VQE) evaluations. DreamQAS employs a recurrent randomized-prior ensemble to predict post-VQE feedback scores relative to an empirical energy frontier, enabling multi-step imagined policy learning over explicit legal circuits. The framework utilizes reliability-controlled loops involving ranking-based activation, uncertainty-aware pessimism, and selective real-VQE verification. Experiments on five molecular tasks demonstrate that DreamQAS achieves lower mean frozen-policy energy errors and requires significantly fewer real VQE calls compared to baseline methods.

Entities (12)

DreamQAS · method · 98%VQE · algorithm · 95%RL-QAS · task · 92%BeH2-8q · molecule · 90%LiH-4q · molecule · 90%CRLQAS · baseline · 88%GQE · baseline · 88%HyRLQAS · baseline · 88%NoImag · baseline · 88%QuantumDARTS · baseline · 88%TFQAS · baseline · 88%HamQASBench · dataset · 85%

Relation Signals (7)

DreamQAS → evaluatedon → LiH-4q

confidence 95% · We study five molecular QAS tasks from the HamQASBench evaluation framework: LiH-4q...

DreamQAS → evaluatedon → BeH2-8q

confidence 95% · We study five molecular QAS tasks from the HamQASBench evaluation framework: ... BeH2-8q ...

DreamQAS → uses → VQE

confidence 95% · DreamQAS ... learns only the expensive post-VQE feedback.

DreamQAS → outperforms → NoImag

confidence 92% · DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks... it uses 1.6x to 2.0x fewer real VQE calls

DreamQAS → outperforms → HyRLQAS

confidence 90% · DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks

DreamQAS → outperforms → CRLQAS

confidence 90% · DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks

DreamQAS → uses → HamQASBench

confidence 90% · We study five molecular QAS tasks from the HamQASBench evaluation framework

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Reinforcement-learning-based quantum architecture search (RL-QAS) repeatedly optimizes a variational quantum eigensolver (VQE) after extending a circuit, although circuit construction and action legality are deterministic and known. We introduce DreamQAS, a model-based RL framework that preserves these exact circuit dynamics and learns only the expensive post-VQE feedback. A recurrent randomized-prior ensemble predicts an oracle-free score relative to an empirical energy frontier and supports multi-step imagined policy learning over explicit legal circuits. Ranking-based activation, uncertainty-aware pessimism and truncation, and selective real-VQE verification form a reliability-controlled learning loop. Under a common 15,000-episode budget and frozen evaluation for the RL methods, DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks and the second-lowest on one. At fine-error targets reached by all seeds of both methods, it uses 1.6x to 2.0x fewer real VQE calls on four tasks and 10.6x fewer on BeH2-8q. Counterfactual action-ranking utility increases across all five tasks, with a mean increase of 0.346 and a 95 percent confidence interval of [0.185, 0.507], while direct greedy and beam use of the same model does not recover the gains of imagined policy learning. Ensemble disagreement also improves risk-coverage over random rejection on all three probed tasks. These results establish a world-model design for QAS whose value lies in decision-useful feedback rather than exact energy prediction.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

93,262 characters extracted from source content.

Expand or collapse full text

DreamQAS: Learning a Decision-Useful World Model for VQE-Efficient Quantum Architecture Search Jiayang Niu1, Yan Wang1, Jie Li1, Ke Deng1, Azadeh Alavi1, Muhammad Usman2, Yongli Ren1,* 1School of Computing Technologies, RMIT University 2Quantum Systems, Data61, CSIRO *Corresponding author: yongli.ren@rmit.edu.au Abstract Reinforcement-learning-based quantum architecture search (RL-QAS) repeatedly optimizes a variational quantum eigensolver (VQE) after extending a circuit, although circuit construction and action legality are deterministic and known. We introduce DreamQAS, a model-based RL framework that preserves these exact circuit dynamics and learns only the expensive post-VQE feedback. A recurrent randomized-prior ensemble predicts an oracle-free score relative to an empirical energy frontier and supports multi-step imagined policy learning over explicit legal circuits. Ranking-based activation, uncertainty-aware pessimism and truncation, and selective real-VQE verification form a reliability-controlled learning loop. Under a common 15,000-episode budget and frozen evaluation for the RL methods, DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks and the second-lowest on one. At fine-error targets reached by all seeds of both methods, it uses 1.6×1.6×–2.0×2.0× fewer real VQE calls on four tasks and 10.6×10.6× fewer on BeH2-8q. Counterfactual action-ranking utility increases across all five tasks (Δρ=0.346 ρ=0.346, 95% CI [0.185,0.507][0.185,0.507]), while direct greedy and beam use of the same model does not recover the gains of imagined policy learning. Ensemble disagreement also improves risk–coverage over random rejection on all three probed tasks. These results establish a world-model design for QAS whose value lies in decision-useful feedback rather than exact energy prediction. 1 Introduction The variational quantum eigensolver (VQE) estimates molecular ground-state energies by optimizing the continuous parameters of a quantum circuit ansatz (Peruzzo et al., 2014; Kandala et al., 2017; Tilly et al., 2022). Its accuracy and trainability depend strongly on the ansatz architecture, yet the best architecture varies with the molecule, Hamiltonian, and hardware. Quantum architecture search (QAS) addresses this design problem by constructing circuits from a prescribed gate space using differentiable, discrete, generative, or learned search procedures (Zhang et al., 2022; Du et al., 2022; Lu et al., 2023; Nakaji et al., 2024; He et al., 2024). The evaluation bottleneck is especially severe in RL-QAS. A policy appends one legal gate at a time, and the environment evaluates each new circuit prefix by optimizing its parameters with VQE. Policy training repeats this operation over thousands of trajectories. The resulting cost is structurally asymmetric: appending a gate and updating action legality are exact and inexpensive, whereas post-optimization feedback is expensive and initially unknown. This suggests a QAS-specific model-based formulation that preserves the known circuit transition and learns only the feedback that would otherwise require another VQE call. Turning this asymmetry into useful synthetic experience is not equivalent to fitting a globally accurate energy regressor. First, an RL-QAS decision is conditional: the model must distinguish legal continuations of the same circuit prefix, even when its errors are nonuniform across the full architecture space. Second, deployable training feedback cannot assume access to the exact ground-state energy, and its useful scale changes as the empirical search frontier improves. Third, imagination shifts the policy toward model-preferred circuits, so systematic errors can be amplified across consecutive decisions. A QAS world model must therefore represent ordered prefixes, learn an oracle-free decision signal, and expose reliability information that controls when and how imagined feedback is used. Predictor-assisted QAS amortizes evaluation by learning a surrogate for architecture quality (Zhang et al., 2021; He et al., 2023; Deng et al., 2023; Soloviev et al., 2024; Martyniuk et al., 2025). Such predictors are often used to rank or screen candidates. A reusable circuit-construction policy poses a different problem: predicted feedback must provide credit across a sequence of gate decisions. Directly following the circuit that a surrogate scores best can concentrate search in model-biased regions. In contrast, a world model can generate internal experience for policy learning (Ha and Schmidhuber, 2018; Janner et al., 2019; Hafner et al., 2020), but generic world-model agents usually learn latent transitions together with rewards or values. QAS permits a sharper factorization: the circuit is kept explicit, its transition remains exact, and only the post-VQE feedback is modeled. We propose DreamQAS, a QAS-specific world-model framework built around this factorization. A recurrent ensemble represents each variable-length circuit prefix and predicts an oracle-free, frontier-relative feedback score. Starting from VQE-verified replay prefixes, the actor constructs multi-step imagined trajectories using exact legal transitions and learned feedback. A ranking gate activates imagination when the model becomes decision-ready; ensemble disagreement supplies pessimism and confidence-based truncation; and selected promising or uncertain circuits are periodically verified using real VQE and returned to training. The paper follows four connected questions. First, does DreamQAS improve final QAS quality and the number of real VQE calls required to reach a target error? Second, does the feedback model acquire action-level decision utility? Third, can the same model be used directly as a surrogate searcher, or is multi-step imagined policy learning essential? Fourth, can ensemble disagreement monitor where model feedback is risky? Under a common 15,000-episode RL budget and frozen evaluation, DreamQAS has the lowest mean final error on four tasks and the second-lowest on one. At fine-error targets reached by every seed of both methods, it uses 1.6×1.6×–2.0×2.0× fewer real VQE calls than NoImag on four tasks and 10.6×10.6× fewer on BeH2-8q. Meanwhile, final action-ranking utility improves over its start checkpoint by Δρ=0.346 ρ=0.346 across tasks, and disagreement-based rejection lowers risk by 24%24\%–46%46\% at half coverage. Our contributions are: • We formulate RL-QAS as a known-dynamics, expensive-feedback problem and introduce a feedback-only world model that never requires the exact ground-state energy during training or search. • We develop a QAS-aware recurrent ensemble, frontier-relative target, decision-oriented replay, and multi-step imagination scheme, integrated with ranking activation, pessimism, truncation, and selective verification. • Across five molecular tasks, we demonstrate strong common-budget final quality and substantial real-VQE savings over matched internal controls, then explain these gains through counterfactual action utility, same-model deployment controls, transition-matched horizons, and uncertainty risk–coverage. 2 Related Work Adaptive VQE constructs an ansatz iteratively by selecting chemistry-motivated or qubit operators from a pool, often using measured gradients (Grimsley et al., 2019; Tang et al., 2021). QAS instead treats the architecture itself as a search object in a prescribed gate space and has been approached through differentiable supercircuits (Zhang et al., 2022; Wu et al., 2023), discrete search (Du et al., 2022), generative construction (Nakaji et al., 2024), and training-free structural criteria (He et al., 2024). These approaches automate circuit design under different structural assumptions. Our setting focuses on gate-level sequential construction, where evaluating each newly visited prefix requires a complete post-optimization feedback signal. RL-QAS makes this sequential structure explicit: a partial circuit is the state, and a legal gate insertion or modification is the action. Prior work has studied value-based ansatz construction, curriculum learning, robustness to hardware errors, joint discrete–continuous decisions, and controlled agent comparisons (Ostaszewski et al., 2021; Patel et al., 2024; Niu et al., 2025; Ikhtiarudin et al., 2025). Benchmarking efforts further expose differences among search spaces, evaluators, and protocols (Lu et al., 2023; Niu et al., 2026). These methods improve how the policy explores or optimizes the architecture space, but fresh policy experience is still predominantly obtained from newly VQE-evaluated circuits. DreamQAS is complementary: it targets the cost of producing that feedback and reuses verified experience to train the construction policy. Predictor-assisted QAS amortizes circuit evaluation by learning performance models for candidate screening, active label acquisition, progressive search, or architecture benchmarking (Zhang et al., 2021; He et al., 2023; Deng et al., 2023; Soloviev et al., 2024; Martyniuk et al., 2025). Neural predictors can substantially reduce the number of labeled circuit evaluations, while training-free proxies avoid supervised labels altogether (Zhang et al., 2021; He et al., 2024). Their usual deployment unit is a complete candidate whose score guides selection or acquisition. Sequential policy learning requires a different form of utility: the model must rank legal continuations conditional on a policy-visited prefix, and its predictions alter the future prefix distribution on which it is used. DreamQAS therefore consumes learned feedback through real-anchored imagined trajectories rather than treating the globally best predicted circuit as the answer. Section 5.4 compares these two uses of the same model. Model-based RL has long interleaved real and model-generated experience for learning and planning (Sutton, 1991). Modern approaches use probabilistic ensembles to represent predictive uncertainty (Chua et al., 2018), branch short synthetic rollouts from real data to limit model bias (Janner et al., 2019), or learn latent dynamics for multi-step imagination (Ha and Schmidhuber, 2018; Hafner et al., 2020; Schrittwieser et al., 2020). Uncertainty-penalized objectives provide a complementary defense against policies exploiting unsupported model predictions (Yu et al., 2020), while iterative data aggregation addresses the distribution shift induced by a learned decision rule (Ross et al., 2011). Model-based QAS has also used learned dynamics inside tree search (Rapp et al., 2025). DreamQAS specializes these principles to molecular RL-QAS: symbolic circuit transitions and legality remain explicit and exact, only post-VQE feedback is learned, and ranking activation, ensemble pessimism, rollout truncation, and selective VQE verification regulate how that feedback enters policy learning. 3 Problem Formulation An RL-QAS episode constructs a parameterized circuit one gate at a time. Let st=(a1,…,at)s_t=(a_1,…,a_t) denote the explicit circuit prefix after t steps and (st)A(s_t) its legal-action set. Selecting at+1∈(st)a_t+1 (s_t) gives the known transition st+1=circ(st,at+1)=st⊕at+1.s_t+1=T_circ(s_t,a_t+1)=s_t a_t+1. (1) Evaluating the new prefix is expensive because its continuous parameters must be optimized, E∗(st+1)=minϑ⁡⟨0|U†(st+1,ϑ)HU(st+1,ϑ)∣0⟩.E^*(s_t+1)= _ 0|U (s_t+1, )HU(s_t+1, )|0 . (2) The search objective is to learn a construction policy that discovers a low-energy circuit under a finite budget of real VQE optimizations. This separates an exact, inexpensive circuit transition from an expensive feedback function and motivates DreamQAS to model only the latter. 4 DreamQAS Figure 1 summarizes the learning loop. Real trajectories anchor the policy and provide post-VQE energies for feedback-model training. The model then supplies additional multi-step imagined trajectories without VQE at imagined steps. Reliability signals determine when and how strongly this synthetic feedback enters policy learning, while selective real verification expands the trusted data distribution. Figure 1: DreamQAS training loop. Real QAS trajectories provide post-VQE energies for an oracle-free recurrent ensemble. Starting from verified prefixes, exact circuit rules and learned feedback generate multi-step imagined trajectories for policy learning. Ranking readiness, ensemble pessimism, confidence truncation, and selective real-VQE verification form the reliability loop. 4.1 Oracle-Free Feedback World Model The circuit transition in Eq. (1) is already known; the learned component predicts only the post-VQE feedback associated with the resulting prefix. Training must also avoid assuming the exact ground-state energy E0E_0. Let FkF_k be the empirical frontier adopted at the kkth feedback-model refresh and define the displacement in mHa as dt(k)=E∗(st)−Fk10−3Ha.d_t^(k)= E^*(s_t)-F_k10^-3\,Ha. (3) DreamQAS uses the signed-log score yt=Sk(E∗(st))=sign⁡(dt(k))log10⁡(1+|dt(k)|m),m=0.1mHa.y_t=S_k(E^*(s_t))=sign(d_t^(k)) _10\! (1+ |d_t^(k)|m ), m=0.1\,mHa. (4) This strictly monotone transform compresses deviations on both sides of the frontier while preserving their energy ordering. A pending frontier tracks the best observed real optimized energy continuously. At a model refresh it is adopted atomically as FkF_k; the replay buffer stores raw energies, so all labels are recomputed under the current scale. Real policy learning uses the improvement reward rt+1real=yt−yt+1,r_t+1^real=y_t-y_t+1, (5) which is positive when the extended circuit lowers the optimized energy. The exact E0E_0 is used only after training to express frozen-policy evaluation in mHa. The resulting composite model is ℳ^ϕ(st,at+1)=(circ(st,at+1)⏟st+1,Fϕ(st+1)⏟(¯t+1,y¯t+1,σt+1)), M_φ(s_t,a_t+1)= ( T_circ(s_t,a_t+1)_s_t+1, F_φ(s_t+1)_( h_t+1, y_t+1, _t+1) ), (6) where the first component is exact and the second is learned. The full frontier adoption and replay relabeling procedure is given in Appendix A.1. 4.2 QAS-Aware Feedback Learning Circuit prefixes are ordered and variable in length. DreamQAS therefore uses K=3K=3 independently initialized recurrent members. Member k embeds the gate sequence and encodes the complete prefix with a GRU, t(k)=fϕk(a1:t)h_t^(k)=f_ _k(a_1:t). Its prediction combines a trainable head and a frozen randomized prior function (Osband et al., 2018): y^t(k)=gϕk(t(k))+βrpfp(k)(t(k)). y_t^(k)=g_ _k(h_t^(k))+ _rpfp^(k)(h_t^(k)). (7) Independent encoders, per-member bootstrap samples, and randomized priors produce an ensemble mean and disagreement, y¯t=1K∑ky^t(k),σt=Stdk⁡[y^t(k)],¯t=1K∑kt(k). y_t= 1K _k y_t^(k), _t=Std_k[ y_t^(k)], h_t= 1K _kh_t^(k). (8) The actor consumes the detached recurrent representation sg⁡(¯t)sg( h_t), while y¯t y_t and σt _t replace expensive feedback during imagination. Real VQE trajectories enter an all-history replay buffer. Training batches mix elite circuits, priority-weighted feedback regions, stratified targets, and uniform samples so that rare improvements and policy-visited prefixes remain represented. With padding mask qitq_it, member bootstrap mask μit(k) _it^(k), and decision-oriented weight w(yit)w(y_it), the supervised objective is ℒWM=1K∑k=1K∑i,tqitμit(k)w(yit)ℓH(y^it(k)−yit)∑i,tqitμit(k).L_WM= 1K _k=1^K _i,tq_it _it^(k)w(y_it) _H( y_it^(k)-y_it) _i,tq_it _it^(k). (9) Architecture, replay composition, and optimization details are reported in Appendix A.2. 4.3 Multi-Step Imagined Policy Learning Real trajectories always anchor the actor. Let A^treal A_t^real be the standardized advantage derived from discounted real rewards. The real policy objective is ℒreal=−t[log⁡πθ(at+1∣st)A^treal]−centt[ℋ(πθ)].L_real=-E_t\! [ _θ(a_t+1 s_t) A_t^real ]-c_entE_t[H( _θ)]. (10) The action distribution is masked and renormalized over the exact legal set. An imagined rollout starts from a VQE-verified replay prefix. At each imagined step, the actor chooses a legal gate, Eq. (1) appends it to the explicit circuit, and the recurrent ensemble predicts the new feedback. To limit exploitation of uncertain predictions, DreamQAS defines the pessimistic potential Φt=−(y¯t+βpesσt),rt+1imag=Φt+1−Φt. _t=-( y_t+ _pes _t), r_t+1^imag= _t+1- _t. (11) Masked multi-step returns over the trusted part of each trajectory produce A^timag A_t^imag and ℒimag=−∑tztlog⁡πθ(at+1∣st)sg⁡(A^timag)∑tzt,L_imag=- _tz_t _θ(a_t+1 s_t)sg( A_t^imag) _tz_t, (12) where ztz_t masks steps after the legal action space is exhausted or confidence truncation is triggered. Because the reward is a potential difference, policy learning consumes predicted improvements along a trajectory rather than using the model as a terminal circuit scorer. The default horizon is H=15H=15; complete return and rollout specifications are in Appendix A.3. 4.4 Reliability-Controlled Learning The feedback model is introduced into policy learning through a closed loop. First, after a real-data warm-up, a global ranking gate activates imagination when pairwise accuracy on VQE-verified prefixes reaches τpair _pair: Accpair=(u,v)∼Unif⁡()[[(y¯u−y¯v)(yu−yv)>0]].Acc_pair=E_(u,v) (P) [I\! [( y_u- y_v)(y_u-y_v)>0 ] ]. (13) Second, ensemble disagreement enters the pessimistic potential at every imagined step. Third, σt>τσ _t> _σ truncates low-confidence returns. Finally, a small deduplicated mixture of model-preferred and high-disagreement circuits is periodically evaluated with real VQE. These observations are returned to replay in a DAgger-style feedback step (Ross et al., 2011), and every verification call is charged to the real-feedback budget. The combined actor objective is ℒactor=ℒreal+ωim[Nep≥Nwarm∧Accpair≥τpair]ℒimag.L_actor=L_real+ _imI\! [N_ep≥ N_warm _pair≥ _pair ]L_imag. (14) Appendix A provides the complete algorithm, thresholds, verification schedule, and VQE accounting. 5 Experiments We organize the evaluation around four questions: RQ1 final policy quality and real-VQE efficiency; RQ2 learned action-level decision utility; RQ3 the role of multi-step imagined policy learning beyond direct surrogate exploitation; and RQ4 uncertainty-based monitoring of world-model feedback. 5.1 Experimental Setup We study five molecular QAS tasks from the HamQASBench evaluation framework (Niu et al., 2026): LiH-4q, BeH2-6q, LiH-6q, BeH2-8q, and BeH2-10q. LiH-4q allows up to 40 construction steps and the remaining tasks allow 50. The 4- and 6-qubit tasks use COBYLA with at most 1,000 optimizer iterations; the 8- and 10-qubit tasks use GPU-accelerated ROTOSOLVE with three sweeps. All episode-based RL methods use 15,000 construction episodes. Each cell contains five independent training seeds; the frozen policy is evaluated for 100 fresh episodes per seed, and tables report the cross-seed mean and sample standard deviation of mean frozen-policy energy error. DreamQAS and its internal controls use the exact ground-state energy only after training to convert evaluation energies to mHa. Training curves include every real VQE call, including selective verification. Internal controls share the environment, action space, VQE optimizer, and real episode budget. DreamQAS-RL removes the feedback model and imagination; NoImag retains recurrent representation and feedback learning without imagined policy updates; NoDAG uses uncertainty-aware imagination but does not return selectively verified samples to replay. All methods use the same Hamiltonians, circuit-depth caps, task-level optimizer families, and five-run protocol. The RL methods additionally share the 15,000-episode budget and frozen-evaluation metric. CRLQAS (Patel et al., 2024) and HyRLQAS (Niu et al., 2025) retain their original E0E_0-informed learning rules. GQE (Nakaji et al., 2024), TFQAS (He et al., 2024), and QuantumDARTS (Wu et al., 2023) use their native search budgets and provide final-quality context only, not matched-VQE efficiency comparisons. Complete molecular specifications, gate spaces, VQE configurations, baseline provenance, run protocols, and statistical procedures are provided in Appendix B; complete run inventories and data lineage are provided in Appendix H. 5.2 RQ1: Search Quality and Real-VQE Efficiency Method LiH-4q BeH2-6q LiH-6q BeH2-8q BeH2-10q RL methods: common 15,000-episode budget and frozen evaluation DreamQAS 0.053±0.0240.053\!±\!0.024 0.058±<0.0010.058\!±\!<0.001 11.4±0.4711.4\!±\!0.47 2.17±0.002.17\!±\!0.00 1.03±0.29¯ 1.03\!±\!0.29 NoImag 0.221±0.0620.221\!±\!0.062 0.328±0.120.328\!±\!0.12 13.6±3.513.6\!±\!3.5 2.35±0.402.35\!±\!0.40 2.18±0.912.18\!±\!0.91 DreamQAS-RL 0.391±0.0740.391\!±\!0.074 0.098±0.0610.098\!±\!0.061 26.3±1.126.3\!±\!1.1 3.12±0.963.12\!±\!0.96 9.88±13.09.88\!±\!13.0 CRLQAS 4.73±1.304.73\!±\!1.30 1.78±1.301.78\!±\!1.30 17.3±3.917.3\!±\!3.9 5.04±2.405.04\!±\!2.40 2.65±1.902.65\!±\!1.90 HyRLQAS 16.5±2416.5\!±\!24 309±690309\!±\!690 215±410215\!±\!410 18.4±2318.4\!±\!23 0.897±<0.0010.897\!±\!<0.001 Method-specific search procedures; final-quality context GQE 128±79128\!±\!79 235±220235\!±\!220 108±45108\!±\!45 261±130261\!±\!130 345±400345\!±\!400 TFQAS 32.7±1.332.7\!±\!1.3 20.3±4.020.3\!±\!4.0 36.0±0.6236.0\!±\!0.62 56.7±1056.7\!±\!10 77.9±5.477.9\!±\!5.4 QuantumDARTS 2.74±0.912.74\!±\!0.91 1.26±2.401.26\!±\!2.40 10.4±5.110.4\!±\!5.1 38.0±4938.0\!±\!49 113±120113\!±\!120 Table 1: Mean frozen-policy energy error in mHa (mean ± sample std over five seeds; lower is better). Bold and underline mark the lowest and second-lowest means within the RL block. CRLQAS and HyRLQAS retain their original E0E_0-informed learning rules. The lower block uses method-specific search procedures and is shown as final-quality context without cross-block ranking. QuantumDARTS reports one final architecture per independent search; its variation is across five searches rather than frozen-policy rollout variation. Within the RL block of Table 1, DreamQAS has the lowest mean error on LiH-4q, BeH2-6q, LiH-6q, and BeH2-8q and the second-lowest on BeH2-10q. It reaches chemical accuracy (1.6mHa1.6\,mHa) on LiH-4q, BeH2-6q, and BeH2-10q. Against the matched oracle-free internal controls, the full method improves mean frozen-policy energy error on every task. The method-specific lower block is retained as broader final-quality context and does not support a matched-resource ranking. Figure 2: Training quality versus cumulative real VQE calls on three representative tasks. Curves show the across-seed median and IQR of the trailing-100-episode mean; all verification calls are included. Arrows denote visual gaps between median curves at the stated error levels; all numerical speedup claims use the seed-level sustained-crossing estimator. Complete five-task curves, reach counts, and seed ranges are reported in Appendix C. At the finest task-specific targets reached by all five seeds of both methods, DreamQAS requires 1.6×1.6×–2.0×2.0× fewer real VQE calls than NoImag on LiH-4q, BeH2-6q, LiH-6q, and BeH2-10q, and 10.6×10.6× fewer on BeH2-8q. The complete seed-level sustained crossings, reach counts, and four-level error ladders are reported in Appendix C. DreamQAS is not uniformly faster at loose targets; its advantage grows as the target tightens. Thus imagination primarily accelerates fine-error refinement, where search must repeatedly improve already competitive circuits. The model apparatus adds approximately 10.910.9–12.7%12.7\% wall-clock overhead relative to NoImag in same-load state-vector timing runs, while verification is its largest single component. Because verification VQE is included on the efficiency axis, the reported savings account for the real evaluations used to maintain the world model. Full VQE, optimizer-evaluation, model-query, and runtime accounting is given in Appendix C. 5.3 RQ2: Learned Decision Utility For each policy-visited real prefix, we construct up to ten legal next-action continuations, score them with the frozen feedback model, and evaluate the same candidates with matched real VQE. We measure within-prefix Spearman correlation ρact _act, normalized regret of the model-ranked top action, and its win rate against matched random legal actions. Task actstart ρ_ act start act1/4 ρ_ act^1/4 actfinal ρ_ act final final↓NReg final\! >final↑WM>rand final\! LiH-4q 0.234±0.1060.234\!±\!0.106 0.469±0.0340.469\!±\!0.034 0.683±0.0920.683\!±\!0.092 0.026±0.0460.026\!±\!0.046 0.942±0.0400.942\!±\!0.040 BeH2-6q 0.078±0.0720.078\!±\!0.072 0.175±0.0890.175\!±\!0.089 0.226±0.0730.226\!±\!0.073 0.152±0.0560.152\!±\!0.056 0.649±0.1060.649\!±\!0.106 LiH-6q 0.155±0.0550.155\!±\!0.055 0.330±0.0360.330\!±\!0.036 0.436±0.0870.436\!±\!0.087 0.054±0.0480.054\!±\!0.048 0.707±0.0780.707\!±\!0.078 BeH2-8q 0.074±0.0890.074\!±\!0.089 0.417±0.1150.417\!±\!0.115 0.515±0.0990.515\!±\!0.099 0.000±0.0000.000\!±\!0.000 0.973±0.0600.973\!±\!0.060 BeH2-10q 0.000±0.0510.000\!±\!0.051 0.384±0.0590.384\!±\!0.059 0.411±0.0540.411\!±\!0.054 0.000±0.0000.000\!±\!0.000 0.978±0.0310.978\!±\!0.031 Table 2: Oracle-free counterfactual action utility at the common 15,000-episode budget. Bold highlights the final action-ranking measurement, which increases from the start checkpoint on every task. Entries are mean ± sample standard deviation over five seeds. Probe construction, tie handling, and per-seed tests are reported in Appendix D. Table 2 shows a consistent training trajectory: mean ρact _act increases from start to quarter budget and again from quarter to final on all five tasks. Treating tasks as the cross-task unit, the mean final-minus-start improvement is Δρ=0.346 ρ=0.346 (95% CI [0.185,0.507][0.185,0.507], p=0.004p=0.004). Final normalized regret is at most 0.1520.152, and the model-ranked continuation beats a matched random legal action with probability 0.6490.649–0.9780.978. The feedback model therefore acquires the local ordering information required for policy improvement. The oracle-free target preserves this decision direction without access to E0E_0. On the same real rollout transitions, its reward has Spearman correlation 0.9730.973–1.0001.000 with the corresponding FCI-referenced reward. Together, the reward audit and the counterfactual probe show that the empirical frontier supplies a practical training signal while the learned model improves its action-level ranking over time. Detailed reward, action-probe, and calibration analyses are in Appendix D. 5.4 RQ3: Beyond Direct Surrogate Search Action-level ranking alone does not determine how the model should be used. We first freeze the same feedback model and compare imagined policy learning with direct greedy variants. We then hold the trusted imagined-transition budget fixed and vary how those transitions are organized into horizons. LiH-4q LiH-6q BeH2-8q (a) Same feedback model, different deployment NoImag 0.2210.221 13.613.6 2.352.35 WM-Greedy, β=+1β=+1 6.426.42 36.936.9 2.172.17 WM-Greedy, β=−1β=-1 5.915.91 32.032.0 2.172.17 DreamQAS-NoDAG 0.0730.073 11.411.4 2.532.53 (b) Transition-matched imagined horizon H=1H=1 1.223±1.8041.223\!±\!1.804 19.032±10.28119.032\!±\!10.281 2.175±0.0002.175\!±\!0.000 H=5H=5 0.137±0.0310.137\!±\!0.031 11.756±1.01211.756\!±\!1.012 2.175±0.0012.175\!±\!0.001 H=15H=15 0.073±0.0360.073\!±\!0.036 11.866±1.95011.866\!±\!1.950 2.440±0.4362.440\!±\!0.436 Table 3: Same-model deployment and transition-matched horizons. Panel (a) compares direct and policy-based use of the same feedback model; panel (b) matches trusted imagined transitions across horizons. Values are mean frozen-policy energy errors in mHa; lower is better. Bold marks the imagined-policy result in (a) and multi-step improvements over H=1H=1 in (b). Direct use of the same feedback model does not recover the policy-learning gain. On LiH-4q and LiH-6q, DreamQAS-NoDAG reaches 0.0730.073 and 11.4mHa11.4\,mHa, whereas direct greedy deployment gives 5.915.91–6.426.42 and 32.032.0–36.9mHa36.9\,mHa. Beam and exploratory deployment controls show the same separation (Appendix E). The actor benefits from feedback differences across many real-anchored trajectories rather than repeatedly selecting the circuit with the most favorable absolute prediction. The horizon control isolates the role of multi-step credit assignment. At a matched trusted-transition budget, H=5H=5 and H=15H=15 both improve substantially over H=1H=1 on LiH-4q and LiH-6q. The H=5H=5 and H=15H=15 outcomes are comparable, so the mechanism result is that multi-step imagination outperforms one-step imagination, rather than that a universally longest rollout is required. Complete deployment variants, paired comparisons, and transition accounting are provided in Appendix E. 5.5 RQ4: Monitoring World-Model Reliability We evaluate whether ensemble disagreement can rank prediction risk on candidates that are periodically checked with real VQE. To separate this test from disagreement-based candidate acquisition, risk–coverage is computed on the value-selected verification stratum. Risk is the mean absolute error in the oracle-free frontier-score space after retaining the stated fraction of candidates. Figure 3: Risk–coverage of oracle-free ensemble disagreement. Rejecting candidates with high disagreement lowers retained-set prediction risk relative to random rejection on LiH-4q, LiH-6q, and BeH2-8q. Insets report AURC across five seeds. At 50% coverage, disagreement reduces risk relative to no rejection by 24%, 26%, and 46%, respectively. Complete AURC values, confidence intervals, and sampling details are in Appendix F. Figure 3 shows that disagreement improves over random rejection on all three tasks. At 50% coverage it lowers risk by 24% on LiH-4q, 26% on LiH-6q, and 46% on BeH2-8q relative to retaining every candidate. LiH-6q also exposes the value of separating confidence from predicted score: retaining the predicted-best half raises risk by 34%, while disagreement-based rejection lowers it by 26%. This connects the monitoring result to RQ3, where direct greedy search targets precisely the model-preferred region. Disagreement is used continuously through the pessimistic potential and selectively through confidence truncation and verification. Its strongest end-to-end ablation appears on BeH2-10q, where removing uncertainty control raises final error from 1.031.03 to 2.24mHa2.24\,mHa. The complete AURC table, threshold activation, selective-verification analysis, and component ablations are reported in Appendices F and G. 6 Discussion and Limitations The experiments support a system-level account of decision-useful world modeling for QAS. The recurrent prefix representation provides a state for variable-length circuits; the oracle-free target and replay strategy turn real VQE outcomes into relative feedback; and multi-step imagination converts that feedback into policy improvement. The action probe verifies that useful local ordering is learned, while the same-model deployment and horizon controls show that this information is most effective when used for multi-step credit assignment rather than direct surrogate exploitation. The reliability mechanisms complete this loop. Ranking readiness controls when imagination begins, ensemble pessimism shapes every imagined reward, truncation limits low-confidence credit, and verification connects selected imagined regions back to real VQE. The risk–coverage result provides direct evidence that disagreement carries actionable information about feedback error. Across the component study, imagination provides the broadest contribution, while uncertainty and replay mechanisms contribute complementary, task-dependent improvements; the full breakdown is in Appendix G. Three limitations define the present scope. First, all experiments use state-vector simulation, so hardware noise and queue-dependent wall-clock cost remain to be evaluated. Second, some tasks reach an ansatz or search-space floor, limiting the resolution of late-stage comparisons. Third, ensemble disagreement provides a useful ranking of prediction risk but is not a complete probabilistic calibration guarantee. 7 Conclusion DreamQAS treats molecular RL-QAS as a known-dynamics, expensive-feedback problem. It preserves exact legal circuit transitions, learns an oracle-free recurrent ensemble of post-VQE feedback, and converts that feedback into multi-step imagined policy gradients. Across five tasks, it achieves a strong final-quality profile while reaching matched error levels with fewer real VQE calls than a non-imaginative control. Counterfactual probes show that the model learns action-level utility; direct use of the same model cannot replace imagined policy learning; and ensemble disagreement identifies risky feedback. World models for QAS therefore need not reproduce the full environment or act as exact energy oracles: they should preserve known structure and learn the feedback required for reliable sequential decisions. References K. Chua, R. Calandra, R. McAllister, and S. Levine (2018) Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems 31. Cited by: §2. M. Deng, Z. He, S. Zheng, Y. Zhou, F. Zhang, and H. Situ (2023) A progressive predictor-based quantum architecture search with active learning. The European Physical Journal Plus 138 (10), p. 905. Cited by: §1, §2. Y. Du, T. Huang, S. You, M. Hsieh, and D. Tao (2022) Quantum circuit architecture search for variational quantum algorithms. npj Quantum Information 8 (1), p. 62. Cited by: §1, §2. H. R. Grimsley, S. E. Economou, E. Barnes, and N. J. Mayhall (2019) An adaptive variational algorithm for exact molecular simulations on a quantum computer. Nature communications 10 (1), p. 3007. Cited by: §2. D. Ha and J. Schmidhuber (2018) World models. arXiv preprint arXiv:1803.10122 2 (3), p. 440. Cited by: §1, §2. D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi (2020) Dream to control: learning behaviors by latent imagination. In International Conference on Learning Representations, Cited by: §1, §2. Z. He, M. Deng, S. Zheng, L. Li, and H. Situ (2024) Training-free quantum architecture search. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38, p. 12430–12438. Cited by: Table 7, §1, §2, §2, §5.1. Z. He, X. Zhang, C. Chen, Z. Huang, Y. Zhou, and H. Situ (2023) A gnn-based predictor for quantum architecture search: z. he et al.. Quantum Information Processing 22 (2), p. 128. Cited by: §1, §2. A. Ikhtiarudin, A. Das, P. Thakkar, and A. Kundu (2025) BenchRL-qas: benchmarking reinforcement learning algorithms for quantum architecture search. In Proceedings of the AAAI Symposium Series, Vol. 7, p. 358–367. Cited by: §2. M. Janner, J. Fu, M. Zhang, and S. Levine (2019) When to trust your model: model-based policy optimization. Advances in neural information processing systems 32. Cited by: §1, §2. A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J. M. Chow, and J. M. Gambetta (2017) Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets. nature 549 (7671), p. 242–246. Cited by: §1. X. Lu, K. Pan, G. Yan, J. Shan, W. Wu, and J. Yan (2023) Qas-bench: rethinking quantum architecture search and a benchmark. In International conference on machine learning, p. 22880–22898. Cited by: §1, §2. D. Martyniuk, J. Jung, D. Barta, and A. Paschke (2025) Benchmarking quantum architecture search with surrogate assistance. arXiv preprint arXiv:2506.06762. Cited by: §1, §2. K. Nakaji, L. B. Kristensen, R. Kemmoku, J. A. Campos-Gonzalez-Angulo, M. G. Vakili, H. Huang, M. Bagherimehrab, C. Gorgulla, F. Wong, A. McCaskey, et al. (2024) The generative quantum eigensolver (gqe) and its application for ground state search. arXiv preprint arXiv:2401.09253. Cited by: Table 7, §1, §2, §5.1. J. Niu, A. Karim, Y. Wang, J. Li, K. Deng, A. Alavi, M. Usman, and Y. Ren (2026) HamQASBench: a hamiltonian-informed diagnostic benchmark for evaluating quantum architecture search. arXiv preprint arXiv:2607.04845. Cited by: §2, §5.1. J. Niu, Y. Wang, J. Li, K. Deng, A. Alavi, M. Usman, and Y. Ren (2025) Hybrid action reinforcement learning for quantum architecture search. arXiv preprint arXiv:2511.04967. Cited by: Table 7, §2, §5.1. I. Osband, J. Aslanides, and A. Cassirer (2018) Randomized prior functions for deep reinforcement learning. Advances in neural information processing systems 31. Cited by: §A.2, §4.2. M. Ostaszewski, L. M. Trenkwalder, W. Masarczyk, E. Scerri, and V. Dunjko (2021) Reinforcement learning for optimization of variational quantum circuit architectures. Advances in neural information processing systems 34, p. 18182–18194. Cited by: §2. Y. J. Patel, A. Kundu, M. Ostaszewski, X. Bonet-Monroig, V. Dunjko, and O. Danaci (2024) Curriculum reinforcement learning for quantum architecture search under hardware errors. arXiv preprint arXiv:2402.03500. Cited by: Table 7, §2, §5.1. A. Peruzzo, J. McClean, P. Shadbolt, M. Yung, X. Zhou, P. J. Love, A. Aspuru-Guzik, and J. L. O’brien (2014) A variational eigenvalue solver on a photonic quantum processor. Nature communications 5 (1), p. 4213. Cited by: §1. F. Rapp, D. A. Kreplin, M. F. Huber, and M. Roth (2025) Reinforcement learning-based architecture search for quantum machine learning. Machine Learning: Science and Technology 6 (1), p. 015041. Cited by: §2. S. Ross, G. Gordon, and D. Bagnell (2011) A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, p. 627–635. Cited by: §2, §4.4. J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, et al. (2020) Mastering atari, go, chess and shogi by planning with a learned model. Nature 588 (7839), p. 604–609. Cited by: §2. V. P. Soloviev, V. Dunjko, C. Bielza, P. Larrañaga, and H. Wang (2024) Trainability maximization using estimation of distribution algorithms assisted by surrogate modelling for quantum architecture search. EPJ Quantum Technology 11 (1), p. 69. Cited by: §1, §2. R. S. Sutton (1991) Dyna, an integrated architecture for learning, planning, and reacting. ACM Sigart Bulletin 2 (4), p. 160–163. Cited by: §2. H. L. Tang, V. Shkolnikov, G. S. Barron, H. R. Grimsley, N. J. Mayhall, E. Barnes, and S. E. Economou (2021) Qubit-adapt-vqe: an adaptive algorithm for constructing hardware-efficient ansätze on a quantum processor. PRX Quantum 2 (2), p. 020310. Cited by: §2. J. Tilly, H. Chen, S. Cao, D. Picozzi, K. Setia, Y. Li, E. Grant, L. Wossnig, I. Rungger, G. H. Booth, et al. (2022) The variational quantum eigensolver: a review of methods and best practices. Physics Reports 986, p. 1–128. Cited by: §1. W. Wu, G. Yan, X. Lu, K. Pan, and J. Yan (2023) Quantumdarts: differentiable quantum architecture search for variational quantum algorithms. In International conference on machine learning, p. 37745–37764. Cited by: Table 7, §2, §5.1. T. Yu, G. Thomas, L. Yu, S. Ermon, J. Y. Zou, S. Levine, C. Finn, and T. Ma (2020) Mopo: model-based offline policy optimization. Advances in neural information processing systems 33, p. 14129–14142. Cited by: §2. S. Zhang, C. Hsieh, S. Zhang, and H. Yao (2021) Neural predictor based quantum architecture search. Machine Learning: Science and Technology 2 (4), p. 045027. Cited by: §1, §2. S. Zhang, C. Hsieh, S. Zhang, and H. Yao (2022) Differentiable quantum architecture search. Quantum Science & Technology 7 (4), p. 045023. Cited by: §1, §2. Appendix A Method and Algorithm Details A.1 Oracle-Free Feedback Construction The QAS environment exposes an exact circuit transition and an expensive post-VQE feedback function. For a real optimized energy E, DreamQAS stores the raw energy and constructs the supervision label only when the world model is refreshed. Let FpendingF_pending be the running minimum over every real optimized energy observed in policy rollouts or selective verification. At a world-model refresh, DreamQAS atomically adopts Fk←FpendingF_k← F_pending (15) and increments the scale version. The adopted frontier remains fixed until the next refresh, so every label in a supervised update uses the same reference. This pending/adopted separation makes the supervision coordinate system explicit. A newly observed low energy changes the frontier-relative score of every stored example, not only the newest transition. Updating the reference inside a world-model optimization block would therefore mix labels from different scale versions and confound model residuals with target movement. DreamQAS instead stores raw energies, adopts one frontier per refresh, and relabels replay atomically. Historical circuits remain usable after the search improves because their targets can always be reconstructed under the current version. For an energy E, define dk(E)=E−Fk10−3Ha,Sk(E)=sign⁡(dk(E))log10⁡(1+|dk(E)|m),m=0.1mHa.d_k(E)= E-F_k10^-3\,Ha, S_k(E)=sign(d_k(E)) _10\! (1+ |d_k(E)|m ), m=0.1\,mHa. (16) The transform is strictly monotone, continuous, and differentiable at the frontier. Raw energies in replay are relabeled using Eq. (16) after each frontier adoption. The corresponding positive companion scale smHa=m10Sk(E)s_\,mHa=m10^S_k(E) is used by the oracle-free quality sampler. Real trajectory rewards are score differences, rt+1real=Sk(Et)−Sk(Et+1).r_t+1^real=S_k(E_t)-S_k(E_t+1). (17) No ground-state energy enters frontier construction, model supervision, replay sampling, action selection, or checkpoint selection. For d≥0d≥ 0, the score has the same reward differences as the legacy log10⁡(m+d) _10(m+d) transform because the two differ only by the constant log10⁡m _10m. For d<0d<0, the signed-log branch compresses improvements beyond the current frontier logarithmically instead of extending the positive branch linearly. This is the transform used in all oracle-free runs reported in the paper. A.2 World-Model Architecture and Replay The feedback model is an ensemble of K=3K=3 fully independent members. Each member contains its own gate embedding, two-layer GRU, and prediction head LayerNorm→Linear(h,128)→SiLU→Linear(128,1).LayerNorm (h,128) (128,1). (18) The default GRU hidden size is 384384, with task-specific sizes 256256 on LiH-4q and 512512 on BeH2-10q. Each prediction combines a trainable head and a frozen randomized prior function (Osband et al., 2018), scaled by βrpf=3 _rpf=3. During each supervised update, every member receives an independent 80%80\% bootstrap mask. The actor feature is the mean of the members’ recurrent representations; the predicted feedback and epistemic signal are the ensemble mean and population standard deviation. The recurrent state represents the complete ordered circuit prefix rather than a learned transition state. After an imagined gate is appended by the exact QAS transition, each ensemble member encodes the resulting explicit gate sequence and predicts only its post-VQE feedback. Imagined rollouts therefore do not autoregress through a learned circuit-dynamics model, and transition error cannot compound with rollout length. Independent encoders, bootstrap masks, and randomized priors provide three sources of ensemble diversity. The actor consumes the mean recurrent feature with a stopped gradient, so policy updates cannot reshape the feedback representation to make imagined returns easier to optimize. The all-history replay sampler draws four complementary subsets: pelite=0.25,ppriority=0.35,pstratified=0.20,puniform=0.20.p_elite=0.25, p_priority=0.35, p_stratified=0.20, p_uniform=0.20. (19) Priority combines recency, oracle-free quality, and the most recent world-model error. Stratification covers ten target deciles. Decision-oriented inverse-density reweighting smooths a 40-bin histogram of frontier scores with a Gaussian kernel of width two bins and applies weights proportional to the inverse square root of density, normalized to mean one and capped at 5050. The four replay subsets address different failure modes of QAS feedback data. Elite sampling preserves rare low-energy prefixes; priority sampling revisits recent or poorly predicted policy-visited regions; target stratification keeps the dominant plateaus from occupying an entire update; and uniform sampling maintains coverage outside these selected regions. Inverse-density weighting changes the optimization emphasis within the resulting batch but never changes the stored raw energy or frontier-relative target. It is therefore a decision-oriented allocation of supervised capacity, rather than a new reward definition. The weighted per-member Huber loss is ℒWM=1K∑k=1K∑i,tqitμit(k)w(yit)ℓH(y^it(k)−yit)∑i,tqitμit(k).L_WM= 1K _k=1^K _i,tq_it _it^(k)w(y_it) _H( y_it^(k)-y_it) _i,tq_it _it^(k). (20) Group Parameter Value World model ensemble members / GRU layers 33 / 22 embedding / default hidden size 6464 / 384384 hidden overrides LiH-4q: 256256; BeH2-10q: 512512 randomized-prior scale 3.03.0 optimizer / learning rate / weight decay Adam / 10−310^-3 / 10−510^-5 refresh interval / gradient steps 2020 iterations / 200200 Actor hidden size / layers 512512 / 33 optimizer / learning rate Adam / 3×10−53× 10^-5 discount / entropy coefficient 0.990.99 / 10−310^-3 gradient clipping 10.010.0 Replay batch size / elite capacity 6464 / 10001000 elite / priority / stratified / uniform .25/.35/.20/.20.25/.35/.20/.20 DIR bins / smoothing / maximum weight 4040 / 2.02.0 / 5050 Table 4: World-model, actor, and replay hyperparameters used by the oracle-free full method. Task-specific settings not listed here are given in Section B. A.3 Imagined Policy Learning and Reliability Controls Each update collects four real policy episodes. Every appended gate is followed by a real VQE optimization, so an episode of realized depth d costs d real VQE calls. Real REINFORCE uses a batch-mean baseline and standardized real advantages; no learned critic is used. Once the ranking gate is active, 6464 replay prefixes are sampled from the most recent 200200 episodes. Each prefix is cut at a uniformly sampled position and extended for up to H=15H=15 exact legal transitions. The pessimistic potential and imagined reward are Φt=−(y¯t+βpesσt),rt+1imag=Φt+1−Φt, _t=-( y_t+ _pes _t), r_t+1^imag= _t+1- _t, (21) with βpes=1 _pes=1. Imagined returns use γ=0.99γ=0.99 and λ=0.95λ=0.95. A trajectory is truncated when the exact legal-action set becomes empty or σt>0.60 _t>0.60; the rejected step receives no credit. The global ranking gate is evaluated every 20 iterations after at least 500 real episodes. It samples 50,000 pairs from at most 200 recent held-out episodes and enables imagination when pairwise ranking accuracy reaches 0.700.70. Every 20 iterations, selective verification generates fresh imagined candidates, deduplicates them, selects five with the best predicted value and five with the largest disagreement, and replays each selected circuit through the real VQE environment from the empty circuit. Verified trajectories are returned to replay in the full method. These controls act at distinct levels of the learning loop. The ranking gate is a global readiness test that determines whether model-generated policy updates are enabled at all. Pessimism continuously modifies feedback at every accepted imagined step, whereas truncation determines how far a particular trajectory is trusted. Selective verification operates at the candidate level: value-based selection checks regions the policy is encouraged to exploit, and disagreement-based selection checks potential blind spots. Returning both strata as raw-energy trajectories closes the loop without changing the exact circuit transition or exposing the policy to E0E_0. Algorithm 1 Oracle-free DreamQAS training 1: Initialize actor πθ _θ, ensemble FϕF_φ, replay D, pending/adopted frontiers, and VQE counters 2: for training iteration i=1,…,Ni=1,…,N do 3: Collect four real QAS episodes with exact legal transitions 4: Run VQE after each applied gate; add raw energies and circuits to D; update the pending frontier 5: Update πθ _θ using real REINFORCE returns 6: if i is a world-model refresh iteration then 7: Adopt the pending frontier and relabel replay from raw energies 8: Train each ensemble member for 200 bootstrapped replay updates 9: Evaluate pairwise ranking readiness on held-out real prefixes 10: end if 11: if warm-up and ranking gate permit imagination then 12: Sample 64 verified replay prefixes and generate exact-transition imagined rollouts of horizon at most 15 13: Compute pessimistic feedback, confidence masks, and imagined multi-step returns 14: Update πθ _θ using the combined real and imagined loss 15: end if 16: if i is a selective-verification iteration then 17: Select five predicted-best and five highest-disagreement circuits 18: Replay them using real VQE and charge the calls to the real budget 19: Add verified trajectories to replay 20: end if 21: end for The implementation maintains three counters. vqe_calls contains real rollout steps plus full-method selective-verification replays and is the reported acceleration budget. calib_vqe_calls records the identical diagnostic probe in controls that do not feed verified data back and is excluded from their training budget. eval_vqe_calls records frozen-policy evaluation and is excluded for every method. Appendix B Tasks, Baselines, and Evaluation Protocol B.1 Molecular Tasks and QAS Environment Table 5 records the molecular geometries and basis sets used in the five tasks. LiH-4q, LiH-6q, and BeH2-6q use STO-3G. BeH2-8q and BeH2-10q correspond to the HamQASBench entries BeH2_631G and BeH2_6311G, respectively. Task Geometry (Å) Basis n_q LiH-4q Li(0,0,0)(0,0,0); H(0,0,3.4)(0,0,3.4) STO-3G 4 BeH2-6q Be(0,0,0)(0,0,0); H(0,0,1.33)(0,0,1.33); H(0,0,−1.33)(0,0,-1.33) STO-3G 6 LiH-6q Li(0,0,0)(0,0,0); H(0,0,2.2)(0,0,2.2) STO-3G 6 BeH2-8q Be(0,0,0)(0,0,0); H(0,0,1.326)(0,0,1.326); H(0,0,−1.326)(0,0,-1.326) 6-31G 8 BeH2-10q Be(0,0,0)(0,0,0); H(0,0,1.326)(0,0,1.326); H(0,0,−1.326)(0,0,-1.326) 6-311G 10 Table 5: Molecular geometries and basis sets. The LiH-4q, LiH-6q, and BeH2-6q geometries follow the experiment geometry table; the BeH2-8q and BeH2-10q rows follow the HamQASBench molecule-generation table. LiH-4q BeH2-6q LiH-6q BeH2-8q BeH2-10q Mapping parity JW JW JW JW Symmetry taper Yes No No Yes Yes Exact E0E_0 (Ha) −7.78908889-7.78908889 −14.86158918-14.86158918 −7.84487909-7.84487909 −15.76150376-15.76150376 −15.76512417-15.76512417 Pauli terms 100 34 118 60 155 Hamiltonian dimension 1616 6464 6464 256256 10241024 Maximum circuit depth 40 50 50 50 50 Action-space size 24 48 48 80 120 VQE optimizer COBYLA COBYLA COBYLA ROTOSOLVE ROTOSOLVE Inner budget 1000 iters 1000 iters 1000 iters 3 sweeps 3 sweeps Training backend CPU c128 CPU c128 CPU c128 GPU c64 GPU c64 Main episodes 15,000 15,000 15,000 15,000 15,000 Training seeds 5 5 5 5 5 Table 6: Recorded Hamiltonian, search-space, and optimization settings. JW denotes the Jordan–Wigner mapping. The mapping and taper entries for LiH-4q, BeH2-6q, and LiH-6q follow the experiment geometry/mapping table; the 8q/10q taper flags follow the run configuration. Frozen evaluation of GPU-trained tasks is recomputed on a complex128 reference backend. For n qubits, the action dictionary contains n(n−1)n(n-1) directed CNOT actions and 3n3n single-qubit rotations, giving n(n+2)n(n+2) actions in total. CNOTs are unparameterized. Each rotation introduces one angle, and VQE reoptimizes all angles after every gate placement. The legality mask forbids immediate repetition of the same single-qubit rotation or the same directed CNOT. The same stateless legality implementation is used by real and imagined rollouts. Training terminates an episode at the task-specific curriculum threshold or at the maximum structural depth. Frozen evaluation disables the accuracy stop and always constructs to the structural termination condition. Thus policies are compared under the same evaluation trajectory length rather than under their training-time stopping behavior. B.2 Baselines and Statistical Protocol Internal controls share the oracle-free training signal, task environment, action space, VQE implementation, and 15,000-episode main checkpoint. DreamQAS-RL is the model-free policy. NoImag keeps the recurrent circuit representation and supervised feedback training but removes imagined policy updates. NoDAG keeps ranking-gated, uncertainty-controlled imagination but discards selectively verified samples. Additional controls remove decision-oriented reweighting or both uncertainty uses. Method Algorithm family Search budget Training reference CRLQAS (Patel et al., 2024) N-step DQN with curriculum 15,000 episodes Exact E0E_0 HyRLQAS (Niu et al., 2025) Hybrid REINFORCE 15,000 episodes Exact E0E_0 GQE (Nakaji et al., 2024) Transformer generative search 1,000–1,500 epochs Native objective TFQAS (He et al., 2024) Training-free screening 11,250 candidates Native objective QuantumDARTS (Wu et al., 2023) Differentiable search 500–800 epochs Native objective Table 7: Compact baseline configuration summary. All methods use the same Hamiltonians, circuit-depth caps, task-level optimizer families, and five-run protocol. Full run configurations are provided with the supplementary artifact. Table 7 records only the settings needed to interpret the comparison. CRLQAS and HyRLQAS retain their original E0E_0-informed learning rules under the common 15,000-episode RL budget. The remaining baselines use method-native search units, so their results provide final-quality context only. Real-VQE efficiency claims are restricted to the exactly matched DreamQAS internal controls. At the main checkpoint, each frozen policy runs 100 fresh stochastic evaluation episodes per training seed. The energy error of an evaluation episode is defined by the lowest post-VQE energy among its circuit prefixes. The per-seed score is the mean over these 100 episode-level errors, and tables report mean ± sample standard deviation across five training seeds. We refer to this quantity as the mean frozen-policy energy error. During frozen evaluation, E0E_0 is used only to convert the selected energy to a physical error in mHa. Training-efficiency curves use the trailing-100-episode mean within each seed and pointwise median and IQR across seeds. Seed-level crossings are evaluated every 50 episodes after the trailing window is full. For a fixed error y, the crossing is the first point at which a seed remains below y for three consecutive reported points. Seed-level summaries take the median only over seeds that reach y and always report the reach count; unreached seeds are right-censored and never imputed. A crossing is reported only when at least three of five seeds reach it. A ≤ marker denotes a left-censored crossing: at least one contributing seed had already reached the target at the first admissible full-window point, so the displayed call count is an upper bound. Paired quality comparisons use percentile bootstrap intervals on the paired mean difference. Action-utility growth additionally reports Holm-adjusted one-sample t-test p-values and a task-cluster-aware summary using the five task means as the experimental units. Risk–coverage computes AURC per seed and then applies paired bootstrap contrasts across five seeds. Appendix C Complete Performance and Efficiency Results C.1 Full VQE-Efficiency Curves and Crossings The main paper presents LiH-4q, LiH-6q, and BeH2-8q. Figure 4 supplies the remaining BeH2-6q and BeH2-10q curves under the same trailing-window and real-VQE accounting. The figures aggregate curves pointwise across seeds for visualization, whereas every numerical speedup claim below first computes a sustained crossing within each seed and only then aggregates across reached seeds. Figure 4: Additional oracle-free VQE-efficiency curves. Training quality is plotted against cumulative real VQE calls for BeH2-6q and BeH2-10q. Curves use the same across-seed median and IQR construction as the main-text figure. Task / target DreamQAS NoImag Reach Saving LiH-4q, 1mHa1\,mHa 166,396 266,292 5/5 vs. 5/5 1.60×1.60× BeH2-6q, 1mHa1\,mHa 237,538 426,911 5/5 vs. 5/5 1.80×1.80× LiH-6q, 20mHa20\,mHa 212,738 418,657 5/5 vs. 5/5 1.97×1.97× BeH2-8q, 3mHa3\,mHa 39,714 420,128 5/5 vs. 5/5 10.58×10.58× BeH2-10q, 5mHa5\,mHa 30,427 50,625 5/5 vs. 5/5 1.66×1.66× Table 8: Fine-target seed-level sustained crossings used for the main-text efficiency claim. Calls are medians over reached seeds; every selected cell has full reach for both methods. Saving is NoImag divided by DreamQAS and is not averaged across tasks. Table 8 selects the finest task-specific target reached by all five seeds of both matched oracle-free methods. It therefore avoids comparing a survivor-biased median from a partially reaching method with a fully reaching method. The full ladders in Tables 9–13 report every pre-specified target. Each cell gives median cumulative real training VQE calls and reach count on the first line, followed by the min–max range over reached seeds. DreamQAS-family counts include selective-verification VQE and exclude frozen-evaluation VQE; LiH-6q is capped at the locked 15,000-episode budget. Method ≤ ≤ 10\,mHa ≤ ≤ 5\,mHa ≤ ≤ 2\,mHa ≤ ≤ 1\,mHa Oracle-free internal campaign DreamQAS 15,563 (5/5) [12,551–16,475] 26,823 (5/5) [25,181–36,502] 91,653 (5/5) [62,542–94,747] 166,396 (5/5) [150,005–186,911] NoImag 18,206 (5/5) [8,054–22,114] 33,246 (5/5) [23,574–38,988] 126,924 (5/5) [94,252–150,645] 266,292 (5/5) [225,211–294,966] DreamQAS-RL 6,968 (5/5) [5,729–21,851] 53,775 (5/5) [19,417–255,156] 184,694 (5/5) [133,948–365,808] 252,045 (5/5) [207,953–410,515] Canonical external reference CRLQAS 67,887 (5/5) [51,998–87,710] 131,745 (5/5) [75,601–135,479] – (0/5) – (0/5) HyRLQAS 18,668 (5/5) [16,214–23,912] 42,683 (5/5) [26,258–228,412] 149,900 (4/5) [66,529–188,432] – (0/5) Table 9: LiH-4q sustained-crossing ladder. Cell format is median calls (reach), followed by the reached-seed min–max range. Method ≤ ≤ 10\,mHa ≤ ≤ 5\,mHa ≤ ≤ 2\,mHa ≤ ≤ 1\,mHa Oracle-free internal campaign DreamQAS 17,882 (5/5) [17,275–28,288] 55,161 (5/5) [30,026–58,084] 172,719 (5/5) [151,191–230,552] 237,538 (5/5) [196,712–328,375] NoImag 16,039 (5/5) [15,199–22,365] 77,798 (5/5) [59,852–133,100] 290,576 (5/5) [256,133–345,756] 426,911 (5/5) [393,764–514,785] DreamQAS-RL 24,468 (5/5) [20,703–32,221] 134,990 (5/5) [34,596–200,903] 274,842 (5/5) [257,069–293,816] 373,258 (5/5) [312,813–425,345] Canonical external reference CRLQAS 110,633 (5/5) [76,232–137,771] 154,472 (5/5) [80,674–187,433] 264,993 (4/5) [217,858–442,842] – (1/5) HyRLQAS 9,691 (5/5) [7,848–11,350] 181,812 (5/5) [25,478–474,415] 183,658 (4/5) [41,109–335,371] 186,714 (4/5) [57,210–363,272] Table 10: BeH2-6q sustained-crossing ladder. Formatting follows Table 9. Method ≤ ≤ 50\,mHa ≤ ≤ 30\,mHa ≤ ≤ 20\,mHa ≤ ≤ 15\,mHa Oracle-free internal campaign DreamQAS 9,645 (5/5) [9,460–10,863] 19,460 (5/5) [15,782–27,628] 212,738 (5/5) [163,799–255,392] 239,289 (5/5) [234,646–301,878] NoImag 10,458 (5/5) [9,769–10,844] 15,340 (5/5) [9,769–29,692] 418,657 (5/5) [301,236–703,450] 479,310 (4/5) [333,836–509,356] DreamQAS-RL 8,885 (5/5) [8,547–9,391] 9,391 (5/5) [8,689–25,722] – (0/5) – (0/5) Canonical external reference CRLQAS ≤5,050≤ 5,050 (5/5) [5,050–5,050] 102,550 (5/5) [77,550–132,550] 265,050 (5/5) [100,050–703,665] – (2/5) HyRLQAS 7,151 (5/5) [5,128–7,385] 23,255 (5/5) [19,167–43,049] – (1/5) – (0/5) Table 11: LiH-6q sustained-crossing ladder at the locked 15,000-episode budget. The ≤ entry is left-censored and is an upper bound. Method ≤ ≤ 20\,mHa ≤ ≤ 10\,mHa ≤ ≤ 5\,mHa ≤ ≤ 3\,mHa Oracle-free internal campaign DreamQAS 23,885 (5/5) [21,262–34,472] 25,494 (5/5) [22,144–37,314] 38,400 (5/5) [25,316–72,726] 39,714 (5/5) [27,984–129,365] NoImag 25,461 (5/5) [21,035–30,931] 33,769 (5/5) [26,830–203,238] 281,678 (5/5) [43,890–343,209] 420,128 (5/5) [136,128–502,705] DreamQAS-RL 45,225 (5/5) [30,223–64,163] 51,824 (5/5) [31,252–73,455] 52,080 (5/5) [36,261–84,196] 80,105 (5/5) [65,589–151,659] Canonical external reference CRLQAS 60,050 (5/5) [55,021–77,541] 70,021 (5/5) [60,050–110,040] 130,021 (5/5) [117,550–417,536] 305,048 (4/5) [142,550–432,423] HyRLQAS 22,951 (5/5) [4,615–29,312] 20,920 (4/5) [4,615–31,436] 22,442 (4/5) [4,724–33,827] 23,507 (4/5) [4,724–36,163] Table 12: BeH2-8q sustained-crossing ladder. Formatting follows Table 9. Method ≤ ≤ 30\,mHa ≤ ≤ 15\,mHa ≤ ≤ 10\,mHa ≤ ≤ 5\,mHa Oracle-free internal campaign DreamQAS 27,486 (5/5) [16,370–29,093] 28,727 (5/5) [17,121–30,032] 29,699 (5/5) [17,121–31,011] 30,427 (5/5) [18,542–32,015] NoImag 25,168 (5/5) [15,219–31,448] 29,715 (5/5) [15,219–34,418] 32,486 (5/5) [21,387–50,147] 50,625 (5/5) [38,516–84,838] DreamQAS-RL 67,204 (4/5) [49,028–223,080] 72,882 (4/5) [53,156–320,680] 73,530 (4/5) [60,083–320,680] 77,178 (3/5) [65,672–83,404] Canonical external reference CRLQAS 22,016 (5/5) [16,387–28,424] 26,147 (5/5) [24,622–33,180] 28,241 (5/5) [26,500–36,264] 34,844 (5/5) [26,688–43,615] HyRLQAS 27,271 (5/5) [12,237–30,887] 28,195 (5/5) [13,657–32,996] 28,299 (5/5) [13,657–32,996] 28,400 (5/5) [13,657–33,893] Table 13: BeH2-10q sustained-crossing ladder. Formatting follows Table 9. The ladders show that the matched DreamQAS advantage is concentrated in fine-error refinement rather than coarse basin entry. At loose targets NoImag or DreamQAS-RL can cross first, whereas tightening the target increases the Full-versus-NoImag saving to the values summarized in Table 8. DreamQAS-RL also reaches loose LiH-6q targets but reaches neither 20mHa20\,mHa nor 15mHa15\,mHa in any seed. This separates cheap entry into a coarse-quality region from reliable refinement using learned feedback and imagination. CRLQAS and HyRLQAS share the prefix-VQE counting unit but use canonical E0E_0-informed rewards and stopping rules. Their rows are therefore descriptive training-curve references, not matched causal speed comparisons. In particular, a low reached-seed crossing need not imply strong frozen-policy quality: on BeH2-8q, HyRLQAS reaches several targets on four seeds with low call counts yet has substantially worse frozen-policy error in Table 1. GQE, TFQAS, and QuantumDARTS are omitted because their native search units do not admit the same prefix-VQE crossing estimator. Final policy quality and fixed-error crossings measure different properties. The former asks what a frozen policy produces at the common checkpoint; the latter asks how much real feedback was required before training reached a specified quality level. A crossing is counted only after the curve remains below its target for three reported points, which prevents a single favorable episode from defining the cost. Seed-level summaries retain the reach count and right-censor unreached runs, so a ratio is reported only where the target is supported by the observed training trajectories. This is why the efficiency claim is stated at task-specific achieved errors rather than at one universal threshold. C.2 Computational Cost Task Arm Real VQE Rollout/enc. WM train Imagine Other LiH-4q Full 92.8%92.8\% 3.0%3.0\% 1.8%1.8\% 1.6%1.6\% 0.8%0.8\% NoImag 94.7%94.7\% 3.3%3.3\% 1.9%1.9\% 0.0%0.0\% 0.0%0.0\% LiH-6q Full 89.0%89.0\% 5.0%5.0\% 2.5%2.5\% 2.2%2.2\% 1.3%1.3\% NoImag 90.9%90.9\% 6.1%6.1\% 2.9%2.9\% 0.0%0.0\% 0.0%0.0\% Table 14: Steady-state phase shares from controlled state-vector timing runs (iterations ≥300≥ 300). Absolute seconds are omitted because both arms ran under shared node load; phase shares and matched-load ratios are the intended quantities. Verification accounts for approximately 11.1%11.1\% and 8.7%8.7\% of total runtime on LiH-4q and LiH-6q, respectively. Including verification, imagination, and the incremental difference from the recurrent model apparatus, the full method adds 12.7%12.7\% over NoImag on LiH-4q and 10.9%10.9\% on LiH-6q. Verification is the largest single added component. These percentages describe state-vector training and are not hardware wall-clock estimates. The timing decomposition also separates computational overhead from real-VQE efficiency. Recurrent encoding, world-model fitting, and imagination add classical work, while verification adds both classical replay work and charged real VQE calls. The crossing analysis already includes those verification calls; the reported feedback savings therefore do not treat model maintenance as free. Appendix D Oracle-Free Signal and Decision Utility D.1 Oracle-Free Reward Audit We recompute the oracle-free and FCI-referenced rewards on the identical real rollout transitions used for training: rOF(t)=S(Et)−S(Et+1),rFCI(t)=log10⁡(Et−E0)−log10⁡(Et+1−E0).r_OF(t)=S(E_t)-S(E_t+1), r_FCI(t)= _10(E_t-E_0)- _10(E_t+1-E_0). (22) The FCI quantity is a read-only diagnostic and never enters training. Both scores are strictly increasing in energy, so an actual energy change has the same reward direction under both transforms. Table 15 measures how closely their magnitudes are ordered on transitions whose logged energy changes. Task Moved/seed (rOF,rFCI) ρ(r_OF,r_FCI) Sign [Δr]E[ r] (Δr)SD( r) |Δr|P95| r| LiH-4q 426,757 0.990±0.0020.990± 0.002 0.9998 −0.0072-0.0072 0.1189 0.1645 BeH2-6q 190,261 0.988±0.0040.988± 0.004 0.9995 −0.0223-0.0223 0.1997 0.4274 LiH-6q 247,351 0.998±0.0020.998± 0.002 0.9992 +0.0051+0.0051 0.0751 0.0609 BeH2-8q 70,443 0.990±0.0040.990± 0.004 0.9995 +0.0999+0.0999 0.4105 0.7266 BeH2-10q 65,318 0.995±0.0040.995± 0.004 0.9993 +0.1411+0.1411 0.3387 0.6924 Table 15: Same-transition oracle-free reward audit, aggregated across the full training trajectory. Statistics are computed per seed and then averaged across five seeds. The residual deviation from unit sign agreement comes from six-digit logging precision on near-zero changes. The ranking correlation remains between 0.9880.988 and 0.9980.998 over the full trajectories and between 0.9730.973 and 1.0001.000 in the late stage. The largest magnitude differences occur near the empirical frontier. LiH-6q has the smallest late-stage magnitude distortion but the largest adopted-frontier lag, indicating that moving-reference nonstationarity is distinct from per-step reward distortion. The reward audit tests the target, not the learned model. Monotonicity gives direction agreement analytically, while the measured Spearman correlation asks whether the relative magnitudes of improvements are preserved along the actual training distribution. The later counterfactual probe asks a separate question—whether a fitted ensemble can rank several legal continuations from the same prefix. Keeping these tests separate distinguishes the quality of the oracle-free supervision signal from the quality of the model learned from it. For sensitivity, Table 16 compares independently trained oracle-free and canonical campaigns. This is not a paired intervention, but it quantifies the task-dependent cost of removing E0E_0 from training. Task Oracle-free Canonical Relative change LiH-4q 0.053±0.0240.053± 0.024 0.038±0.0570.038± 0.057 +42%+42\% BeH2-6q 0.0580.058 0.0580.058 0%0\% LiH-6q 13.4±2.413.4± 2.4 10.1±2.010.1± 2.0 +33%+33\% BeH2-8q 2.172.17 2.172.17 0%0\% BeH2-10q 1.03±0.291.03± 0.29 1.70±1.101.70± 1.10 −40%-40\% Table 16: Cross-campaign sensitivity of mean frozen-policy energy error in mHa. The campaigns differ in training signal and stochastic draws; the table is a sensitivity analysis rather than a paired causal comparison. D.2 Counterfactual Action-Utility Protocol and Tests For each of 15 policy-visited real prefixes per seed, the probe samples up to ten legal next actions. The frozen ensemble predicts each continuation, and the same candidates are evaluated using real VQE. Within each prefix, the probe computes Spearman ranking correlation, normalized regret NReg=e(aWM)−mina⁡e(a)maxa⁡e(a)−mina⁡e(a),NReg= e(a_WM)- _ae(a) _ae(a)- _ae(a), (23) and the frequency with which the WM-selected action is no worse than matched random legal choices. Per-prefix statistics are averaged within seed before the five-seed summary. Task ρ Bootstrap CI t p_t Holm p_Holm Wilcoxon p_Wilcoxon LiH-4q +0.449+0.449 [+0.343,+0.554][+0.343,+0.554] .0018 .0059 .0625 BeH2-6q +0.148+0.148 [+0.057,+0.238][+0.057,+0.238] .0481 .0481 .1250 LiH-6q +0.281+0.281 [+0.170,+0.386][+0.170,+0.386] .0105 .0211 .0625 BeH2-8q +0.441+0.441 [+0.340,+0.538][+0.340,+0.538] .0015 .0059 .0625 BeH2-10q +0.411+0.411 [+0.352,+0.470][+0.352,+0.470] .0003 .0014 .0625 Task-cluster summary +0.346+0.346 [+0.185,+0.507][+0.185,+0.507] .0040.0040 – – Table 17: Growth in action-ranking utility from start to the 15,000-episode checkpoint. Per-task ptp_t values are two-sided one-sample t-tests over five paired seed differences; Holm adjusts this five-test family. Bootstrap CIs and t-test p-values are separately computed summaries. The final row treats the five task means as the experimental units. All five paired changes are positive and remain below 0.050.05 after Holm adjustment. The distribution-free Wilcoxon results are included because with n=5n=5 its minimum attainable two-sided p-value is 0.06250.0625. The cluster-aware summary is therefore the primary cross-task test used in the main paper. The probe uses ordinal ranks and excludes constant candidate vectors from the correlation. Constant vectors still contribute to NReg and WM>>random; for a fully tied candidate set NReg is defined as zero. The final BeH2-8q and BeH2-10q candidate sets contain extensive ties, so their NReg values are interpreted together with ρact _act rather than in isolation. The three action metrics are complementary. Spearman correlation evaluates the ordering of all sampled legal continuations, normalized regret evaluates the cost of the model’s top-ranked choice, and WM>>random evaluates whether that choice improves on a matched uninformed decision. The increase in rank correlation from the start checkpoint to the operating checkpoint shows that the ordering signal is acquired during training; the final regret and WM>>random values then quantify whether that learned ordering supports useful choices. D.3 Auxiliary World-Model Diagnostics Task Pairwise fidelity Calibration MAE (σ,|error|) ρ(σ,|error|) LiH-4q 0.949±0.0060.949± 0.006 0.396±0.0150.396± 0.015 0.289±0.0370.289± 0.037 LiH-6q 0.790±0.0170.790± 0.017 0.338±0.1270.338± 0.127 0.296±0.1060.296± 0.106 BeH2-8q 0.963±0.0170.963± 0.017 0.599±0.2370.599± 0.237 0.720±0.0410.720± 0.041 Table 18: Final logged oracle-free world-model diagnostics. Calibration MAE is measured in the frontier-score space and should not be numerically compared with canonical log-error calibration. The ranking gate observes high pairwise fidelity on all three diagnostic tasks. Depth-stratified endpoint analyses are reported only as diagnostic accuracy versus circuit depth; they are distinct from the controlled imagined-horizon experiment in Appendix E. Appendix E Direct Deployment and Horizon Controls E.1 Same-Model Deployment Controls Arm LiH-4q LiH-6q BeH2-8q NoImag 0.221±0.0620.221± 0.062 13.6±3.513.6± 3.5 2.35±0.402.35± 0.40 WM-Greedy, β=+1β=+1 6.42±1.26.42± 1.2 36.9±0.036.9± 0.0 2.17±0.02.17± 0.0 WM-Greedy, β=−1β=-1 5.91±1.45.91± 1.4 32.0±11.032.0± 11.0 2.17±0.02.17± 0.0 WM-Greedy, β=−2β=-2 – 36.9±0.036.9± 0.0 – DreamQAS-NoDAG 0.073±0.0360.073± 0.036 11.4±0.711.4± 0.7 2.53±0.492.53± 0.49 Table 19: Mean frozen-policy energy error in mHa at 15,000 episodes. All model-based deployment rows use the same learned feedback capability; only its use for direct selection or imagined policy learning changes. To isolate deployment from model training, Table 20 starts from a single frozen NoDAG checkpoint for each seed and changes only the rule that converts its ensemble predictions into actions. Greedy ϵ=0ε=0 constructs one deterministic trajectory, greedy ϵ>0ε>0 adds exploratory action sampling, and beam search retains the ten highest-scoring prefixes at each depth. Beam search uses no real VQE for expansion or pruning; its final trajectory is evaluated only after the search terminates. Task Checkpoint Actor Greedy ϵ=0ε=0 Greedy ϵ>0ε>0 Beam B=10B=10 LiH-4q 15k 0.073±0.0360.073± 0.036 4.280±1.9664.280± 1.966 3.378±0.6223.378± 0.622 4.434±0.0004.434± 0.000 LiH-6q 30k 11.055±1.94911.055± 1.949 31.974±10.98231.974± 10.982 34.265±5.91634.265± 5.916 36.885±0.00036.885± 0.000 BeH2-8q 15k 2.529±0.4952.529± 0.495 2.175±0.0002.175± 0.000 2.175±0.0012.175± 0.001 2.175±0.0002.175± 0.000 Table 20: Same-frozen-WM deployment control: post-VQE energy error in mHa (mean ± sample std across five seeds). The actor and exploratory-greedy columns average 100 frozen rollouts per seed; deterministic greedy and beam produce one circuit per seed. Within each task, all four rules use identical NoDAG world-model weights and the same checkpoint; only deployment changes. The LiH-6q control uses the shared 30,000-episode checkpoint available for all four rules and is separate from the 15,000-episode main comparison. Lower is better. Relative to WM-Greedy with β=+1β=+1, NoDAG improves paired error by 6.351mHa6.351\,mHa on LiH-4q (95% CI [5.319,6.949][5.319,6.949] in magnitude) and 25.516mHa25.516\,mHa on LiH-6q (95% CI [25.006,26.084][25.006,26.084]); all five seeds favor imagined policy learning. Reversing or increasing optimism does not close the gap. In the stricter same-frozen-WM control, the learned actor improves over beam search by 4.361mHa4.361\,mHa on LiH-4q (95% CI [4.332,4.388][4.332,4.388] in magnitude) and 25.830mHa25.830\,mHa on LiH-6q (95% CI [24.475,27.488][24.475,27.488]); all five paired seeds favor the actor. The corresponding actor–exploratory-greedy gaps are 3.305mHa3.305\,mHa (95% CI [2.768,3.747][2.768,3.747]) and 23.210mHa23.210\,mHa (95% CI [19.438,25.601][19.438,25.601]). BeH2-8q is saturated and does not distinguish the deployment rules. This result should be read together with the counterfactual action probe. DreamQAS’s ensemble can acquire useful local rankings while direct greedy use of its absolute predictions still performs poorly. Greedy deployment commits to one model-preferred continuation at each prefix; imagined policy learning instead aggregates predicted improvement signals over many replay-anchored trajectories before changing the actor. The comparison therefore isolates how the same feedback capability is consumed, rather than comparing a strong model with a deliberately weakened surrogate. E.2 Transition-Matched Horizon Control The H=1H=1 and H=5H=5 arms receive an explicit transition budget equal to the trusted imagined transitions used by the reference H=15H=15 arm: 954 per update on LiH-4q, 960 on LiH-6q, and 601 on BeH2-8q. Thus the comparison changes how model queries are organized into trajectories, not their total number. Task Contrast Paired Δ 95% CI Interpretation LiH-4q H1−H15H1-H15 +1.150+1.150 [+0.223,+2.763][+0.223,+2.763] multi-step better LiH-4q H5−H15H5-H15 +0.064+0.064 [+0.033,+0.090][+0.033,+0.090] sub-chemical LiH-6q H1−H15H1-H15 +7.166+7.166 [+1.226,+13.785][+1.226,+13.785] multi-step better LiH-6q H5−H15H5-H15 +0.379+0.379 [−0.412,+1.452][-0.412,+1.452] comparable BeH2-8q H1−H15H1-H15 −0.265-0.265 [−0.536,+0.000][-0.536,+0.000] saturated Table 21: Paired horizon contrasts at a matched trusted-transition budget. Positive H1−H15H1-H15 means the one-step arm has higher error. Values are in mHa. The non-saturated LiH tasks support the same mechanism conclusion: organizing trusted model transitions into multi-step trajectories improves final policy quality relative to independent one-step feedback. The comparison between H=5H=5 and H=15H=15 does not establish a universal preference for the longest horizon. Because every imagined gate uses the exact circuit transition and the trusted transition count is matched, this control does not exchange a larger model-query budget for better quality. It changes only whether trusted feedback terms are organized as independent one-step updates or as temporally connected returns. The LiH contrasts therefore identify multi-step credit assignment as the relevant mechanism; the saturated BeH2-8q task does not resolve horizon quality at its floor. Appendix F Uncertainty and Verification F.1 Risk–Coverage Construction and AURC The risk–coverage sample contains candidates selected by predicted value (selection tag top or both); candidates admitted solely by disagreement are excluded. Ensemble disagreement therefore does not determine membership in the analyzed sample. For each seed and rejection rule, candidates are sorted by the rule, retained at coverage levels from 0.100.10 to 1.001.00, and risk is the retained mean absolute prediction error in frontier-score units. AURC is the normalized trapezoidal area under the resulting curve. Task n/seed Disagreement Value Inverse value Random LiH-4q 935 0.2927±0.03270.2927± 0.0327 0.3009±0.03940.3009± 0.0394 0.4312±0.02510.4312± 0.0251 0.3686±0.02050.3686± 0.0205 LiH-6q 1870 0.2900±0.14070.2900± 0.1407 0.5264±0.23590.5264± 0.2359 0.2832±0.10550.2832± 0.1055 0.3720±0.14620.3720± 0.1462 BeH2-8q 935 0.1301±0.13220.1301± 0.1322 0.2060±0.04470.2060± 0.0447 0.3157±0.26800.3157± 0.2680 0.2332±0.14850.2332± 0.1485 Table 22: Per-seed AURC (mean ± sample std over five seeds; lower is better). Bold marks the lowest mean rule for each task. Inverse value is a diagnostic control rather than a deployable uncertainty score. Task Contrast AURC 95% CI Better seeds LiH-4q disagreement −- value −0.0081-0.0081 [−0.0202,+0.0034][-0.0202,+0.0034] 4/5 LiH-4q disagreement −- random −0.0759-0.0759 [−0.0875,−0.0642][-0.0875,-0.0642] 5/5 LiH-6q disagreement −- value −0.2364-0.2364 [−0.3177,−0.1552][-0.3177,-0.1552] 5/5 LiH-6q disagreement −- random −0.0821-0.0821 [−0.0991,−0.0586][-0.0991,-0.0586] 5/5 BeH2-8q disagreement −- value −0.0759-0.0759 [−0.1352,+0.0126][-0.1352,+0.0126] 4/5 BeH2-8q disagreement −- random −0.1031-0.1031 [−0.1180,−0.0902][-0.1180,-0.0902] 5/5 Table 23: Paired AURC contrasts using 10,000 percentile-bootstrap resamples over five seeds. Negative values favor ensemble disagreement. Bold marks intervals that exclude zero. At 50%50\% coverage, disagreement changes risk from 0.36170.3617 to 0.27600.2760 on LiH-4q (−24%-24\%), from 0.37450.3745 to 0.27590.2759 on LiH-6q (−26%-26\%), and from 0.23570.2357 to 0.12700.1270 on BeH2-8q (−46%-46\%). On LiH-6q, keeping the predicted-best half instead increases risk to 0.50110.5011 (+34%+34\%), while keeping the predicted-worst half lowers it to 0.24800.2480 (−34%-34\%). Restricting this analysis to value-selected candidates prevents the evaluation set from being defined by the same disagreement score used to rank risk. AURC then measures whether disagreement orders absolute prediction error better than the controls; it does not require disagreement to be a calibrated error probability. This ordering property is exactly what the deployed controls use: pessimism consumes disagreement continuously, while truncation rejects the upper tail of uncertain imagined steps. F.2 Threshold and Verification Behavior At the deployed threshold σ≤0.60σ≤ 0.60, the value-selected and disagreement-selected strata retain 98%/79%98\%/79\% of candidates on LiH-4q, 99%/99%99\%/99\% on LiH-6q, and 76%/34%76\%/34\% on BeH2-8q. Thus hard truncation is most active on BeH2-8q, while uncertainty remains active on every task through the continuous pessimistic potential. Verified stratum LiH-4q BeH2-6q LiH-6q BeH2-8q BeH2-10q All +0.011+0.011 −0.015-0.015 −0.089-0.089 −0.477-0.477 +0.137+0.137 Value-selected −0.029-0.029 −0.014-0.014 −0.108-0.108 −0.485-0.485 −0.003-0.003 Disagreement-selected +0.052+0.052 −0.017-0.017 −0.062-0.062 −0.392-0.392 +0.277+0.277 Table 24: Late-budget Full-minus-NoDAG change in mean absolute calibration error per verification event. Negative values indicate lower error when verified samples are returned to replay. Bold marks cells whose paired bootstrap interval excludes zero in the improving direction. Complete intervals are supplied in the artifact. The clearest correction effect appears on BeH2-8q, including both the value-selected and disagreement-selected strata. Other tasks show mixed population changes, consistent with treating selective verification as a targeted feedback route rather than the sole source of the final-quality gain. Verification complements rather than duplicates the risk gate. The value-selected stratum checks regions likely to be exploited, while the disagreement-selected stratum probes regions the ensemble identifies as blind spots. Their real VQE outcomes are returned as ordinary raw-energy trajectories, so any correction enters later world-model updates through the same frontier versioning and replay pipeline as on-policy experience. Appendix G Component Ablations The oracle-free component campaign evaluates five arms at the same 15,000-episode checkpoint. Each cell contains five training seeds and uses the same mean frozen-policy energy error as the main table. Arm LiH-4q BeH2-6q LiH-6q BeH2-8q BeH2-10q Full 0.053±0.0240.053± 0.024 0.0580.058 11.4±0.4711.4± 0.47 2.172.17 1.03±0.291.03± 0.29 −-DAgger 0.073±0.0360.073± 0.036 0.0580.058 11.4±0.7011.4± 0.70 2.53±0.492.53± 0.49 1.03±0.291.03± 0.29 −-DIR 0.125±0.0700.125± 0.070 0.0580.058 11.1±0.6311.1± 0.63 2.172.17 1.03±0.291.03± 0.29 −-uncertainty 0.053±0.0480.053± 0.048 0.0580.058 11.2±1.011.2± 1.0 2.53±0.492.53± 0.49 2.24±1.62.24± 1.6 −-imagination† 0.221±0.0620.221± 0.062 0.328±0.120.328± 0.12 13.6±3.513.6± 3.5 2.35±0.402.35± 0.40 2.18±0.912.18± 0.91 Table 25: Oracle-free component ablation: mean frozen-policy energy error in mHa at 15,000 episodes. Bold marks the best mean on the non-saturated tasks. The BeH2-6q and BeH2-8q columns reach search-space floors and are not used to rank components. †Removing imagination also disables DAgger because verification is tied to surrogate imagination. Removed component Task 95% CI Interpretation DIR LiH-4q +0.072+0.072 [+0.006,+0.152][+0.006,+0.152] improves quality DIR LiH-6q −0.380-0.380 [−0.606,−0.105][-0.606,-0.105] task-dependent reversal Uncertainty BeH2-10q +1.213+1.213 [+0.129,+2.738][+0.129,+2.738] improves quality Imagination† LiH-4q +0.168+0.168 [+0.124,+0.209][+0.124,+0.209] improves quality Imagination† BeH2-6q +0.271+0.271 [+0.175,+0.366][+0.175,+0.366] saturated final cell Imagination† BeH2-10q +1.158+1.158 [+0.515,+1.673][+0.515,+1.673] improves quality Table 26: Component-removal contrasts whose paired interval excludes zero. Δ= = ablated arm minus Full, so positive values indicate that removal increases error. The DIR result is heterogeneous across tasks. Imagination has the broadest final-quality contribution and also accounts for the largest fine-error VQE-efficiency gains. Uncertainty has a clear non-saturated effect on BeH2-10q. DIR improves LiH-4q but reverses direction on LiH-6q, showing that label-density correction interacts with the task’s feedback distribution. The −-uncertainty arm jointly removes pessimism and hard truncation, so these two uses are not causally separated by the present campaign. Similarly, the ranking gate is part of the deployed system but has no independent zero-threshold arm. The component results are most informative when combined with the targeted mechanism experiments. The same-model deployment and matched-horizon controls isolate multi-step use of feedback more directly than the −-imagination arm, which also removes verification feedback. The uncertainty contrast on BeH2-10q aligns with the independent risk–coverage evidence that disagreement carries useful error-ordering information. DIR has the expected benefit on LiH-4q but reverses on LiH-6q, showing that redistributing supervised capacity is task-dependent even when the target itself is unchanged. Saturated columns demonstrate that the system reaches the available floor, but they do not supply resolution for ranking components at convergence. Appendix H Reproducibility and Artifact Inventory H.1 Hardware and Software Environment All DreamQAS and baseline experiments ran on AWS EC2 g6.24xlarge instances using the shared crlqas_env Conda environment. Each run was assigned one GPU and used no cross-GPU communication. Tables 27 and 28 record the execution environment; OpenFermion and PySCF were used only for offline molecule generation or validation and were not part of the training loop. Item Configuration Instance AWS EC2 g6.24xlarge CPU AMD EPYC 7R13; 48 physical cores, two hardware threads per core (9696 vCPUs), one NUMA node Memory 363 GiB GPU 4×4× NVIDIA L4, 23,034 MiB each; compute capability 8.9 (Ada Lovelace) GPU interconnect PCIe Gen4 ×16× 16, without NVLink; one GPU per run and no cross-GPU communication Driver NVIDIA 580.126.16; driver-reported CUDA 13.0 Storage 123 GiB root EBS and 3.4 TiB local ephemeral NVMe used for run outputs Table 27: Hardware used for DreamQAS and the five baseline campaigns. Component Version or configuration Operating system Ubuntu 24.04.4 LTS; Linux kernel 6.17.0-1007-aws Runtime Python 3.10.20; shared Conda environment crlqas_env PyTorch 2.10.0+cu128; CUDA runtime 12.8; cuDNN 9.10.2 NumPy / SciPy NumPy 2.2.6; SciPy 1.15.2 Qulacs 0.6.13 GPU build; QuantumStateGpu available; no separate qulacs-gpu wheel installed Baseline utilities PennyLane 0.42.3; tqdm 4.67.3; Matplotlib 3.10.8; Weights & Biases 0.25.1 (disabled at runtime with --use-wandb 0) Offline chemistry OpenFermion 1.7.1 and PySCF 2.13.1, used only for molecular generation and validation Table 28: Software environment. DreamQAS itself imports PyTorch, NumPy, SciPy, and Qulacs; the remaining packages support baselines, logging, or offline molecular preparation. H.2 Training Schedule and Randomness Every main arm uses five training seeds, indexed 0–4. An iteration contains four real episodes, a real actor update, optional imagination, and periodic world-model refresh and verification. The common reported checkpoint is 15,000 episodes. Frozen evaluation uses separate evaluation seeds and 100 episodes per training seed. Randomness enters actor initialization and sampling, ensemble initialization, per-member bootstrap masks, replay sampling, imagined-prefix selection, and frozen evaluation. Ranking-gate pairs use a fixed random generator. The risk–coverage random baseline uses a molecule-specific CRC32 seed so its value does not depend on task iteration order. H.3 Logged Artifacts and Data Lineage Artifact Principal fields Paper use metrics.jsonl iteration, VQE counters, training errors, model loss, ranking gate, frontier version, timing fields efficiency curves, cost, frontier lag, timing trajectory.jsonl circuit step, action, raw energy, reward, frontier score, evaluation-only true error reward audit and depth diagnostics calibration.jsonl selection stratum, predicted and real scores, absolute error, disagreement, circuit length risk–coverage and verification analysis imag.jsonl imagined return, advantage, horizon, disagreement, seed count, transition budget horizon matching and model-query accounting fidelity.jsonl pairwise accuracy, gate state, episode count, model loss ranking-readiness diagnostics eval_traces*.jsonl episode-level minimum-prefix energies, evaluation count, checkpoint, cumulative training cost all frozen-policy energy error tables t1a_probe.json action correlation, regret, random comparison, horizon and advantage diagnostics counterfactual action utility Table 29: Run artifacts and their role in the paper. Every displayed number is derived from stored run outputs rather than reconstructed from plotted pixels. The final lineage audit resolves all 23 analysis chains to a training run, checkpoint, seed set, and reducer. The main oracle-free tables use the current t1a_probe.json action statistics, the order-independent risk baseline, the common 15,000-episode evaluation, and the trailing-window width W=100W=100. Canonical development campaigns are kept separate from the oracle-free main campaign in every table. H.4 Software and Hardware Boundary The 4- and 6-qubit tasks use a CPU complex128 Qulacs path; the 8- and 10-qubit tasks train with a GPU batched complex64 VQE path and are reevaluated using a complex128 reference implementation. Dependency versions, CPU/GPU model, driver version, and final repository commit should be exported from the frozen submission environment together with the code release. The wall-clock study in Appendix C is a same-load state-vector comparison and does not estimate queueing or execution time on a quantum processor.

Instant research discovery