Paper deep dive
Structure-aware Reinforcement Learning for Protein Directed Evolution
Zikun Nie, Suyuan Zhao, Yizhen Luo, Siqi Fan, Zaiqing Nie
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Protein optimization remains a longstanding goal in life sciences. Existing machine learning-assisted directed evolution (MLDE) methods primarily rely on sequence-only features, overlooking the critical spatial constraints and co-evolutionary interactions encoded in protein structures. However, directly integrating structural information remains challenging due to the scarcity of reliable mutant structures. To address these issues, we propose StructEvo, a novel structure-aware reinforcement learning framework for protein directed evolution. StructEvo employs a delta-structure fusion encoder to approximate mutant structure features via feature differences, enabling dynamic incorporation of spatial knowledge. The vast mutation space is then decomposed into manageable subspaces through a structure-aligned hierarchical action network, while a geometric constraint further stabilizes delta feature learning. Our approach outperforms prior state-of-the-art methods by 9.2% and 16.3% on two challenging optimization benchmarks, and further identifies an experimentally validated epistasis pattern in GFP, highlighting the importance of structural guidance for effective protein directed evolution.
Tags
Links
- Source: https://arxiv.org/abs/2609.39048v1
- Canonical: https://arxiv.org/abs/2609.39048v1
Trouble viewing inline? Open PDF directly →
Full Text
115,626 characters extracted from source content.
Expand or collapse full text
Structure-aware Reinforcement Learning for Protein Directed Evolution Zikun Nie Affiliation: Institute of AI Industry Research (AIR), Tsinghua University Affiliation: Department of Computer Science and Technology, Tsinghua University Email: nzk24@mails.tsinghua.edu.cn Suyuan Zhao Affiliation: Institute of AI Industry Research (AIR), Tsinghua University Affiliation: Department of Computer Science and Technology, Tsinghua University Email: zhaosy23@mails.tsinghua.edu.cn Yizhen Luo Affiliation: Institute of AI Industry Research (AIR), Tsinghua University Affiliation: Department of Computer Science and Technology, Tsinghua University Email: yz-luo22@mails.tsinghua.edu.cn Siqi Fan Affiliation: Institute of AI Industry Research (AIR), Tsinghua University Email: zaiqing@air.tsinghua.edu.cn Zaiqing Nie †thanks: Corresponding author Affiliation: Institute of AI Industry Research (AIR), Tsinghua University Affiliation: Pharmolix Inc. Abstract Protein optimization remains a longstanding goal in life sciences. Existing machine learning–assisted directed evolution (MLDE) methods primarily rely on sequence-only features, overlooking the critical spatial constraints and co-evolutionary interactions encoded in protein structures. However, directly integrating structural information remains challenging due to the scarcity of reliable mutant structures. To address these issues, we propose StructEvo, a novel structure-aware reinforcement learning framework for protein directed evolution. StructEvo employs a delta-structure fusion encoder to approximate mutant structure features via feature differences, enabling dynamic incorporation of spatial knowledge. The vast mutation space is then decomposed into manageable subspaces through a structure-aligned hierarchical action network, while a geometric constraint further stabilizes delta feature learning. Our approach outperforms prior state-of-the-art methods by 9.2% and 16.3% on two challenging optimization benchmarks, and further identifies an experimentally validated epistasis pattern in GFP, highlighting the importance of structural guidance for effective protein directed evolution. 1 Introduction Proteins are fundamental functional units of biological systems. Compared to de novo protein design, optimizing existing proteins through mutations to improve desired functional properties (i.e., fitness) has greater practical value for real-world applications [1], ranging from antibody affinity maturation [2, 3] to enzyme catalytic engineering [4, 5]. In practice, the most widely adopted approach for protein optimization is directed evolution [6, 7, 8], which iteratively modifies proteins by random mutagenesis, expression, screening and selection. This process relies on large-scale in vivo expression and in vitro screening, making it costly and time-consuming [9]. Figure 1: (a) Illustration of the active learning pipeline for directed evolution. Starting from an initial pool, candidates are evaluated by an oracle and then update the pool, after which a proxy is iteratively refined to guide candidate proposals. The figure highlights two key challenges for the proposal policy: how to model the fine-grained local fitness landscape, and how to effectively leverage structural knowledge. (b) Performance comparison on GFP-hard benchmark. StructEvo achieves the highest fitness while maintaining comparable diversity to other high-fitness baselines. Since directed evolution operates on amino acid sequences, existing machine learning-assisted directed evolution (MLDE) approaches [10, 11, 12, 13, 14, 15, 16] are predominantly sequence-oriented, focusing on modeling the rugged fitness landscape that maps protein sequences to fitness values. Early methods [14, 15] represent protein sequences by one-hot encodings or hand-crafted features, while recent approaches [13, 16] leverage protein language models (PLMs) to capture evolutionary priors. However, these sequence-only approaches often overlook the intrinsic spatial context in protein structures, limiting their ability to capture functionally relevant interactions. By explicitly encoding physical and geometric constraints, three-dimensional structures provide rich co-evolutionary interaction signals such as spatial proximity and functional motifs, which are essential for protein activity [17]. Motivated by this, we integrate protein structure as a complementary modality to sequence, facilitating more fine-grained geometric guidance for navigating local fitness landscapes. However, leveraging structural knowledge presents two key challenges. (1) Reliable mutant structures are generally unavailable. Though a representative experimentally resolved structure may be available for a protein family, mutant structures remain scarce in databases. Meanwhile, existing structure prediction models, such as the AlphaFold [18, 19, 20, 21] and Protenix series [22, 23, 24], are computationally costly [25, 26] and often insensitive to point mutations, frequently producing nearly identical structures for similar sequences [27, 28]. Consequently, it remains impractical to directly capture structural variations at coordinate level, especially in directed evolution settings where even a few selected mutations may induce noticeable backbone deviations. To address this, we model structural changes at the feature level by converting differences in PLM representations, which have been shown to contain rich mutational effects [29]. In this way, we transform the static structure into dynamic guidance during optimization. (2) It remains unclear how to effectively translate structural signals into mutation decisions. Jointly optimizing mutation positions and substituted amino acids results in a large action space, making it difficult to learn reliable structural guidance. A key observation is that protein structure is more informative for where to mutate than for mutate to what, as it provides position-specific spatial context such as residue interactions in three-dimensional space. Motivated by this insight, we decompose mutation decisions into two sequential processes: position selection and amino acid refinement. This design enables structure to guide position decisions, while providing cleaner gradient signals in two low-dimensional subspaces. Based on these insights, we propose StructEvo, a structure-aware reinforcement learning framework for protein directed evolution. StructEvo introduces a novel paradigm for incorporating dynamic structural knowledge into mutation proposal policy, consisting of three components: (1) a delta-structure fusion encoder that approximates mutant structure features via delta representations and integrates sequence and structure information via a cross-attention module; (2) a structure-aligned hierarchical action network that decomposes action space and enables structure-guided mutation sampling; and (3) a geometric constraint applied on delta representations to improve stability. We validate our work through comprehensive experiments on two combinatorial mutation benchmarks and two more challenging full-length benchmarks. Across all benchmarks, StructEvo consistently outperforms strong baselines, and achieves relative fitness gains of up to 9.2% on AAV-hard and 16.3% on GFP-hard (Figure 1b), demonstrating its effectiveness for protein directed evolution. We further present an independently discovered triple-mutation epistasis pattern in GFP validated by existing wet-lab experiments, highlighting the potential of our method for real-world applications. Our code and model are open-sourced at https://github.com/Skyyyyyalker/StructEvo. Our contributions are summarized as follows: • We propose StructEvo, a structure-aware reinforcement learning framework that introduces a novel paradigm for incorporating structural knowledge into mutation proposal policy. • We introduce a delta-structure fusion encoder, a structure-aligned hierarchical action network and a geometric constraint to enable effective structural guidance. • We achieve relative fitness gains of 9.2% and 16.3% over the prior strongest baselines on two hard benchmarks, and identify a validated triple-mutation pattern in GFP, highlighting the importance of structural guidance for protein directed evolution. 2 Related work Figure 2: Architecture of the StructEvo framework. A delta-structure fusion encoder approximates mutant structure features via delta representations and integrates sequence and structure information. A structure-aligned hierarchical action network then decomposes action space and enables structure-guided mutation sampling. A geometric constraint loss is calculated to improve robustness. The proxy model remains frozen and provides reward signals during policy updates. A.A.: amino acid. Machine learning–assisted directed evolution (MLDE). Evolutionary search-based methods simulate natural selection through iterative mutation and recombination [11], with extensions that encourage proximal exploration via distance penalties [12]. Energy-based methods learn smooth fitness landscapes as energy models to guide sampling through gradient ascent [30, 31, 32]. Generative model-based methods explicitly model the mutant distributions using variational autoencoders [10, 33] or generative flow networks [34], with recent work further introducing controllable generation mechanisms [35, 36]. While previous methods primarily rely on sequence-only features, we incorporate structural knowledge as complementary guidance for high-fitness mutant proposal. Reinforcement learning (RL) for protein optimization. Recent studies increasingly formulate protein optimization as a sequential decision-making problem and leverage RL to guide mutation policies [37]. Wang et al. [38] generate new sequences through self-play and Monte Carlo tree search, while Wang et al. [16] draw physicochemical priors from amino acid knowledge graph to guide mutation decisions. Lee et al. [13] further optimize mutants in a continuous latent space by treating mutations as small perturbations. Unlike prior methods, our approach introduces delta representations to capture dynamic structural variation within RL policy, enabling fine-grained optimization. Multi-modal protein fitness prediction. Fitness landscape modeling focuses on predicting functional fitness values for given protein variants [39, 40]. Beyond sequence representations, prior work incorporate wild-type structure utilizing Graph Neural Network [41, 42] or inverse-folding models [43, 44, 45]. Recent studies further explore sequence-structure fusion by joint vocabulary [46] or structure quantization module [47]. Additional modalities such as surface topology and multiple sequence alignment (MSA) are also leveraged [48, 49]. Compared to static fitness prediction, our work focuses on the more challenging task of dynamic protein evolution in an active learning setting. 3 Task formulations 3.1 Directed evolution in an active learning setting In both in vitro and in silico settings, the directed evolution is often conducted in an active learning setting [38], where the mutants start from an initial pool and are then optimized iteratively over a total of R rounds. Within each round, the optimization proceeds through four key components (Figure 1a): Oracle evaluation. Let V denote the vocabulary of the 20 standard amino acids. A protein sequence of length L is defined as X=(x1,…,xL)∈=LX=(x^1,...,x^L) =V^L, where each residue xi∈x^i . A black-box oracle ℱ:→ℝF:X is a function that maps a protein sequence X to a scalar fitness value ℱ(X)F(X), thereby defining the fitness landscape. In MLDE, the oracle corresponds to costly wet-lab experiments for evaluating candidate sequences. Consequently, the number of oracle queries is strictly limited to N per round, where N≪||N |X|. Candidate pool update. A candidate pool D is iteratively updated to collect high-fitness mutants. The initial pool 0=(Xn,ℱ(Xn))n=1|0|D_0=\(X_n,F(X_n))\_n=1^|D_0| contains only a small set of low-fitness candidates. At the beginning of each round i, newly proposed mutants are evaluated and incorporated into the pool: i=i−1∪(Xn,ℱ(Xn))n=1N.D_i=D_i-1∪\(X_n,F(X_n))\_n=1^N. (1) Proxy model update. Due to the limited number of oracle queries, a trainable proxy model fϕ(⋅)f_φ(·), parameterized by ϕφ, is used to approximate the landscape defined by the oracle. After updating the candidate pool, the proxy is iteratively fine-tuned on iD_i by minimizing the mean squared error loss: ℒ(ϕ)=1|i|∑X∈i(fϕ(X)−ℱ(X))2.L(φ)= 1|D_i| _X _i (f_φ(X)-F(X) )^2. (2) Proposal policy. Guided by the fine-tuned proxy model fϕf_φ and starting from the current candidate pool iD_i, a proposal policy generates a batch of N novel mutants at each round. These mutants are then evaluated by the oracle and subsequently update the candidate pool for the next round. The main objective of directed evolution is to identify a batch of K high-fitness protein mutants under a fixed budget of R rounds. A summary of all symbols is provided in Appendix A.1 for reference. 3.2 Reinforcement learning-based directed evolution We formulate protein optimization as a reinforcement learning (RL) problem. By modeling the multi-point mutations as a Markov Decision Process (MDP), RL enables learning a policy that sequentially selects mutations to achieve high fitness. State. At step t, the state st=(xt1,…,xtL)s_t=(x_t^1,...,x_t^L) is a protein mutant with L residues, where xti∈x_t^i . Action. An action at=(atpos,attype)a_t=(a_t^pos,a_t^type) at step t represents a single-point mutation applied to the current state sts_t. The action ata_t can be factorized into two parts: the mutation position atposa_t^pos and the type of substituted amino acid attypea_t^type. In our task, the transition function P(st+1∣st,at)P(s_t+1 s_t,a_t) is deterministic and the state is updated by: xt+1i=attype,i=atpos,xti,otherwise.x_t+1^i= casesa_t^type,&i=a_t^pos,\\ x_t^i,&otherwise. cases Reward. Reward signals are provided by the proxy model. In our work, we adopt a sparse reward setting, where the reward rt=r(st,at)r_t=r(s_t,a_t) is assigned only at trajectory termination. A trajectory terminates either when the policy finds a mutant better than the starting one, or when a predefined maximum number of steps mstepm_step is reached. Specifically, for a trajectory ending at step T, the reward is rT=fϕ(sT+1)r_T=f_φ(s_T+1), while intermediate rewards are set to rt=0r_t=0 for t<Tt<T. We emphasize that rewards are generated solely by the proxy model, and the oracle is used only for final evaluation. Training Objective. We aim to learn a policy π(at∣st)π(a_t s_t) that maximizes the expected return over trajectories. Under the sparse reward setting, the training objective is formulated as: maxπτ∼π[∑t=0Tγtrt]=maxπτ∼π[γTfϕ(sT+1)], _π\;E_τ π [ _t=0^Tγ^tr_t ]= _π\;E_τ π [γ^Tf_φ(s_T+1) ], (3) where τ=(s0,a0,…,sT,aT)τ=(s_0,a_0,…,s_T,a_T) denotes a mutation trajectory terminating at step T, and γ is the discount factor. We optimize the policy using Proximal Policy Optimization (PPO; 50), which has been proved effective in protein optimization tasks [13, 16]. The collected candidates are then ranked by proxy-predicted fitness and evaluated by the oracle for subsequent refinement in the next round. Within the reinforcement learning framework, our work primarily focuses on improving the key component—the mutation proposal policy, as described in the following section. 4 Methodology The main goal of our work is to incorporate structural knowledge into protein mutation policy that maps states to actions. In this section, we present the proposed StructEvo framework, highlighting three main components: (1) a delta-structure fusion encoder that fuses sequential and structural features for a given mutant (Sec. 4.1), (2) a structure-aligned hierarchical action network that receives the fused feature and samples high-fitness mutations (Sec. 4.2), and (3) a geometric constraint between delta features to improve robustness and stability (Sec. 4.3). 4.1 Delta-structure fusion encoder The delta-structure fusion encoder integrates both sequence and structural knowledge to produce a fine-grained representation of protein mutants. As illustrated in Figure 2, it consists of a protein sequence encoder ℰseqE_seq, a protein structure encoder ℰstrE_str, a geometric projector fprojf_proj and a cross-attention module. We adopt ESM-2 [51], a protein language model (PLM) pretrained on large-scale evolutionary sequence data, to initialize the sequence encoder and capture mutant sequence features. For structure encoding, we employ ESM-IF [43] due to its simplicity and efficiency, as it captures both topology and geometric constraints of protein structures. Notably, the StructEvo framework is model-agnostic and easily extensible to different protein encoders. Formulation of delta sequence feature. Since reliable mutant structures are generally unavailable, we adopt a reference structure SrefS^ref with the highest reliability as a structural scaffold for the corresponding mutant family (see Appendix D.1), with its sequence denoted as the reference sequence XrefX^ref. Prior work [29] has shown that differences between PLM representations capture rich information of mutational effect. Therefore, we introduce the delta sequence feature Δhseq h_seq as: Δhseq=hseq−hseqref=ℰseq(X)−ℰseq(Xref). h_seq=h_seq-h_seq^ref=E_seq(X)-E_seq(X^ref). (4) X,XrefX,X^ref denote the sequences of the current and the reference proteins, and hseq,hseqrefh_seq,h_seq^ref are the corresponding sequence representations, where dseqd_seq denotes its dimensionality. Projection into structural manifold. We speculate that the mutational knowledge contained in the delta sequence feature reflects corresponding structural changes. Accordingly, we introduce a projector fproj:ℝL×dseq→ℝL×dstrf_proj:R^L× d_seq ^L× d_str to map the delta sequence feature from the sequence space to the structural manifold, yielding the delta structure feature Δhstr h_str. Both the sequence and structural representations are aligned to residue-level embeddings of the protein backbone, allowing an addition operator to get approximate mutant structure features h^str h_str. These procedures are formulated as: Δhstr=fproj(Δhseq),h^str=hstrref+Δhstr=ℰstr(Sref)+Δhstr. h_str=f_proj( h_seq), h_str=h^ref_str+ h_str=E_str(S^ref)+ h_str. (5) Here, Sref∈ℝL×3×3S^ref ^L× 3× 3 is the backbone coordinates of the reference structure, where each residue is represented by the three-dimensional positions of its N, Cα, and C atoms. To ensure stability, we further impose a geometric constraint between delta features, as described below in Sec. 4.3. Structure-conditioned feature fusion. To integrate sequence and structure information, we employ a cross-attention module where the mutant sequence feature hseqh_seq serves as the query, and the approximated structure feature h^str h_str serves as the key and value. The asymmetric design allows sequence features to selectively query and attend to structural context. The resulting fusion feature hfusionh_fusion encodes both sequential and structural information of the mutant, serving as input to the downstream hierarchical action network for mutation sampling. 4.2 Structure-aligned hierarchical action network In RL formulation, the Markov decision process decomposes the vast mutation space into a sequence of single-step decision problems. Each action corresponds to a single-point mutation a=(apos,atype)a=(a^pos,a^type), where the mutation position apos∈1,…,La^pos∈\1,...,L\ and the substituted amino acid atype∈a^type , with L as the protein length and V as the standard amino acid vocabulary. As depicted in Figure 2, the decision process is decomposed into two hierarchical steps: Mutation position sampling. Since mutation position is more tightly coupled with the structural context than amino acid type, prioritizing position selection aposa^pos enables the policy to better exploit structure-derived insights. A position action network πθ1 _ _1 receives the fusion feature hfusionh_fusion as input and outputs a categorical distribution over mutation position subspace, from which the position action apos∼πθ1(⋅∣hfusion)a^pos _ _1(· h_fusion) is sampled. Substituted amino acid sampling. Once the position aposa^pos is determined, the corresponding local representation at the specific position is then extracted and detached from the fusion feature. An amino acid type action network πθ2 _ _2 then operates on the detached context to sample the substituted amino acid atype∼πθ2(⋅∣hfusion,apos)a^type _ _2(· h_fusion,a^pos). Here, the detach operation blocks gradients backpropagating into the shared fusion encoder and the position network. This allows πθ1 _ _1 to be optimized solely based on position selection, resulting in a cleaner learning signal (see Appendix A.3). 4.3 Geometric constraint for delta features Geometric constraint. In the delta-structure fusion encoder, the delta sequence feature is projected to obtain a corresponding delta structure feature. We expect the consistency between structure variations and sequence perturbations: small mutational effects captured by PLM should not lead to excessively large structural deviations, and vice versa. To encourage this, we introduce a per-residue geometric constraint between the two delta features, formulated as: ℒgeo=Δhstr,Δhseq[1L∑i=1L(‖Δhstri‖dstr−c∗‖Δhseqi‖dseq)2],L_geo=E_ h_str, h_seq [ 1LΣ^L_i=1 ( \| h^i_str\| d_str-c* \| h^i_seq\| d_seq )^2~ ], (6) where c is the scaling factor, and ∥⋅∥\|·\| denotes the ℓ2 _2-norm. This constraint encourages a more stable and faithful mapping to preserve variation magnitude. From another perspective, the geometric constraint can be viewed as a stronger form of Lipschitz continuity [52] in metric space analysis, as detailed in Appendix A.4. Overall training objective. The overall RL training objective consists of two parts: ℒtotal=ℒpolicy+λentropy⋅ℒentropy+λvalue⋅ℒvalue⏟standard PPO loss+λgeo⋅ℒgeo⏟geometric constraint.L_total= L_policy+ _entropy·L_entropy+ _value·L_value_standard PPO loss+ _geo·L_geo_geometric constraint. (7) This objective jointly optimizes the mutation policy while enforcing consistency between delta sequence and structure features. Algorithm 1 summarizes the overall mutation proposal pipeline. 5 Experiments In this section, we demonstrate that StructEvo is effective on directed evolution tasks across two combinatorial benchmarks (Sec. 5.1) and two more challenging full-length mutation benchmarks (Sec. 5.2). We further provide an in-depth analysis of our model (Sec. 5.3) and a case study that reveals high-fitness mutation patterns at green fluorescent protein (Sec. 5.4). 5.1 Optimization on combinatorial benchmarks Table 1: Results on GB1 and PhoQ benchmarks. We report average results and std across 5 independent runs. ∗: results are derived from Wang et al. [16]. n.a.: not reported in the original paper. GB1 PhoQ Method Max↑ Mean↑ NDCG↑ Max↑ Mean↑ NDCG↑ CMAES 0.59± 0.17 0.13± 0.04 0.75± 0.02 0.29± 0.14 0.05± 0.01 0.69± 0.02 BO 0.67± 0.10 0.17± 0.02 0.77± 0.01 0.29± 0.06 0.05± 0.01 0.70± 0.01 AdaLead 0.64± 0.13 0.30± 0.09 0.74± 0.02 0.27± 0.11 0.14± 0.02 0.67± 0.02 PEX 0.66± 0.03 0.36± 0.02 0.76± 0.01 0.40± 0.03 0.10± 0.02 0.71± 0.02 MLDE∗ 0.68± .a. 0.20± .a. 0.79± .a. 0.36± .a. 0.10± .a. 0.79± .a. ftMLDE (EVmutation)∗ 0.94± .a. 0.41± .a. 0.83± .a. 0.44± .a. 0.11± .a. 0.80± .a. ftMLDE (Transformer)∗ 0.93± .a. 0.42± .a. 0.81± .a. 0.42± .a. 0.11± .a. 0.82± .a. EvoPlay∗ 0.84± .a. 0.46± .a. 0.85± .a. 0.47± .a. 0.14± .a. 0.80± .a. CLADE∗ 0.84± .a. 0.46± .a. 0.86± .a. 0.47± .a. 0.09± .a. 0.78± .a. CLADE2.0∗ 0.94± .a. 0.49± .a. 0.88± .a. 0.40± .a. 0.15± .a. 0.81± .a. KnowRLM∗ 0.97± 0.06 0.56± 0.02 0.88± 0.02 0.66± .a. 0.16± .a. 0.82± .a. StructEvo (ours) 0.99± 0.01 0.59± 0.01 0.93± 0.01 0.64± 0.07 0.19± 0.02 0.86± 0.01 Datasets. We first evaluate StructEvo on two widely recognized four-site combinatorial datasets, GB1 and PhoQ. The GB1 dataset [53] consists of 149,361 variants at four epistatic sites (V39, D40, G41 and V54) of the human protein GB1 domain, with fitness defined as binding affinity to the antibody IgG-Fc. Only 2.4% variants have higher fitness than the wild-type protein, highlighting the sparsity of the GB1 fitness landscape. The PhoQ dataset [54] contains 140,517 variants of the PhoQ histidine kinase mutated at four key interfacial residues (A284, V285, S288 and T289), with fitness reflecting both kinase and phosphatase responsiveness of PhoP/PhoQ regulatory system on signal transduction. Compared to GB1, the PhoQ landscape is even more rugged—98.7% mutants with fitness below 0.1 and 63.7% with zero, making it a particularly challenging task. All fitness values are min-max normalized to [0, 1]. Details of benchmarks are provided in Appendix C.1. Baselines. We compare our method with various strong baselines, including (1) probability-based methods, BO [55] and CMA-ES [56]; (2) evolutionary search-based methods, AdaLead [11], PEX [12], MLDE [53] and two variants of ftMLDE [57]; (3) cluster-based methods, CLADE [14] and its advanced version CLADE2.0 [15], and (4) recent RL-based methods, EvoPlay [38] and KnowRLM [16]. More details of baselines can be found in Appendix C.3. Metrics. We report the maximum and mean fitness over a combined set of candidates, including N initial mutants, N∗RN*R proposed mutants across R rounds, and the top N mutants ranked by the final proxy. Here, R denotes the total optimization rounds and N is the number of proposed candidates per round. Given that the ground truth fitness is fully available in combinatorial tasks, we further evaluate the ranking quality of proxy by normalized discounted cumulative gain (NDCG). Formal definitions of metrics are provided in Appendix C.5. Experimental settings. Following prior work [16], we set R=3R=3 and N=96N=96. The starting N mutants are sampled by CLADE [14]. The oracle is the ground-truth dataset, with rare missing instances assigned a value of 0. All experiments are repeated over 5 independent runs. Results and analysis. We present performance comparisons at round 3 in Table 1, with more detailed results of each round in Appendix E.1. We observe that: (1) StructEvo achieves state-of-the-art performance on GB1, outperforming the strongest RL-based baseline (KnowRLM) by relative gains of 5.7% on NDCG, demonstrating its strong capability in navigating fine-grained fitness landscapes and guiding mutant proposals. (2) Though the reference structure of PhoQ is predicted by AlphaFold2 [18] (with an average pLDDT of 82.94, see Appendix D.1) rather than experimentally resolved, StructEvo still attains a comparable performance with KnowRLM on PhoQ benchmark, with relative gains of 3.7% on NDCG and 18.8% on mean fitness. This observation shows the robustness of our framework, while also indicating the importance of accurate structural information for landscape exploration. (3) Probabilistic and evolutionary methods perform poorly under extremely scarce data. The key strength of PEX, proximal preference, becomes less significant in the constrained four-site space. Cluster-based methods leverage prior knowledge to reduce aimless exploration, while RL-based methods further improve performance by explicitly modeling the sequential decision-making process. Hence, we emphasize the significance of combining structural priors for effective protein optimization under sparse-data scenario. Table 2: Performance on AAV and GFP benchmarks. The results include average and standard deviation across 5 independent runs. ∗: results are derived from Kirjner et al. [30]. †: results are derived from Lee et al. [13]. -: not reported in the original paper. AAV medium AAV hard Method Mean↑ Max↑ Diversity Novelty Mean↑ Max↑ Diversity Novelty CMA-ES 0.05± 0.00 0.40± 0.01 20.9± 0.5 17.2± 0.5 0.05± 0.00 0.32± 0.03 20.4± 0.7 18.6± 0.5 BO 0.64± 0.03 0.72± 0.04 8.1± 0.2 9.4± 0.9 0.62± 0.03 0.70± 0.03 8.6± 0.2 10.3± 0.3 PEX 0.65± 0.01 0.74± 0.00 5.5± 0.5 5.3± 0.3 0.63± 0.02 0.75± 0.01 5.7± 0.5 6.2± 0.4 GGS∗ 0.51± 0.01 - 4.0± 0.2 5.4± 0.5 0.60± 0.02 - 4.5± 0.5 7.0± 0.0 AdaLead 0.74± 0.04 0.80± 0.05 5.2± 0.9 7.6± 0.4 0.76± 0.03 0.81± 0.03 4.0± 0.4 8.9± 0.6 VLGPO 0.69± 0.01 0.76± 0.01 3.1± 0.2 5.7± 0.1 0.71± 0.01 0.78± 0.01 3.3± 0.2 6.9± 0.1 LatProtRL† 0.71± 0.02 - 5.4± 0.8 6.0± 0.2 0.66± 0.01 - 6.0± 0.8 7.0± 0.6 GFN-δ-CS 0.77± 0.02 0.81± 0.02 4.4± 1.1 5.8± 1.5 0.76± 0.03 0.80± 0.03 4.0± 0.7 6.7± 1.1 StructEvo 0.82± 0.01 0.86± 0.01 4.5± 0.7 8.6± 0.2 0.83± 0.03 0.87± 0.04 4.5± 0.3 8.4± 0.5 GFP medium GFP hard Method Mean↑ Max↑ Diversity Novelty Mean↑ Max↑ Diversity Novelty CMA-ES -0.04± 0.01 0.58± 0.00 172.7± 2.1 154.3± 4.9 -0.03± 0.00 0.16± 0.03 173.8± 2.0 218.6± 7.6 BO 0.30± 0.07 0.42± 0.07 26.4± 5.6 37.7± 12.1 0.26± 0.07 0.39± 0.04 36.0± 2.7 71.7± 10.1 PEX 0.64± 0.04 0.84± 0.04 8.3± 2.6 6.8± 1.9 0.46± 0.05 0.62± 0.05 7.0± 1.2 10.3± 1.7 GGS∗ 0.76± 0.01 - 3.7± 0.2 5.0± 0.1 0.74± 0.00 - 3.6± 0.0 6.9± 0.1 AdaLead 0.93± 0.01 0.99± 0.02 5.2± 0.9 9.5± 1.2 0.75± 0.04 0.81± 0.04 5.4± 0.6 13.5± 1.1 VLGPO 0.93± 0.03 1.02± 0.01 2.8± 0.2 5.3± 0.0 0.79± 0.02 0.95± 0.00 1.9± 0.0 6.2± 0.0 LatProtRL† 0.93± 0.00 - 4.6± 0.4 5.5± 0.1 0.85± 0.01 - 4.8± 0.3 7.0± 0.4 GFN-δ-CS 1.06± 0.04 1.09± 0.03 7.0± 0.4 9.5± 1.9 0.86± 0.06 0.90± 0.05 4.3± 1.6 8.2± 1.5 StructEvo 1.11± 0.02 1.17± 0.02 6.3± 0.8 11.9± 0.8 1.00± 0.04 1.05± 0.05 4.9± 0.3 11.4± 0.6 Figure 3: Visualization of mean (top row) and maximum (bottom row) fitness across rounds. The curves indicate the average results, and the shaded regions indicate the standard deviation. 5.2 Optimization on full-length benchmarks Datasets. We further evaluate StructEvo on two full-length protein mutation benchmarks, GFP and AAV. Compared to combinatorial tasks, these are more challenging as mutations span the full sequence, leading to an exponentially larger search space. The GFP dataset [58] measures log-fluorescence intensity of 56,806 Aequorea victoria green fluorescent protein variants, with a mutation space size of ||=20237|X|=20^237. The AAV dataset [59] focuses on engineering a 28-amino acid segment of the VP1 capsid protein of adeno-associated virus, comprising 44,156 variants from a space of ||=2028|X|=20^28. Following Kirjner et al. [30], we partition both benchmarks into medium and hard regimes. The initial fitness distribution under the hard setting is more distant from the high-fitness range than the medium setting, leading to increased optimization difficulty. See more details in Appendix C.2. Baselines. In addition to baselines for combinatorial benchmarks introduced above, we further compare our work with energy model-based methods, GGS [30], as well as recent approaches that optimize in latent spaces, including LatProtRL [13], VLGPO [35] and GFN-δ-CS [36]. Implementation details are provided in Appendix C.4. Metrics. We report the mean and maximum fitness of the top K sequences proposed over R rounds. Fitness values are evaluated by the oracle and min-max normalized using the ground truth dataset. Notably, normalized fitness may exceed 1.0, as the oracle can assign higher fitness to previously unseen mutants than the maximum observed in the ground-truth dataset. We additionally choose distance-based metrics, diversity and novelty, as described in Appendix C.6. Experimental settings. Following prior work [30], we set the total rounds to R=15R=15, the number of candidates proposed per round to N=256N=256, and the evaluation budget to K=128K=128. The oracle models are derived from Kirjner et al. [30]. Each method is run with 5 independent seeds. Results and analysis. Table 2 and Figure 3 show comparisons between StructEvo and baselines on full-length benchmarks. We observe that: (1) StructEvo consistently achieves state-of-the-art fitness across all four scenarios, surpassing the strongest baseline by relative gains of 9.2% and 16.3% in mean fitness on the AAV and GFP hard settings, respectively. Furthermore, the larger improvements on hard settings compared to medium ones highlight the increasing importance of structural guidance when high-fitness data is severely limited. (2) While LatProtRL converges earlier, StructEvo continues to improve across rounds and ultimately surpasses it, indicating that structural information provides better guidance for exploring more fine-grained high-fitness regions. (3) We note that higher diversity and novelty do not simply imply better performance, as pointed out in previous work [13]. For instance, CMA-ES attains the highest diversity and novelty but yields dysfunctional mutants with near-zero fitness. In contrast, StructEvo achieves the highest fitness while maintaining comparable or higher diversity and novelty than other high-fitness methods, reflecting a more effective trade-off between exploration and exploitation. 5.3 Ablation studies Settings Inputs Mean↑ Max↑ Time(s)↓ X XrefX^ref SrefS^ref AAV hard StructEvo ✓ ✓ ✓ 0.83 0.87 0.337 (1) ✓ ✓ 0.80 0.84 0.323 (2) ✓ ✓ 0.75 0.79 0.297 (3) ✓ 0.75 0.81 0.303 (4) w/o hierarchy 0.77 0.81 0.583 GFP hard StructEvo ✓ ✓ ✓ 1.00 1.05 0.825 (1) ✓ ✓ 0.97 1.01 0.819 (2) ✓ ✓ 0.83 0.89 0.753 (3) ✓ 0.84 0.93 0.697 (4) w/o hierarchy 0.75 0.82 3.515 Table 3: Results of ablation studies. X,XrefX,X^ref: mutant and reference sequence, SrefS^ref: reference structure. w/o: without. Figure 4: A triple-mutation mutant in GFP. The reference (green) and the mutant (blue) structure are rendered by PyMOL. We examine the impact of each component in StructEvo by: (1) removing delta module by directly fusing mutant sequence with the static reference structure via cross-attention, (2) removing structure and fusing two sequence features, (3) relying solely on mutant sequence, and (4) replacing the hierarchical action network with a flat network that jointly samples both actions. We also report the runtime cost per candidate for each setting. As shown in Table 3 and Appendix E.2, removing delta module degrades performance, while removing structure leads to a larger drop (9.6% mean fitness on AAV-hard and 16.0% on GFP-hard). Substituting structure with reference sequence yields no improvements. Moreover, sampling in the joint space significantly degrades fitness and incurs much longer runtime, with both drawbacks amplified as protein length increases. These findings validate each component in our framework. We refer readers to additional in-depth analyses of structure prediction models, proxy robustness, and geometric constraint sensitivity in Appendix F. 5.4 Case study: a triple-mutation epistasis in GFP In GFP, we identify a triple-mutation pattern F64L/V163A/I171V (PDB: 8DPD; Figure 4). From 3,840 proposed mutants over 15 rounds, 271 mutants with this triple mutation achieve substantially higher fitness (average 0.91) than 1,834 single (0.07) and 427 double mutants (0.23), revealing a pronounced epistasis effect. Interestingly, this pattern has been validated and explained by wet-lab experiments [60, 61]: residue 64 is near the chromophore, while 163 and 171 are located in the proximal region of β-barrel; all mutations result in smaller side-chains, facilitating a synergistic tighter packing. This promotes energy dissipation via photon emission and ultimately boosts fluorescence intensity. We further find a higher cosine similarity of this true mutant structure feature to the approximated feature (0.85) than to the reference (0.73), supporting the validity of delta module. Notably, none of the baselines recover this pattern, highlighting the advantage of structure in capturing long-range co-evolutionary signals that are typically missed by sequence-only approaches. 6 Conclusions In this work, we propose StructEvo, a structure-aware reinforcement learning framework for protein directed evolution. By approximating mutant structure features via delta representations and decomposing the action space via a hierarchical action network, StructEvo introduces a new paradigm for incorporating structural knowledge into mutation proposal. Comprehensive experiments on both combinatorial and full-length benchmarks show consistent performance improvements over strong baselines, demonstrating the value of dynamic structural context for efficient protein optimization. Several limitations remain: (1) extending to protein complexes rather than monomers; (2) expanding single-object fitness to multi-objective properties; and (3) integrating feedback from wet-lab experiments, which we leave for future work. While StructEvo shows potential in real-world applications, we emphasize safety concerns that any proteins generated by our method should be complemented with rigorous experimental validation and careful human inspection before practical use. Acknowledgments This research is supported by the Innovative Drug Research and Development National Science and Technology Major Project (No.2025ZD1803101), the Wuxi Research Institute of Applied Technologies, Tsinghua University (Grant 20242001120), and PharMolix Inc. References [1] M. Soskine and D. S. Tawfik (2010) Mutational effects and the evolution of new protein functions. Nature Reviews Genetics 11 (8), p. 572–582. Cited by: §1. [2] H. Roost, M. F. Bachmann, A. Haag, U. Kalinke, V. Pliska, H. Hengartner, and R. M. Zinkernagel (1995) Early high-affinity neutralizing anti-viral igg responses without further overall improvements of affinity.. Proceedings of the National Academy of Sciences 92 (5), p. 1257–1261. Cited by: §1. [3] V. Manivel, N. C. Sahoo, D. M. Salunke, and K. V. Rao (2000) Maturation of an antibody response is governed by modulations in flexibility of the antigen-combining site. Immunity 13 (5), p. 611–620. Cited by: §1. [4] D. W. Anderson, F. Baier, G. Yang, and N. Tokuriki (2021) The adaptive landscape of a metallo-enzyme is shaped by environment-dependent epistasis. Nature Communications 12 (1), p. 3867. Cited by: §1. [5] R. J. Fox and G. W. Huisman (2008) Enzyme optimization: moving from blind evolution to statistical exploration of sequence–function space. Trends in biotechnology 26 (3), p. 132–138. Cited by: §1. [6] F. H. Arnold (1998) Design by directed evolution. Accounts of chemical research 31 (3), p. 125–131. Cited by: §1. [7] M. S. Packer and D. R. Liu (2015) Methods for the directed evolution of proteins. Nature Reviews Genetics 16 (7), p. 379–394. Cited by: §1. [8] Y. Wang, P. Xue, M. Cao, T. Yu, S. T. Lane, and H. Zhao (2021) Directed evolution: methodologies and applications. Chemical reviews 121 (20), p. 12384–12444. Cited by: §1. [9] L. Yuan, I. Kurek, J. English, and R. Keenan (2005) Laboratory-directed protein evolution. Microbiology and Molecular Biology Reviews 69 (3), p. 373–392. Cited by: §1. [10] D. Brookes, H. Park, and J. Listgarten (2019) Conditioning by adaptive sampling for robust design. In International conference on machine learning, p. 773–782. Cited by: §1, §2. [11] S. Sinai, R. Wang, A. Whatley, S. Slocum, E. Locane, and E. D. Kelsic (2020) AdaLead: a simple and robust adaptive greedy search algorithm for sequence design. arXiv preprint arXiv:2010.02141. Cited by: §C.3, §C.3, §1, §2, §5.1. [12] Z. Ren, J. Li, F. Ding, Y. Zhou, J. Ma, and J. Peng (2022) Proximal exploration for model-guided protein sequence design. In International Conference on Machine Learning, p. 18520–18536. Cited by: §C.3, §1, §2, §5.1. [13] M. Lee, L. F. Vecchietti, H. Jung, H. J. Ro, M. Cha, and H. M. Kim (2024) Robust optimization in protein fitness landscapes using reinforcement learning in latent space. In International Conference on Machine Learning, p. 26976–26990. Cited by: §C.4, §C.4, §1, §2, §3.2, §5.2, §5.2, Table 2, Table 2. [14] Y. Qiu, J. Hu, and G. Wei (2021) Cluster learning-assisted directed evolution. Nature computational science 1 (12), p. 809–818. Cited by: §C.3, §C.5, §1, §5.1, §5.1. [15] Y. Qiu and G. Wei (2022) Clade 2.0: evolution-driven cluster learning-assisted directed evolution. Journal of chemical information and modeling 62 (19), p. 4629–4641. Cited by: §C.3, §1, §5.1. [16] Y. Wang, Q. Zhang, M. Qin, X. Zhuang, X. Li, Z. Gong, Z. Wang, Y. Zhao, J. Yao, K. Ding, et al. (2024) Knowledge-aware reinforced language models for protein directed evolution. In Forty-first International Conference on Machine Learning, Cited by: §C.3, §C.3, Table 10, Table 10, Table 9, Table 9, §1, §2, §3.2, §5.1, §5.1, Table 1, Table 1. [17] H. Hegyi and M. Gerstein (1999) The relationship between protein structure and function: a comprehensive survey with application to the yeast genome. Journal of molecular biology 288 (1), p. 147–164. Cited by: §1. [18] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al. (2021) Highly accurate protein structure prediction with alphafold. nature 596 (7873), p. 583–589. Cited by: §D.1, §1, §5.1. [19] R. Evans, M. O’neill, A. Pritzel, N. Antropova, A. Senior, T. Green, A. Žídek, R. Bates, S. Blackwell, J. Yim, et al. (2021) Protein complex prediction with alphafold-multimer. biorxiv, p. 2021–10. Cited by: §1. [20] J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. (2024) Accurate structure prediction of biomolecular interactions with alphafold 3. Nature 630 (8016), p. 493–500. Cited by: §1. [21] J. Cheng, G. Novati, J. Pan, C. Bycroft, A. Žemgulytė, T. Applebaum, A. Pritzel, L. H. Wong, M. Zielinski, T. Sargeant, et al. (2023) Accurate proteome-wide missense variant effect prediction with alphamissense. Science 381 (6664), p. eadg7492. Cited by: §1. [22] B. A. A. Team, X. Chen, Y. Zhang, C. Lu, W. Ma, J. Guan, C. Gong, J. Yang, H. Zhang, K. Zhang, et al. (2025) Protenix-advancing structure prediction through a comprehensive alphafold3 reproduction. BioRxiv, p. 2025–01. Cited by: §F.1, §1. [23] Y. Zhang, C. Gong, H. Zhang, W. Ma, Z. Liu, X. Chen, J. Guan, L. Wang, Y. Yang, Y. Xia, and W. Xiao (2026) Protenix-v1: toward high-accuracy open-source biomolecular structure prediction. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2026/02/22/2026.02.05.703733.1.full.pdf Cited by: §1. [24] Y. Zhang, C. Gong, J. Sun, J. Guan, M. Ren, S. Xue, H. Zhang, W. Ma, Z. Liu, X. Chen, and W. Xiao (2026) Protenix-v2: broadening the reach of structure prediction and biomolecular design. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2026/04/11/2026.04.10.717613.full.pdf Cited by: §1. [25] J. Kim, M. Kwon, and J. Zhao (2025) Alphafold3 workload characterization: a comprehensive analysis of bottlenecks and performance scaling. In 2025 IEEE International Symposium on Workload Characterization (IISWC), p. 478–488. Cited by: §1. [26] F. Zhu, A. Nowaczynski, R. Li, J. Xin, Y. Song, M. Marcinkiewicz, S. B. Eryilmaz, J. Yang, and M. Andersch (2024) ScaleFold: reducing alphafold initial training time to 10 hours. In Proceedings of the 61st ACM/IEEE Design Automation Conference, p. 1–6. Cited by: §1. [27] M. A. Pak, K. A. Markhieva, M. S. Novikova, D. S. Petrov, I. S. Vorobyev, E. S. Maksimova, F. A. Kondrashov, and D. N. Ivankov (2023) Using alphafold to predict the impact of single mutations on protein stability and function. Plos one 18 (3), p. e0282689. Cited by: §1. [28] F. Luppino, S. Lenz, C. F. W. Chow, and A. Toth-Petroczy (2025) Deep learning tools predict variants in disordered regions with lower sensitivity. BMC genomics 26 (1), p. 367. Cited by: §1. [29] Y. Luo, Z. Nie, M. Hong, S. Zhao, H. Zhou, and Z. Nie (2024) MutaPLM: protein language modeling for mutation explanation and engineering. Advances in Neural Information Processing Systems 37, p. 79783–79818. Cited by: §1, §4.1. [30] A. Kirjner, J. Yim, R. Samusevich, S. Bracha, T. S. Jaakkola, R. Barzilay, and I. R. Fiete (2024) Improving protein optimization with smoothed fitness landscapes. In ICLR, Cited by: §C.2, §C.4, §C.4, §D.2, §2, §5.2, §5.2, §5.2, Table 2, Table 2. [31] N. C. Frey, D. Berenberg, K. Zadorozhny, J. Kleinhenz, J. Lafrance-Vanasse, I. Hötzel, Y. Wu, S. Ra, R. Bonneau, K. Cho, et al. (2024) PROTEIN discovery with discrete walk-jump sampling. In 12th International Conference on Learning Representations, ICLR 2024, Cited by: §2. [32] T. V. Tran, N. K. Ngo, V. T. D. Nguyen, and T. S. Hy (2024) Latentde: latent-based directed evolution accelerated by gradient ascent for protein sequence design. In NeurIPS 2024 Workshop on AI for New Drug Modalities, Cited by: §2. [33] D. H. Brookes and J. Listgarten (2018) Design by adaptive sampling. arXiv preprint arXiv:1810.03714. Cited by: §2. [34] M. Jain, E. Bengio, A. Hernandez-Garcia, J. Rector-Brooks, B. F. Dossou, C. A. Ekbote, J. Fu, T. Zhang, M. Kilgour, D. Zhang, et al. (2022) Biological sequence design with gflownets. In International Conference on Machine Learning, p. 9786–9801. Cited by: §C.4, §2. [35] L. Bogensperger, D. Narnhofer, A. Allam, K. Schindler, and M. Krauthammer (2025) A variational perspective on generative protein fitness optimization. In Forty-second International Conference on Machine Learning, Cited by: §C.4, §C.4, §2, §5.2. [36] H. Kim, M. Kim, T. Yun, S. Choi, E. Bengio, A. Hernández-García, and J. Park (2025) Improved off-policy reinforcement learning in biological sequence design. In Forty-second International Conference on Machine Learning, Cited by: §C.4, §2, §5.2. [37] C. Angermueller, D. Dohan, D. Belanger, R. Deshpande, K. Murphy, and L. Colwell (2019) Model-based reinforcement learning for biological sequence design. In International conference on learning representations, Cited by: §2. [38] Y. Wang, H. Tang, L. Huang, L. Pan, L. Yang, H. Yang, F. Mu, and M. Yang (2023) Self-play reinforcement learning guides protein engineering. Nature Machine Intelligence 5 (8), p. 845–860. Cited by: §C.3, §2, §3.1, §5.1. [39] C. Dallago, J. Mou, K. E. Johnston, B. J. Wittmann, N. Bhattacharya, S. Goldman, A. Madani, and K. K. Yang (2021) FLIP: benchmark tasks in fitness landscape inference for proteins. bioRxiv, p. 2021–11. Cited by: §2. [40] P. Notin, A. Kollasch, D. Ritter, L. Van Niekerk, S. Paul, H. Spinner, N. Rollins, A. Shaw, R. Orenbuch, R. Weitzman, et al. (2023) Proteingym: large-scale benchmarks for protein fitness prediction and design. Advances in neural information processing systems 36, p. 64331–64379. Cited by: §2. [41] P. Hermosilla and T. Ropinski (2022) Contrastive representation learning for 3d protein structures. arXiv preprint arXiv:2205.15675. Cited by: Appendix B, §2. [42] C. Chen, J. Zhou, F. Wang, X. Liu, and D. Dou (2023) Structure-aware protein self-supervised learning. Bioinformatics 39 (4), p. btad189. Cited by: Appendix B, §2. [43] C. Hsu, R. Verkuil, J. Liu, Z. Lin, B. Hie, T. Sercu, A. Lerer, and A. Rives (2022) Learning inverse folding from millions of predicted structures. In International conference on machine learning, p. 8946–8970. Cited by: Appendix B, §D.2, §2, §4.1. [44] J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. Wicky, A. Courbet, R. J. de Haas, N. Bethel, et al. (2022) Robust deep learning–based protein sequence design using proteinmpnn. Science 378 (6615), p. 49–56. Cited by: Appendix B, §2. [45] K. K. Yang, N. Zanichelli, and H. Yeh (2023) Masked inverse folding with sequence transfer for protein representation learning. Protein Engineering, Design and Selection 36, p. gzad015. Cited by: Appendix B, §2. [46] J. Su, C. Han, Y. Zhou, J. Shan, X. Zhou, and F. Yuan (2024) SaProt: protein language modeling with structure-aware vocabulary. In ICLR, Cited by: Appendix B, §2. [47] M. Li, Y. Tan, X. Ma, B. Zhong, H. Yu, Z. Zhou, W. Ouyang, B. Zhou, P. Tan, and L. Hong (2024) Prosst: protein language modeling with quantized structure and disentangled attention. Advances in Neural Information Processing Systems 37, p. 35700–35726. Cited by: Appendix B, §D.2, §2. [48] Z. Zhang, P. Notin, Y. Huang, A. Lozano, V. Chenthamarakshan, D. Marks, P. Das, and J. Tang (2024) Multi-scale representation learning for protein fitness prediction. Advances in Neural Information Processing Systems 37, p. 101456–101473. Cited by: §2. [49] Y. Tan, R. Wang, B. Wu, L. Hong, and B. Zhou (2025) From high-throughput evaluation to wet-lab studies: advancing mutation effect prediction with a retrieval-enhanced model. Bioinformatics 41 (Supplement_1), p. i401–i409. Cited by: §2. [50] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: §A.3, §D.2, §3.2. [51] Z. Lin, H. Akin, R. Rao, B. Hie, Z. Zhu, W. Lu, N. Smetanin, A. dos Santos Costa, M. Fazel-Zarandi, T. Sercu, S. Candido, et al. (2022) Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv. Cited by: Appendix B, §D.2, §4.1. [52] K. Asadi, D. Misra, and M. Littman (2018) Lipschitz continuity in model-based reinforcement learning. In International conference on machine learning, p. 264–273. Cited by: §A.4, §4.3. [53] Z. Wu, S. J. Kan, R. D. Lewis, B. J. Wittmann, and F. H. Arnold (2019) Machine learning-assisted directed protein evolution with combinatorial libraries. Proceedings of the National Academy of Sciences 116 (18), p. 8852–8858. Cited by: §C.3, §5.1, §5.1. [54] A. I. Podgornaia and M. T. Laub (2015) Pervasive degeneracy and epistasis in a protein-protein interface. Science 347 (6222), p. 673–677. Cited by: §5.1. [55] J. T. Wilson, R. Moriconi, F. Hutter, and M. P. Deisenroth (2017) The reparameterization trick for acquisition functions. arXiv preprint arXiv:1712.00424. Cited by: §C.3, §5.1. [56] N. Hansen, S. D. Müller, and P. Koumoutsakos (2003) Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary computation 11 (1), p. 1–18. Cited by: §C.3, §5.1. [57] B. J. Wittmann, Y. Yue, and F. H. Arnold (2021) Informed training set design enables efficient machine learning-assisted directed protein evolution. Cell systems 12 (11), p. 1026–1045. Cited by: §C.3, §5.1. [58] K. S. Sarkisyan, D. A. Bolotin, M. V. Meer, D. R. Usmanova, A. S. Mishin, G. V. Sharonov, D. N. Ivankov, N. G. Bozhanova, M. S. Baranov, O. Soylemez, et al. (2016) Local fitness landscape of the green fluorescent protein. Nature 533 (7603), p. 397–401. Cited by: §5.2. [59] D. H. Bryant, A. Bashir, S. Sinai, N. K. Jain, P. J. Ogden, P. F. Riley, G. M. Church, L. J. Colwell, and E. D. Kelsic (2021) Deep diversification of an aav capsid protein by machine learning. Nature Biotechnology 39 (6), p. 691–696. Cited by: §5.2. [60] S. Delagrave, R. E. Hawtin, C. M. Silva, M. M. Yang, and D. C. Youvan (1995) Red-shifted excitation mutants of the green fluorescent protein. Bio/technology 13 (2), p. 151–154. Cited by: §5.4. [61] J. Pédelacq, S. Cabantous, T. Tran, T. C. Terwilliger, and G. S. Waldo (2006) Engineering and characterization of a superfolder green fluorescent protein. Nature biotechnology 24 (1), p. 79–88. Cited by: §5.4. [62] R. M. Rao, J. Liu, R. Verkuil, J. Meier, J. Canny, P. Abbeel, T. Sercu, and A. Rives (2021) MSA transformer. In International conference on machine learning, p. 8844–8856. Cited by: Appendix B. [63] J. Meier, R. Rao, R. Verkuil, J. Liu, T. Sercu, and A. Rives (2021) Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in neural information processing systems 34, p. 29287–29303. Cited by: Appendix B. [64] N. Brandes, D. Ofer, Y. Peleg, N. Rappoport, and M. Linial (2022) ProteinBERT: a universal deep-learning model of protein sequence and function. Bioinformatics 38 (8), p. 2102–2110. Cited by: Appendix B. [65] P. Notin, M. Dias, J. Frazer, J. Marchena-Hurtado, A. N. Gomez, D. Marks, and Y. Gal (2022) Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In International Conference on Machine Learning, p. 16990–17017. Cited by: Appendix B. [66] J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), p. 4171–4186. Cited by: Appendix B. [67] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu (2020) A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32 (1), p. 4–24. Cited by: Appendix B. [68] E. Bengio, M. Jain, M. Korablyov, D. Precup, and Y. Bengio (2021) Flow network based generative models for non-iterative diverse candidate generation. Advances in neural information processing systems 34, p. 27381–27394. Cited by: §C.4. [69] M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter (2015) Efficient and robust automated machine learning. In Advances in Neural Information Processing Systems 28 (2015), p. 2962–2970. Cited by: §C.5. [70] M. Norouzi, D. J. Fleet, and R. R. Salakhutdinov (2012) Hamming distance metric learning. Advances in neural information processing systems 25. Cited by: §C.6. [71] H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov, and P. E. Bourne (2000) The protein data bank. Nucleic acids research 28 (1), p. 235–242. Cited by: §D.1. [72] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann (2021) Stable-baselines3: reliable reinforcement learning implementations. Journal of machine learning research 22 (268), p. 1–8. Cited by: §D.2. Appendix Appendix A Details of StructEvo A.1 Symbol definitions Table 4: Definition of symbols used in our work. Category Symbol Definition Protein Representation L the protein length X the protein sequence S the protein structure X the mutation space V the amino acid vocabulary xix_i the amino acid identity of residue i ai,bi,cia_i,b_i,c_i the three-dimensional coordinates of residue i ℰseq,ℰstr(⋅)E_seq,E_str(·) the sequence and structure encoder hseq,hstr,hfusionh_seq,h_str,h_fusion the sequence feature, structure feature, and the fusion feature Δhseq,Δhstr h_seq, h_str the delta sequence and structure feature h^str h_str the approximated mutant structure feature dseq,dstrd_seq,d_str the hidden size of sequence and structure features Active Learning R the number of total rounds N the number of proposed candidates per round ℱ(⋅)F(·) the oracle function fϕ(⋅)f_φ(·) the proxy model parameterized by ϕφ D the mutant candidate pool 0D_0 the initial mutant candidate pool Reinforcement Learning and Policy A the action space sts_t the state at step t ata_t the action at step t atpos,attypea_t^pos,a_t^type the mutation position and substituted amino acid type at step t rtr_t the reward calculated by the proxy at step t γ the discount rate πθ1,πθ2 _ _1, _ _2 the parameterized position and amino acid type action networks ,τT,τ the collected trajectory mstepm_step the maximum number of steps in a trajectory A.2 Algorithm for StructEvo framework pipeline We provide a full description of the StructEvo pipeline in Algorithm 1. The outer loop is the active learning process, where selected mutant candidates are evaluated by the oracle and then iteratively update the candidate pool, followed by proxy model fine-tuning. The inner loop shows the reinforcement learning procedure within each round, including how the mutation trajectories are collected, how rewards are assigned, and the policy network is updated. Algorithm 1 The StructEvo pipeline 1: Input: Total rounds R, number of proposed candidates per round N, black-box oracle ℱF, starting candidates set 0=Xn,ℱ(Xn)n=1|0|D_0=\X_n,F(X_n)\_n=1^|D_0|, reference structure SrefS^ref and sequence XrefX^ref, sequence encoder ℰseqE_seq, structure encoder ℰstrE_str, trajectory max length mstepm_step, total max step mtotalm_total 2: Initialize current round i←0i← 0 3: while i<Ri<R do 4: // active learning loop begin 5: Fine-tune the proxy model fϕf_φ with validated candidates iD_i 6: Initialize candidate buffer ←∅D← 7: Initialize trajectory buffer ←∅T← 8: Initialize total steps ttotal←0t_total← 0 9: while ttotal<mtotalt_total<m_total do 10: // RL loop begin 11: Sample an initial sequence X0∼(i)X_0 TopK(D_i) 12: Initialize state s0←X0s_0← X_0, reward threshold rinit←fϕ(s0)r_init← f_φ(s_0) 13: step t←0t← 0, trajectory τ←∅τ← , FLAGstop←FalseFLAG_stop← False 14: while not FLAGstopFLAG_stop do 15: // delta-structure fusion encoder 16: Delta sequence feature Δhseq,t←ℰseq(st)−ℰseq(Xref) h_seq,t _seq(s_t)-E_seq(X^ref) 17: Delta structure feature Δhstr,t←fproj(Δhseq,t) h_str,t← f_proj( h_seq,t) 18: Approximated mutant structure feature h^str,t←ℰstr(Sref)+Δhstr,t h_str,t _str(S^ref)+ h_str,t 19: Calculate geometric constraint loss ℒgeo(Δhstr,t,Δhseq,t)L_geo( h_str,t, h_seq,t) 20: Fusion feature hfusion,t←CrossAttention(ℰseq(st),h^str,t,h^str,t)h_fusion,t← CrossAttention(E_seq(s_t), h_str,t, h_str,t) 21: // hierarchical action network 22: Sample mutation position atpos∼πθ1(⋅∣hfusion,t)a_t^pos _ _1(· h_fusion,t) 23: Sample substituted amino acid type attype∼πθ2(⋅∣hfusion,t,atpos)a_t^type _ _2(· h_fusion,t,a_t^pos) 24: Apply mutation to obtain next state st+1←transition(st,(atpos,attype))s_t+1← transition(s_t,(a_t^pos,a_t^type)) 25: Assign reward rt←fϕ(st+1)r_t← f_φ(s_t+1) 26: if rt>rinitr_t>r_init or t≥mstept≥ m_step then 27: FLAGstop←TrueFLAG_stop← True 28: end if 29: if not FLAGstopFLAG_stop then 30: rt←0r_t← 0 // sparse reward for intermediate steps 31: end if 32: τ←τ∪st,at,rtτ←τ∪\s_t,a_t,r_t\ 33: t←t+1t← t+1 34: ttotal←ttotal+1t_total← t_total+1 35: end while 36: Update ←∪τT ∪τ 37: Update ←∪stD ∪\s_t\ 38: end while 39: Update policy by ℒPPO+λgeo⋅ℒgeoL_PPO+ _geo·L_geo with collected trajectories T 40: Select N novel candidates ranked by proxy Xnn=1N←(,fϕ())\X_n\^N_n=1← TopN(D,f_φ(D)) 41: Evaluate Xnn=1N\X_n\^N_n=1 by oracle ℱF 42: Update candidate pool i←i−1∪Xn,ℱ(Xn)n=1ND_i _i-1∪\X_n,F(X_n)\^N_n=1 43: i←i+1i← i+1 44: end while 45: return proposed candidates RD_R A.3 The detach operator As introduced in Sec. 4.2, in the proposed structure-aligned hierarchical action network, the localized feature is extracted and detached from the fusion feature. We next show how the detach operation blocks gradient flow and leads to a cleaner learning signal. For simplicity, we use h below to denote hfusionh_fusion in the main text. As introduced in Schulman et al. [50], the training objective of PPO consists of three components: LtCLIP+VF+S(θ)=^t[LtCLIP(θ)−c1LtVF(θ)+c2S[πθ](st)],L_t^CLIP+VF+S(θ)= E_t [L_t^CLIP(θ)-c_1L_t^VF(θ)+c_2S [ _θ ](s_t) ], (8) where the main objective LtCLIP(θ)L_t^CLIP(θ) is: LtCLIP(θ)=^t[min(rt(θ)A^t,clip(rt(θ), 1−ϵ, 1+ϵ)A^t)],L_t^CLIP(θ)= E_t [ (r_t(θ)\, A_t,\;clip (r_t(θ),\,1-ε,\,1+ε )\, A_t ) ], (9) with the probability ratio rt(θ)r_t(θ) defined as: rt(θ)=πθ(at∣st)πθold(at∣st).r_t(θ)= _θ(a_t s_t) _ _old(a_t s_t). (10) In our hierarchical action network, the mutation action ata_t is decomposed into at=(atpos,attype)a_t=(a_t^pos,a_t^type), which is sampled sequentially through two action networks πθ1 _ _1 and πθ2 _ _2. Therefore, the probability of sampled action ata_t at given state sts_t corresponds to: πθ(at∣st)=p(atpos,attype)=πθ1(atpos∣h)⋅πθ2(attype∣h,atpos). _θ(a_t s_t)=p(a_t^pos,a_t^type)= _ _1(a_t^pos h)· _ _2(a_t^type h,a_t^pos). (11) During policy optimization, gradients of the PPO objective are computed. We focus on the joint log-likelihood component ∇θ1,θ2logp(apos,atype) _ _1, _2 p(a^pos,a^type) associated with the action network. Under the hierarchical regime, it factorizes as: ∇θ1,θ2logp(apos,atype) _ _1, _2 p(a^pos,a^type) (12) =∇θ1,θ2log[πθ1(apos∣h)⋅πθ2(atype∣h,apos)] = _ _1, _2 [ _ _1(a^pos h)· _ _2(a^type h,a^pos) ] (13) =∇θ1,θ2[logπθ1(apos∣h)+logπθ2(atype∣h,apos)] = _ _1, _2 [ _ _1(a^pos h)+ _ _2(a^type h,a^pos) ] (14) =∇θ1logπθ1(apos∣h)+∇θ2logπθ2(atype∣h,apos), = _ _1 _ _1(a^pos h)+ _ _2 _ _2(a^type h,a^pos), (15) where the sampling operation blocks the gradients of the amino acid decision back-propagating to the position action network. The gradients from πθ2 _ _2 cannot influence the preceding feature fusing module, since the detaching operation enforces ∇hπθ2(atype∣h,apos)=0 _h~ _ _2(a^type h,a^pos)=0. As a result, the position network πθ1 _ _1 is optimized to capture the overall contribution of selecting mutation sites, yielding a cleaner and more stable learning signal for position selection. This decomposition enables the structure-enhanced features to more directly and effectively guide the identification of functional mutation positions, thereby reducing learning complexity and improving data efficiency under limited experimental feedback. A.4 The geometric constraint In Sec. 4.3, we introduce a per-residue geometric constraint loss between the delta sequence feature Δhseq∈ℝL×dseq h_seq ^L× d_seq and delta structure feature Δhstr∈ℝL×dstr h_str ^L× d_str, formulated as: ℒgeo(Δhstr,Δhseq)=1L∑i=1L(‖Δhstri‖dstr−c∗‖Δhseqi‖dseq)2,L_geo( h_str, h_seq)= 1LΣ^L_i=1 ( \| h^i_str\| d_str-c* \| h^i_seq\| d_seq )^2, (16) where c is the scaling factor. Here, we provide an intuitive explanation of how the proposed geometric constraint meets the upper bound of Lipschitz continuity. We refer readers to Asadi et al. [52] for a more rigorous analysis on how the Lipschitz continuity benefits to smooth mappings and more stable learning signals in RL framework. For simplicity, we denote the delta feature matrices as: =Δhseq=xiji=1,…,Lj=1,…,dseq,=Δhstr=yiji=1,…,Lj=1,…,dstr.x= h_seq=\x_ij\_i=1,...,L^j=1,...,d_seq, = h_str=\y_ij\_i=1,...,L^j=1,...,d_str. (17) The corresponding ℓ2 _2 norms are defined as: ‖=∑i=1L∑j=1dseqxij2,‖i‖=∑j=1dseqxij2,‖=∑i=1L∑j=1dstryij2,‖i‖=∑j=1dstryij2.\|x\|= _i=1^L _j=1^d_seqx_ij^2, \|x_i\|= _j=1^d_seqx_ij^2, \|y\|= _i=1^L _j=1^d_stry_ij^2, \|y_i\|= _j=1^d_stry_ij^2. (18) Under this notation, the geometric constraint loss is formulated as: ℒgeo=1L∑i=1L(‖i‖dstr−c⋅‖i‖dseq)2.L_geo= 1L _i=1^L( \|y_i\| d_str-c· \|x_i\| d_seq)^2. (19) We consider the idealized global minimum where ℒgeo=0L_geo=0. This implies: ‖i‖=(c⋅dstrdseq)⋅‖i‖for alli∈1,…,L.\|y_i\|= (c· d_str d_seq )·\|x_i\| all i∈\1,…,L\. (20) We denote c0=c⋅dstrdseqc_0=c· d_str d_seq. Squaring both sides, we obtain: ∑j=1dstryij2=‖i‖2=c02⋅‖i‖2=c02⋅∑j=1dseqxij2for alli∈1,…,L. _j=1^d_stry_ij^2=\|y_i\|^2=c_0^2·\|x_i\|^2=c_0^2· _j=1^d_seqx_ij^2 all i∈\1,…,L\. (21) Summing over i from 11 to L, we obtain: ∑i=1L∑j=1dstryij2=∑i=1Lc02⋅‖i‖2=c02⋅∑i=1L∑j=1dseqxij2. _i=1^L _j=1^d_stry_ij^2= _i=1^Lc_0^2·\|x_i\|^2=c_0^2· _i=1^L _j=1^d_seqx_ij^2. (22) Thus, ‖2=c02⋅‖2.\|y\|^2=c_0^2·\|x\|^2. (23) Therefore, ‖=c0⋅‖.\|y\|=c_0·\|x\|. (24) We define the projection mapping from delta sequence manifold =Δhseqx= h_seq to delta structure manifold =Δhstry= h_str as ℳ:↦M:x . By definition, the mapping ℳM is K-Lipschitz continuous if any input x satisfies: ‖ℳ()−ℳ(ref)‖≤K⋅‖−ref‖.\|M(x)-M(x^ref)\|≤ K·\|x-x^ref\|. (25) In the context of our delta representation, any delta feature is defined as the difference Δhseq=hseq−hseqref h_seq=h_seq-h_seq^ref. Accordingly, the delta feature of the reference protein is ref=Δhseqref=hseqref−hseqref=x^ref= h_seq^ref=h_seq^ref-h_seq^ref=0. This simplifies Eq. 25 to: ‖=‖ℳ()‖≤K⋅‖.\|y\|=\|M(x)\|≤ K·\|x\|. (26) Comparing Eq. 24 and Eq. 26, we meet the upper bound of the Lipschitz continuity with the Lipschitz constant: K=c0=c⋅dstrdseq.K=c_0=c· d_str d_seq. (27) Thus, the geometric constraint loss ℒgeoL_geo enables model to a stronger condition than Lipschitz continuity, encouraging consistent sensitivity in the projection mapping and leading to more stable learning signals for RL training process. We also conduct experiments to investigate the sensitivity of this geometric constraint, as detailed in Appendix F.3. Appendix B Protein fitness landscape modeling Protein fitness landscape modeling aims to learn a fine-grained mapping from proteins to fitness values. Sequence-based models [62, 63, 51, 64, 65] formulate protein residues as discrete tokens analogous to natural language, and are pre-trained on large-scale sequence databases via masked language modeling (MLM; 66), enabling zero-shot fitness prediction. Structure-based models [41, 42] typically leverage Graph Neural Network (GNN; 67) to explicitly encode three-dimensional protein structures as nodes and edges. Inverse-folding models [43, 44, 45] captured structural representations to predict protein sequence. More recently, Hybrid models incorporate both sequence and structure to obtain more informative protein representations. Su et al. [46] introduced a concatenated structure-aware vocabulary, while Li et al. [47] proposed a structure quantization module with a disentangled attention mechanism for modality fusion. Appendix C Experiments details C.1 Combinatorial mutation benchmarks A description of GB1 and PhoQ dataset is listed at Table 5. Fitness of GB1 dataset measures the binding affinity of human protein GB1 domain to the antibody IgG-Fc, while fitness of PhoQ refers to kinase and phosphatase responsiveness of PhoP/PhoQ regulatory system. The key advantage of these datasets is that they are sufficiently large to almost exhaustively cover the theoretical four-site combinatorial space (204=160,00020^4=160,000), thereby serving as reliable assessments for MLDE methods. Table 5: Description of GB1 and PhoQ benchmarks. The last three columns report the proportion of mutants whose normalized fitness is lower than a given threshold, highlighting the sparsity and challenge of these benchmarks. a: amino acids. Name UniProt ID Protein Length Dataset Size Mutation Sites <0.3 <0.1 = 0 GB1 P19909 56a 149,361 V39,D40, G41,V54 99.30% 97.30% 19.74% PhoQ P23837 486a 140,517 A284,V285, S288,T289 99.96% 98.65% 63.70% C.2 Full-length mutation benchmarks Following the settings of Kirjner et al. [30], both GFP and AAV tasks are partitioned into two levels, medium and hard, by controlling the initial variant pool. For medium level, initial mutants are sampled from the 20th–40th fitness percentiles of the full dataset with a minimum pairwise Hamming distance of 6, resulting in 2,139 and 2,828 initial variants for AAV and GFP, respectively. For hard level, initial sequences are drawn from the bottom 30th percentile with a minimum distance of 7, yielding 3,448 and 2,426 variants for AAV and GFP, respectively. Table 6 and 7 describe the statistic of four initial sets. Figure 5 provides a visualized comparison on fitness distributions among all data, medium initial sets and hard initial sets. Table 6: Description of initial sets in the AAV benchmark. Mean Fitness is the mean normalized fitness of top 128 sequences in subsets. Level Range(%) Gap Size Mean Fitness Medium 20-40th 6 2139 0.376 Hard <30th 7 3448 0.326 Table 7: Description of initial sets in the GFP benchmark. Mean Fitness is the mean normalized fitness of top 128 sequences in subsets. Level Range(%) Gap Size Mean Fitness Medium 20-40th 6 2828 0.232 Hard <30th 7 2426 0.092 Figure 5: Kernel density estimation (KDE) of fitness distributions for AAV and GFP benchmarks. C.3 Baselines for combinatorial benchmarks We evaluate various baselines on four-site combinatorial benchmarks, GB1 and PhoQ. CMA-ES [56]. The Covariance Matrix Adaptation Evolution Strategy optimizes protein sequences through a continuous relaxation of the discrete mutation space, where each sequence is represented by a one-hot encoding and optimized in a continuous domain. BO [55]. Bayesian optimization formulates protein directed evolution as a black-box optimization problem by updating an acquisition function after each round. The implementation of BO is derived from FLEXS benchmark [11]. AdaLead [11]. This work adopts an adaptive greedy search algorithm for directed evolution by iteratively selecting sequences whose fitness exceeds a dynamically updated threshold based on the previous batch, and generating new candidates via mutation and recombination of these high-performing variants. MLDE [53]. This work first incorporates machine learning into the combinatorial directed evolution workflow by training an ensemble model to increase throughput with in-silico modeling. ftMLDE [57]. This work evaluates how different protein encoding methods and training set partitioning strategies affect the performance of protein directed evolution, with EVmutation and MSATransformer adopted as our baselines following Wang et al. [16]. CLADE [14]. This work proposes an unsupervised hierarchical clustering strategy to explore the combinatorial mutation space, followed by in silico screening of selected mutants using a supervised proxy model. CLADE 2.0 [15]. This work extends CLADE by improving the initial sampling stage through ensembling multiple evolutionary scores, resulting in more robust performance with reduced sensitivity to hyperparameter choices. PEX [12]. This work introduces proximal exploration, which biases evolutionary search toward high-fitness mutants with low mutation counts by applying a distance-based preference around the wild-type sequence. EvoPlay [38]. This work models protein directed evolution by a neural network-guided Monte Carlo tree search (MCTS) procedure, which is more interpretable than optimization methods that performed in a latent space. KnowRLM [16]. This work introduces an Amino Acid Knowledge Graph (AAKG) together with a dynamic window mechanism to guide mutation selection using physicochemical properties, optimizing by PPO. C.4 Baselines for full-length benchmarks This section introduces additional baselines for full-length benchmarks. GGS [30]. This work employs a graph-based smoothing approach to train a smoothed proxy model, which defines a discrete energy function for mutant sampling via Gibbs-with-Gradients. Following prior work [13, 35] on the GFP and AAV benchmarks, we use the provided checkpoints of both the smoothed proxy and the oracle in this work. LatProtRL [13]. This work encodes protein mutants into latent representations using a pretrained variational encoder–decoder and treats these latent features as states for PPO, with actions defined as small continuous perturbations in the latent space, which improves sample efficiency by reducing the dimension of the mutation space. VLGPO [35]. This work introduces Variational Latent Generative Protein Optimization, which optimizes the sampling strategy in a continuous latent space using a flow-matching prior. As the original work demonstrates better performance when using an unsmoothed fitness predictor proposed in [30], we follow this choice and train the model for 15 rounds in an active learning setting. GFN-δ-CS [36]. GFN-AL [34] leverages Generative Flow Networks (GFlowNets; 68) within an active learning loop for protein optimization. This work further improves GFN-AL by introducing a controllable conservation factor δ to explicitly balance exploration and exploitation. In our experiments, we set δ=0.01δ=0.01 to prioritize fitness optimization. Since these methods rely on pretraining an encoder–decoder or performing optimization in continuous latent spaces, we carefully considered their compatibility and, for fairness and clarity of comparison, did not include them as baselines for the combinatorial benchmarks. C.5 Metrics for combinatorial benchmarks The mean and maximum fitness are computed over the combined candidate set, which includes 96 mutants from initial set sampled by CLADE [14] via an unsupervised hierarchical clustering approach, 288 mutants proposed across three rounds (N=96N=96 per round), as well as the top 96 mutants selected by the proxy model after the final training round, resulting in a total of 480 candidates. The normalized discounted cumulative gain (NDCG) is calculated between predictions and the ground truth fitness in the entire mutation space with size of 204=160,00020^4=160,000. We compute NDCG using metrics.ndcg_score implemented in scikit-learn framework [69]. C.6 Metrics for full-length benchmarks The mean and maximum fitness are computed on the top-K of the proposed sequences across 15 rounds, where K is set to 128. The diversity of N proposed sequences =Xii=1ND=\X_i\_i=1^N is defined as mean pairwise Hamming distance [70] among sequences: Diversity()=1N(N−1)∑i=1N∑j=1,j≠iNHamming(i,j).Diversity(D)= 1N(N-1) _i=1^N _j=1,j≠ i^N Hamming(i,j). (28) The novelty of N proposed sequences =Xii=1ND=\X_i\_i=1^N is defined as the mean minimum Hamming distance from each proposed sequence to the initial set 0D_0: Novelty()=1N∑i=1N(minX∗∈0Hamming(Xi,X∗)).Novelty(D)= 1N _i=1^N ( _X^* _0 \ Hamming(X_i,X^*) \ ). (29) Appendix D Implementation details D.1 Structure process Table 8: Information of the reference protein structures used in our work. Protein Source Structure ID Chain ID Position Length GB1 PDB 2GI9 A 1–56 56 PhoQ AFDB AF-Q8FIB8-F1-v6 A 188–376 189 AAV PDB 3NG9 A 561–588 28 GFP PDB 1GFL A 2–238 237 The sources of protein structures used in this work are listed in Table 8. Experimental structures of GB1, AAV and GFP are obtained from the Protein Data Bank (PDB) [71], the largest global database of biological macromolecular structures. As the crystal structure of PhoQ is unavailable in the PDB, we select a high-confidence predicted structure with average pLDDT of 82.94 from the AlphaFold Protein Structure Database (AFDB) [18] instead. D.2 Training details We employed the open-source ESM-2 [51] as PLM for encoding protein sequences, and ESM-IF [43] as the inverse folding model for encoding structures. Our code is developed within python 3.11 and PyTorch framework, running on a single NVIDIA A800 80G GPU. Proxy model training details. For four-site combinatorial benchmarks, we adopt the structure-aware ProSST [47] as our proxy model. ProSST incorporates a structure quantization module and a transformer-based disentangled attention mechanism to better align sequence and structural features. We append a regression head to the backbone, with a hidden size of 128 and a scalar as output, to convert it into a fitness predictor. During all fine-tuning and inference process of ProSST, we utilize the reference structure coordinates as structural inputs. For full-length benchmarks, we use the provided checkpoints of the smoothed proxy in Kirjner et al. [30] by convention. The proxy is based on a convolutional neural network (CNN) architecture and encodes protein sequences to one-hot embeddings as inputs. For all benchmarks, the proxy model is fine-tuned on the newly proposed mutant candidates for 30 epochs after each round, using a batch size of 96 and a learning rate of 2×10−32× 10^-3. The total compute time for an individual experimental run is around 2 hours. RL training details. The reinforcement learning procedure is optimized using Proximal Policy Optimization (PPO) [50], supplemented by the Stable-Baseline3 [72] framework. For all benchmarks, we train the PPO using 8 parallel environments, with a clipping range of 0.3, entropy coefficient of 0.0, discount factor of 0.99, learning rate of 3×10−43× 10^-4, and batch size of 64. The max episode length mstepm_step is set to 3 for each round. The number of total training steps is set to 3,000 for combinatorial benchmarks and 15,000 for full-length benchmarks. For geometric constraint, the weight λgeo _geo is set to 0.05 and the scaling factor c is set to 1. Appendix E Additional experiment results E.1 Full results on 4-site benchmarks We provide results on 3 optimization rounds at Table 9 and Table 10. Table 9: Results on GB1 benchmarks across 3 rounds. The best scores bolded and the second best underlined. All baseline results are derived from Wang et al. [16]. Round 1 Round 2 Round 3 Method Max↑ Mean↑ NDCG↑ Max↑ Mean↑ NDCG↑ Max↑ Mean↑ NDCG↑ MLDE 0.650 0.183 0.767 0.680 0.217 0.794 0.684 0.203 0.789 ftMLDE (EVmutation) 0.725 0.233 0.791 0.770 0.280 0.814 0.935 0.414 0.833 ftMLDE (Transformer) 0.761 0.239 0.792 0.814 0.298 0.819 0.932 0.416 0.813 EvoPlay 0.837 0.433 0.826 0.834 0.457 0.839 0.840 0.460 0.847 CLADE 0.785 0.309 0.801 0.802 0.303 0.803 0.835 0.458 0.857 CLADE 2.0 0.886 0.376 0.808 0.903 0.419 0.858 0.935 0.491 0.879 KnowRLM 0.931 0.494 0.851 0.972 0.534 0.862 0.972 0.562 0.884 StructEvo (ours) 0.942 0.514 0.870 0.973 0.540 0.902 0.990 0.587 0.932 Table 10: Results on PhoQ benchmarks across 3 rounds. The best scores bolded and the second best underlined. All baseline results are derived from Wang et al. [16]. Round 1 Round 2 Round 3 Method Max↑ Mean↑ NDCG↑ Max↑ Mean↑ NDCG↑ Max↑ Mean↑ NDCG↑ MLDE 0.309 0.069 0.753 0.364 0.087 0.775 0.361 0.095 0.791 ftMLDE (EVmutation) 0.297 0.072 0.754 0.431 0.114 0.807 0.436 0.115 0.804 ftMLDE (Transformer) 0.346 0.073 0.756 0.414 0.108 0.802 0.422 0.117 0.815 EvoPlay 0.443 0.119 0.782 0.444 0.135 0.786 0.474 0.143 0.804 CLADE 0.319 0.070 0.759 0.441 0.089 0.762 0.467 0.089 0.777 CLADE 2.0 0.345 0.118 0.781 0.383 0.125 0.779 0.399 0.148 0.814 KnowRLM 0.486 0.129 0.821 0.532 0.152 0.816 0.658 0.157 0.819 StructEvo (ours) 0.530 0.158 0.844 0.581 0.169 0.848 0.640 0.186 0.856 E.2 Full ablation results on full-length benchmarks See Table 11 for full ablation results. Ablation settings are: (1) removing delta module by directly fusing mutant sequence with the static reference structure via cross-attention, (2) removing structure and fusing two sequence features, (3) relying solely on mutant sequence, and (4) replacing the hierarchical action network with a flat network that jointly samples both actions. The results on the two medium settings show trends consistent with those on the hard setting, as analyzed in Sec. 5.3. Method Mean↑ Max↑ Diversity Novelty AAV medium StructEvo 0.82± 0.01 0.86± 0.01 4.5± 0.7 8.6± 0.2 (1) w/o delta 0.79± 0.00 0.83± 0.01 5.5± 0.8 7.7± 0.0 (2) w/o structure 0.77± 0.01 0.81± 0.02 4.7± 1.3 8.1± 0.7 (3) w/o reference 0.77± 0.01 0.81± 0.02 4.8± 2.0 7.8± 0.0 (4) w/o hierarchy 0.78± 0.01 0.80± 0.01 5.3± 1.3 7.9± 0.6 AAV hard StructEvo 0.83± 0.03 0.87± 0.04 4.5± 0.3 8.4± 0.5 (1) w/o delta 0.80± 0.03 0.84± 0.05 4.9± 0.6 8.3± 0.7 (2) w/o structure 0.75± 0.03 0.79± 0.04 4.9± 1.8 8.6± 0.7 (3) w/o reference 0.75± 0.01 0.81± 0.01 7.5± 1.4 8.7± 0.4 (4) w/o hierarchy 0.77± 0.02 0.81± 0.02 5.2± 1.4 7.9± 0.3 GFP medium StructEvo 1.11± 0.02 1.17± 0.02 6.3± 0.8 11.9± 0.8 (1) w/o delta 1.09± 0.02 1.13± 0.02 6.2± 1.3 10.9± 1.1 (2) w/o structure 1.05± 0.02 1.09± 0.02 7.4± 1.7 10.7± 0.5 (3) w/o reference 1.06± 0.05 1.10± 0.05 6.2± 1.6 11.5± 1.6 (4) w/o hierarchy 0.91± 0.01 0.98± 0.01 10.5± 2.5 7.4± 1.4 GFP hard StructEvo 1.00± 0.04 1.05± 0.05 4.9± 0.3 11.4± 0.6 (1) w/o delta 0.97± 0.05 1.01± 0.05 4.7± 0.3 11.0± 1.1 (2) w/o structure 0.83± 0.04 0.89± 0.02 5.7± 1.2 11.7± 0.8 (3) w/o reference 0.84± 0.06 0.93± 0.05 5.9± 0.9 11.3± 1.2 (4) w/o hierarchy 0.75± 0.02 0.82± 0.01 13.8± 2.2 13.0± 1.8 Table 11: Full results of ablation studies. w/o: without. Appendix F Additional in-depth analysis We discuss additional ablation effects in this section, including analysis on structure prediction models (Sec. F.1), proxy model robustness (Sec. F.2), and weights for geometric constraint (Sec. F.3). F.1 Analysis on structure prediction models In this section, we assess the sensitivity of structure prediction models to point mutations on GFP mutants. We choose Protenix [22] as the representative structure prediction model for assessment. Compared to recent well-known models such as AlphaFold3, Protenix is completely open-source while achieving comparable performance across multiple structure prediction benchmarks. Specifically, we perform stratified sampling on the GFP dataset by partitioning mutant sequences into 10 bins according to their fitness values, and randomly selecting 10 mutant sequences from each bin, resulting in a total of 100 sequences. The average Hamming distance of these 100 sequences is 7.5, indicating a moderate level of mutational diversity for evaluating model sensitivity. For each sequence, we predict its structure using the default Protenix inference configuration, including MSA construction via UniRef100, 5 Pairformer cycles, and 200 diffusion steps. We adopt the protenix_base_default_v1.0.0 version and run all experiments on a single NVIDIA A800 80GB GPU. Due to the expensive computational costs, we only generate one structure sample per sequence. The entire MSA search and structure prediction process for 100 sequences takes approximately 13 hours, making full-scale structure prediction prohibitive within the RL training loop, which requires evaluating substantially more candidates. Under this setting, we evaluate two aspects. First, we examine the Pearson and Spearman correlations (implemented by scipy.stats) between three commonly used confidence metrics and fitness values: • pLDDT (predicted Local Distance Difference Test): Higher pLDDT values indicate greater confidence in local structural predictions at the residue level. • GpDE (Global predicted Distance Error): Lower distance errors indicate more reliable predictions. Therefore, we use the negative values of GpDE scores to compute correlations. • pTM (predicted TM-score): Values closer to 1 indicate higher confidence in the overall structural prediction, capturing the reliability of global structure. The results are shown in Figure 6. All three confidence metrics exhibit consistently weak correlations with fitness, indicating a poor alignment between structure prediction confidence and protein functionality. Notably, the highest observed correlation (0.29 for pLDDT-Pearson) remains below 0.3, suggesting that confidence scores are not sufficiently sensitive to the subtle structural variations induced by point mutations. Figure 6: Correlation between Confidence Scores and Fitness. To further investigate the sensitivity of predicted structures, we conduct two analyses: • Reference-based difference: We calculate the structure deviation between the 100 predicted mutant structures with the reference structure, and examine the correlation between the structural deviation and fitness value. • Pairwise difference: For each pair in the 100 predicted mutant structures (a total of 4950 pairs), we examine the correlation between structural difference and the corresponding fitness difference. The fitness difference is defined as the absolute fitness gap. In both analyses, the structural difference is measured by CαCα-RMSD (root mean square deviation) with rigid-body alignment implemented by the BioPython.Superimposer module. Figure 7: Samples of reference-based difference (left) and pairwise difference (right). The results are shown in Figure 7. We find both structural differences reveal negligible relationships with fitness values. In the reference-based setting, structural deviation from the reference shows weak negative correlations with fitness (Pearson -0.19, Spearman -0.12). In the pairwise setting, the correlation is much lower (Pearson 0.03, Spearman 0.02). The messy scatter distributions further indicate weak relationship between structural variations and functional changes. Notably, these correlations are even weaker than those of confidence scores. This suggests that current structure prediction outputs yield weak relationships to point mutations, limiting their utility for modeling fine-grained fitness landscape and guiding protein optimization. Given the high computational costs of structure prediction, a potential acceptable strategy is to periodically update the reference structure with newly predicted mutant structure, for example every T steps or when the accumulated mutation distance exceeds a threshold. However, this approach requires careful evaluation of update moments. We leave this exploration to future work. F.2 Robustness analysis of the proxy model To examine the robustness of proxy model, we conduct experiments on the GFP-hard task under several settings: • Default: The proxy model provides rewards during RL. At each round, the proxy selects top-N candidates, which are then evaluated by the oracle and used to iteratively refine the proxy. • Oracle-full: The proxy is replaced by the oracle to provide rewards and can be queried without constraint. This represents an ideal upper bound of the proxy. • Oracle-limited: The proxy is replaced by the oracle, but the number of queries is strictly limited as default setting (i.e., N per round). The results are listed below: Table 12: Proxy robustness analysis on GFP-hard task. Mean↑ Max↑ Diversity Novelty Default 1.00± 0.04 1.05± 0.05 4.9± 0.3 11.4± 0.6 Oracle-full 1.07± 0.01 1.12± 0.02 12.9± 2.9 13.1± 1.2 Oracle-limited 0.41± 0.03 0.49± 0.04 16.7± 4.2 5.8± 0.9 We observe that oracle-full achieves the best performance as an idealized upper bound setting. However, this setting is impractical in MLDE, as oracle queries correspond to costly wet-lab experiment budgets. In contrast, oracle-limited shows a significant performance drop, indicating that the RL agent collects insufficient trajectories under limited signals. This further demonstrates that the frequent rewards from the proxy model are crucial for effective evolution. We will include a more comprehensive analysis of proxy effects on different rounds and tasks in future work. F.3 Sensitivity analysis of the geometric constraint weight As introduced in Sec. 4.3, the overall RL loss consists of PPO loss and geometric constraint loss: ℒtotal=ℒpolicy+λentropy⋅ℒentropy+λvalue⋅ℒvalue⏟standard PPO loss+λgeo⋅ℒgeo⏟geometric constraint.L_total= L_policy+ _entropy·L_entropy+ _value·L_value_standard PPO loss+ _geo·L_geo_geometric constraint. (30) To investigate the effect of the geometric constraint, we vary its weight λgeo _geo and conduct experiments on GFP-hard task. Each setting is evaluated over 5 independent runs. The results are shown in Table 13. Table 13: Hyperparameter analysis of geometric constraint weight on GFP-hard task. Mean↑ Max↑ Diversity Novelty λgeo=0 _geo=0 0.984± 0.081 1.017± 0.072 5.101± 1.087 9.648± 1.795 λgeo=0.01 _geo=0.01 0.987± 0.052 1.031± 0.036 4.878± 0.312 11.427± 1.134 λgeo=0.05 _geo=0.05 (default) 0.998± 0.041 1.047± 0.046 4.868± 0.336 11.425± 0.602 λgeo=0.1 _geo=0.1 0.970± 0.041 1.010± 0.043 5.289± 0.358 10.596± 0.846 λgeo=0.5 _geo=0.5 0.925± 0.043 0.991± 0.031 6.452± 1.534 10.677± 0.785 λgeo=1 _geo=1 0.931± 0.034 0.974± 0.027 5.115± 0.970 9.706± 0.115 We observe that: (1) increasing λgeo _geo generally improves training stability, as reflected by reduced standard deviations across mean and max fitness metrics, indicating more stable optimization. (2) λgeo=0.05 _geo=0.05 achieves the best performance on fitness metrics, while both larger and smaller values degrade performance. This is likely because larger weights overly restrict policy updates, whereas smaller weights provide insufficient regularization, leading to noisier gradients and less stable optimization. (3) We found that novelty decreases with larger λgeo _geo, likely because stronger constraint limits exploration and favors more proximal candidates. Overall, we select λgeo=0.05 _geo=0.05 as the default setting reported in the main experiments. NeurIPS Paper Checklist 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: We clearly state the scope and contributions of both communities of protein optimization and computer science in the abstract and introduction. Guidelines: • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 2. Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss limitations at Sec. 6 . Guidelines: • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate “Limitations” section in their paper. • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 3. Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [N/A] Justification: The paper does not include theoretical results. Guidelines: • The answer [N/A] means that the paper does not include theoretical results. • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced. • All assumptions should be clearly stated or referenced in the statement of any theorems. • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 4. Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: The paper has disclosed all the necessary information to reproduce the main experimental results, including model architecture (Sec. 4), algorithm (Sec. A.2), and implementation details (Sec. D). Guidelines: • The answer [N/A] means that the paper does not include experiments. • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provide our code in the supplemental material. Guidelines: • The answer [N/A] means that paper does not include experiments requiring code. • Please see the NeurIPS code and data submission guidelines (https://neurips.c/public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.c/public/guides/CodeSubmissionPolicy) for more details. • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 6. Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results? Answer: [Yes] Justification: The experimental settings are presented in Sec. 5.1 and Sec. 5.2. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 7. Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: We report standard deviation on results across 5 independent runs. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates). • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We provide the needed computer resources in Sec. D. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper). 9. Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.c/public/EthicsGuidelines? Answer: [Yes] Justification: We conduct our work in accordance with the NeurIPS Code of Ethics. Guidelines: • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics. • If the authors answer [No] , they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The societal impacts are in Sec. 6. Guidelines: • The answer [N/A] means that there is no societal impact of the work performed. • If the authors answer [N/A] or [No] , they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 11. Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)? Answer: [N/A] Justification: We do not release a high-risk pretrained generative model; generated candidates require oracle evaluation, wet-lab validation, and human inspection before use. Guidelines: • The answer [N/A] means that the paper poses no such risks. • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We cite the original paper and clearly state assets details at Appendix C.1 and C.2. Guidelines: • The answer [N/A] means that the paper does not use existing assets. • The authors should cite the original paper that produced the code package or dataset. • The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., C-BY 4.0) should be included for each asset. • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset’s creators. 13. New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [N/A] Justification: This paper does not release new assets. Guidelines: • The answer [N/A] means that the paper does not release new assets. • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 14. Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [N/A] Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines: • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [N/A] Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines: • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 16. Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required. Answer: [N/A] Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components. Guidelines: • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components. • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.