Paper deep dive
MFG: Multi-turn Feedback Guided Molecular Optimization
Trieu Nguyen, Frazier N. Baker, Ruoxi Gao, Daniel Adu-Ampratwum, Xia Ning
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-objective molecular optimization is a critical process in drug discovery that aims to modify a given molecule to simultaneously improve multiple properties while maintaining a similar overall structure. This process often requires iterative modifications guided by computational or experimental feedback. Recent large language model (LLM)-based methods have begun to explore this iterative approach; however, these methods are not explicitly trained to leverage such feedback, while existing approaches primarily focus on single-turn molecular optimization. To address this gap, we introduce Multi-turn Feedback Guided (MFG) molecular optimization, a reinforcement learning framework that trains an LLM to progressively optimize molecules using feedback from previous turns. MFG introduces a feedback vector capturing both property improvements and source molecule similarity, alongside a feedback-aligned reward function that promotes progressive improvement across turns. We apply MFG over $\text{GeLLM}^{3}\text{O}$, a state-of-the-art LLM model for multi-objective molecular optimization, evaluate $\text{GeLLM}^{3}\text{O-MFG}$ on ten multi-objective molecular optimization tasks. The results demonstrate that $\text{GeLLM}^{3}\text{O-MFG}$ achieves the highest average similarity-constrained success rate ($81.50%$), outperforming the base model $\text{GeLLM}^{3}\text{O}$ ($70.81%$).
Tags
Links
- Source: https://arxiv.org/abs/2610.18851v1
- Canonical: https://arxiv.org/abs/2610.18851v1
Trouble viewing inline? Open PDF directly →
Full Text
64,711 characters extracted from source content.
Expand or collapse full text
MFG: Multi-turn Feedback Guided Molecular Optimization Trieu Nguyen Affiliation: Department of Computer Science and Engineering, The Ohio State University Frazier N. Baker Affiliation: Department of Computer Science and Engineering, The Ohio State University Ruoxi Gao Affiliation: Department of Computer Science and Engineering, The Ohio State University Daniel Adu-Ampratwum Affiliation: Division of Medicinal Chemistry and Pharmacognosy, The Ohio State University Correspondence: ning.104@osu.edu Xia Ning Affiliation: Department of Computer Science and Engineering, The Ohio State University Affiliation: Department of Biomedical Informatics, The Ohio State University Affiliation: Translational Data Analytics Institute, The Ohio State University Affiliation: Division of Medicinal Chemistry and Pharmacognosy, The Ohio State University Correspondence: ning.104@osu.edu Abstract Multi-objective molecular optimization is a critical process in drug discovery that aims to modify a given molecule to simultaneously improve multiple properties while maintaining a similar overall structure. This process often requires iterative modifications guided by computational or experimental feedback. Recent large language model (LLM)-based methods have begun to explore this iterative approach; however, these methods are not explicitly trained to leverage such feedback, while existing approaches primarily focus on single-turn molecular optimization. To address this gap, we introduce Multi-turn Feedback Guided (MFG) molecular optimization, a reinforcement learning framework that trains an LLM to progressively optimize molecules using feedback from previous turns. MFG introduces a feedback vector capturing both property improvements and source molecule similarity, alongside a feedback-aligned reward function that promotes progressive improvement across turns. We apply MFG over GeLLM3OGeLLM^3O, a state-of-the-art LLM model for multi-objective molecular optimization, evaluate GeLLM3O-MFGGeLLM^3O-MFG on ten multi-objective molecular optimization tasks. The results demonstrate that GeLLM3O-MFGGeLLM^3O-MFG achieves the highest average similarity-constrained success rate (81.50%), outperforming the base model GeLLM3OGeLLM^3O (70.81%70.81\%). 1 Introduction Multi-objective molecular optimization (MOMO) Sun et al. (2022) is a critical process in drug discovery Hughes et al. (2011) that aims to modify a given molecule to simultaneously improve multiple properties while maintaining a similar overall structure. Such improvements are necessary because even promising molecules may suffer from poor solubility, insufficient selectivity, or toxicity that hinders their development into viable drugs. Achieving these improvements often requires iterative structural modifications to balance multiple, potentially competing molecular properties. By iteratively reasoning on how to modify molecules, chemists can rely on not only their expert knowledge, but also computational Nicolaou and Brown (2013) or experimental feedback. Large language models (LLMs) have shown promise on similar iterative tasks through multi-turn reasoning Naveed et al. (2025), providing a promising mechanism to mimic this approach to MOMO. Recent efforts Ye et al. (2025); Zuo et al. (2025) have used LLMs for MOMO. Several methods Liu et al. (2023); Le et al. (2024) incorporated iterative feedback at inference-time, prompting LLMs to modify molecules over multiple turns while incorporating feedback on the modified molecules of previous turns. However, these methods are not trained on how to leverage such feedback. Instead, they rely only on the LLMs’ general-purpose pretraining and basic reasoning capabilities. Other methods have used supervised finetuning methods Dey et al. (2025) and reinforcement learning (RL) Liu et al. (2025) to train models for MOMO. However, training is limited to non-iterative, single-turn generation, with no specialized training on multi-turn feedback. Here, we introduce MFG, a Multi-turn Feedback Guided framework for MOMO. MFG formulates MOMO as a multi-turn problem with feedback, using RL to train an LLM to iteratively optimize molecules by leveraging feedback from previous turns. MFG introduces a feedback vector that captures both property improvements and similarity to the given source molecule at each turn, providing dynamically updated feedback to guide the LLM policy throughout the optimization process. MFG’s RL training is guided by a reward function that aligns with this feedback while also encouraging progressive improvement across turns. During inference, MFG can use its trained model to generate multiple candidate molecules at each turn and select the best molecule for further turns, carefully balancing property improvement and similarity. Together, MFG serves as a powerful new framework for MOMO with feedback. To evaluate MFG, we instantiate GeLLM3O-MFG, using GeLLM3O, a state-of-the-art molecular optimization LLM, as the base model for MFG’s RL training. We compare GeLLM3O-MFG against GeLLM3O and other existing molecular optimization baselines on the MuMOInstruct dataset Dey et al. (2025), a benchmark featuring diverse property combinations and both in-distribution and out-of-distribution MOMO tasks. Across all tasks, GeLLM3O-MFG achieves the highest average similarity-constrained success rate (81.50%), outperforming the best baseline (BioMedGPT-Mol, 77.81%77.81\%) and GeLLM3O (70.81%70.81\%). An ablation study shows that multi-turn feedback and RL training each contribute a distinct advantage to MFG. Additionally, a case study illustrates how GeLLM3O-MFG adapts its optimization strategy based on feedback. GeLLM3O-MFG is publicly available at https://anonymous.4open.science/r/MFG-85BD/. 2 Related Work RL for Molecular optimization Existing molecular optimization LLMs such as GeLLM3O Dey et al. (2025) have explored instruction tuning of general-purpose LLMs on large-scale MOMO MOMO tasks (see Appendix A.1 for a summary of more related work). However, this supervised learning approach may limit model’s ability to explore chemical space beyond the distribution of training examples. Recent works have leveraged RL to further align molecular optimization LLMs with desired molecule properties. For example, DrugImproverGPT Liu et al. (2025) employed reward-based RL to explore molecule modifications for improving multiple molecule properties. However, the sparsity of high-reward molecules in chemical space can make effective exploration challenging. To address this challenge, RePO Li et al. (2026) leveraged reference optimized molecules to provide LLMs with templates of successful optimization. Despite different designs of RL methods, existing works primarily optimize source molecules in a single turn, treating each optimization as an independent process. Our work considers a new setting in which molecules are iteratively optimized over multiple turns, and develops a novel multi-turn training framework for this setting. To the best of our knowledge, our work is the first to improve molecular optimization LLMs through multi-turn RL. 3 MFG Framework Figure 1: Overall figure of Multi-turn Feedback Guided (MFG) MOMO MOMO . 3.1 Problem Formulation Multi-objective molecular optimization (MOMO MOMO ) seeks to modify a source molecule C0 C_0 into a structurally similar molecule C while improving m properties =(1,2,⋯,m) $ P $=( $ p $_1, $ p $_2,·s, $ p $_m). Existing LLM-based approaches commonly formulate this task as a single-turn problem, where an LLM directly generates C in a single response when prompted with C0 C_0 and P . However, this single-turn approach offers limited opportunity to adjust the optimization strategy. MOMO can be very challenging, as structural changes that improve one property may degrade another, and improvement patterns may not remain consistent across different C0 C_0 . To address this challenge, we reformulate MOMO MOMO as a multi-turn problem with feedback, where an LLM iteratively generates optimized molecules for a given C0 C_0 and P , adjusting its strategies based on previous turn’s property change and similarity to C0 C_0 . Under this formulation, we consider the LLM with parameters θ as a policy πθ _θ that decides how to generate a molecule CtC_t at any turn t given C0 C_0 , P , and the sequence of previous turns up to t−1t-1, denoted as 1:t−1 $ τ $_1:t-1. Each turn t $ τ $_t is defined as t=(Ct,t) $ τ $_t=(C_t, $ f $_t), where t∈ℝm+1 $ f $_t ^m+1 is a feedback vector measuring the quality of CtC_t, with the first m entries indicating degradation or improvement in each property objective ∈ $ p $∈ $ P $, and the final entry representing the similarity to C0 C_0 . This provides πθ _θ with valuable insight into each specific MOMO MOMO problem, allowing it to adjust its molecular optimization strategy through trial-and-error. An optimal πθ _θ leverages this feedback across all turns to maximize the average improvement on the property set P and maintain strong similarity to C0 C_0 . To obtain such a policy, we formulate the following optimization problem: maxπθt,Ct∼πθ(⋅|C0,,1:t−1)[R(Ct)], _ _θE_t,C_t _θ(·| $ C_0 $, $ P $, $ τ $_1:t-1) [R(C_t) ], (1) where t∈1,…,Tt∈\1,…,T\ is the number of turns up to maximum T and R(Ct)R(C_t) is a reward function specifically designed for MOMO MOMO (defined in Section 3.2.2). This formulation distributes the optimization of multiple property objectives across turns, with reward signals guiding progressive improvement throughout the process. An overview of MFG is illustrated in Figure 1. 3.2 MFG Training MFG trains πθ _θ to leverage feedback for multi-turn MOMO MOMO by maximizing an expected reward R(Ct)R(C_t) (Section 3.2.2). Specifically, we use group-relative policy optimization (GRPO) Shao et al. (2024) to train πθ _θ by sampling multiple candidate molecules CtC_t at each turn t conditioned on C0C_0, P, and τ1:t−1 _1:t-1, and updating πθ _θ based on their relative rewards. To optimize the expectation over t in Eq. 1, MFG samples t∼Uniform(1,T)t (1,T) during training, ensuring that πθ _θ is trained to leverage feedback at every turn in multi-turn MOMO. MFG employs on-policy training, where the previous turns τ1:t−1 _1:t-1 are generated using the same πθ _θ that generates CtC_t. However, MFG treats τ1:t−1 _1:t-1 as fixed conditioning context rather than optimizing its generation based on R(Ct)R(C_t). Together, these design choices allow πθ _θ to learn effective feedback-guided molecular optimization strategies by comparing candidate molecules generated under the same conditioning context. As πθ _θ is updated during training, the conditioning of previous turns τ1:t−1 _1:t-1, which are generated on-policy and used as conditioning context, also reflect the policy’s changing output distribution. 3.2.1 Candidate Advancement Criterion To encourage progressive improvements across turns, MFG defines a candidate selection criterion Advance(⋅)Advance(·) with two purposes: (1) during training, identifying the best molecule from previous turns as a reference for progressive improvement; and (2) during inference, selecting a promising molecule to carry forward, along with its feedback, to guide subsequent optimization. Specifically, Advance(⋅)Advance(·) is a hierarchical selection criterion that balances improvements across multiple target properties with structural similarity to C0 C_0 through prioritized comparisons. It prioritizes molecules satisfying a predefined similarity threshold δ to C0 C_0 , followed by molecules that improve all target properties. When no candidate improves all properties, those improving more properties are preferred, preventing large gains in one property from outweighing broader improvements. Together, Advance(⋅)Advance(·) prioritizes the breadth of property improvements over their magnitude while preserving structural similarity (see Appendix A.2 for details). 3.2.2 Reward Function MFG defines a reward function R(Ct∣C0,,Ct−1∗)R(C_t C_0, $ P $,C_t-1^*) that encourages property improvement, structural similarity to C0C_0, and progressive improvements over the best molecule from the previous turns. The goal is to prioritize structural similarity as a prerequisite for rewarding property improvements, while encouraging the model to progressively generate better molecules across turns. Specifically, R(Ct)R(C_t) (with conditioning omitted for simplicity) is defined as: R(Ct)=s−2,if s≤δ,Rbase(Ct)+Rbest(Ct),otherwise,R(C_t)= casess-2,&if s≤δ,\\ R_base(C_t)+R_best(C_t),&otherwise, cases (2) where s=Sim(Ct,C0)s=Sim(C_t,C_0) denotes the Tanimoto Bajusz et al. (2015) similarity between CtC_t and C0C_0. As shown in Eq 2, when s≤δs≤δ, R(Ct)R(C_t) assigns a similarity-dependent penalty, without accounting for property improvements . This prevents large property gains from compensating for excessive structural deviations. Moreover, this penalty complements our algorithm choice, GRPO, which updates the policy based on relative rewards among candidates in a group. Even when all candidates within a group breach the similarity threshold, those with higher similarity to C0 C_0 receive higher rewards, providing a learning signal that encourages the model to generate more structurally similar molecules. For molecules that meet the similarity threshold, we introduce Rbase(⋅)R_base(·) to incentivize both property improvements and similarity to C0C_0, and is defined as follows: Rbase(Ct)=λ(Ct)((Ct)+(Ct)).R_base(C_t)=λ(C_t) (G(C_t)+S(C_t) ). (3) Here, (⋅)G(·) and (⋅)S(·) measure the average property improvement on P and the similarity to C0 C_0 , respectively, each normalized by the corresponding value of the reference optimized molecule (details in Section 4.2). The normalization addresses differences in the numerical scales of different properties and encourages πθ _θ to generate molecules outperforming the reference molecule. λ(Ct)∈[1,2)λ(C_t)∈[1,2) amplifies the reward only when CtC_t improves all target properties over C0C_0, with a larger bonus for higher structural similarity. Unlike (⋅)G(·), which measures average property improvement, λ(⋅)λ(·) encourages simultaneous improvement across competing properties rather than large gains in a few at the expense of others. Details of (⋅)G(·), (⋅)S(·), and λ(⋅)λ(·) are provided in Appendix A.3. In addition to evaluating CtC_t with respect to C0C_0, Rbest(⋅)R_best(·) in Eq 2 compares CtC_t with the best molecule Ct−1∗C^*_t-1 from the previous t−1t-1 turns, identified using Advance(⋅)Advance(·) described in Section 3.2.1. Rbest(⋅)R_best(·) is defined as follows: Rbest(Ct)=[Rbase(Ct)−Rbase(Ct−1∗)]+,R_best(C_t)= [R_base(C_t)-R_base(C_t-1^*) ]_+, (4) where [⋅]+=max(⋅,0)[·]_+= (·,0) denotes the positive-part operator. This sub-reward provides a bonus when CtC_t achieves a higher base reward than Ct−1∗C_t-1^*, encouraging progressive improvement across turns. 3.3 MFG Inference At inference time, MFG performs iterative molecular optimization using the trained policy πθ _θ, conditioned on the source molecule C0 C_0 , target property set P , initially empty set of previous turns τ. At each turn t, MFG samples N candidates and applies the selection criterion Advance(⋅)Advance(·) introduced in Section 3.2.1 to obtain Ct=Advance(Ct1,…,CtN)C_t=Advance(\C_t^1,…,C_t^N\). The selected molecule and its feedback vector tf_t, capturing property changes and structural similarity relative to C0 C_0 , form the updated τ1:t=τ1:t−1∪(Ct,t) _1:t= _1:t-1∪\(C_t,f_t)\. This process repeats until turn T, with each selected molecule and its feedback incorporated into the conditioning context for the next turn. The final optimized molecule CTC_T is then selected from all T×NT× N generated candidates according to the criteria described in Section 4.3. Unlike the final output selection, Advance(⋅)Advance(·) contributes to feedback construction during inference by determining which intermediate molecule and its associated feedback are carried forward to subsequent turns. This selection aims to carry forward a promising intermediate molecule that provides a meaningful direction for subsequent optimization, allowing future turns to build on partial progress. Thus, Advance(⋅)Advance(·) supports feedback-guided MOMO by limiting the propagation of unfavorable candidates and retaining informative feedback that helps the model adapt across turns. 4 Experimental Settings 4.1 GeLLM3O-MFG We use MFG to train GeLLM3O-MFG, a multi-turn MOMO MOMO model based on GeLLM3O-P(6)Llama, a state-of-the-art single-turn MOMO MOMO model. GeLLM3O-P(6)Llama is particularly suitable for training with MFG because its previous training on single-turn MOMO MOMO has allowed it to acquire substantial knowledge of structure-property relationships. Implementation details of GeLLM3O-MFG are provided in Appendix A.4. 4.2 Dataset GeLLM3O-MFG is trained and evaluated using the MuMOInstruct benchmark dataset following the training/testing split established by GeLLM3O. MuMOInstruct includes multi-property optimization tasks on combinations of six target properties: blood-brain barrier permeability (B), dopamine receptor D2 binding affinity (D), human intestinal absorption (H), mutagenicity (M), penalized logP, a measure of lipophilicity (P), and quantitative estimate of drug-likeness (Q). The optimization objective is to increase the values of B, D, H, P, and Q, while decreasing that of M. Each task consists of a set of source molecules, paired with their corresponding reference optimized molecules that we use for reward normalization in Eq 3. Specifically, the property improvements achieved by the reference optimized molecule relative to the source molecule are used to normalize (⋅)G(·), while its structural similarity to the source molecule is used to normalize (⋅)S(·) (details in Appendix A.3). The evaluation set consists of five in-distribution (IND) tasks (BDP, BDQ, BPQ, DPQ, and BDPQ) and five out-of-distribution (OOD) tasks (MPQ, BDMQ, BHMQ, BMPQ, and HMPQ). IND tasks evaluate the model’s optimization capability on property combinations seen during training, while OOD tasks assess its zero-shot generalization to unseen property combinations. 4.3 Evaluation Metrics Following GeLLM3O, we evaluate optimization performance using three complementary metrics: (1) Success Rate (SRα SR_α), which is the proportion of source molecules successfully optimized to improve all property objectives while maintaining Tanimoto similarity of at least α to the source molecule; (2) Similarity (Sim) that is the average Tanimoto similarity between the optimized molecules and their corresponding source molecules; and (3) Relative Improvement (RI) measuring the average relative change across the target properties with respect to the source molecules (Appendix A.5). Higher SRα SR_α, Sim and RI are desirable, indicating better overall optimization performance. For each source molecule C0 C_0 , we select the optimized molecule C as the candidate with the highest RI among those that improve all target properties and satisfy the similarity threshold α. If no such candidate exists, the optimization is considered unsuccessful. 4.4 Baselines We evaluate GeLLM3O-MFG against the base model GeLLM3O-P(6)Llama, and two categories of LLMs with different levels of chemistry-specific training: general-purpose LLMs and LLMs for chemistry which are trained on large-scale chemistry instruction tasks. We use GeLLM3O-P(6)Llama to directly assess the improvement introduced by MFG while keeping the underlying model architecture fixed. In terms of general-purpose LLMs, we select two of the largest and most capable open-weight models available to us at the time of evaluation, Qwen3.5-9B (Qwen Team, 2026) and Ministral-3-8B (Liu et al., 2026). They serve as references for molecular optimization without adaptation to the chemistry domain. We choose LlaSMol (Yu et al., 2024) and BioMedGPT-Mol (Zuo et al., 2025) as LLMs for chemistry baselines due to their reported strong performance on MOMO MOMO tasks. For fair comparison, all baselines use identical inputs, inference settings (Section 4.5), and output processing procedures, with model and prompt details provided in Appendix A.6. 4.5 Inference Settings We evaluate all methods under both single-turn and multi-turn settings, denoted as ST and MT, respectively. For baselines originally designed for ST optimization, we extend them to the MT setting by applying iterative inference with the same feedback vector and selection criterion as MFG. This allows us to separate the contribution of MFG inference from MFG training. Following the evaluation protocol of GeLLM3O, we generate a maximum of 20 molecules throughout the optimization process for each source molecule. For T-turn optimization, we generate an equal number of candidates at each turn. To ensure a comprehensive evaluation, we independently evaluate each method with T∈1,2,3,4T∈\1,2,3,4\ turns. We report the results corresponding to the highest SR across the evaluated values of T for each task. The full results for each value of T are provided in Appendix A.7. 5 Results 5.1 Performance on MOMO MOMO tasks 5.1.1 Overall comparison with baselines Table 1: Overall Performance on IND Tasks Model Setting BDP BDQ BPQ DPQ BDPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI General-purpose LLMs Qwen3.5-9B ST 32.80 0.76 0.98 18.80 0.73 0.54 90.40 0.75 0.58 15.60 0.72 0.51 11.00 0.75 0.54 MT 36.40 0.72 0.95 21.60 0.70 0.60 90.40 0.75 0.58 19.80 0.69 0.36 12.20 0.76 0.51 Ministral-3-8B ST 27.60 0.71 0.74 24.00 0.67 0.45 64.80 0.68 0.52 12.80 0.68 0.28 8.60 0.72 0.56 MT 37.60 0.68 0.70 24.60 0.66 0.69 66.00 0.65 0.65 13.60 0.68 0.36 11.60 0.70 0.47 LLMs for Chemistry LlaSMol ST 5.00 0.65 0.66 1.60 0.56 0.79 27.40 0.65 0.66 4.40 0.66 0.37 2.40 0.73 0.42 MT 6.60 0.62 0.96 1.60 0.56 0.79 30.00 0.62 0.72 5.00 0.63 0.37 3.60 0.64 0.48 BioMedGPT-Mol ST 69.20 0.62 2.82 79.00 0.62 4.40 97.20 0.61 1.23 47.80 0.63 1.58 38.00 0.62 2.03 MT 74.20 0.58 3.36 79.00 0.62 4.40 97.20 0.61 1.23 47.80 0.63 1.58 43.00 0.59 1.88 GeLLM3O GeLLM3O ST 66.00 0.62 2.77 65.00 0.64 3.88 84.40 0.60 1.28 43.80 0.61 1.84 39.00 0.60 2.26 MT 66.00 0.62 2.77 71.40 0.60 4.37 84.40 0.60 1.28 43.80 0.61 1.84 39.00 0.60 2.26 GeLLM3O-MFG ST 83.20 0.62 3.62 73.00 0.63 4.30 89.20 0.62 1.25 48.40 0.63 1.97 44.00 0.61 2.50 MT 84.20 0.61 4.24 86.20 0.61 5.32 91.20 0.60 1.32 54.60 0.60 2.49 53.40 0.59 3.19 • ST denotes the original single-turn setting and MT denotes the multi-turn setting. ↑ denotes that a higher value of the metric is desirable. For each metric, the best performance is in bold and the second-best performance is underlined. Tables 1 and 2 present a comprehensive comparison between GeLLM3O-MFG and all the baselines on IND and OOD MOMO MOMO tasks, respectively. Overall, GeLLM3O-MFG achieves the highest SR0.5 SR_ 0.70.5 on 6 of the 10 tasks and highest RI on 9 tasks. Specifically, it achieves higher average SR0.5 SR_ 0.70.5 (81.50%81.50\% vs. 77.81%77.81\%) and RI (2.632.63 vs. 2.082.08) than the strongest baseline, BioMedGPT-Mol. Given that the inference settings are consistent across all methods, the superior performance of GeLLM3O-MFG can be attributed to MFG’s training design, rather than the inference strategy alone. GeLLM3O-MFG also maintains an average Sim of 0.600.60, matching GeLLM3O (0.600.60) and BioMedGPT-Mol (0.600.60), indicating that stronger property optimization does not compromise average structural similarity. This further indicates that the similarity-sensitive reward assist the optimization process in preserving desired molecular structures. Table 2: Overall Performance on OOD Tasks Model Setting MPQ BDMQ BHMQ BMPQ HMPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI General-purpose LLMs Qwen3.5-9B ST 75.80 0.69 0.41 17.00 0.76 0.41 72.03 0.73 1.03 75.39 0.71 0.47 81.25 0.73 0.54 MT 75.80 0.69 0.41 23.40 0.71 0.43 77.12 0.69 1.03 75.39 0.71 0.47 81.25 0.69 0.68 Ministral-3-8B ST 47.80 0.65 0.37 15.40 0.66 0.52 51.69 0.66 1.07 47.12 0.65 0.47 34.38 0.67 0.59 MT 49.60 0.64 0.39 23.20 0.64 0.76 57.63 0.63 1.03 56.54 0.65 0.49 40.62 0.63 0.69 LLMs for Chemistry LlaSMol ST 17.40 0.61 0.38 0.20 0.67 0.0 15.25 0.62 1.78 15.71 0.61 0.51 31.25 0.63 0.69 MT 18.20 0.60 0.41 0.80 0.58 0.68 20.34 0.63 1.02 18.32 0.64 0.50 31.25 0.63 0.69 BioMedGPT-Mol ST 94.40 0.60 0.74 65.60 0.61 2.48 88.98 0.62 2.25 91.10 0.60 0.87 91.67 0.62 1.14 MT 94.40 0.60 0.74 65.60 0.61 2.48 91.53 0.59 2.53 91.62 0.60 0.92 93.75 0.59 1.64 GeLLM3O GeLLM3O ST 83.40 0.60 0.76 64.40 0.62 2.66 82.20 0.60 2.81 84.82 0.59 0.96 87.50 0.59 1.51 MT 83.40 0.60 0.76 65.60 0.59 3.06 82.20 0.60 2.81 84.82 0.59 0.96 87.50 0.59 1.51 GeLLM3O-MFG ST 88.40 0.60 0.78 80.20 0.62 2.77 80.51 0.60 2.74 86.91 0.62 0.92 88.54 0.60 1.37 MT 88.40 0.60 0.78 83.60 0.61 3.50 90.68 0.57 2.87 89.01 0.60 0.99 93.75 0.58 1.62 • The metrics, notations and formatting have the same meanings as those in Table 1. 5.1.2 Comparison with GeLLM3O As shown in Tables 1 and 2, GeLLM3O-MFG consistently outperforms its base model, GeLLM3O, across all IND and OOD tasks. Under the MT setting, GeLLM3O-MFG outperforms GeLLM3O across all IND and OOD tasks, achieving an average increase of 10.69%10.69\% in SR0.5 SR_ 0.70.5 and 0.47 in RI. Moreover, the gap between ST and MT performance is substantially larger for GeLLM3O-MFG than for GeLLM3O. Moving from ST to MT increases GeLLM3O-MFG’s average SR0.5 SR_ 0.70.5 by 5.27% (compared with only 0.76 for GeLLM3O) and improves RI on 9 tasks (compared with only 2 for GeLLM3O), while decreasing average Sim by only 0.02. As both models share the same backbone and are evaluated under identical inference settings, the larger gains from ST to MT setting suggest that MFG’s training better equips GeLLM3O-MFG to leverage intermediate molecules and their feedback to guide subsequent optimization. These findings are also consistent with MFG’s reward design, which promotes progressive improvement over previously generated molecules. Table 3: Ablation Study Model Setting BDP BDQ BPQ DPQ BDPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI GeLLM3O-RL ST 79.00 0.61 3.67 70.60 0.63 4.94 90.60 0.60 1.31 47.20 0.62 2.00 41.80 0.62 2.28 MT 79.40 0.58 4.18 80.60 0.59 5.67 90.60 0.60 1.31 47.20 0.62 2.00 49.20 0.59 3.14 GeLLM3O-MFG ST 83.20 0.62 3.62 73.00 0.63 4.30 89.20 0.62 1.25 48.40 0.63 1.97 44.00 0.61 2.50 MT 84.20 0.61 4.24 86.20 0.61 5.32 91.20 0.60 1.32 54.60 0.60 2.49 53.40 0.59 3.19 SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI GeLLM3O-RL ST 64.60 0.67 2.96 59.60 0.67 4.03 79.60 0.67 1.06 32.80 0.67 1.58 28.00 0.69 1.25 MT 64.60 0.67 3.17 64.00 0.67 4.34 79.60 0.67 1.08 32.80 0.67 1.93 30.20 0.69 1.90 GeLLM3O-MFG ST 68.60 0.67 2.90 61.20 0.67 3.65 78.60 0.67 1.03 35.60 0.68 1.43 29.60 0.68 1.65 MT 69.20 0.67 3.43 69.40 0.67 4.56 78.60 0.67 1.09 37.60 0.68 1.80 32.40 0.68 2.00 Model Setting MPQ BDMQ BHMQ BMPQ HMPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI GeLLM3O-RL ST 88.80 0.59 0.83 69.80 0.62 3.01 88.14 0.59 2.87 90.05 0.61 0.99 92.71 0.59 1.44 MT 88.80 0.59 0.83 81.60 0.59 3.77 88.98 0.58 3.12 90.05 0.61 0.99 95.83 0.56 1.75 GeLLM3O-MFG ST 88.40 0.60 0.78 80.20 0.62 2.77 80.51 0.60 2.74 86.91 0.62 0.92 88.54 0.60 1.37 MT 88.40 0.60 0.78 83.60 0.61 3.50 90.68 0.57 2.87 89.01 0.60 0.99 93.75 0.58 1.62 SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI SR0.6↑ SR_0.6 Sim↑ Sim RI↑ RI GeLLM3O-RL ST 71.20 0.66 0.72 58.00 0.66 2.15 68.64 0.67 2.25 71.20 0.67 0.83 72.92 0.65 1.25 MT 71.20 0.66 0.73 62.60 0.66 2.69 68.64 0.67 2.28 71.20 0.67 0.90 72.92 0.65 1.25 GeLLM3O-MFG ST 75.20 0.66 0.67 68.80 0.67 1.97 63.56 0.68 2.36 71.20 0.68 0.78 71.88 0.67 1.13 MT 75.20 0.66 0.74 68.80 0.67 2.64 68.64 0.68 2.36 72.77 0.68 0.86 71.88 0.67 1.14 • The metrics, notations and formatting have the same meanings as those in Table 1. 5.1.3 Comparison with other LLM baselines Compared with LlaSMol, GeLLM3O-MFG achieves vastly higher SR0.5 SR_ 0.70.5 and RI across all tasks. Under MT, LlaSMol averages 13.57%13.57\% and 0.660.66 for SR0.5 SR_ 0.70.5 and RI, whereas GeLLM3O-MFG achieves 81.50%81.50\% and 2.632.63, respectively. This performance gap suggests that training on a broad collection of chemistry tasks alone may not develop strong molecular optimization capabilities or the ability to leverage property feedback and adapt optimization strategies accordingly. GeLLM3O-MFG also substantially outperforms the general-purpose LLM baselines, achieving an average SR0.5 SR_ 0.70.5 of 81.50%81.50\% under MT setting compared with 51.34%51.34\% for Qwen3.5-9B, the strongest general-purpose LLM baseline. Moreover, GeLLM3O-MFG consistently achieves higher RI across all tasks, despite maintaining slightly lower but still substantial similarity to the source molecules. These results reinforce the importance of chemistry-specific or even task-specific training for developing strong MOMO MOMO capabilities. 5.2 Ablation Study To investigate the contribution of MT feedback-guided training beyond single-turn RL training, we compare GeLLM3O-MFG with GeLLM3O-RL. Both models are initialized from GeLLM3O and trained using GRPO under identical settings. However, GeLLM3O-RL uses ST training (T == 11) without intermediate feedback. Because there are no intermediate molecules in this setting, we only maximize Rbase(⋅)R_base(·) (Eq. 3) during GeLLM3O-RL’s training. To better study the differences between GeLLM3O-MFG and GeLLM3O-RL in terms of similarity-constrained optimization, we evaluate SR under two different similarity thresholds: 0.5 and 0.6, denoted as SR0.5 SR_ 0.70.5 and SR0.6 SR_ 0.70.6, respectively. Figure 2: An optimization case on BDPQ with T=2T=2. (a) The source molecule. (b) The best optimized molecule at turn t=1t=1. (c) The best optimized molecule at turn t=2t=2. The modified fragments are highlighted. 5.2.1 Overall comparison of GeLLM3O-MFG and GeLLM3O-RL As shown in Table 3, ST RL training alone leads to notable improvements in MOMO performance. Under ST inference, GeLLM3O-RL achieves a higher average SR0.5 SR_ 0.70.5 than the base model GeLLM3O (75.87%75.87\% vs. 70.05%70.05\%). Given that both models share the same backbone and inference setting, these results highlight the effectiveness of our reward function in guiding RL training. Specifically, as GeLLM3O-RL was trained with ST setting, it indicates that the RbaseR_base component can provide a strong learning signal for MOMO MOMO . Building on this improvement, GeLLM3O-MFG further outperforms GeLLM3O-RL under MT inference, achieving higher average SR0.5 SR_ 0.70.5 (81.50%81.50\% vs. 79.23%79.23\%) and SR0.6 SR_ 0.70.6 (64.45%64.45\% vs. 61.78%61.78\%). Moreover, moving from ST to MT increases SR0.5 SR_ 0.70.5 and SR0.6 SR_ 0.70.6 by approximately 5.275.27 and 2.03%2.03\% for GeLLM3O-MFG, compared with smaller gains of 3.363.36 and 1.121.12 for GeLLM3O-RL, respectively. As both models were trained with GRPO and follow identical inference setting, these larger gains highlight the additional benefits of MFG’s multi-turn training, which incorporates intermediate feedback and the RbestR_best reward component to encourage progressive improvements. Together, these findings suggest that while RbaseR_base provides a useful foundation for RL-based molecular optimization, MFG’s multi-turn training design further enhances the model’s ability to leverage feedback and progressively refine molecules across turns. 5.2.2 Performance under different similarity thresholds The advantage of MFG becomes more pronounced under a more stringent similarity threshold α=0.6α=0.6 as demonstrated in Table 3. In the ST setting, GeLLM3O-MFG improves over GeLLM3O-RL by only 0.37%0.37\% on average at SR0.5 SR_ 0.70.5, but by 1.77%1.77\% at SR0.6 SR_ 0.70.6. Under the MT setting, the corresponding advantage increases from approximately 2.28%2.28\% at SR0.5 SR_ 0.70.5 to 2.67%2.67\% at SR0.6 SR_ 0.70.6. Both models use (⋅)S(·) (Eq. 3) in their reward function to encourage structural similarity, thus the larger performance gains under stricter similarity constraints suggest that MFG’s multi-turn training enables the GeLLM3O-MFG to better leverage intermediate feedback to adjust its optimization strategies. The higher success rates also reflect GeLLM3O-MFG’s ability to better balance property improvement and structural similarity preservation. 5.3 Case Study Figure 2 shows a successful two-turn optimization by GeLLM3O-MFG on four properties: B, D, P, and Q. In this optimization, the first turn substantially improves B, P, and Q, while the second turn focuses on improving D. The source molecule (Figure 2a) contains multiple polyphenolic esters that contribute to low lipophilicity and high molecular weight, both of which reduce B and Q. In the first turn, GeLLM3O-MFG reduces polarity by hydrolyzing one phenolic ester and replacing another with a 1,1,1-trifluoropropyl ester. These modifications also reduce the molecular weight from 718 to 461, which improves Q by aligning better with molecular weight guidelines for drug-like molecules, such as Lipinski’s Rule of Five Lipinski et al. (2001). However, the changes introduced in this turn change very little to improve the molecule’s predicted DRD2 activity, increasing from 0.03 to 0.05. Based on this feedback, GeLLM3O-MFG prioritizes D in the second turn, replacing the free dihydrobenzofuran carboxylic acid with a 4,4-difluoropiperidinyl group and hydrolyzing the 1,1,1-trifluoropropyl ester. This produces a substantial improvement on D, while also further improving B, P, and Q. The improvement on D may be driven in part by the introduction of a cyclic amine, which helps the molecule better resemble patterns in established drugs targeting DRD2. Taken together, this multi-turn optimization example illustrates how GeLLM3O-MFG shifts priorities across turns based on feedback, resulting in higher-quality candidate molecules after multiple turns. 6 Conclusion In this work, we introduce MFG for molecular optimization, a multi-turn RL framework that trains LLMs to iteratively optimize molecules using previously generated candidates and their property feedback. By training a SOTA molecular optimization LLM, GeLLM3O, with MFG, we develop GeLLM3O-MFG. Across 10 MOMO MOMO tasks, GeLLM3O-MFG outperforms GeLLM3O and achieves competitive performance against existing baselines, improving molecular properties while preserving structural similarity. Extensive analyses demonstrate MFG’s effectiveness in leveraging multi-turn feedback for progressive molecular optimization. Together, these findings highlight the advantages of MFG in training LLMs to adapt their molecular optimization strategies across turns. Acknowledgments This project was made possible, in part, by support from the National Science Foundation grant no. IIS-2435819 (X.N.), the National Library of Medicine grant no. 1R01LM014385 (X.N.), and the Sanofi iDEA-TECH Awards North America (X.N.). Any opinions, findings, conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the funding agency. The authors would like to thank Benjamin Burns for their proofreading of the manuscript. Limitations Despite the strong performance of MFG in multi-objective molecular optimization, several limitations remain. (1) Our experiments are conducted on MuMOInstruct, which includes MOMO MOMO tasks involving six target properties. Although GeLLM3O-MFG demonstrates strong generalization to out-of-distribution (OOD) tasks with unseen property combinations, its effectiveness on additional property objectives remains unexplored. (2) MFG relies on property oracles to provide intermediate property feedback and compute rewards for generated molecules. The reliability of this feedback depends on the accuracy of the property oracles, and the improvements observed in our evaluations have not been experimentally validated. (3) During training, the reward function uses the target molecule provided by MuMOInstruct to normalize property improvements and structural similarity. This requirement may limit the applicability of the current training framework to datasets without corresponding target molecules. (4) We apply MFG to GeLLM3O to develop GeLLM3O-MFG. Although the results demonstrate the effectiveness of MFG on this model, further evaluations on other molecular optimization LLMs are needed to establish the generalizability of our framework. Ethical considerations MFG for molecular optimization aims to improve multiple property objectives through iterative generation guided by intermediate property feedback. Although our framework is developed for molecular optimization rather than harmful applications, its ability to generate and optimize candidate molecules may also be used to produce potentially unsafe compounds. Moreover, improvements in target property objectives and structural similarity to the source molecule do not necessarily indicate that the generated molecules are safe for practical use. We therefore emphasize that molecules generated by MFG should undergo careful safety evaluation by expert chemists before being considered for synthesis or experimental testing. Any practical application of the framework and its generated molecules should also adhere to relevant safety regulations, ethical guidelines, and established professional practices. References Bajusz et al. (2015) Dávid Bajusz, Anita Rácz, and Károly Héberger. 2015. Why is tanimoto index an appropriate choice for fingerprint-based similarity calculations? Journal of cheminformatics, 7(1):20. Chen et al. (2021) Ziqi Chen, Martin Renqiang Min, Srinivasan Parthasarathy, and Xia Ning. 2021. A deep generative model for molecule optimization via one fragment modification. Nature machine intelligence, 3(12):1040--1049. Dey et al. (2025) Vishal Dey, Xiao Hu, and Xia Ning. 2025. GeLLM³O: Generalizing large language models for multi-property molecule optimization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25192--25221, Vienna, Austria. Association for Computational Linguistics. Fu et al. (2021) Tianfan Fu, Cao Xiao, Xinhao Li, Lucas M Glass, and Jimeng Sun. 2021. Mimosa: Multi-constraint molecule sampling for molecule optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 125--133. Hughes et al. (2011) JP Hughes, S Rees, SB Kalindjian, and KL Philpott. 2011. Principles of early drug discovery. British Journal of Pharmacology, 162(6):1239--1249. Irwin et al. (2022) Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum. 2022. Chemformer: a pre-trained transformer for computational chemistry. Machine Learning: Science and Technology, 3(1):015022. Le et al. (2024) Khiem Le, Ting Hua, and Nitesh V Chawla. 2024. Agentdrug: Utilizing large language models in an agentic workflow for zero-shot molecular editing. arXiv preprint arXiv:2410.13147. Li et al. (2026) Xuan Li, Zongze Li, Jiangchao Yao, Yu Rong, Lu Zhang, Bo Han, and 1 others. 2026. Reference-guided policy optimization for molecular optimization via llm reasoning. In International Conference on Learning Representations, volume 2026, pages 65266--65297. Lipinski et al. (2001) Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. 2001. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings1pii of original article: S0169-409x(96)00423-1. the article was originally published in advanced drug delivery reviews 23 (1997) 3–25.1. Advanced Drug Delivery Reviews, 46(1):3--26. Special issue dedicated to Dr. Eric Tomlinson, Advanced Drug Delivery Reviews, A Selection of the Most Highly Cited Articles, 1991-1998. Liu et al. (2026) Alexander H Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, and 1 others. 2026. Ministral 3. arXiv preprint arXiv:2601.08584. Liu et al. (2023) Shengchao Liu, Jiongxiao Wang, Yijin Yang, Chengpeng Wang, Ling Liu, Hongyu Guo, and Chaowei Xiao. 2023. Chatgpt-powered conversational drug editing using retrieval and domain feedback. arXiv preprint arXiv:2305.18090. Liu et al. (2025) Xuefeng Liu, Songhao Jiang, Siyu Chen, Zhuoran Yang, Yuxin Chen, Ian Foster, and Rick Stevens. 2025. Drugimprovergpt: A large language model for drug optimization with fine-tuning via structured policy optimization. arXiv preprint arXiv:2502.07237. Naveed et al. (2025) Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2025. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology, 16(5):1--72. Nicolaou and Brown (2013) Christos A. Nicolaou and Nathan Brown. 2013. Multi-objective optimization methods in drug design. Drug Discovery Today: Technologies, 10(3):e427--e435. Qwen Team (2026) Qwen Team. 2026. Qwen3.5: Towards native multimodal agents. Shao et al. (2024) Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Sun et al. (2022) Duxin Sun, Wei Gao, Hongxiang Hu, and Simon Zhou. 2022. Why 90 Acta Pharmaceutica Sinica B, 12(7):3049--3062. Wu et al. (2024) Zhenxing Wu, Odin Zhang, Xiaorui Wang, Li-Ni Fu, Huifeng Zhao, Ji-Ke Wang, Hongyan Du, Dejun Jiang, Yafeng Deng, Dongsheng Cao, Chang-Yu Hsieh, and Tingjun Hou. 2024. Leveraging language model for advanced multiproperty molecular optimization via prompt engineering. Nature Machine Intelligence, 6:1359 -- 1369. Ye et al. (2025) Geyan Ye, Xibao Cai, Houtim Lai, Xing Wang, Junhong Huang, Longyue Wang, Wei Liu, and Xiangxiang Zeng. 2025. Drugassist: A large language model for molecule optimization. Briefings in Bioinformatics, 26(1):bbae693. Yu et al. (2024) Botao Yu, Frazier N Baker, Ziqi Chen, Xia Ning, and Huan Sun. 2024. Llasmol: Advancing large language models for chemistry with a large-scale, comprehensive, high-quality instruction tuning dataset. arXiv preprint arXiv:2402.09391. Zuo et al. (2025) Chenyang Zuo, Siqi Fan, and Zaiqing Nie. 2025. Biomedgpt-mol: Multi-task learning for molecular understanding and generation. arXiv preprint arXiv:2512.04629. Appendix A Appendix A.1 Related Work Molecular optimization Computational methods for molecular optimization has been explored through a variety of paradigms, including molecule string translation and graph-based generation. Chemformer Irwin et al. (2022) and Prompt-MolOpt Wu et al. (2024) used transformer models to translate input molecule strings into optimized molecules. Modof Chen et al. (2021) and MIMOSA Fu et al. (2021) instead formulated molecular optimization as a graph modification problem and designed graph neural network-based methods to edit substructures. However, these methods are developed for specific objectives and may have limited generalizability across diverse optimization tasks. More recently, LLMs have shown the potential for molecular optimization by treating molecule string as a chemical language. For example, ChatDrug Liu et al. (2023) and Re2DF Le et al. (2024) designed inference-time methods to iteratively refine molecules based on LLM dialogue with external domain feedback. DrugAssist Ye et al. (2025) and GeLLM3O Dey et al. (2025) further instruction-tuned general-purpose LLMs specifically for molecular optimization. BioMedGPT-Mol Zuo et al. (2025) incorporated molecular optimization as one of multiple chemical tasks in a foundation chemical language model. Building on these ideas, our work finetunes a molecular optimization LLM via reinforcement learning (RL) so that it can be applied multiple times to generate progressively improved molecules. A.2 Selection Criterion Advance(⋅)Advance(·) ranks the intermediate candidates according to the criterion in Table A.1, listed in descending order of priority. Each criterion is considered only when higher-priority criteria cannot identify a unique candidate. Table A.1: Selection criteria of Advance(⋅)Advance(·) in descending order of priority. Priority Selection Criterion 1 Chemical validity 2 Similarity to C0 C_0 ≥δ≥δ 3 Improvement on all properties in P 4 Number of improved properties 5 Similarity to C0 C_0 ≥δh≥ _h 6 Relative property improvement 7 Similarity to C0 C_0 A.3 Reward Function Definition of (⋅)G(·). For each source molecule C0 C_0 , we use the reference molecule CrefC_ref provided by MuMOInstruct to normalize the improvement in each property p∈p∈ $ P $: (Ct∣p)=clip(dpΔyp(Ct,C0)dpΔyp(Ctarget,C0),−κ,κ),G(C_t p)=clip ( d_p y_p(C_t, $ C_0 $)d_p y_p(C_target, $ C_0 $),-κ,κ ), (5) where Δyp(A,B)=yp(A)−yp(B) y_p(A,B)=y_p(A)-y_p(B) denotes the difference in property p between molecules A and B, dp∈−1,1d_p∈\-1,1\ specifies the desired optimization direction, and κ is the clipping threshold. We compute the average normalized improvement across all property objectives: (Ct)=1||∑p∈(Ct∣p).G(C_t)= 1| $ P $| _p∈ $ P $G(C_t p). (6) By using CrefC_ref for normalization, property improvements are scaled to balance the contributions of different property objectives. Definition of (⋅)S(·). We similarly normalize the structural similarity CrefC_ref: (Ct)=1,Sim(Ct,C0)≥δh,Sim(Ct,C0)Sim(Cref,C0),otherwise,S(C_t)= cases1,&Sim(C_t, $ C_0 $)≥ _h,\\[3.0pt] Sim(C_t, $ C_0 $)Sim(C_ref, $ C_0 $),&otherwise, cases (7) where Sim(A,B)∈[0,1]Sim(A,B)∈[0,1] denotes the Tanimoto similarity between molecules A and B. The threshold-based design encourages structural similarity while preventing it from dominating the optimization objective. Definition of λ(⋅)λ(·). The scaling factor λt _t provides an additional bonus when CtC_t improves all target properties over C0 C_0 , as indicated by A(Ct)=[dpΔyp(Ct,C0)>0,∀p∈]A(C_t)=I [d_p y_p(C_t, $ C_0 $)>0,\ ∀ p∈ $ P $ ]. We then define λ(Ct)=1+A(Ct)Sim(Ct,C0).λ(C_t)=1+A(C_t)\,Sim(C_t, $ C_0 $). (8) Thus, λ(⋅)>1λ(·)>1 only when CtC_t simultaneously improves all properties, with a larger λ(⋅)λ(·) for higher similarity. Validity Penalties. An output is considered invalid if it does not contain exactly one tagged SMILES string, fails RDKit parsing or kekulization, or duplicates the source molecule or any previously generated molecule. We assign a reward of −2-2 when CtC_t is invalid. For valid molecules, we set the minimum reward to −1-1 to prevent them from receiving penalties comparable to those for invalid generations, despite suboptimal property improvements. Table A.2: Overall Performance on IND Tasks Across Turn Settings Model T BDP BDQ BPQ DPQ BDPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI General-purpose LLMs Qwen3.5-9B 1 32.80 0.76 0.98 18.80 0.73 0.54 90.40 0.75 0.58 15.60 0.72 0.51 11.00 0.75 0.54 2 36.40 0.72 0.95 21.60 0.70 0.60 88.60 0.71 0.65 19.80 0.69 0.36 12.20 0.76 0.51 3 35.40 0.71 0.97 19.80 0.70 0.44 81.80 0.71 0.61 16.60 0.70 0.35 11.60 0.72 0.51 4 34.60 0.70 1.04 19.80 0.71 0.51 75.80 0.71 0.62 15.40 0.69 0.33 11.80 0.71 0.60 Ministral-3-8B 1 27.60 0.71 0.74 24.00 0.67 0.45 64.80 0.68 0.52 12.80 0.68 0.28 8.60 0.72 0.56 2 33.60 0.68 0.81 24.60 0.66 0.69 66.00 0.65 0.65 13.60 0.68 0.36 11.60 0.70 0.47 3 36.60 0.67 0.84 23.75 0.67 0.62 60.60 0.65 0.63 12.00 0.67 0.35 11.20 0.68 0.53 4 37.60 0.68 0.70 23.40 0.66 0.65 57.40 0.65 0.63 12.00 0.67 0.32 11.00 0.68 0.56 LLMs for Chemistry LlaSMol 1 5.00 0.65 0.66 1.60 0.56 0.79 27.40 0.65 0.66 4.40 0.66 0.37 2.40 0.73 0.42 2 6.40 0.64 0.81 0.80 0.54 0.66 30.00 0.62 0.72 5.00 0.63 0.37 3.60 0.67 0.42 3 6.40 0.62 1.04 0.80 0.53 0.68 28.78 0.63 0.65 5.00 0.63 0.37 3.60 0.64 0.48 4 6.60 0.62 0.96 0.80 0.53 0.67 25.60 0.63 0.66 3.80 0.67 0.28 2.60 0.65 0.48 BioMedGPT-Mol 1 69.20 0.62 2.82 79.00 0.62 4.40 97.20 0.61 1.23 47.80 0.63 1.58 38.00 0.62 2.03 2 73.80 0.59 3.10 63.80 0.59 4.20 96.60 0.59 1.26 43.80 0.61 2.07 43.00 0.59 1.88 3 74.20 0.58 3.36 60.40 0.60 4.31 95.40 0.59 1.25 41.80 0.61 2.05 42.00 0.59 1.91 4 73.60 0.58 3.28 58.40 0.59 4.46 94.80 0.58 1.24 41.40 0.60 2.04 42.00 0.59 2.06 GeLLM3O GeLLM3O 1 66.00 0.62 2.77 65.00 0.64 3.88 84.40 0.60 1.28 43.80 0.61 1.84 39.00 0.60 2.26 2 63.60 0.59 3.10 71.40 0.60 4.18 76.40 0.58 1.28 41.00 0.58 2.20 37.20 0.58 3.00 3 62.60 0.57 3.37 71.40 0.60 4.37 75.60 0.58 1.26 42.60 0.58 2.20 34.60 0.57 3.29 4 61.20 0.57 3.21 70.80 0.59 4.39 75.00 0.58 1.23 40.60 0.57 2.47 33.40 0.57 3.22 GeLLM3O-RL 1 79.00 0.61 3.67 70.60 0.63 4.94 90.60 0.60 1.31 47.20 0.62 2.00 41.80 0.62 2.28 2 79.40 0.58 4.18 79.40 0.60 5.88 89.00 0.58 1.35 46.80 0.59 2.77 49.20 0.59 3.14 3 78.80 0.57 4.34 80.40 0.58 5.87 88.40 0.57 1.36 44.80 0.58 2.89 48.40 0.58 3.09 4 77.40 0.57 4.30 80.60 0.59 5.67 87.40 0.57 1.36 44.40 0.58 2.68 47.20 0.58 3.07 GeLLM3O-MFG 1 83.20 0.62 3.62 73.00 0.63 4.30 89.20 0.62 1.25 48.40 0.63 1.97 44.00 0.61 2.50 2 84.20 0.61 4.24 85.60 0.61 5.22 91.20 0.60 1.32 54.20 0.61 2.55 51.40 0.59 2.84 3 83.20 0.61 3.90 84.60 0.61 5.40 89.60 0.59 1.34 54.40 0.60 2.47 53.40 0.59 3.19 4 83.00 0.61 3.92 86.20 0.61 5.32 89.40 0.60 1.32 54.60 0.60 2.49 51.80 0.59 3.16 • The results with optimal T for each model and task is highlighted in yellow. Other metrics, notations, and formatting follow Table 1. Table A.3: Overall Performance on OOD Tasks Across Turn Settings Model T MPQ BDMQ BHMQ BMPQ HMPQ SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI SR0.5↑ SR_0.5 Sim↑ Sim RI↑ RI General-purpose LLMs Qwen3.5-9B 1 75.80 0.69 0.41 17.00 0.76 0.41 72.03 0.73 1.03 75.39 0.71 0.47 81.25 0.73 0.54 2 68.00 0.66 0.46 23.40 0.71 0.43 77.12 0.69 1.03 73.82 0.67 0.54 81.25 0.69 0.68 3 63.20 0.66 0.43 21.20 0.69 0.41 73.73 0.69 1.03 62.83 0.66 0.55 72.92 0.68 0.73 4 60.20 0.66 0.45 21.40 0.69 0.42 70.34 0.70 0.98 59.16 0.67 0.52 70.83 0.67 0.74 Ministral-3-8B 1 47.80 0.65 0.37 15.40 0.66 0.52 51.69 0.66 1.07 47.12 0.65 0.47 34.38 0.67 0.59 2 49.60 0.64 0.39 23.20 0.64 0.76 57.63 0.63 1.03 56.54 0.65 0.49 40.62 0.63 0.69 3 43.00 0.63 0.38 18.60 0.65 0.62 46.61 0.63 0.81 52.36 0.66 0.44 40.62 0.64 0.58 4 42.40 0.63 0.38 16.00 0.65 0.69 38.98 0.63 0.89 50.26 0.65 0.43 35.42 0.66 0.54 LLMs for Chemistry LlaSMol 1 17.40 0.61 0.38 0.20 0.67 0.00 15.25 0.62 1.78 15.71 0.61 0.51 31.25 0.63 0.69 2 18.20 0.60 0.41 0.60 0.59 0.33 20.34 0.63 1.02 18.32 0.64 0.50 25.00 0.61 0.65 3 17.00 0.60 0.39 0.80 0.58 0.68 15.25 0.63 1.38 16.23 0.63 0.49 29.17 0.62 0.70 4 15.80 0.61 0.36 0.60 0.63 0.55 12.71 0.60 1.14 13.61 0.63 0.51 21.88 0.61 0.71 BioMedGPT-Mol 1 94.40 0.60 0.74 65.60 0.61 2.48 88.98 0.62 2.25 91.10 0.60 0.87 91.67 0.62 1.14 2 81.00 0.59 0.81 57.20 0.60 2.64 91.53 0.59 2.53 91.62 0.60 0.92 93.75 0.59 1.64 3 79.00 0.59 0.80 54.60 0.59 2.65 89.83 0.58 2.60 88.48 0.60 0.94 86.46 0.58 1.42 4 78.20 0.59 0.80 52.20 0.59 2.79 87.29 0.59 2.42 84.82 0.59 0.92 86.46 0.58 1.47 GeLLM3O GeLLM3O 1 83.40 0.60 0.76 64.40 0.62 2.66 82.20 0.60 2.81 84.82 0.59 0.96 87.50 0.59 1.51 2 76.20 0.58 0.76 65.60 0.59 3.06 66.10 0.57 3.33 70.16 0.58 1.03 73.96 0.57 1.69 3 74.40 0.58 0.75 62.80 0.58 3.12 63.56 0.56 3.37 67.02 0.57 1.04 68.75 0.56 1.41 4 74.40 0.58 0.74 61.00 0.58 3.26 62.71 0.56 3.21 66.49 0.58 0.98 70.83 0.56 1.38 GeLLM3O-RL 1 88.80 0.59 0.83 69.80 0.62 3.01 88.14 0.59 2.87 90.05 0.61 0.99 92.71 0.59 1.44 2 82.40 0.58 0.88 81.60 0.59 3.77 88.98 0.58 3.12 87.43 0.57 1.11 95.83 0.56 1.75 3 80.00 0.58 0.87 81.20 0.58 3.87 87.29 0.57 2.95 84.82 0.56 1.12 94.79 0.55 1.73 4 78.80 0.58 0.87 79.60 0.58 3.85 86.44 0.57 2.95 83.77 0.56 1.09 93.75 0.55 1.70 GeLLM3O-MFG 1 88.40 0.60 0.78 80.20 0.62 2.77 80.51 0.60 2.74 86.91 0.62 0.92 88.54 0.60 1.37 2 87.20 0.60 0.83 83.60 0.61 3.50 87.29 0.59 2.78 89.01 0.60 0.99 90.62 0.58 1.70 3 85.00 0.59 0.84 80.80 0.61 3.60 88.14 0.58 2.85 86.91 0.59 1.02 93.75 0.58 1.62 4 85.00 0.59 0.84 81.20 0.61 3.69 90.68 0.57 2.87 85.34 0.59 1.04 91.67 0.58 1.42 • The results with optimal T for each model and task is highlighted in yellow. Other metrics, notations, and formatting follow Table 1. A.4 Implementation Details Training We use a group size of four for GRPO per prompt. For groups with uninformative rewards, we apply dynamic resampling with up to three attempts. We set the upper clipping parameter to 0.280.28, the KL coefficient to 0.10.1, and the maximum completion length to 96 tokens. Training uses fused AdamW with β1=0.9 _1=0.9, β2=0.95 _2=0.95, ϵ=10−8ε=10^-8, weight decay of 0.10.1, and gradient clipping at 1.01.0. For the training data, we sample 100 source molecules for each tasks in MuMOInstruct, excluding the 5 OOD tasks. Training GeLLM3O-MFG took approximately 36 hours using two NVIDIA A100 GPUs. For parameter-efficient training, we use LoRA with rank 16, scaling parameter 32, and dropout 0.05. We apply LoRA to the attention and Feed-Forward Network (FFN) projection layers in every Transformer block, while keeping all pretrained model parameters frozen. Training of GeLLM3O-RL follows the same setup as GeLLM3O-MFG, but excludes generated context, feedback, and the reward bonus (Eq. 4). In terms of hyperparameters in reward function, κ in Eq. 5 is set to 1.251.25, δh _h in Eq. 7 is 0.60.6 and δ in Eq. 2 is 0.40.4. Inference As we discussed in Section 4.5, N is set to 20/T20/T. Candidate selection criterion uses the same δ and δh _h values as training. A.5 Evaluation Metrics Definition of RI. Following GeLLM3O, RI is defined as follows, RI=1||∑p∈RIp, RI= 1| $ P $| _p∈ $ P $RI_p, (9) where RIpRI_p is the relative improvement on property p and is defined as follows, RIp=dpyp(M)−yp(S)yp(S).RI_p=d_p y_p(M)-y_p(S)y_p(S). (10) dpd_p is the direction coefficient and is set to +1+1 when property p is optimized to increase and to −1-1 otherwise. A.6 Prompt and Feedback Templates All models receive the same initial prompt specifying the source molecule and target property objectives. The initial prompt follows the format: Your task is to modify the given molecule to adjust specific molecular properties while keeping structural changes as minimal as possible. Your response should only contain a valid SMILES representation of the modified molecule enclosed with <SMILES> </SMILES> tag. %%% Input: <SMILES>[source molecule]</SMILES> %%% Adjust: [property objectives] In the MT setting, each valid generation is followed by feedback on its structural similarity to the source molecule and improvements in each target property, as described in Section 3.1. The feedback is structured as follows: - Similarity to input molecule: [value] ([band]) - [property]: [source] -> [candidate] ([change], [status]; [direction]) Here, we categorize structural similarity into four bands: very high for similarity at least 0.6, high for [0.5,0.6)[0.5,0.6), moderate for [0.4,0.5)[0.4,0.5), and low below 0.4. The feedback is also tailored based on whether a best molecule has been identified in previous turns. If no best molecule exists, the model is instructed to improve properties that have not yet improved while preserving previously achieved improvements and structural similarity to C0 C_0 . Otherwise, the best molecule Ct∗C_t^* is directly provided in the prompt as a tagged SMILES string, guiding the model to further improve upon Ct∗C_t^*. For invalid generations, we provide corrective feedback specifying the reason for failure. This allows the model to identify generation errors and refine its molecular generation strategy in subsequent turns. A.7 Additional Analysis of Experimental Results Tables A.2 and A.3 provide detailed performance results across different numbers of T, illustrating variations in the optimal T across tasks and models. Overall, GeLLM3O-MFG has a higher average optimal T across the 10 tasks (2.70), compared with GeLLM3O (1.30) and GeLLM3O-RL (1.80). This suggests that MFG trains GeLLM3O to effectively adapt its optimization strategies across turns, allowing improvements to accumulate over successive turns. From T=1T=1 to T=4T=4, the average SR of GeLLM3O decreases from 70.05%70.05\% to 61.64%61.64\%, while GeLLM3O-RL improves slightly from 75.87%75.87\% to 75.94%75.94\%. In contrast, GeLLM3O-MFG shows a more pronounced improvement from 76.24%76.24\% to 79.89%79.89\%, demonstrating greater benefits from additional optimization turns. These results suggest that MFG trains the model to better leverage multi-turn feedback for progressive molecular optimization, with a task-dependent optimal T. A.8 Artifact Sources and Licenses Table A.4 summarizes the sources, licenses, and accessibility of the models, datasets, and implementation used in our experiments. We report the licenses specified by the respective artifact providers. For artifacts without an explicitly stated license, we indicate this accordingly. Table A.4: Licenses and sources of artifacts used in our study. Artifact Source License Type Accessibility MuMOInstruct https://huggingface.co/datasets/NingLab/MuMOInstruct C BY 4.0 Dataset Llama-3.1-8B-Instruct https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct Llama 3.1 Community License Checkpoint GeLLM3O-P(6)Llama https://huggingface.co/NingLab/GeLLMO-P6-Llama C BY-NC 4.0 Checkpoint Qwen3.5-9B https://huggingface.co/Qwen/Qwen3.5-9B Apache 2.0 Checkpoint Ministral-3-8B-Instruct-2512-BF16 https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-BF16 Apache 2.0 Checkpoint LlaSMol-Mistral-7B https://huggingface.co/osunlp/LlaSMol-Mistral-7B C BY 4.0 LoRA Adapter BioMedGPT-Mol https://huggingface.co/PharMolix/BioMedGPT-Mol Not specified Checkpoint