Paper deep dive
DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction
Long Xu, Junping Guo, Jianbo Zhao, Jianbo Lu, Yuzhong Peng
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/26/2026, 2:23:42 AM
Summary
DMMRL is a novel framework for molecular property prediction that utilizes variational autoencoders (VAEs) to disentangle molecular representations into shared (structure-relevant) and private (modality-specific) latent spaces. By integrating graph, sequence, and geometric modalities through a gated attention fusion mechanism, the model improves interpretability and predictive performance across benchmark datasets.
Entities (5)
Relation Signals (3)
DMMRL → performs → Molecular Property Prediction
confidence 100% · DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction
DMMRL → uses → Variational Autoencoders
confidence 100% · DMMRL, which employs variational autoencoders to disentangle molecular representations
DMMRL → includes → Gated Attention Fusion
confidence 95% · we design a gated attention fusion mechanism that adaptively combines shared factors
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Molecular property prediction constitutes a cornerstone of drug discovery and materials science, necessitating models capable of disentangling complex structure-property relationships across diverse molecular modalities. Existing approaches frequently exhibit entangled representations--conflating structural, chemical, and functional factors--thereby limiting interpretability and transferability. Furthermore, conventional methods inadequately exploit complementary information from graphs, sequences, and geometries, often relying on naive concatenation that neglects inter-modal dependencies. In this work, we propose DMMRL, which employs variational autoencoders to disentangle molecular representations into shared (structure-relevant) and private (modality-specific) latent spaces, enhancing both interpretability and predictive performance. The proposed variational disentanglement mechanism effectively isolates the most informative features for property prediction, while orthogonality and alignment regularizations promote statistical independence and cross-modal consistency. Additionally, a gated attention fusion module adaptively integrates shared representations, capturing complex inter-modal relationships. Experimental validation across seven benchmark datasets demonstrates DMMRL's superior performance relative to state-of-the-art approaches. The code and data underlying this article are freely available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.21108v1
- Canonical: https://arxiv.org/abs/2603.21108v1
Trouble viewing inline? Open PDF directly →
Full Text
50,221 characters extracted from source content.
Expand or collapse full text
DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction Long Xu1, Junping Guo1, Jianbo Zhao1, Jianbo Lu1, and Yuzhong Peng2* Abstract Molecular property prediction constitutes a cornerstone of drug discovery and materials science, necessitating models capable of disentangling complex structure-property relationships across diverse molecular modalities. Existing approaches frequently exhibit entangled representations—conflating structural, chemical, and functional factors—thereby limiting interpretability and transferability. Furthermore, conventional methods inadequately exploit complementary information from graphs, sequences, and geometries, often relying on naive concatenation that neglects inter-modal dependencies. In this work, we propose DMMRL, which employs variational autoencoders to disentangle molecular representations into shared (structure-relevant) and private (modality-specific) latent spaces, enhancing both interpretability and predictive performance. The proposed variational disentanglement mechanism effectively isolates the most informative features for property prediction, while orthogonality and alignment regularizations promote statistical independence and cross-modal consistency. Additionally, a gated attention fusion module adaptively integrates shared representations, capturing complex inter-modal relationships. Experimental validation across seven benchmark datasets demonstrates DMMRL’s superior performance relative to state-of-the-art approaches. The code and data underlying this article are freely available at https://github.com/xulong0826/DMMRL. I Introduction Molecular property prediction is vital for drug discovery, enabling efficient identification of promising compounds before synthesis [21, 23, 15, 19]. Accurate predictions of solubility, toxicity, and bioactivity accelerate development while reducing costs. With expanding chemical databases and computational resources, data-driven methods have become essential for understanding structure-property relationships [31]. As the chemical space continues to grow exponentially, computational models are increasingly relied upon to prioritize candidates, reduce experimental workload, and guide rational molecular design. These models not only facilitate virtual screening and lead optimization but also enable the exploration of novel chemical scaffolds that may be inaccessible through traditional experimental approaches. In addition, predictive modeling empowers researchers to systematically explore vast chemical spaces, identify structure-activity relationships, and optimize molecular properties with unprecedented efficiency, thereby accelerating the pace of innovation in both pharmaceutical and materials science domains. Despite these advances, current approaches face two key limitations that hinder their practical utility and scientific insight [29]. First, representation entanglement occurs when structural, chemical, and functional factors become intertwined within learned representations, obscuring which molecular aspects drive specific properties. This entanglement fundamentally constrains model interpretability by conflating distinct causal factors that influence molecular behavior. When multiple structural and electronic properties are encoded in overlapping embedding dimensions, the resulting representations cannot effectively isolate the independent contributions of specific molecular substructures, functional groups, or electronic properties. This not only obscures the mechanistic basis of predictions but also restricts transferability to new tasks where different molecular factors may become relevant. The inability to attribute predictions to specific molecular features ultimately limits the trustworthiness and scientific insight provided by the models, particularly in safety-critical domains like drug discovery where mechanistic understanding is essential for rational design and regulatory approval. Second, inadequate multi-modal integration persists despite the complementary nature of different molecular representations [27, 26]. Graph structures capture topological connectivity but lack spatial information; sequence notations encode chemical composition but overlook geometric relationships; conformational representations provide spatial context but may not efficiently encode functional group patterns. Most existing methods either focus exclusively on single modalities, discarding complementary information sources, or employ simplistic concatenation strategies that fail to capture complex inter-modal dependencies and conditional relationships. This deficiency is particularly problematic for properties that emerge from the interplay of multiple structural factors—for instance, molecular binding affinity depends simultaneously on spatial conformation, electronic charge distribution, and functional group arrangement. Effective integration must adaptively weight different modalities according to their task-specific relevance while preserving their complementary nature. Previous methods exhibit specific shortcomings in addressing these challenges. Traditional descriptor-based approaches offer interpretability but perform poorly with complex patterns and require extensive domain expertise [22, 4]. These handcrafted features may not generalize well to novel chemical spaces or capture subtle structure-property relationships, limiting their applicability in modern drug discovery pipelines. In contrast, graph neural networks encode molecular topology effectively but often produce entangled representations that obscure the contributions of individual factors [14, 2, 17]. While GNNs have advanced the field by modeling molecular connectivity, they frequently lack mechanisms to separate property-relevant features from modality-specific noise, resulting in black-box models with limited interpretability. Similarly, standard deep learning approaches create compact representations without addressing disentanglement or multi-modal integration, further exacerbating the challenges of interpretability and adaptability to new tasks. Building on these limitations, recent advances in variational autoencoders (VAEs) have demonstrated that explicit factor separation can significantly enhance interpretability, generalization, and transferability [16]. By enforcing structured latent spaces, VAEs can disentangle shared and private factors, allowing models to focus on the most informative features for property prediction while systematically discarding irrelevant or confounding information. This disentanglement not only improves predictive performance but also provides clearer insights into the molecular determinants of specific properties. Concurrently, multi-modal learning has emerged as a powerful paradigm for leveraging diverse information sources. Graphs encode connectivity, sequences capture chemical syntax, and geometric representations provide spatial context [32, 24]. Integrating these modalities can yield richer and more robust molecular representations, enabling models to capture complex structure-property relationships that are inaccessible to single-modality approaches. However, significant challenges remain in disentangling molecular graphs and fusing modalities while maintaining interpretability. Effective multi-modal fusion must not only combine information but also preserve the independence and complementarity of each modality, ensuring that the resulting representations are both informative and interpretable for downstream tasks. Achieving this balance is critical for advancing molecular property prediction and unlocking new opportunities in drug discovery and materials science. To address these fundamental limitations, we propose DMMRL (Disentangled Multi-Modal Representation Learning), a framework integrating variational autoencoders with multi-modal fusion mechanisms. All in all, the main contributions of DMMRL are as follows. • We develop a feature disentanglement mechanism using VAE modules that separates shared (common structural) from private (modality-specific) factors within each modality. • We design a gated attention fusion mechanism that adaptively combines shared factors from all modalities, capturing complex inter-modal relationships. • Extensive evaluation on seven benchmark datasets demonstrating DMMRL’s superior performance compared to state-of-the-art methods. Ablation studies also demonstrated the effectiveness of the proposed techniques. I Materials and methods I-A Problem formulation The molecular property prediction problem in a multi-modal context can be formally defined as follows [15]. Given a collection of molecules ℳ=mii=1NM=\m_i\_i=1^N, each molecule mim_i is represented through multiple complementary modalities: a molecular graph GiG_i that captures atom-bond connectivity patterns and topological structures, a sequence representation SiS_i (typically SMILES notation) that encodes linear chemical syntax and composition rules, and a geometric conformation CiC_i that reflects three-dimensional spatial arrangements including bond lengths, angles, and torsional relationships. The objective is to derive a predictive function f(⋅)f(·) that effectively maps these multi-modal inputs to molecular property labels yiy_i, represented as f(Gi,Si,Ci)→yif(G_i,S_i,C_i)→ y_i. The central methodological challenge lies in the effective integration of these complementary information sources while ensuring that the learned representations exhibit both interpretability through factor separation and appropriate disentanglement of property-relevant features from modality-specific artifacts. Figure 1 illustrates the overall architecture of our proposed DMMRL framework. I-B Molecular Encoder Module To process molecular data from different perspectives, we employ three established encoders following methodologies in [24]: Sequence Encoder. For SMILES sequences, a bidirectional LSTM followed by a transformer encoder architecture is implemented. The LSTM effectively captures local chemical contexts from tokenized SMILES strings, extracting sequential dependencies and substructure motifs that are important for chemical interpretation. Subsequently, the transformer encoder leverages self-attention mechanisms to model long-range dependencies and global context, which are essential for comprehending the overall molecular structure and capturing interactions between distant atoms or functional groups within the sequence. This hybrid architecture enables the model to learn both local and global features from SMILES representations, providing a comprehensive encoding of chemical syntax and semantics. hiseq h_i^seq =concat(LSTM→(xi),LSTM←(xi)) =concat ( LSTM(x_i),\, LSTM(x_i) ) (1) Hs H_s =Transformer(hiseqi=1T) =Transformer(\h_i^seq\_i=1^T) (2) Graph Encoder. For molecular graphs, a Communicative Message Passing Neural Network (CMPNN) is implemented that systematically updates atom and bond representations through structured information exchange protocols. CMPNN enables each atom and bond to iteratively aggregate information from their neighbors, allowing the encoder to capture complex topological structures, chemical connectivity patterns, and local environments. By facilitating communication between nodes and edges, CMPNN effectively models the intricate relationships present in molecular graphs, such as aromaticity, ring structures, and branching, which are critical for accurate property prediction. mv(k) m_v^(k) =AGGREGATE(huv(k−1):u∈(v)) =AGGREGATE(\h_uv^(k-1):u (v)\) (3) hv(k) h_v^(k) =COMMUNICATE(hv(k−1),mv(k)) =COMMUNICATE(h_v^(k-1),m_v^(k)) (4) Geometry Encoder. For three-dimensional conformations, a geometric graph neural network is utilized that incorporates spatial information including distances and angles between atoms. This encoder not only considers the connectivity of atoms but also explicitly models geometric features such as bond lengths, bond angles, and torsional angles, which are crucial for understanding molecular shape and spatial arrangement. By integrating these geometric descriptors, the encoder captures essential three-dimensional arrangement information that complements the topological and chemical features derived from other modalities, enabling the model to better predict properties that are sensitive to molecular conformation. mv(k) m_v^(k) =AGG(hu(k−1),huv(k−1):u∈(v)) =AGG (\h_u^(k-1),h_uv^(k-1):u (v)\ ) (5) hv(k) h_v^(k) =COMBINE(hv(k−1),mv(k)) =COMBINE(h_v^(k-1),m_v^(k)) (6) Figure 1: Overview of the DMMRL framework. I-C Disentangled Representation VAE Module With these specialized encoders extracting modality-specific representations, we introduce a novel disentangled representation learning module based on variational autoencoders (VAEs) to address the fundamental challenge of representation entanglement. Unlike conventional autoencoders or standard disentanglement frameworks—which typically separate latent factors without explicit consideration of modality relevance or predictive utility—our approach systematically decomposes each modality’s features into two complementary latent spaces: shared (structure-relevant) and private (modality-specific). This design is grounded in the insight that molecular information comprises both generalizable, property-relevant structural patterns and modality-specific artifacts that may introduce noise. The key innovation of our method lies in the principled information bottleneck and targeted regularization strategy [9]. Rather than simply reconstructing input data or enforcing generic factor separation, our VAE module constrains latent capacity and imposes statistical independence between shared and private spaces, ensuring that only the most predictive molecular features are encoded in the shared space. Orthogonality and alignment losses further enforce that shared representations capture cross-modal consistency while private spaces retain modality-specific details. This is in contrast to ordinary AE-based approaches, which often fail to disentangle predictive and non-predictive factors or allow leakage of irrelevant information into the prediction pathway. Specifically, for each modality m∈s,g,hm∈\s,g,h\ (corresponding to sequence, graph, and geometry representations), the encoder transforms input representations HmH_m through a pathway that includes normalization, non-linearity, and dimensionality reduction: hm h_m =Dropout(LayerNorm(ReLU(Linear(Hm)))) =Dropout(LayerNorm(ReLU(Linear(H_m)))) (7) The encoder then generates parameters (mean and log-variance) for the shared latent distribution, with the private space following an analogous process: μmshared,logσmshared _m^shared, _m^shared =Linearshared(hm) =Linear_shared(h_m) (8) Rather than deterministically mapping to fixed points, the reparameterization technique facilitates stochastic sampling from the parameterized distribution. This probabilistic encoding enhances regularization effectiveness and prevents posterior collapse by maintaining gradient flow during backpropagation: zmshared z_m^shared =μmshared+σmshared⊙ϵshared = _m^shared+ _m^shared _shared (9) where σm=exp(0.5⋅logσm) _m= (0.5· _m) and ϵ∼(0,I)ε (0,I) represents Gaussian noise. Log-variances are constrained to [−10,10][-10,10] for numerical stability while preserving expressivity. The resulting complementary latent spaces serve distinct functional purposes within the framework: • Shared latent space zmshared∈ℝdsz_m^shared ^d_s: Encapsulates structural information that remains consistent across modalities and is directly relevant for property prediction. This space captures fundamental molecular characteristics that determine physicochemical properties regardless of representation format, such as electron distribution patterns, stereochemical configurations, and functional group arrangements. • Private latent space zmprivate∈ℝdpz_m^private ^d_p: Encodes modality-specific details that are not directly pertinent to the target property, including representation artifacts, encoding biases, and modality-dependent contextual information. For example, the graph modality’s private space may encode specific node ordering schemes, while the sequence modality’s private space might capture SMILES syntax conventions that lack direct relevance to physical properties. Moreover, only the shared latent representations are propagated to the downstream prediction head, establishing a strict information bottleneck that compels the model to discard irrelevant modality-specific details. This mechanism not only enhances interpretability and robustness but also ensures that the learned representations are optimally aligned with the property prediction task. By integrating these innovations, our framework achieves superior disentanglement and predictive performance compared to traditional autoencoder-based and factor separation methods. I-D Gated Attention Fusion Building upon the disentangled representations obtained from the VAE module, we introduce a gated attention fusion mechanism that represents a significant innovation over conventional multi-modal fusion strategies [5, 18]. Unlike fixed-weight concatenation or averaging, which assign static and often suboptimal importance to each modality, our approach employs a dynamic, context-aware gating system. This mechanism adaptively modulates the flow of information from each modality based on the specific molecular context and structural characteristics, allowing the model to learn optimal modality contributions for each prediction task. The core innovation lies in the use of learnable attention gates, which are generated through a multi-layer perceptron (MLP) and normalized via softmax [25]. This enables the model to capture complex inter-modal dependencies and assign probabilistic weights to each modality, ensuring that the sum of contributions equals one. As a result, the fusion process is highly flexible and molecule-specific, reflecting the fact that different molecular properties may be best predicted by different types of information. The computational pathway of this mechanism consists of three critical stages, each corresponding to the following equations: Context-aware weight generation: gate_weights =Softmax(MLP(zmsharedm=1M)) =Softmax(MLP(\z_m^shared\_m=1^M)) (10) The shared latent vectors from all modalities are first passed through a multi-layer perceptron (MLP), which captures inter-modal dependencies and relative information content. The softmax operation normalizes these outputs into probabilistic weights, ensuring that the sum of contributions from all modalities equals one. This enables the model to dynamically adjust the importance of each modality for every molecule, reflecting the fact that different molecular properties may be best predicted by different types of information. Weighted aggregation of shared representations: fused =∑i=1Mgate_weightsi⋅zishared = _i=1^Mgate\_weights_i· z_i^shared (11) The shared representations are then combined according to the learned gate weights, amplifying the signal from the most relevant modalities while suppressing less informative ones. This selective fusion is crucial for handling cases where, for example, geometric features dominate in predicting binding affinity, while graph features are more informative for solubility or toxicity. Residual connection integration: output =FFN(fused+1M∑i=1Mzishared) =FFN (fused+ 1M _i=1^Mz_i^shared ) (12) A residual connection adds the average of all shared representations to the fused output before passing it through a feedforward neural network (FFN). This design ensures that baseline information from all modalities is preserved, preventing the loss of potentially useful features due to overly aggressive gating. The residual pathway also stabilizes training by maintaining gradient flow and mitigating vanishing gradient issues. This architecture specifically addresses the heterogeneity of molecular data, where certain modalities may contain more relevant information for specific properties or molecular classes than others. For instance, geometric representations may provide superior information for properties dependent on spatial arrangements (such as protein binding), while graph representations might better capture properties related to functional groups and connectivity patterns. By exclusively utilizing shared representations and excluding private representations from the prediction pathway, this mechanism reinforces the fundamental disentanglement principle while enabling adaptive multi-modal information integration. I-E Prediction Head Architecture Once multi-modal features are integrated through the gated attention fusion mechanism, a prediction head is employed to map these integrated features to molecular property outputs. The prediction head is implemented as a feedforward neural network with the following structure: y y =Linear(ReLU(Linear(output))) =Linear(ReLU(Linear(output))) (13) For classification tasks, the output undergoes sigmoid activation to generate probability scores, whereas for regression tasks, raw output values are utilized directly. The prediction head architecture remains deliberately straightforward, thereby focusing the model’s capacity on learning meaningful disentangled representations rather than complex prediction mappings. Task-specific loss functions are applied for prediction: ℒlabel=1||∑i∈BCE(yi,σ(y^i))classification1||∑i∈(yi−y^i)2regression _label= cases 1|V|Σ _i BCE(y_i,σ( y_i))&classification\\ 1|V|Σ _i (y_i- y_i)^2®ression cases (14) where V denotes valid samples, addressing the missing labels issue in molecular datasets. I-F Training and Loss Function To ensure effective learning of both disentangled representations and accurate property predictions, our objective combines property prediction with principled constraints designed to promote interpretable, disentangled representations [13, 30]. The total loss function integrates prediction with multiple specialized regularization terms, each targeting a specific aspect of disentanglement and multi-modal consistency: ℒtotal= _total=\ ℒlabel+βℒKLshared+λℒMMDprivate _label+ _KL^shared+ _MMD^private +γℒalign+δℒortho+ηℒrecon + _align+ _ortho+ _recon (15) where weights (β, λ, γ, δ, η) are hyperparameters or learnable coefficients that balance the influence of each loss component during optimization. Variational Disentanglement Losses. Three complementary regularization terms are employed to ensure the latent spaces are both informative and disentangled: ℒKLshared _KL^shared =1M∑m=1MKL(q(zmshared|Hm)∥p(zmshared)) = 1M _m=1^MKL(q(z_m^shared|H_m)\|p(z_m^shared)) (16) ℒMMDprivate _MMD^private =1M∑m=1MMMD(zmprivate,(0,I)) = 1M _m=1^MMMD(z_m^private,N(0,I)) (17) ℒrecon _recon =1M∑m=1M‖Hm−H^m‖22 = 1M _m=1^M\|H_m- H_m\|_2^2 (18) - ℒKLsharedL_KL^shared is the Kullback-Leibler (KL) divergence between the approximate posterior q(zmshared|Hm)q(z_m^shared|H_m) and the prior p(zmshared)p(z_m^shared) (typically standard normal). This term acts as an information bottleneck, regularizing the shared latent space to prevent overfitting and encouraging the model to encode only the most salient, property-relevant features in zmsharedz_m^shared. - ℒMMDprivateL_MMD^private is the Maximum Mean Discrepancy (MMD) between the private latent variables zmprivatez_m^private and a standard normal distribution. This encourages the private latent space to match a target distribution, promoting diversity and flexibility in capturing modality-specific variations while discouraging leakage of shared information into private spaces. - ℒreconL_recon is the reconstruction loss, measuring the mean squared error between the original encoder output HmH_m and its reconstruction H^m H_m from the latent variables. This ensures that the latent representations retain sufficient information for accurate reconstruction, counterbalancing the compression imposed by the KL and MMD terms. Representation Relationship Losses. Two additional terms enforce structural constraints on the relationships between latent spaces: ℒalign _align =1Npairs∑i<jInfoNCE(zishared,zjshared) = 1N_pairs _i<jInfoNCE(z_i^shared,z_j^shared) (19) ℒortho _ortho =1M∑m=1M|⟨z^mshared,z^mprivate⟩| = 1M _m=1^M| z_m^shared, z_m^private | (20) - ℒalignL_align is an alignment loss based on the InfoNCE contrastive objective, computed over all pairs of shared latent representations from different modalities. By maximizing agreement between zisharedz_i^shared and zjsharedz_j^shared, this term encourages the shared spaces to capture consistent, modality-invariant information, thus facilitating effective multi-modal fusion. - ℒorthoL_ortho is an orthogonality constraint, penalizing the inner product between normalized shared and private representations within each modality. This enforces statistical independence between zmsharedz_m^shared and zmprivatez_m^private, ensuring that property-relevant and modality-specific information are disentangled. Together, these loss components guide the model to learn representations that are not only predictive but also interpretable, disentangled, and robust across modalities. The careful design and integration of these terms are crucial for achieving the dual goals of high predictive accuracy and meaningful latent structure in multi-modal molecular property prediction. I Experimental Settings I-A Datasets and Parameter Setting Dataset. To rigorously evaluate the DMMRL framework, comprehensive experiments were conducted on seven benchmark datasets from the MoleculeNet collection [28]. These datasets were specifically selected to represent diverse molecular properties and prediction challenges. The experimental corpus spans various chemical domains including pharmaceuticals, environmental compounds, and small drug-like molecules, thereby providing a robust testbed for assessing model generalizability across diverse chemical spaces. Table I summarizes the key statistics of the datasets used in our experiments, including the number of tasks, sample sizes, task types, and evaluation metrics. The selected datasets cover both classification (e.g., BACE, BBBP, ClinTox, Tox21) and regression tasks (e.g., ESOL, FreeSolv, Lipo), ensuring a comprehensive assessment of model performance across different molecular property prediction scenarios. The diversity in dataset size and property type allows us to systematically analyze the robustness and adaptability of DMMRL under various real-world conditions. TABLE I: TABLE I. Summary of datasets. Dataset Tasks samples Type Metric BACE 1 1513 Classification ROC-AUC BBBP 1 2040 Classification ROC-AUC ClinTox 2 1478 Classification ROC-AUC Tox21 12 7831 Classification ROC-AUC ESOL 1 1127 Regression RMSE FreeSolv 1 639 Regression RMSE Lipo 1 4200 Regression RMSE Parameter Setting. To ensure methodological consistency and fair comparison with existing approaches, datasets were partitioned using random splitting according to established MoleculeNet protocols, with an 8:1:1 ratio for training, validation, and test sets, respectively. Each experiment was replicated 10 times with different random seeds to ensure statistical robustness, with results reported as mean and standard deviation of area under the receiver operating characteristic curve (ROC-AUC) for classification tasks and root mean squared error (RMSE) for regression tasks. For DMMRL, we set the number of training epochs to 200 and use a batch size of 64 unless otherwise specified. The Noam learning rate scheduler is adopted, with an initial and final learning rate of 1×10−31× 10^-3 or 1×10−41× 10^-4, and a maximum learning rate of 2×10−32× 10^-3 or 2×10−42× 10^-4, selected based on validation performance. All regularization weights, including β for the KL divergence term, λ for the MMD term, γ for the alignment loss, δ for the orthogonality constraint, and η for the reconstruction loss, are learnable parameters with initial values set to 0.1. All coefficients are selected based on validation performance and grid search to ensure optimal model training. All hidden dimensions are set to 256. The dimensions of shared and private latent features are selected according to the specific task and dataset, and these settings can be found in detail in our released code. All activation functions are ReLU, and Layer Normalization is applied throughout the network [3, 1]. Residual connections and feedforward neural networks are incorporated for feature enhancement and stabilization [8]. The Adam optimizer is used for training, and gradient clipping is applied to ensure stability [11]. All experiments were conducted on a workstation equipped with an Intel Core i7-14700KF processor (20 cores, 28 threads), 64GB of system memory, and an NVIDIA GeForce RTX 4060 Ti GPU with 16GB VRAM. Under these settings, training the full DMMRL model on all datasets required approximately 2 days to complete. TABLE I: Molecular property prediction results on classification and regression datasets with baselines and DMMRL. Methods ROC-AUC ↑ (Higher is better) RMSE ↓ (Lower is better) BACE BBBP ClinTox Tox21 ESOL FreeSolv Lipophilicity RNN - 0.902± 0.015 0.915± 0.009 0.806± 0.007 0.743± 0.020 1.108± 0.146 0.770± 0.025 Transformer - 0.944± 0.011 0.954± 0.003 0.813± 0.013 0.767± 0.079 1.021± 0.102 0.900± 0.023 GCN - 0.690± 0.036 0.807± 0.051 0.829± 0.041 0.970± 0.050 1.400± 0.135 - Weave - 0.671± 0.065 0.832± 0.023 0.820± 0.024 0.610± 0.055 1.220± 0.250 - MPNN - 0.910± 0.032 0.881± 0.037 0.844± 0.014 0.702± 0.042 1.242± 0.249 0.645± 0.075 N-Gram - 0.912± 0.013 0.855± 0.037 0.842± 0.027 1.100± 0.160 2.512± 0.190 0.876± 0.033 GROVER - 0.955± 0.003 0.929± 0.178 0.842± 0.009 0.911± 0.116 1.987± 0.072 0.643± 0.030 TrimNet 0.841± 0.043 0.889± 0.020 0.948± 0.030 - 0.770± 0.071 1.639± 0.406 1.202± 0.032 FP-GNN 0.881± 0.028 0.935± 0.027 0.840± 0.038 0.815± 0.024 0.675± 0.332 0.905± 0.649 0.625± 0.152 GraSeq - 0.942± 0.012 0.918± 0.005 0.810± 0.004 0.652± 0.039 0.865± 0.032 0.648± 0.041 MvMRL 0.891± 0.019 0.962± 0.011 0.975± 0.019 0.845± 0.011 0.601± 0.055 0.832± 0.128 0.634± 0.018 SGGRL 0.917± 0.020 0.967± 0.010 0.956± 0.016 0.837± 0.013 0.575± 0.057 0.847± 0.116 0.617± 0.025 DMMRL(Ours) 0.925± 0.012 0.968± 0.013 0.935± 0.021 0.842± 0.013 0.535± 0.068 0.825± 0.089 0.599± 0.033 I-B Method Comparison For comprehensive evaluation, DMMRL was systematically compared with twelve established baseline approaches spanning sequence-based, graph-based, geometry-based, and multi-modal molecular representation methodologies. Specifically, Transformer [22] and RNN [4] represent SMILES sequence-based models. TrimNet [14], FP-GNN [2], GCN [12], Weave [10], MPNN [6], and N-Gram [17] constitute graph neural network methods of varying architectural complexity. Additionally, GROVER [20] implements sophisticated message-passing operations on directed graphs. GraSeq [7] combines molecular SMILES and graph representations in a dual-modality approach. Finally, MvMRL [32] and SGGRL [24] represent state-of-the-art multi-modal approaches that integrate graph and three-dimensional geometry information. I-C Results and Analysis Our comprehensive evaluation demonstrates that DMMRL achieved superior performance on five of the seven benchmark datasets (BACE, BBBP, ESOL, FreeSolv, and Lipophilicity), as shown in Table I. For bioactivity classification tasks, DMMRL reached 92.5% and 96.8% ROC-AUC on BACE and BBBP respectively, representing improvements of 0.8 and 0.1 percentage points over the next best methods. For physicochemical property regression tasks, RMSE reductions of 6.9%, 0.8%, and 2.9% were observed on ESOL, FreeSolv, and Lipophilicity respectively compared to state-of-the-art alternatives. Notably, performance advantages were most pronounced on structurally diverse datasets with sparse annotations (BACE, FreeSolv), where effective feature disentanglement becomes particularly critical. On the BACE dataset, which consists of binding affinity data for β-secretase inhibitors spanning diverse chemical scaffolds, DMMRL demonstrated a substantial 0.8 percentage point improvement in ROC-AUC over the previously best-performing model. Similarly, for the challenging FreeSolv dataset (hydration free energy predictions with only 639 samples), DMMRL reduced RMSE by 0.8% compared to the next best method. This empirical finding confirms our theoretical proposition that disentanglement mechanisms are especially valuable when training data is limited or exhibits high structural variability, as they enable the model to more effectively identify and isolate generalizable structure-property relationships from modality-specific noise. The performance improvement pattern across datasets correlates strongly with chemical diversity and data sparsity metrics, suggesting that DMMRL’s disentanglement approach specifically addresses the challenges of learning from heterogeneous molecular data. By forcing the model to encode only the most relevant molecular features in the shared latent space, DMMRL effectively implements an inductive bias that favors generalizable patterns over dataset-specific correlations or artifacts. Interestingly, DMMRL exhibited consistently low standard deviations across multiple evaluation runs, particularly for BBBP (±0.013) and Tox21 (±0.013), indicating that the model produces reliable predictions across different data subsets—a critical advantage for early-stage drug discovery applications where predictive reliability across diverse chemical scaffolds is essential for effective compound prioritization. The combined improvements in both accuracy and consistency demonstrate that disentangled representations enable more robust structure-property modeling, particularly for the complex, multi-factorial molecular properties that characterize pharmaceutical and materials science applications. TABLE I: Ablation study of DMMRL components across classification and regression tasks. Dataset LBL BOT ALL (DMMRL) Classification Tasks (ROC-AUC ↑ ) BACE 0.906± 0.117 0.911± 0.012 0.925± 0.012 BBBP 0.931± 0.011 0.966± 0.048 0.968± 0.013 ClinTox 0.917± 0.036 0.918± 0.018 0.935± 0.021 Tox21 0.822± 0.032 0.840± 0.022 0.842± 0.013 Regression Tasks (RMSE ↓ ) ESOL 0.614± 0.066 0.563± 0.027 0.535± 0.068 FreeSolv 0.878± 0.175 0.855± 0.085 0.825± 0.089 Lipophilicity 0.615± 0.030 0.612± 0.016 0.599± 0.033 I-D Ablation Studies To systematically validate the contribution of each architectural component motivated by our theoretical framework, we conducted detailed ablation experiments with three progressively more complete model configurations: LBL (prediction pathway only, without disentanglement or regularization), BOT (adding variational bottleneck with KL and MMD regularization), and ALL (complete DMMRL with alignment and orthogonality constraints). Table I summarizes these results across all datasets. The ablation analysis revealed consistent and incremental performance improvements with each additional component. The transition from LBL to BOT yielded particularly substantial gains across multiple datasets (3.5 percentage point ROC-AUC improvement for BBBP; 8.3% RMSE reduction for ESOL), demonstrating that the variational constraints effectively function as an information bottleneck that filters representation noise. By imposing distributional regularization on the latent spaces, the BOT configuration forces the model to prioritize encoding the most statistically relevant features while discarding irrelevant details. This filtering effect is particularly pronounced for the ESOL dataset, where solubility prediction depends primarily on specific functional groups and electronic properties rather than complete molecular structure, making effective feature selection crucial for accurate prediction. The transition from BOT to ALL (complete DMMRL) provided further performance improvements, particularly for datasets with complex structure-property relationships such as BACE (1.4 percentage point ROC-AUC increase) and ClinTox (1.7 percentage point increase). This demonstrates that the alignment and orthogonality constraints contribute meaningfully to representation quality by enforcing statistical independence between shared and private factors while ensuring consistency across modalities. For datasets like BACE, where binding affinity depends on precise spatial arrangements and electronic complementarity with the target protein, the additional constraints help isolate the most pharmacologically relevant molecular features by aligning consistent structural patterns across modalities while separating them from modality-specific artifacts. Notably, the complete DMMRL configuration also demonstrated substantially lower prediction variance compared to the partial configurations, particularly for the BBBP and Tox21 datasets. This variance reduction indicates that the full set of constraints not only improves average performance but also enhances model robustness and reliability—a critical consideration for applications in drug discovery where consistent predictions across diverse chemical scaffolds are essential for effective compound prioritization and development decision-making. I-E Discussion Our experimental results clearly demonstrate that even without employing more sophisticated feature encoders, the feature disentanglement strategy alone yields substantial performance improvements. Specifically, while SGGRL employs contrastive learning to align multi-modal features after feature extraction(i.e., sequence, graph, and geometry encoders), our work utilizes the same multi-modal encoders as SGGRL but incorporates a shared-private feature disentanglement strategy, enabling more precise extraction of property-relevant features. This is further confirmed by our ablation experiments, which systematically validate the effectiveness of the disentanglement approach. DMMRL achieves the best performance on five out of seven benchmark datasets, clearly outperforming existing methods in the majority of property prediction tasks. The consistent performance improvements across diverse datasets strongly suggest that molecular features are inherently entangled, and conventional alignment and fusion strategies are insufficient compared to our disentanglement approach. Notably, the advantage of DMMRL is most pronounced on datasets with high structural diversity and limited training samples, indicating that disentanglement provides a particularly strong inductive bias when learning from heterogeneous or sparse data. By focusing on the most generalizable structural features in the shared space while isolating modality-specific artifacts in private spaces, DMMRL effectively filters out noise that would otherwise contaminate the prediction pathway. In summary, our work provides compelling evidence that representation disentanglement is a highly effective strategy for advancing molecular property prediction, and may be even more impactful than simply developing increasingly complex encoder architectures. The ability to systematically separate shared property-relevant features from private modality-specific information establishes a new paradigm for interpretable and robust multi-modal molecular representation learning. IV Conclusion In this paper, we propose DMMRL, directly addressing the dual challenges of representation entanglement and inadequate multi-modal integration identified in molecular property prediction. By combining feature disentanglement through variational autoencoders with adaptive fusion mechanisms, our approach enhances both predictive performance and interpretability. Extensive experiments across diverse benchmark datasets demonstrate that DMMRL not only achieves state-of-the-art results but also produces more stable and generalizable predictions, especially on structurally diverse and data-sparse tasks. The ablation studies further validate the necessity of each module, showing that variational regularization, alignment, and orthogonality constraints collectively contribute to robust and interpretable molecular representations. Future work includes incorporating domain knowledge into regularization terms to further guide the disentanglement process and improve model interpretability. Enhancing fusion mechanisms to capture higher-order and non-linear interactions among modalities may unlock deeper insights into complex structure-property relationships. Adapting the framework for multi-task learning could enable simultaneous prediction of multiple molecular properties, increasing practical utility in drug discovery and materials science. Additionally, while this work focuses on random splitting for evaluation, exploring scaffold-based splitting could further assess the model’s ability to generalize to novel molecular scaffolds and unseen chemical spaces, providing a more rigorous test of real-world applicability. Overall, DMMRL lays a solid foundation for future research in interpretable and generalizable multi-modal molecular representation learning. References [1] J. L. Ba, J. R. Kiros, and G. E. Hinton (2016) Layer normalization. External Links: 1607.06450, Link Cited by: §I-A. [2] H. Cai, H. ZMolecular property predictionhang, D. Zhao, J. Wu, and L. Wang (2022) FP-gnn: a versatile deep learning architecture for enhanced molecular property prediction. Briefings in bioinformatics 23 (6). Cited by: §I, §I-B. [3] Y. Chen, X. Dai, M. Liu, D. Chen, L. Yuan, and Z. Liu (2020) Dynamic relu. In European conference on computer vision, p. 351–367. Cited by: §I-A. [4] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. Cited by: §I, §I-B. [5] R. Dey and F. M. Salem (2017) Gate-variants of gated recurrent unit (gru) neural networks. In 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS), p. 1597–1600. Cited by: §I-D. [6] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl (2017) Neural message passing for quantum chemistry. In International conference on machine learning, p. 1263–1272. Cited by: §I-B. [7] Z. Guo, W. Yu, C. Zhang, M. Jiang, and N. V. Chawla (2020) GraSeq: graph and sequence fusion learning for molecular property prediction. In Proceedings of the 29th ACM international conference on information & knowledge management, p. 435–443. Cited by: §I-B. [8] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778. Cited by: §I-A. [9] S. Hu, Z. Lou, X. Yan, and Y. Ye (2024) A survey on information bottleneck. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (8), p. 5325–5344. Cited by: §I-C. [10] S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley (2016) Molecular graph convolutions: moving beyond fingerprints. Journal of computer-aided molecular design 30 (8), p. 595–608. Cited by: §I-B. [11] D. P. Kingma (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Cited by: §I-A. [12] T. Kipf (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. Cited by: §I-B. [13] M. Lee and V. Pavlovic (2021) Private-shared disentangled multimodal vae for learning of latent representations. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, p. 1692–1700. Cited by: §I-F. [14] P. Li, Y. Li, C. Hsieh, S. Zhang, X. Liu, H. Liu, S. Song, and X. Yao (2021) TrimNet: learning molecular representation from triplet messages for biomedicine. Briefings in Bioinformatics 22 (4). Cited by: §I, §I-B. [15] Z. Li, M. Jiang, S. Wang, and S. Zhang (2022) Deep learning methods for molecular representation and property prediction. Drug Discovery Today 27 (12), p. 103373. Cited by: §I, §I-A. [16] Z. Liao, L. Xie, H. Mamitsuka, and S. Zhu (2023) Sc2Mol: a scaffold-based two-step molecule generator with variational autoencoder and transformer. Bioinformatics 39 (1), p. btac814. Cited by: §I. [17] S. Liu, M. F. Demirel, and Y. Liang (2019) N-gram graph: simple unsupervised representation for graphs, with applications to molecules. Advances in neural information processing systems 32. Cited by: §I, §I-B. [18] Z. Niu, G. Zhong, and H. Yu (2021) A review on the attention mechanism of deep learning. Neurocomputing 452, p. 48–62. Cited by: §I-D. [19] Y. Peng, Z. Zhang, Q. Jiang, J. Guan, and S. Zhou (2019) Top: towards better toxicity prediction by deep molecular representation learning. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), p. 318–325. Cited by: §I. [20] Y. Rong, Y. Bian, T. Xu, W. Xie, Y. Wei, W. Huang, and J. Huang (2020) Self-supervised graph transformer on large-scale molecular data. Advances in neural information processing systems 33, p. 12559–12571. Cited by: §I-B. [21] P. Schneider, W. P. Walters, A. T. Plowright, N. Sieroka, J. Listgarten, R. A. Goodnow Jr, J. Fisher, J. M. Jansen, J. S. Duca, T. S. Rush, et al. (2020) Rethinking drug design in the artificial intelligence era. Nature reviews drug discovery 19 (5), p. 353–364. Cited by: §I. [22] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: §I, §I-B. [23] W. P. Walters and R. Barzilay (2020) Applications of deep learning in molecule generation and molecular property prediction. Accounts of chemical research 54 (2), p. 263–270. Cited by: §I. [24] Z. Wang, T. Jiang, J. Wang, and Q. Xuan (2024) Multi-modal representation learning for molecular property prediction: sequence, graph, geometry. arXiv preprint arXiv:2401.03369. Cited by: §I, §I-B, §I-B. [25] P. B. Weerakody, K. W. Wong, G. Wang, and W. Ela (2021) A review of irregular time series data handling with gated recurrent neural networks. Neurocomputing 441, p. 161–178. Cited by: §I-D. [26] J. Wu, Y. Su, A. Yang, J. Ren, and Y. Xiang (2023) An improved multi-modal representation-learning model based on fusion networks for property prediction in drug discovery. Computers in biology and medicine 165, p. 107452. Cited by: §I. [27] T. Wu, Y. Tang, Q. Sun, and L. Xiong (2023) Molecular joint representation learning via multi-modal information of smiles and graphs. IEEE/ACM transactions on computational biology and bioinformatics 20 (5), p. 3044–3055. Cited by: §I. [28] Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande (2018) MoleculeNet: a benchmark for molecular machine learning. External Links: 1703.00564 Cited by: §I-A. [29] J. Xia, L. Zhang, X. Zhu, Y. Liu, Z. Gao, B. Hu, C. Tan, J. Zheng, S. Li, and S. Z. Li (2023) Understanding the limitations of deep models for molecular property prediction: insights and solutions. Advances in Neural Information Processing Systems 36, p. 64774–64792. Cited by: §I. [30] J. Xu, Y. Ren, H. Tang, X. Pu, X. Zhu, M. Zeng, and L. He (2021) Multi-vae: learning disentangled view-common and view-peculiar visual representations for multi-view clustering. In Proceedings of the IEEE/CVF international conference on computer vision, p. 9234–9243. Cited by: §I-F. [31] S. Yang, Z. Li, G. Song, and L. Cai (2021) Deep molecular representation learning via fusing physical and chemical information. Advances in neural information processing systems 34, p. 16346–16357. Cited by: §I. [32] R. Zhang, Y. Lin, Y. Wu, L. Deng, H. Zhang, M. Liao, and Y. Peng (2024) MvMRL: a multi-view molecular representation learning method for molecular property prediction. Briefings in Bioinformatics 25 (4), p. bbae298. Cited by: §I, §I-B.