Autoencoders in Molecular Design: A Comprehensive Overview

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Nov 26 2023

In an article published in the journal Nature, researchers delved into autoencoders and their algorithmic foundations and applications, emphasizing their role in data-driven molecular representation and constructive molecular design.

Background

In molecular informatics and drug discovery, the application of deep learning, notably autoencoders, has showcased remarkable versatility. These unsupervised neural networks, conceived in the late 1980s, play a crucial role in reconstructing meaningful molecular representations. Autoencoders excel in two primary tasks: creating data-driven digital representations of molecules and facilitating de novo molecular design. The flexibility of autoencoders in handling diverse molecular data types has led to advancements, particularly with the introduction of the variational autoencoder (VAE).

The VAE's ability to generate new samples from training set distributions has revolutionized generative deep learning in molecular informatics. The study explored the foundational concepts of molecular autoencoders, detailing their architecture, training mechanisms, and the influential role of VAEs. Moreover, the diverse landscape of molecular representations, from descriptors to graph-based methods and string-based representations like simplified molecular-input line-entry systems (SMILES), was discussed.

The selection of an appropriate molecular representation emerged as a critical decision, impacting the performance and applicability of computational models. Researchers aimed to provide a comprehensive understanding of autoencoders in molecular informatics, highlighting their strengths, challenges, and the evolving landscape in drug discovery.

Autoencoders for Molecular Representation

Autoencoders play a pivotal role in molecular informatics, offering a mechanism to serialize and compress complex molecular data into fixed-length vectors of real numbers. This latent space representation proved valuable in various applications:

• Dimensionality Reduction and Visualization: Autoencoders nonlinearly reduce data dimensionality, aiding in the visualization of complex datasets for enhanced data exploration and analysis.
• Preprocessing for Downstream Tasks: The latent space served as a preprocessing step for downstream prediction tasks, enhancing the performance of subsequent machine learning models.
• Improved Performance Over Linear Models: Autoencoders captured nonlinear relationships, outperforming traditional linear models, especially with large and complex datasets.
• Computational Efficiency: They were computationally efficient, making them suitable for handling large datasets in scenarios with limited computational resources.

In molecular informatics, autoencoders were extensively applied with SMILES strings as input, resulting in successful fixed-length vector representations in the latent space. This approach allowed for data augmentation and molecular similarity assessments. Co-learning with property prediction models enriched the latent space, enhancing predictive performance.

Autoencoders were versatile, extending to hybrid models combining graph-based and SMILES-based representations. They effectively captured 3D conformational information and could serve as a pretraining step for downstream prediction tasks. Additionally, the decoder in autoencoders proved versatile, facilitating tasks such as transforming molecular representations back into SMILES format for better interpretability and broader applicability.

Generative Autoencoders in Molecular Design

Autoencoders, particularly VAEs, are widely used in de novo molecule design. The choice of molecular representation in training these models significantly impacted their architecture and challenges. SMILES strings were commonly used, allowing diverse neural network architectures, including convolutional and recursive networks.

Early VAE applications faced challenges in ensuring the validity of reconstructed SMILES strings. Solutions included incorporating formal grammar, modifying sampling procedures, and introducing alternatives like Self-Referencing Embedded Strings (SELFIES). Recent advances explored Graph Neural Networks (GNNs) as an alternative, explicitly encoding molecular structure. Initial challenges in generating entire molecular graphs at once led to constructive models, allowing step-wise generation with intermediate validity checks.

Several graph-based molecule generation approaches emerged:

• Sequential graph generation: Building the molecular graph node by node, ensuring validity at each step.
• Two-step generation: First, generating nodes or fragments independently, then determining connectivity in a second step.
• Coarse-grained fragments: Using chemical or data-based fragments for hierarchical reconstruction, improving accuracy for larger molecules.

Deep learning was applied to generate energetically favorable molecular conformers, with VAE models conditioned on 2D graphs. This approach outperformed traditional heuristics, offering computational efficiency for subsequent geometry optimization.

Reconstructing both molecular graph and 3D geometry from the latent space was challenging. Various methods used matrix-based representations, GNNs, and equivariant graph convolutions for effective 3D conformation reconstruction.

Targeted Molecule Generation

Autoencoders facilitate de novo molecular design by:

1. Conditional VAE (CVAE): Generates molecules conditioned on specific properties
2. Semi-supervised VAE (SSVAE): Leverages both labeled and unlabeled data, enhancing chemical diversity.
3. Transfer learning: Uses the latent space to represent training property predictors.
4. Optimization Techniques: Decodes molecules from 'painted' latent space, optimizing for desired properties.

Methods for diverse molecule generation include:

• Sampling from Latent Space: Exploring chemical space around seed molecules.
• Diversity Layer: Promoting chemical diversity in generated molecules.
• Latent Space Optimization: Biasing latent space towards desirable properties through active learning or dataset biasing.

Comparison with Other Generative Models

Autoencoders were compared with other generative models like RNNs, GANs, Normalizing Flows, and Reinforcement Learning. Combinations of autoencoders with GANs (Adversarial Autoencoders), RNNs, and Reinforcement Learning were explored, providing a rich set of techniques for generating novel molecules in drug discovery and design. Each model exhibited strengths, and their combination could contribute to further advancements in molecular informatics.

These generative models had applications in diverse tasks, including molecule validity, novelty, and property optimization. While autoencoders were effective, their integration with other models broadened the scope of molecular design and drug discovery.

Conclusion

In conclusion, the autoencoder, originally designed for data compression, found new applications in molecular representation by introducing VAEs. This revival allowed unbiased exploration of chemical space, departing from human-designed features. The autoencoder's compression of input data into compact vectors held the potential for understanding pharmacological phenomena and drug-like chemical space. VAEs, with a meaningful latent space, contributed to de novo drug design, enabling the targeting of molecules with specific properties.

They played a crucial role in complex computational pipelines, capturing and transforming complex information for downstream tasks. The adaptability of autoencoder architecture to various data structures in drug design, beyond molecular representations, showcases their versatility. Ongoing efforts are needed to address challenges with increasingly complex data structures and fully unlock autoencoder potential in advancing drug discovery and molecular design efforts.