RetNet: Revolutionizing Gas Adsorption Predictions in Metal-Organic Frameworks

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Feb 5 2024

In a paper published in the journal Scientific Reports, researchers emphasized the potential of metal-organic frameworks (MOFs) for gas adsorption due to their inherent porosity. The study introduced a novel machine-learning (ML) framework to address the challenge of selecting optimal candidates from various structures.

Using the potential energy surface (PES) as the sole descriptor, voxelized and processed through a 3D convolutional neural network (CNN), the model outperformed traditional geometric descriptor models for predicting gas adsorption in MOFs. Notably, it demonstrated superior performance with two orders of magnitude less training data and showcased transferability to covalent organic frameworks (COFs), highlighting its generic applicability beyond reticular chemistry.

Related Work

Past research in reticular chemistry has harnessed the vast chemical space offered by MOFs, particularly in gas adsorption applications such as carbon capture and storage. However, the challenge lies in efficiently identifying optimal MOF candidates from extensive databases. ML has proven effective, with geometric descriptors commonly used for gas adsorption predictions.

Voxelized PES and CNN

The approach to computing the voxelized PES involves overlaying a 3D grid onto the material's unit cell. The grid size, denoted as N×N×N, is chosen as a trade-off between resolution and computational cost. Each voxel at a specific grid point (i,j,k) is then designated the interaction energy resulting from a probe molecule's presence at that position about the framework atoms. The grid size and potential type are treated as hyperparameters to manage information content and computational cost trade-offs.

Filling all voxels with energy values obtained from ab-initio calculations is the final step in representing the PES. It involves utilizing a spherical probe molecule and approximating host-guest interactions with the Lennard-Jones potential.  A CNN processes the voxelized PES in the proposed framework to predict gaseous adsorption properties. CNNs, specialized for image-like data, utilize convolutional layers for hierarchical feature extraction and pooling layers for downsampling.

The convolutional layers consist of filters performing template matching to extract features, with deeper layers capturing higher levels of abstraction. Pooling layers, such as max pooling, downsample feature maps by substituting outputs in a small neighborhood with a summary statistic, reducing computational load and the risk of overfitting.

The voxelized PES, serving as the sole descriptor, provides a data-driven approach to feature extraction, eliminating the need for manual feature selection. The convolutional and pooling layers of the CNN facilitate the extraction of meaningful features from the 3D energy image, allowing the model to capture essential information about the material's sorption behavior. This approach differs from previous studies using indirect representations of the PES, ensuring maximum information content by directly using the PES as a descriptor. The flexibility of this methodology extends its applicability to various host-guest systems for predicting diverse adsorption properties.

Researchers initiated the process by introducing and utilizing the Python package to facilitate the calculation of energy voxels. The study employs a spherical probe molecule and approximates host-guest interactions with the Lennard-Jones potential for easy calculation. This proof-of-concept study showcases the potential of the proposed framework by predicting gas uptake in MOFs and COFs, demonstrating its transferability and superiority over conventional schemes employing geometric descriptors. The voxelized PES and the CNN architecture provide an efficient and generalizable approach for predicting gaseous adsorption properties in diverse materials.

RetNet Insights, Trends

The visualization of RetNet, the neural network trained on the MOFs dataset, offers insights into its internal processing of energy voxels. Examining feature maps from the first five layers reveals that conv1 and conv2 layers highlight the texture of the material's structure, with conv2 providing a downsampled version of conv1. The MaxPool2 layer precedes two consecutive conv layers, leading to a Flatten layer that converts feature maps into a single vector processed by a fully connected neural network. The output layer extracts a fingerprint from the PES as a linear layer. It utilizes a linear model for predicting gas uptake, a process distinguishable from methods using hand-crafted fingerprints.

Researchers present the learning curves for ML models constructed with energy voxels and geometric descriptors. The CNN model outperforms the MOF dataset's random forest (RF) model, even with a basic PES approximation. Notably, the CNN performs better with two orders of magnitude fewer training samples. Similar trends are observed in the COFs dataset, emphasizing the CNN's enhanced generalization. The learning curves highlight the increased information content of the voxelized PES and the CNN's ability to process image-like data, contributing to its superior performance.

The study discusses the impact of the LJ potential in approximating the voxelized PES for molecules like dinitrogen (N2N2​), underscoring the importance of refining the potential over increasing resolution to maximize information content and model performance. Furthermore, the modular nature of the proposed framework, rooted in ubiquitous interactions, extends its applicability beyond reticular chemistry.

The framework's potential for predicting the properties of organic molecules is suggested by voxelizing the electrostatic potential map for applications such as solubility prediction. Incorporating transfer learning techniques, leveraging knowledge gained from solving original tasks to address new but similar tasks, is highlighted as a means to enhance data efficiency.

Conclusion

To sum up, RetNet's internal processing visualization sheds light on its practical analysis of energy voxels in the MOFs dataset. Learning curves demonstrate the CNN model's superiority over geometric descriptors, showcasing enhanced performance with minimal training samples. Researchers discussed the LJ potential's impact on voxelized PES approximation, highlighting the importance of refining potential representations. The modular framework's versatility extends beyond reticular chemistry, offering potential applications in predicting organic molecule properties.