SCANN - Interpretable Deep Learning for Structure-Property Relationships

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Dec 12 2023

In a paper published in the journal Computational Materials, researchers proposed an interpretable deep learning (DL) architecture incorporating attention mechanisms to overcome limitations in understanding the relationship between material structure and properties.

Evaluating their approach with established and in-house datasets, the models demonstrated strong predictive capabilities comparable to state-of-the-art methods. Their findings highlighted the significance of attention mechanisms in interpreting structure-property relationships, offering the potential for accelerating material design by explicitly identifying crucial features within material structures.

Background

Materials science faces challenges in designing materials with specific properties, often relying on time-consuming experimentation and computational simulations. Materials informatics (MI) emerged to expedite this process by using data-driven methods to extract knowledge from experimental and computational data, facilitating the discovery of superior materials.

Researchers characterized traditional materials based on compositions and structures, but computational experiments have limitations due to exhaustive calculations and specialized knowledge requirements. MI, however, transforms data descriptions into representations for mathematical inference, which is crucial for uncovering patterns in material data. Recent advancements in DL for MI show promise, yet their interpretability remains a challenge. Existing DL models need more transparency, hindering a comprehensive understanding of material-structure-property relationships.

SCANN: Interpretable DL for Materials

The methodology employed in this study focuses on characterizing material structures and developing an interpretable DL architecture, Self-Consistent Attention Neural Network (SCANN), to predict material properties while elucidating structure-property relationships. Researchers perceive the material structure as a collection of local structures composed of a central atom, its neighbors, and their specific arrangement.

O’Keeffe's definition guides the identification of neighboring atoms using Voronoi polyhedron faces, facilitating the segmentation of structures into local arrangements. Geometrical influences among atoms are quantified via a vector representation, capturing distances and solid angles, enabling the conveyance of structural information to SCANN.

SCANN's architecture involves local attention layers to learn representations of local structures iteratively. These layers utilize atomic and geometrical arrangements, capturing long-range interactions and updating representations based on neighboring structures and geometric influences. Multiple local attention layers facilitate information flow among local structures, which is critical for understanding material properties within a structure. Introducing a global attention layer integrates representations of local structures based on attention scores to represent the overall material structure comprehensively.

This layer quantifies attention distributions across local structures, capturing a comprehensive representation of the entire structure. The resulting representation is then employed to predict material properties accurately, while attention scores help interpret the structure-property relationships.

SCANN+ extends SCANN by incorporating the Voronoi solid angle information, refining geometry influences between neighbors, and enhancing model performance and interpretability. Model training involves optimization techniques and a defined loss function, with parameter initialization and regularization strategies to ensure robust training.

Experiments utilize diverse datasets: QM9 for drug-like organic molecules, Fullerene-MD for fullerene molecules, Pt/graphene-MD for platinum-graphene systems, SmFe12-CD for crystalline magnetic materials, and MP-2018.6.13 for crystal structures. Each dataset offers specific properties (such as energy levels, deformation, and formation energies) for evaluating SCANN's predictive capability and interpretability. These datasets, encompassing varied material structures and properties, facilitate a comprehensive assessment of SCANN's performance and interpretative capacity across multiple domains in materials science.

SCANN: Materials Insight through DL

The study introduces the SCANN, a DL architecture tailored for comprehending material structures and predicting their properties. SCANN effectively represents material structures by iteratively learning representations of local atomic arrangements and combining these representations to grasp the overall material structure. Employing attention mechanisms, SCANN excels in capturing long-range interactions among these local structures and unveiling critical structure-property relationships. It differs from conventional methods by using global attention scores to weigh the contributions of individual local structures, enabling more nuanced insights into structure-property connections.

The evaluation of SCANN's performance across various datasets showcases its predictive capabilities and interpretability in diverse material domains. It predicts properties such as energy levels, deformation, and formation energies, demonstrating competitive performance compared to established DL models. SCANN+'s modifications further enhance predictions, especially for properties sensitive to geometric structures. Notably, SCANN's accuracy aligns closely with the chemical accuracy thresholds, underlining its practical applicability in predicting material properties accurately.

Moreover, the study delves into the interpretability of SCANN by examining the learned attention scores of local structures about molecular orbitals, structural deformations, and crystal stability. The correspondence between SCANN-derived attention scores and quantum mechanical results highlights its capacity to unveil structure-property relationships across molecular and crystal structures. This interpretative capability empowers scientists to understand how material properties relate to specific structural features, guiding future material design and discovery efforts.

Conclusion

In summary, the SCANN architecture revolutionizes material understanding and prediction through its iterative learning, capturing local atomic arrangements and global structures. Its attention mechanisms unravel complex structure-property relationships, surpassing traditional methods by weighing individual structural elements' contributions. Across diverse datasets, SCANN demonstrates robust predictive abilities, aligning closely with chemical accuracy thresholds. Its interpretability links attention scores to molecular structures and crystal stability and offers profound insights into material properties, guiding future design and discovery endeavors.