ML-based Data-Driven Framework Uncovers Conservation Laws

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Aug 16 2024

In a paper published in the journal Machine Learning Science and Technology, researchers developed a data-driven computational framework to identify conservation laws in systems without known dynamics. Their method leveraged the stability of singular vectors to noise, enabling accurate identification with minimal data and limited parameter tuning. While the primary focus was biological systems, the framework proved versatile and could be applied to various data science applications, integrating seamlessly with other machine learning (ML) approaches.

Background

Past work has extensively explored analytical and computational methods for identifying conservation laws in dynamical systems, including manifold learning, neural networks, and Koopman theory. However, these approaches often require large data sets and help with noise, particularly in higher-order systems. Additionally, challenges remain in selecting appropriate functional libraries and optimal parameters, which can hinder the accurate recovery of conservation laws.

Conservation Discovery

This work focuses on developing a robust data-driven method to identify all possible conservation laws and their functional forms for systems modeled by differential equations. The method translates the problem into solving the null space of a data matrix derived from time series data, enabling the identification of conservation laws even in noisy environments. Key challenges addressed include choosing the appropriate library, determining the existence of multiple conservation laws, and identifying the minimal data required for accurate analysis. This framework provides a systematic, user-agnostic strategy for discovering conservation laws across various applications.

Handling Perturbation Effects

To develop a robust and stable numerical algorithm for identifying conservation laws, it is crucial to understand the impact of noise on solutions produced by the SVD-based null-space estimation. While Weyl's theorem provides bounds on singular values under perturbations, the practical application requires adapting these results to the specific structure of the Θ-library matrix. Numerical experiments using synthetic data reveal the sensitivity of singular values to Gaussian noise, guiding the selection of measurement points and Θ-library functions.

Theorems like Weyl's and Wedin's, alongside Tikhonov regularization, offer insight into perturbation effects on singular values and vectors, emphasizing the importance of having sufficient observations relative to library functions. This analysis informs the development of a computational framework that automates the identification of conservation laws, ensuring accuracy and stability by appropriately selecting data points and reducing candidate functions.

Optimal Library Selection

To identify the optimal Θ-library for learning conservation laws, the proposed algorithm evaluates a set of candidate libraries by analyzing their corresponding SVDs. Each library's effectiveness is assessed by examining the singular values and vectors to determine which library minimizes extraneous terms while accurately reflecting the conservation laws. The process computes the singular value decomposition for each candidate library, identifies 'ghost' values below a cutoff, and selects the optimal library based on the largest spectral gap between significant and minimal singular values.

Once the optimal Θ-library is identified, the corresponding right singular vectors define the coefficients for the terms in the chosen library. The team refined recovered conservation laws by converting them to their reduced row echelon form (RREF). It simplifies the representation and may reveal insights into the underlying system structure. The algorithm systematically tests each library, computes necessary matrices, and evaluates the number of nonzero entries to ensure that the identified conservation laws are accurate and minimal regarding the number of terms included.

Algorithm Performance Summary

The numerical implementation of algorithm 1 was applied to the examples with noise variance considered across different candidate libraries. The libraries, represented by triples (a, b, c), include combinations of polynomial terms up to order 3, trigonometric, and logarithmic terms. The team evaluated the effectiveness of each candidate library using SVD and analyzed the results based on how the singular values approached a predefined cutoff.

The aim was to select a library that minimizes extraneous terms while accurately reflecting the conservation laws. Data was simulated from known ordinary differential equation (ODE) systems, with derivative data generated using numerical differentiation and Tikhonov regularization applied to manage noise. Researchers plotted the singular values for each library and identified the optimal one based on the largest spectral gap between significant and minimal singular values.

The results showed varying success in recovering conservation laws: first-order polynomial libraries achieved 100% accuracy in example 1, while combinations of polynomials and logarithmic terms were effective for nonlinear laws in example 3. The algorithm also demonstrated its ability to avoid overfitting in systems without true conservation laws and consistently recovered linear conservation laws in the pathway example.

Conclusion

To sum up, a robust data-driven computational framework was developed for identifying conservation laws without prior knowledge of system dynamics. The approach demonstrated that singular vectors' relative stability to noise enabled accurate reconstruction of conservation laws with minimal data and parameter tuning. Although the primary focus was biological systems, the framework proved adaptable to various data science applications and could be integrated with other ML methods. This work highlights the potential of data-driven techniques in advancing the identification of conservation laws across different domains.