Invertible Neural Networks Solve Bayesian Inverse Problems

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Aug 5 2024

In an article published in the journal Machine Learning Science and Technology, researchers introduced physics-informed invertible neural networks (PI-INN) to address Bayesian inverse problems.

PI-INN utilized an INN to model the relationship between parameter fields and solution functions. It decomposed latent variables into expansion coefficients and noise, enabling accurate posterior distribution estimates even without labeled data. The approach included a new loss function to ensure independence between components, validated through numerical experiments.

Background

Inverse problems, prevalent in fields such as seismic tomography and medical imaging, involve inferring system parameters from indirect measurements. Traditional approaches like regularization and Bayesian inference often struggle with computational efficiency and high-dimensional settings.

Variational inference and deep learning offer alternatives but typically require labeled data, which is frequently unavailable. Advances in physics-informed neural networks (PINNs) and INNs have shown potential but still face challenges with data requirements and uncertainty quantification.

This paper introduced a novel approach, PI-INN, to address these limitations. PI-INN integrated a neural operator model with INN techniques to efficiently approximate Bayesian posteriors without relying on labeled data. It employed a neural network framework where the branch network, designed as an INN, mapped the parameter field to a combination of expansion coefficients and noise. This design allowed PI-INN to provide accurate estimates of the posterior distribution of parameters based on the solution function.

Methodology and Implementation of the PI-INN Model

Building on the concept of invertible deep operator network (DeepONets), the PI-INN combined neural operator models with INNs. The key innovation of the PI-INN was the incorporation of noise in the latent space, which helped manage the uncertainty inherent in inverse problems where information was often lost during the parameter-to-solution mapping.

The PI-INN employed a branch network implemented as an INN to model both forward and inverse operators simultaneously. The training process of the PI-INN involved minimizing errors in forward operator estimation and ensuring that the noise in the model maintained statistical independence from the solution function. This approach allowed the PI-INN to efficiently approximate Bayesian posteriors without requiring extensive labeled data or computationally intensive sampling methods.

The model's effectiveness was demonstrated through its ability to solve inverse problems by providing accurate Bayesian approximations of parameter fields based on given solutions.

Advancements of PI-INN Over Invertible DeepONet

Unlike the DeepONet, which established a direct invertible mapping between parameter fields and solution functions, PI-INN modeled the parameter field as a combination of solution and noise components. This adjustment allowed PI-INN to better address information loss and stochastic effects. Training PI-INN involved ensuring independence between solution components and noise, a challenge compounded by the difficulty in assessing independence directly. While maximum mean discrepancy (MMD)-based methods have been used, PI-INN's independence loss function offered a more efficient and flexible approach, as demonstrated by superior results in experiments with labeled data.

Numerical Experiments

Various numerical experiments validated the effectiveness of the PI-INN model. The performance of the PI-INN’s independence loss term was compared with the MMD loss term using inverse kinematics and stochastic diffusion equations. Key metrics included re-simulation error, calibration error, and Wasserstein distance, with results showing that PI-INN generally outperformed INN-MMD and approximate Bayesian computation (ABC) in terms of calibration error and predictive accuracy. The PI-INN’s advantage was especially evident with smaller datasets.

For one-dimensional (1-D) and two-dimensional (2-D) diffusion equations, PI-INN was tested against conditional INN (cINN) and ABC. PI-INN demonstrated superior re-simulation error and calibration metrics compared to cINN, with comparable performance to ABC. In scenarios with Gaussian random fields, PI-INN consistently produced results close to ABC’s, particularly in posterior distribution shapes. The computational cost analysis revealed that while PI-INN required more training time than cINN, it offered efficient performance in generating posterior samples.

Conclusion

In conclusion, the researchers introduced PI-INN, a novel method for solving Bayesian inverse problems using INNs. By integrating a branch network with an independence loss function, PI-INN effectively modelled the relationship between parameter fields and solution functions without relying on extensive labeled data.

Compared to traditional and recent methods like DeepONet, PI-INN demonstrated superior performance in terms of calibration error and predictive accuracy, particularly with smaller datasets. Numerical experiments validated its efficacy in 1-D and 2-D diffusion equations. Future work will focus on enhancing its real-world applicability and addressing challenges such as unmodeled partial differential equations errors and measurement noise.