Revolutionizing Renewable Energy Forecasting with Graph Patch Informer

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Aug 16 2023

Accurate renewable energy forecasting (REF) is crucial for stable power systems. In a recent paper published in the journal Energy, researchers proposed the Graph Patch Informer (GPI) for REF. It features segment-wise self-attention, graph attention networks (GATs), and a self-supervised pre-training approach.

Background

The growing demand for renewable energy due to advancements in various sectors, such as building energy, electricity, and transportation, has led to a surge in interest. Solar, hydropower, wind, and geothermal energy have gained popularity worldwide. However, these sources are susceptible to weather changes, posing challenges to the supply-demand balance. Researchers have invested in REF methods to tackle this. Physical and data-driven models have emerged.

Deep learning, particularly transformer-based models, is promising for REF. Despite successes, long-term modeling remains challenging. Transformer's success in various artificial intelligence (AI) fields motivated its use in REF. Yet, linear models have surprisingly outperformed transformers. To address this, a new model, GPI, is introduced. It mitigates temporal information loss, incorporates GATs, and utilizes self-supervised pre-training.

GPI: A transformer-based approach

In the current study, researchers introduced a novel transformer-based solution for REF, GPI. It uses pre-training followed by fine-tuning. This strategy facilitates abstract-level representation learning for downstream tasks. During pre-training, GPI's core mirrors the Informer Encoder, although it incorporates a patch operation to enhance abstract representation learning. Self-supervised learning augments pre-training, yielding informative weights. For fine-tuning, researchers introduced GATs with a self-adaptive adjacency matrix to enhance inter-temporal information propagation. After pre-training, GAT and linear layers are sequentially applied, refining supervised learning. Once trained, GPI performs end-to-end predictions. Pre-training details encompass patch operation, while Informer Encoder features embedding and multi-head prob attention layers. GAT's self-adaptive adjacency matrix supports inter-temporal correlation, ultimately augmenting transformer-based models for REF tasks.

Empowering REF through GPI

Graph Patch Informer's applicability in REF spans multiple downstream tasks, negating manual feature engineering. This includes wind speed, wind power, solar radiation, and photovoltaic power forecasting. These tasks follow the same end-to-end process, utilizing historical data with weather variables as input and yielding predicted values.

GPI offers several advantages: (1) generalization across tasks, reducing cost in wind-solar power generation; (2) robustness against missing solar data relevant in industry; (3) enhanced accuracy in solar-related forecasting with a longer history. Applying GPI to REF tasks involves predicting renewable energy values from historical data. For instance, solar radiation prediction utilizes global horizontal irradiance, airmass, zenith angle, and temperature as input features and similar approaches are used for wind speed, wind power, and photovoltaic power forecasting.

Experimental evaluation and performance analysis

The proposed model was tested on four downstream tasks: wind speed (WS), solar radiation (SR), wind power (WP), and photovoltaic power (PVP). These tasks were evaluated using respective datasets. Data standardization using the training set mean and standard deviation was employed to avoid future information leakage. To validate the model's effectiveness, five experiments were performed. Various benchmark models were compared, including long short-term memory (LSTM), long short-term time-series network (LSTNet), temporal convolutional network (TCN), DLinear, Transformer, Informer, and Autoformer. Three evaluation metrics, mean absolute error (MAE), mean square error (MSE), and promoting percentages of MSE (PMSE), were used.

In Experiment I, the GPI model's performance was compared to benchmark models in terms of MAE and MSE across different forecasting horizons. GPI consistently outperformed other models, showcasing significant improvements over Autoformer and TCN for various tasks. Specifically, GPI achieved 20.76% to 31.19% lower MAE and 23.67% to 40.75% lower MSE for SR and PVP tasks compared to Autoformer. Similarly, it surpassed TCN by 2.53% to 24.64% on MAE and 5.13% to 31.03% on MSE for WS and WP tasks.

Experiment II focused on the effectiveness of using GAT for transformer-based models. GAT-based variants demonstrated improvements ranging from 0% to 12.33% in MSE over the original Transformer variants. Notably, the performance gain was more pronounced for longer forecasting horizons, indicating that GAT enhances the models' ability to learn inter-temporal dependencies.

Experiment III explored the benefits of self-supervised learning applied to GPI and benchmark models during pre-training. This approach led to performance enhancements in various datasets and models, including GPI, Transformer, and TCN. Applying self-supervised learning improved the models' accuracy on SR, PVP, and WS datasets.

In Experiment IV, the study highlighted additional advantages of self-supervised learning for GPI. The self-supervised learning approach demonstrated its potential to recognize changing patterns even when dealing with missing values, which is particularly beneficial for solar data. Notably, GPI outperformed supervised learning when the input data had missing values.

Experiment V delved into GPI's ability to learn from extended historical information by varying the look-back window size. The accuracy of forecasts generally improved with a longer history for SR and PVP tasks. However, WS and WP tasks exhibited varied results due to the intermittent nature of the data.

Conclusion

In summary, the researchers introduced GPI, a novel transformer-based model addressing challenges in REF. GPI integrates the Informer Encoder with patch operation and two-layer GAT through self-supervised pre-training and fine-tuning. Comprehensive experiments reveal its superiority over benchmarks, achieving state-of-the-art performance.

GAT enhances GPI and other Transformer models, while self-supervised learning aids performance. GPI mitigates missing solar data effects, excels in long-term forecasting, and opens new REF possibilities. Future work entails refining temporal information loss solutions, extending GPI to long-term solar forecasting, and exploring GAT-attention synergy for better representation and interpretability.