DCTN: A Novel DCNN-Transformer Model for Climate Change Impact Evaluation

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Aug 5 2023

In a paper published in the journal Environmental Research, researchers proposed a novel end-to-end deep runoff prediction model called deep convolutional neural network and Transformer (DCTN) that addresses the global challenge of climate change and its impact on runoff patterns. The model combines a deep convolutional neural network and Transformer, allowing it to extract local features from climate data while capturing long-term dependencies through self-attention. Experiments conducted at the Shanjiaodi hydrology station in the Dagu River Basin show a significant improvement of approximately 30.9% compared to traditional models.

Background

Accurate hydrologic forecasting is essential for effective water resource management, considering the complex nature of runoff influenced by factors like soil moisture, atmospheric temperature, and land vegetation. Traditional physically-based and statistical methods have limitations in capturing nonlinear characteristics of runoff. Recent advancements in computer
technologies have led to the popularity of artificial intelligence approaches, including deep learning models like CNN and Transformer. However, existing deep learning-based methods still face challenges in learning long-term
dependencies. 
To address this challenge, the authors propose DCTN, a novel end-to-end deep runoff prediction model that combines DCNN and Transformer. DCTN effectively captures both local and global features, improving prediction performance. The study applies DCTN to forecast runoff in the Dagu River basin and assesses climate change impacts using CMIP6 projections. 

The key contributions of the paper are:

• Introducing the novel DCTN model for reliable monthly runoff prediction, enhancing hydrological forecasting.

• Comparative analysis with five models using historical data from Shanjiaodi hydrological station for monthly runoff prediction.

• Demonstrating superior performance of DCTN over benchmarking and existing deep prediction methods in historical runoff forecasting.

• Providing valuable future runoff projections for the mid-century and late-century, facilitating informed water resources management and planning.

Proposed Method

The proposed runoff prediction model, DCTN combines DCNN and Transformer to extract local feature information and long-term dependencies from raw data. The DCNN layer consists of multilayer convolutional neural networks for feature extraction, while the Transformer layer utilizes self-attention to capture intersequence dependencies. The output layer predicts the final runoff. During training, hyperparameters are tuned using the Adam algorithm, and the best configuration is chosen based on the efficiency of the validation set. In the testing stage, the trained DCTN model predicts runoff.

 The study employs the Stochastic Weather Generator LARSWG6 to replicate monthly rainfall and temperature series under different Shared Socioeconomic Pathways (SSPs) for the study area, enhancing regional climate change simulations.

Dataset and Experimental Analysis

The Dagu River Basin in Qingdao, Shandong Province, experiences a temperate monsoon climate influenced by the East Asian monsoon and the Pacific Ocean. It has abundant rainfall, mainly during the flood season from June to September. The significance of the basin as a primary water source for Qingdao calls for accurate runoff forecasting under different SSPs to plan for potential floods and droughts. The study uses data from meteorological and hydrologic stations to address these challenges. The Dagu River is crucial in providing potable water, but historical flooding events have caused disruptions.

Data normalization ensures consistent scales and accuracy in forecasting monthly runoff. In this study, min-max scaler normalization is applied to the raw data. The common sliding time window processing with a size of 10 and stride of 1 is used to incorporate historical data effectively. Four evaluation metrics are employed to assess the performance of the proposed method. The Adam optimizer with 200 epochs and early stopping (patience of 20) is used to prevent overfitting. The training and test datasets are split in an 8:2 ratio, with the training set of 20% as the validation set. The learning rate is 0.001, and the mean squared error (MSE) function is used as the loss function. Dropout with a rate of 0.2 is applied in each fully convolutional network (FCN) layer. The optimal DCTN structure is determined through grid search.

The simulation results were evaluated by comparing the accuracy of Eight prediction models utilized to simulate monthly streamflow at the Shanjiaodi hydrological station. hydrological station during the test period (2009.1-2014.12). These models include the seven deep prediction models: multilayer perceptron (MLP), DCNN, Bidirectional long and short-term memory (BiLSTM), Bidirectional gated memory unit (BiGRU), Deep Transformer, BiLSTM-Transformer, and the proposed model DCTN. 

To ensure a fair comparison, the number of layers in the deep prediction network is set consistently, with four layers for MLP, BiLSTM, and BiGRU, and the hidden layer units set as (32, 32, 16, 16). Furthermore, the soil and water assessment tool (SWAT), a physical, hydrological model, was introduced to compare its performance with other deep learning models. The relative changes in precipitation ensemble mean values mainly range from 1 to 1.5, indicating an increase in rainfall, with significantly higher values projected for the late century than the mid-century. The other results reveal an overall increase in monthly runoff depth across all scenarios, with varying amplitudes among GCMs. The highest predicted runoff depth shows a 7.6% increase compared to historical averages. Extreme wet periods experience reduced runoff, while extreme drought periods show an increase.

Conclusion

To summarize, this study introduces the DCTN, a novel deep-learning prediction model for hydrology. The combination of DCNN and Transformers captures the local and global patterns in time series data. Compared to other models, DCTN shows superior accuracy, outperforming SWAT by about 30.9% in R2, excelling in Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE), and simulating extreme runoff values effectively.

Forecasting future runoff under climate change scenarios reveals potential risks of spring floods and decreased runoff during the rainy season. These findings significantly affect regional climate change response and long-term runoff forecasting.