MMSTNet: A Breakthrough Model for Enhanced Traffic Prediction

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Sep 22 2023

In a recent publication in the journal Transportation Research Part C: Emerging Technologies, researchers developed a model called MMSTNet, which integrates spatial and temporal attention networks to excel in traffic prediction. The proposed model surpasses current models in real-world dataset evaluations.

Background

Intelligent transportation systems (ITSs) play a vital role in the future development of smart cities. The continuous advancements in data collection, communication, and storage techniques have broadened the applications of ITS in modern traffic management systems. One of the fundamental challenges in implementing ITS is achieving highly accurate traffic prediction, encompassing forecasts of traffic speed, flow, or density within real infrastructure, such as road segments, based on historical data. Accurate traffic prediction serves as a foundation for various downstream tasks within ITS.

Existing models for traffic prediction

Traffic prediction entails forecasting traffic conditions based on historical data, encompassing aspects such as traffic flow, density, or speed for various modes of transportation, including vehicles, trains, subways, and more. In earlier works, research focused on temporal relationships. These methods, including Historical Average (HA) and Autoregressive Integrated Moving Average (ARIMA), employ simple functions such as linear functions for single prediction objects.

Deep neural network (NN) models such as long-short-term memory (LSTM) and gated recurrent unit (GRU) replaced these simple functions with complex ones, enhancing their representation capabilities for time-series tasks like traffic forecasting. However, these models primarily addressed temporal correlations and overlooked spatial correlations between nodes. The spatio-temporal frameworks addressed this issue. They incorporate convolutional neural networks (CNNs) and LSTM or GRU models. While effective in some scenarios, CNNs faced limitations in graph data scenarios, like traffic prediction for road networks. To overcome this challenge, the authors replaced CNNs with graph convolutional networks (GCNs). The GCNs come in two types: spectral-type and spatial-based GCNs, both relying on an adjacency matrix to define node neighborhoods.

Defining macro adjacency between regions can be complex, especially when multiple types of macro-spatial correlations are involved. To address this, a combination of GCN and an attention mechanism was introduced, with the attention mechanism managing macro learning. This approach primarily applies the attention mechanism to macro-spatial learning, mitigating the adjacency definition challenge and reducing computational complexity. Additionally, it is used to adaptively combine different representations learned from various spatial structures and strengthen the learning of local temporal correlations, enhancing the overall predictive capability of the model.

MMSTNet framework for traffic prediction

The data preparation process aims to generate multi-graphs that enable the GCN to capture micro correlations and regions with their embeddings to facilitate macro correlation learning. This process starts with multi-graph generation, where different graphs represent physical relationships between road segments. Two graphs with the same nodes but unequal edge weights are constructed. The first graph signifies physical proximity between road segments, while the second reflects intrinsic pattern correlations based on historical data similarity. Node embeddings are prepared for each graph using the node2vec approach, capturing both global and local structural information. For these embeddings, K-means is applied to obtain regions, and node embeddings within the same region are summed to create regional embeddings. Using temporal data preparation, researchers generated temporal embeddings for each time step using one-hot vectors representing the time of day and day of the week, concatenated for each time step.

Researchers introduced the "MMSTNet" model, designed for macro-micro spatio-temporal predictions. The model takes input from three parts: macro- and micro-level input data, multi-graphs and fusion region embedding, and temporal embedding. The macro and micro input data pass through fully connected (FC) layers before entering a macro-micro learning module, consisting of micro and macro learning blocks, and a fusion module. The micro block employs temporal attention networks to capture global temporal correlations and temporal convolutional networks (TCN) for local correlation learning. A multi-graph GCN captures micro-spatial correlations.

The micro fusion module integrates results from multi-graphs using an attention mechanism. The macro block replaces spatial learning with a spatial attention network. Macro-micro interactions occur in the fusion module, extracting high-dimension spatio-temporal patterns. The final predictions are generated by a stack of FC layers.

Results and analysis

The authors employed two speed datasets, both derived from data provided by the California Department of Transportation Performance Measurement System (PeMS). The first dataset, known as PEMS03, covers the 3rd congressional district of California. It comprises data from 617 sensors, with a total of 43,200 speed records for each road segment. These sensors are strategically placed along the main road in the district. The second dataset, PEMS08, covers the 8th congressional district of California. It consists of data from 474 sensors, with a total of 52,128 speed records per road segment.

Comparative experiments were conducted against various state-of-the-art machine learning models. Furthermore, eight variants of MMSTNet were examined to assess the significance of different model components. Hyperparameters such as learning rates, block sizes, layer sizes, and dimensions were carefully selected to optimize model performance.

The results demonstrate that MMSTNet outperforms most baselines across different time-range predictions, particularly excelling in long-range forecasting. Additionally, the ablation studies underscore the effectiveness of MMSTNet's various components. The model's superiority is further validated through real-time forecasting tasks. The study highlights the need for constant data updates and potential adjustments to the model's framework to adapt to changing transportation networks.

Conclusion

In summary, the study explores traffic prediction using temporal and spatial correlations. The proposed MMSTNet model integrates micro- and macro-spatial correlations with multi-graph GCN and spatial attention. It employs TCN and temporal attention for local and global temporal correlation. MMSTNet outperforms baselines on real-world traffic datasets. Future improvements may include incorporating different GCNs and attention tools, enhancing temporal learning fusion, and expanding MMSTNet's applicability beyond transportation.