The MPDB Dataset: Exploring Driving Behavior with Multimodal Physiological Data

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Apr 22 2024

In an article published in the journal Nature, researchers explored driving behavior analysis using multimodal physiological data collected from 35 participants while they operated a six-degree-of-freedom driving simulator. The data included electroencephalogram (EEG), electrocardiogram (ECG), electromyography (EMG), galvanic skin response (GSR), and eye movement data.

The authors categorized driving behavior into five groups and developed classification models to demonstrate the correlation between physiological data and driving behaviors. They presented the multimodal physiological dataset for analyzing driving behavior (MPDB), offering opportunities for researchers in traffic psychology and behavior.

Background

The field of traffic psychology and behavior studies how human factors impact driving and road safety. According to the National Motor Vehicle Crash Causation Survey, 94% of traffic crashes are linked to inappropriate driver behavior. Previous research has predominantly focused on the influence of the driving state on accidents, exploring areas such as fatigue and distraction. However, the gaps in understanding the direct impact of human reactions and decision-making during driving tasks remain.

Traditionally, driver behavior research has relied on data from vehicle sensors, cameras, and smartphones, which can be subject to inaccuracies, noise, and limitations in capturing nuanced behaviors. Physiological data, such as EEG and ECG, have also been used to study driver states, but there is a lack of datasets that map these signals directly to driver behavior in real-time scenarios. This paper addressed these gaps by introducing a novel dataset that directly mapped multimodal physiological signals to driver behavior during complex driving tasks.

The dataset included high-resolution data from EEG, ECG, EMG, and eye tracking collected during driving tasks on a six-degree-of-freedom driving simulator. By combining physiological signals with explicit driving behavior, this research offered insights into driver cognitive functions and provided a more comprehensive understanding of human decision-making processes in driving. This dataset paved the way for advancements in the study of traffic psychology, driving behavior, and the development of more effective models for enhancing road safety. 

Multimodal Physiological Data Collection Methods for Event-Related Driving Tasks

The authors focused on understanding human driving behavior through a comprehensive event-related driving experiment conducted with 35 voluntary participants. All participants were students or faculty members from Tsinghua University, aged between 20 and 60 years, with an average age of 25 years and at least one year of driving experience.

Before the experiment, all participants were required to maintain proper rest, abstain from drug or stimulant intake, and complete a pretest to ensure they understood the tasks and were comfortable with the experimental setup. The experimental environment included a six-degree-of-freedom motion driving simulator and a circular curtain. The simulator was capable of translating and rotating, allowing participants to experience various driving scenarios realistically. The simulator was equipped with a real car steering wheel, servo motor, and motion control system for a more authentic driving experience.

The road scenario replicated an 11-kilometer route from the actual road near Shunbai Road in Beijing, featuring diverse elements such as sharp curves, urban roads, and multiple lanes. Five driving events were modeled: smooth driving (control), acceleration, deceleration, lane change, and turning. Each driving behavior was triggered by specific events, such as overtaking, sudden lane changes, pedestrian crossing, and static obstacles. Participants were required to respond to these events while maintaining normal driving conditions. All events were marked for precise data analysis.

Various physiological signals, including EEG, ECG, EMG, GSR, and eye-tracking data, were synchronously collected during the driving tasks using advanced data collection systems from Neuracle and Tobii. EEG data were collected using a 64-electrode cap, while ECG signals were recorded using an electrode on the participant's chest. EMG data were acquired from specific muscles involved in driving, and GSR signals were measured using electrodes on the fingers.

Eye-tracking data provided valuable insights into gaze patterns and pupil dilation during driving. The combination of these diverse physiological signals with participant responses to driving events allowed for a comprehensive understanding of human driver behavior, offering a robust dataset for future research in traffic psychology and behavior modeling.

Physiological Data Storage and Analysis for Driving Behavior Evaluation

The MPDB dataset provided data storage and organization details, including raw and preprocessed data and eye-tracking data available on Figshare. The raw dataset contained physiological data from 35 subjects driving for two hours each and was organized in folders for each subject, further divided into different driving behaviors such as deceleration, acceleration, turning, and lane change. Users could frame and preprocess the raw data as per their needs.

In the preprocessed dataset, physiological data samples from each subject were combined into a single file, including five types of behaviors. The raw data were organized by subject number, separating EEG and ECG data collected through the same wireless transmission device. Technical validation included assessing the quality of physiological data, vehicle parameters, and the correlation between physiological data and driving behaviors. The validation process covered quality checks of physiological variables and vehicle parameters, as well as correlation analysis between the two.

Physiological structure analysis involved examining EEG time-frequency domain features of driving behaviors, including using the short-time Fourier transform. Statistical property validation included analyzing the statistical properties of EEG data, such as power spectral densities. The dataset also included EMG, ECG, and GSR signals, which were preprocessed with bandpass filtering and noise removal. Each signal type's waveforms were presented for different subjects and channels, demonstrating normal periodic peaks in ECG and changes in skin conductivity in GSR.

Correlation analysis between physiology and driving behaviors was illustrated with heat maps showing the correlation between five driving behaviors and physiological signals. Eye tracking variable validation examined the subject's gaze patterns during driving, with scatter diagrams showing concentration on the main objects causing events. The dataset's driving environment was designed to mimic actual vehicle conditions, with adjustable seats, a steering wheel, and other vehicle components.

Vehicle parameters such as speed, acceleration, and gear position were recorded, and each subject followed the same route and event triggers, ensuring consistency in the data. Quality control measures included synchronizing event labels with physiological data using a universal serial bus (USB) port. Classification tasks using physiological data and vehicle parameters aimed to identify correlations between similar samples and differences between dissimilar samples.

Models such as linear discriminant analysis and EEGNet were employed to classify data and evaluate accuracy. The use of multimodal data, such as combining EEG, EMG, and ECG signals, improved the classification performance. The MMPNet neural network model outperformed baseline models, showing that multimodal data significantly enhanced classification accuracy compared to using only a single data mode.

Conclusion

In conclusion, the comprehensive validation and analysis of MPDB demonstrated its reliability and effectiveness in evaluating driving behaviors through physiological data. From EEG to EMG and ECG, each modality underwent rigorous validation, ensuring high-quality recordings. Correlation analysis between physiological signals and driving behaviors confirmed their relationship, further reinforced by classification models. Utilizing multimodal data enhanced classification accuracy, underscoring the dataset's robustness.