Multimodal Dataset for Driver Monitoring in Assisted Driving Automation

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Apr 12 2024

In a paper published in the journal Scientific Data, researchers introduced the human dimension in automated driving (manD) 1.0, a dataset for driver monitoring in automated driving. It included data from 50 participants, aged 21 to 65, navigating through five scenarios in a driving simulator.

The automation levels ranged from manual control to conditional automation. ManD 1.0 encompassed environmental and vehicle data and driver state information such as physiology and gaze. It served as a benchmark for data-driven modeling and interaction strategy formulation applications.

Related Work

Past work has focused on gathering comprehensive datasets to assess driver states in various driving contexts accurately. Initially, datasets primarily captured vehicle-related information, but attention shifted towards monitoring driver states like fatigue, attention, and distraction. These datasets evolved to incorporate visual data, physiological metrics, and driver activities. However, existing datasets often target specific factors that require a holistic approach.

Data Preprocessing Overview

The BIOPAC sensor system captures data through nine electroencephalogram (EEG) channels and a single electrocardiogram (ECG) channel, offering a dynamic range of 1000 and operating at a sampling rate of 256. The system maintains an accuracy and resolution of 3.0 peak-to-peak and 0.038 resolution for EEG signals and 3.0 peak-to-peak with a resolution of 0.06 for ECG signals, per the manufacturer's specifications.

The AcqKnowledge 4 software performs initial signal processing by applying a 0.1 highpass filter and a 67 lowpass filter. After the acquisition, researchers implement a data preprocessing step to correct bad channels. Channels presenting extreme average values are detected, and interpolation is applied using the average of neighboring channels. If all neighboring channels exhibit similar compromise, researchers remove the defective channel. Additionally, researchers identify channels with frozen (constant) values and actively interpolate these unchanging segments.

The AcqKnowledge 4 software's initial signal processing includes applying a 0.1 highpass filter and a 67 lowpass filter to the captured data. These filters actively remove unwanted noise and artifacts from the EEG and ECG signals, ensuring that only relevant physiological data remains for further analysis.

After the acquisition phase, researchers actively carry out a data preprocessing step to address potential issues with the captured data. This preprocessing step involves the correction of bad channels, which are channels exhibiting abnormal or extreme values. Channels with extreme average values are detected, and interpolation techniques are applied to replace these values with estimates based on the surrounding channels. When neighboring channels are similarly compromised, the affected channel is removed from further analysis to prevent erroneous interpretations.

Additionally, researchers actively identify channels that exhibit frozen or constant values during preprocessing and interpolate these unchanging data segments to ensure that the entire dataset is free from any artifacts or inconsistencies that could impact the validity of subsequent analyses.

Validation Processes Detailed

This study's technical validation process involved two key steps: quality control and experimental validation. The quality control phase focused on ensuring the availability and reliability of the collected data. For instance, environmental data underwent rigorous monitoring to ensure the accuracy of both the range of data and the sequence of recorded events.

Similarly, vehicle data underwent validation checks, including the order of interaction signals and verification of specific vehicle dynamics parameters. Additionally, the study scrutinized unique attributes such as multi-labeling in driver activity data to ensure the plausibility of overlaps.

Experimental validation aimed to introduce varied driver states through different stimuli, such as emotional video clips, to evoke specific emotions. Statistical analyses, including analysis of variance (ANOVA) and post hoc tests, were conducted to assess the effectiveness of these stimuli in eliciting intended emotions.

Furthermore, surprise was captured through unexpected events at the beginning of driving scenarios, followed by verbal inquiries about drivers' feelings. The researchers assessed the consistency of drivers' responses to these inquiries, although further replication of results. Overall, meticulous validation procedures were employed to ensure the integrity and reliability of the dataset for future analyses and research endeavors.

In addition to quality control and experimental validation, the study emphasized meticulous scrutiny of various data sources and parameters. It included validation checks for physiological data collected using devices like the Empatica E4, where ranges for parameters such as blood volume pulse (BVP), skin conductance level  (SCL), and skin temperature were examined. Moreover, EEG and ECG datasets underwent thorough inspection, including visual observation, to identify anomalies and ensure data accuracy. Seat-pressure-sensor mat data were also scrutinized for pressure reading ranges, ensuring consistency and reliability.

Furthermore, the experimental validation phase incorporated dynamic elements to evoke different emotional states in participants, such as playing emotional video clips and introducing unexpected events during driving scenarios. Statistical analyses were employed to assess the effectiveness of these stimuli in inducing intended emotions, providing valuable insights into driver reactions and responses. The study aimed to enhance understanding of driver behavior and emotional responses in various scenarios through rigorous validation processes and experimental design, contributing to advancements in automotive safety and human-computer interaction research.

Conclusion

To summarize, this study's technical validation process ensured the robustness and reliability of the collected dataset for subsequent analysis and interpretation. Researchers undertook meticulous quality control measures, actively validating environmental, vehicle, and physiological data and conducting experimental validation to assess emotional responses. These efforts provided a solid foundation for researchers to delve deeper into behavioral patterns and insights, contributing to advancements in understanding human behavior in driving scenarios.