Real-Time Assessment of Driver Drowsiness using PPG Signals: A Novel End-to-End Approach

By Aryaman PattnayakReviewed by Susha Cheriyedath, M.Sc.Aug 9 2023

A recent study submitted to the arXiv* server presents an end-to-end pipeline for continuous real-time assessment of driver drowsiness levels by analyzing photoplethysmography (PPG) signals. The approach strategically acquires PPG data through customized sensors embedded in the steering wheel and implements a tailored deep neural network architecture for accurate drowsiness classification.

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.

Background

Drowsy driving represents a major overlooked cause of serious traffic accidents worldwide, contributing to thousands of crashes annually. Conservative estimates indicate it plays a role in 20% of all serious motor vehicle incidents. Developing effective driver monitoring systems that can continually assess and detect diminishing alertness is, therefore, a critical priority for enhancing road safety.

Prior works have explored using heart rate variability (HRV) derived from electrocardiography (ECG) signals to correlate autonomic nervous system activity with fluctuating driver awareness levels. However, ECG acquisition can be highly susceptible to motion artifacts and signal degradation in real on-road environments. This necessitates complex preprocessing pipelines that are computationally expensive and difficult to deploy. 

Recent studies have investigated PPG as a more reliable, non-invasive alternative to ECG for evaluating driver drowsiness through pulse rate variations. PPG provides similar insights into HRV but has key advantages – it requires only a single point of skin contact for clean physiological signal acquisition and is more robust to motion interference. Sophisticated deep learning techniques have also shown tremendous promise in accurately classifying highly complex driver fatigue behaviors from physiological data. However, most existing methods still entail substantial computational and hardware costs, limiting real-time deployment in commercial vehicles.

Novel PPG Sensing Strategy

The researchers strategically designed and developed a customized PPG sensor suite specifically optimized for driver drowsiness monitoring applications. The sensor incorporates a specialized array of near-infrared LED emitters and silicon photomultiplier detectors tailored for reliably capturing blood volume changes in the driver's palm.

The photodetector array consists of a silicon photomultiplier device with thousands of microcells, providing high sensitivity to the backscattered near-IR light. The near-IR LED components leverage advanced InGaN semiconductor technology to emit light focused on the optimal spectral bands for PPG detection.

A key innovation is the strategic placement of this customized PPG sensor suite. It is seamlessly embedded into the steering wheel to continually and unobtrusively acquire driver palm PPG waveforms through natural contact during normal driving. This overcomes the limitations of cumbersome electroencephalogram (EEG) or ECG sensors that require extensive manual sensor placement at multiple body locations. The steering wheel integration enables robust, motion-resilient PPG signal capture even in the presence of artifacts.

Dedicated power management and signal conditioning circuits ensure optimal sensor performance during acquisition. The raw PPG waveform is processed through high-resolution analog-digital conversion and filtering algorithms optimized to isolate the blood volume pulse signal.

By developing a sensor system tailored specifically for robust in-vehicle PPG acquisition, the researchers overcame major challenges in driver physiological signal monitoring. This enables the subsequent deep neural network architecture to analyze the PPG waveform for subtle changes indicating diminished driver alertness.

Multi-Stage Processing Pipeline

A multi-phase pipeline preprocesses the acquired raw PPG signals to extract relevant features before classification include the following:

• FIR filtering isolates the 1-10 Hz range and removes noise interference, including 50 Hz power line frequencies and motion artifacts
• Inspired by hyperspectral imaging, hyper-filtering layers divide the frequency range into 11 sub-bands to extract nuanced pulse waveform characteristics
• Reinforcement learning automatically determines optimal filter frequencies for each sub-band based on maximizing drowsiness detection performance  
• Signal patterns capturing temporal PPG variations are extracted from 4-second windows to feed into the deep network

 Advanced Deep Neural Network

Using the extracted PPG signal patterns, a tailored 1D temporal convolutional neural network (D-CNN) architecture was developed to classify driver drowsiness. The D-CNN employs residual blocks with dilated causal convolutions to effectively capture temporal relationships and dependencies within the input sequence data.

A two-class softmax output layer finally predicts the binary drowsiness state based on the multi-modal preprocessed PPG samples. The network is trained end-to-end using backpropagation to learn highly complex driver fatigue behaviors from the PPG analysis pipeline.

 ​​​​​​Comprehensive experiments using a 70-subject PPG dataset under controlled drowsy and alert conditions achieved over 96% classification accuracy, significantly outperforming other deep learning approaches. Further testing on synthesized PPG waveforms representing different attention levels also strongly validated performance.

Future Prospects

The researchers highlighted plans to implement the system on an embedded platform with edge-based inference for rapid in-vehicle deployment. Fusing the PPG approach with infrared driver imaging, gaze tracking, and other modalities could further boost accuracy and robustness. Expanding the deep neural network architecture using nonlinear processing was also discussed as a promising direction.

Potential future work includes deploying the system in vehicles and exploring fusion with other sensing modalities to improve robustness. Expanding the network architecture using nonlinear processing is another future possibility. Overall, this study introduces a promising solution for passive, real-time driver drowsiness evaluation with the potential to enhance road safety.

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.