In a paper published in the journal European Journal of Operational Research, researchers developed a real-time machine learning (ML) framework to predict and explain workload fluctuations in railway traffic control rooms at Infrabel, Belgium. The researchers found that both high and low workloads impact employee well-being and identified LightGBM as the best-performing model. Shapley additive explanations (SHAP) values helped distinguish workload presence and magnitude by providing decision support to traffic supervisors.
Study: AI-Powered Workload Prediction for Railway Traffic Control Rooms. Image credit: nostal6ie/Shutterstock
Background and Related Work
The European Commission introduced Industry 5.0 as an extension of Industry 4.0, focusing on research and innovation to create a resilient, sustainable, and worker-centric European industry. Industry 5.0 prioritizes worker well-being in addition to efficiency and productivity. In this context, a human-centric approach to digital technologies is emphasized.
In prior studies, digital control rooms in sectors like energy and transportation were identified by highlighting their characteristics, such as human-machine interaction, involvement in safety-critical operations, and variable workloads. These studies emphasized the importance of managing workload due to its impact on employee well-being and operational performance that considered both high and low workload levels. A multi-dimensional concept called Workload can be measured through various methods like task performance assessment, subjective reports, and physiological indicators. Managing workload proactively is challenging due to the unpredictability of future workloads.
Methodology Customization for Digital Control Rooms
The framework is tailored to the specific digital control room setting to recognize that 15-minute intervals often show no workload for certain important tasks. The distinction between workload presence and magnitude, as highlighted by Heckman's work on sample selection issues, is acknowledged. A two-stage approach inspired by recent deep learning models is implemented. The first stage predicts workload presence for each of the six categories, while the second stage predicts workload magnitude for present categories. This customization allows for gaining deep insights into future workload composition and magnitude, crucial for proactive task reallocation.
Model Selection and Tuning
To determine the most suitable models for each stage, a comprehensive benchmark of various ML and deep learning models commonly used in operations research is conducted. These models encompass Decision Trees, Random Forests, eXtreme Gradient Boosting, Light Gradient Boosting Machine, Artificial Neural Network (ANN), Deep Neural Network (DNN), and Long Short-Term Memory encoder-decoder model (LSTM E-D). The evaluation of performance in predicting both workload presence and magnitude uses metrics like the Area Under the Receiver Operator Curve (AUROC/AUC) for presence prediction.
Workload Magnitude Prediction
A similar setup is applied to predict workload magnitude for each category in the second stage. The data is filtered to focus on workload presence in the relevant categories. The tuned model predicts workload magnitude when presence is detected and assigns a value of 0 when absence is predicted. The performance of the second stage is gauged using the Root Mean Squared Error (RMSE) to measure the accuracy of magnitude predictions.
Explainability Through SHAP Values
In addition to accuracy, emphasis is placed on explainability to build trust and elucidate the model's logic. The SHAP framework is employed to explain the contribution of different features to predictions. SHAP values reveal the significance and direction (positive or negative) of feature contributions by providing both local and global insights into model behavior. However, vigilance is maintained regarding potential issues related to highly correlated features, which can lead to misleading interpretations.
Study Results
The accuracy benchmark in both stages of the methodology begins with an evaluation of several ML models for Stage 1, where the presence of workload categories is predicted. These models include Decision Trees, Random Forests, XGBoost, LightGBM, ANN, DNN, and LSTM E-D. The benchmark results show that LightGBM outperforms the other models, making it the best-performing model for predicting and classifying workload presence. The same models are benchmarked in Stage 2, where the magnitude within present workload categories is predicted, and LightGBM emerges as the top performer.
The methodology also prioritizes explainability and utilizes the SHAP framework to highlight feature contributions to predictions. In this global SHAP analysis, the focus is on the top 10 most important features for each category in both stages. Notably, the current workload appears as a crucial feature in predictions. The significance of granular data, which includes various feature types such as HMI and partner exposure, is demonstrated by these global SHAP value plots. This highlights the importance of teamwork dynamics and underscores the value of incorporating partner workload, delays, and experience of partner control room operators. Additionally, the discussion includes SHAP dependence plots. These plots specifically focus on the relationship between partner workload and predicted workload by indicating the empirical usefulness of the two-stage approach and highlighting the importance of controlling for potential bias in explanation caused by the sample selection issue in this specific setting.
Conclusion
In summary, the research addresses high workload variability in digital control rooms by using real-time data analytics to predict and explain employee workload. The approach connects operations research with human resource management and offers valuable insights into workload explanations. Future research directions include exploring the link between task-based and mental workload and investigating workload's impact on safety-critical events. Additionally, further research could focus on incorporating prescriptive optimization models to proactively balance workload among controllers.
Machine Learning Enhances Water Quality Monitoring