In a study featured in the journal Aerospace, researchers reported a comprehensive safety assessment conducted within the Single European Sky Air Traffic Management (ATM) Research 2020 (SESAR2020) project PJ.10-W2-96 Automatic Speech Recognition (ASR), which focused on the past implementation of ASR technology in air traffic control (ATC) centers. This advanced ASR technology enabled the automatic recognition of aircraft callsigns and various ATC commands, enhancing the working environment for controllers.
Study: Advancing Air Traffic Control Safety with Automatic Speech Recognition. Image credit: Generated using DALL.E.3
The safety assessment procedure addressed design requirements across different modes of ATC operations and identified eight functional hazards by analyzing specific use cases. Rigorous assessments, including modeling and simulations, were undertaken to validate ASR's safety benefits, which included reduced controller workload and improved situational awareness. The paper emphasized the importance of analyzing ASR's safety impact in ATC and concluded that it presented no increased safety risks, making it ready for industrial application.
ASR Evolution in ATC
Past works in ATC have focused on the feasibility and validation of ASR technology. The SESAR2020 project PJ.16-04 initially delved into the feasibility of ASR within controlled laboratory settings. Subsequent projects, such as PJ.10-W2-96, shifted their focus towards validating ASR in real operational scenarios. Throughout these initiatives, improving safety and reducing the workload for ATC has remained a central theme. Several research endeavors, including STARFiSH, have explored the application of ASR in various aspects of ATC. These studies have extensively investigated how ASR can enhance ATC operations and minimize human errors, with a primary objective of achieving safety improvements.
Methods for ASR Safety Assessment in ATC
Materials and Methods: The safety assessment conducted during the system development phases of Technology Readiness Level 5 (TRL 5) and TRL 6 aimed to demonstrate the feasibility of implementing ASR technology in ATC operations. The assessment followed the SESAR safety reference material and guidance to ensure compliance with European regulations. Two fundamental approaches were employed to validate the ASR system: the success approach and the failure approach. The success approach evaluated how effectively ASR reduced pre-existing aviation risks and contributed positively to aviation safety. In contrast, the failure approach focused on assessing the risks ASR might introduce in the event of failure, encompassing potential hazards and their implications.
Selected Use Cases: The assessment concentrated on specific use cases operating at Technology Readiness Level 5 (TRL 5) and TRL 6. Researchers chose these use cases to represent ASR technology validated and demonstrated in operational environments. Notable among these were:
- "Highlight of Callsigns (Aircraft Identifier) on the Controller Working Position (CWP) Based on the Recognition of Pilot Voice Communications," where ASR recognized pilot voice signals to identify aircraft callsigns, supporting ASR processed AT Controller (ATCO) in their interactions with flight crews.
- "Highlight of Callsigns on the CWP Based on the Recognition of ATCO Voice Communications," in which ASR processed ATCO voice signals to enhance safety by verifying the correspondence between callsigns and flight radar data labels.
- "Annotation of ATCO Commands," where ATCO voice commands were processed and annotated on the CWP Human-Machine Interface (HMI), improving situational awareness and ensuring safety in flight clearances.
- The "Pre-Filling of Commands in the CWP" use case allowed recognized commands and values to be presented to ATCOs, granting them the ability to accept, reject, or adjust these commands to maintain operational safety.
Safety Assessment Methodology: The primary focus of this assessment was on the "failure" aspect, determining the contribution of ASR to accident risk in case of failure. It began with hazard identification via a comprehensive analysis of use cases. Subject matter experts employed sequence diagrams during walkthroughs to identify potential hazards actively. Subject matter experts actively assessed these identified hazards through a Functional Hazard Assessment. The assessment involved evaluating the effects of hazards on operations, determining the severity of each hazard effect, and specifying safety objectives.
Furthermore, researchers conducted a top-down causal analysis for each functional hazard, actively identifying preventive mitigations to safeguard against the propagation of primary causes.
Low and High Callsign Recognition Exercises: Researchers conducted two distinct validation exercises to evaluate the ASR system under various scenarios—the first exercise aimed to validate ASR's performance for callsign highlighting and command annotation. Researchers analyzed real-life communications between ATCOs and flight crew to assess ASR performance using subjective and objective data. The second exercise focused on quantifying the safety and workload benefits of ASR in different traffic scenarios, medium and heavy density, by comparing its performance to a baseline scenario where commands were manually input. These exercises provided insights into the safety and operational advantages of ASR technology in ATC operations.
Considerations and Limitations
The safety assessment within the SESAR2020 PJ.10-W2-96.2 ASR research focuses on specific ATC scenarios and may not apply universally. It primarily addresses early ASR technology implementation, and the findings require further validation for real-world use, adhering to local regulations. This study did not include long-term assessments of ASR functionality or extensive safety evaluations, and it did not thoroughly examine external factors like noise and radio interference.
ASR's introduction may cause cybersecurity vulnerabilities, necessitating local assessments. Future research should explore ASR in various operational environments and consider its interaction with existing safety tools. The acceptance of ASR by ATCOs and its impact on workload may vary, requiring a holistic evaluation for specific contexts.
Conclusion
To sum up, the safety assessment of ASR application in ATC operational environments revealed that potential hazards had no significant impact on overall ATM safety. Mitigations derived from operational needs ensure ATCO performance while addressing any concerns related to situational awareness or workload. The ASR system's accuracy and timeliness exceeded design requirements, and subjective feedback from ATCOs in real-world settings confirmed its feasibility for integration into existing ATM systems.
Enhancing Science Education with Multimodal Large Language Models