AI-Powered Threat Hunting for Critical Infrastructure Protection

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Aug 11 2023

The proliferation of diverse, complex cyberattacks, including on critical infrastructure (CI), demands proactive defense. Threat hunters, and specialized cyber-security experts, strive to detect evolving threats, but quickly processing vast amounts of data is challenging. Integrating artificial intelligence (AI), specifically machine learning (ML) techniques, aids real-time analysis to differentiate benign from malicious data. In a paper published in the journal Mathematics, researchers designed a distributed, scalable ML-based threat-hunting system targeting CI needs.

Background

The growth of information technology (IT), particularly the internet, has led to its widespread integration in various actions. This necessitates agencies, small and medium enterprises (SMEs), large companies, and CI to ensure their internet connectivity functions seamlessly. Failure could render their services useless, resulting in economic repercussions. Cybercriminals disrupt businesses and engage in activities such as data theft and reputation compromise for financial gain.

Given the severity of cyber threats, CI invests significantly in prevention and mitigation. They bolster IT security teams to prevent data loss, information theft, reputation damage, and business interruptions. Equipping specialists with suitable tools is vital to prevent them from being overwhelmed by real-time data.

Surveys among threat hunters reveal that a substantial portion of actionable data consists of benign employee actions such as web browsing. ML techniques, skilled in extracting patterns from unstructured data, can classify datasets by potential threat, but customization for CI challenges is vital. Studies emphasize context's influence on human pattern recognition, especially under stress. Threat hunters work amid such stress, contending with complex attack scenarios loaded with data, often in unfamiliar settings with variables such as zero-day attacks.

Considering the link between cognitive prediction and context, human bias may err in distinguishing similar attack and non-attack behaviors. ML systems exhibit the potential for unbiased accuracy, empowering threat hunters with insights. The hypothesis generation phase benefits from ML, aiding threat hunters in uncovering concealed patterns.

Advancements in ML for enhanced threat detection and response

The use of machine learning techniques to improve threat detection is rising. One study focuses on leveraging machine learning to accelerate reactions in time-sensitive systems, while others try to build enhanced threat-hunting algorithms for software-defined networks (SDNs). The potential of machine learning to address IoT difficulties with low resources has attracted attention.

Attempts are being made to investigate the role of machine learning in overcoming these restrictions and conducting AI-driven studies of real-time data for social trend spotting. Furthermore, threat-hunting systems based on machine learning have been presented. These architectures involve data gathering, visualization, and ML-assisted indicator of compromise (IoC) creation. However, they lack ways of generating hypotheses from data. A full architecture for threat hunting has been described, confined to a single social network, and a unified approach addresses issues from prior solutions.

Due to unknown threats and unpredictable contexts, these studies illustrate the difficulty of understanding threats. Splunk, Palo Alto Firewalls, IBM X-Force Exchange, and Anomali ThreatStream are examples of industrial products that use ML for threat hunting. A modular, scalable, and secure architecture is thought to be optimal for building a threat-hunting system for critical infrastructure. Prioritizing user identification, data encryption, and blockchain technologies provides operational integrity and transparency.

Leveraging ML for threat detection

Machine learning (ML) techniques are instrumental in aiding threat hunters, even without extensive data science expertise. Yet, challenges arise when clear guidelines for selecting appropriate techniques are lacking, potentially hindering rather than aiding. Research in this field offers insights into the ML landscape, essential for security experts striving to enhance threat detection.

A range of ML methods are relevant in cybersecurity. Clustering techniques such as k-means, affinity propagation, and density-based spatial clustering of applications with noise (DBSCAN) identify patterns and group data. Neural networks such as long short-term memory recurrent neural networks (LSTM RNN), bidirectional LSTM, continuous RNNs with adversarial training (C-RNN-GAN), and graph neural networks (GNNs) excel in sequential pattern capture and complex data processing. Natural language processing (NLP) methods such as Bidirectional Encoder Representations from Transformers (BERT) and term frequency-inverse document frequency (TF-IDF) handle language tasks.

Support from data analysts or basic ML knowledge is crucial for threat hunters to effectively apply these techniques. ML aids in tactics, techniques, and procedures discovery, behavior analysis, alert prioritization, streamlining processes, and bolstering cybersecurity. ML's adaptable application, aligned with threat hunters' goals, enhances overall efficacy.

Prototype development

After finalizing the architecture, a system prototype was developed to assess its effectiveness in securing critical infrastructure (CI) using machine learning (ML). Python was chosen for its ML library compatibility and CUDA optimization for efficient processing. Autonomous components aligned with the architecture were containerized using Docker after testing, and their configurations were set through environmental variables.

The prototype underwent rigorous evaluation, simulating real-world scenarios with VMware vSphere. It emulated tasks such as gate operations, user interactions, temperature changes, and server requests. Validation employed ML techniques and staged attacks, fine-tuning systems for log analysis and utilizing MITRE CALDERA (the Microsoft Windows security framework) for threat profiling.

ML techniques, including NLP, C-RNN-GAN, and GNNs, were employed for attack identification. While effective, enhancements involve integrating Python with C++ for optimization and evaluating performance under resource constraints, bolstering the prototype's resilience.

Conclusion

In summary, a robust architecture for CI protection with adaptable ML-based anomaly detection and hypothesis generation was chosen. Multiple ML techniques, such as NLP, C-RNN-GAN, and GNNs, were integrated and validated for their efficacy in simulated environments, showcasing the prototype's effectiveness against various threats. Further efforts aim to refine ML methods for resource-constrained situations and extend their application into hybrid scenarios encompassing both cyber and physical domains.