In the modern digital era, the rising complexity of cyberattacks has increased the difficulties of developing effective tools to detect them, which poses a significant challenge to cybersecurity. Artificial intelligence (AI) techniques, specifically ensemble learning, deep learning (DL), and machine learning (ML), can be utilized to detect such attacks more effectively and efficiently. ML-, DL-, and ensemble-based intrusion detection systems (IDS) can significantly improve detection accuracy. This article discusses different AI-based IDS and recent developments in this field.
Image credit: Generated using DALL.E.3
Importance of AI-based IDS
Intrusion detection primarily refers to identifying ongoing or attempted attacks on a computer network or system. Major steps in the intrusion detection process include data collection, data reduction, behavior classification, reporting, and response.
Data reduction involves analyzing a collection of data to identify the most important data components to reduce the storage requirements, communications overhead, and processing time, while behavior classification involves identifying intruders and attackers. IDS are utilized extensively in the cyber security field to detect vulnerabilities and threats and to mitigate and prevent them in computer networks.
AI techniques can be used in IDS to efficiently perform crucial tasks in the intrusion detection process. For instance, different ML approaches have been used to categorize various types of attacks and detect anomalies. Statistical and ML algorithms are highly effective for intrusion detection.
DL, an advanced form of ML algorithm, has received significant attention with the rising amount of critical data in the cloud. DL possesses many advantages, including automatic feature learning, which implies the automatic extraction of features and training of a model using large amounts of data. An efficient DL algorithm can be used to effectively address the issue of zero-day attacks and false positives.
Different AI-based IDSs
Supervised ML-based IDS: Support vector machine (SVM), k-nearest neighbors (KNN), Naive Bayes (NB), random forest (RF), and logistic regression can be used in IDS. For instance, 91.6%, 36.65%, 12%, and 22% detection rate has been realized for denial of service (DoS), Probe, user-to-root (U2R), and root-to-local (R2L) attacks, respectively, using SVM on KDD’99 dataset.
Similarly, 99.31% accuracy and 99.20% detection rate were achieved using logarithm marginal density ratios transformation (LMDRT)-SVM on NSL-KDD dataset, while 99.89% accuracy was attained for DoS, Probe, U2R, and R2L attacks using KNN on KDD dataset.
98.59% and 97% accuracy were realized for DoS, Probe, U2R, and R2L on the CIC-IDS2017 dataset and NSL-KDD dataset, respectively, using the NB classifier. Moreover, 98.5%, 83%, 96.78%, 97.49%, and 99.64% accuracies were achieved using KNN, NB algorithm, RF and logistic regression, RF, and RF, respectively, on NSL-KDD, UNSWNB-15, and AWID datasets for DoS, Probe, U2R, R2L, and 802.11 MAC layer attacks.
Unsupervised ML-based IDS: The unsupervised learning technique utilizes clustering based on similarities and can identify the hidden patterns in the unlabeled dataset. Fuzzy C means clustering and K-means clustering algorithms can be used in IDS.
For instance, 92.77% accuracy was achieved for DoS, Probe, U2R, and R2L attacks using K-Means + RF on NSL-KDD, while a detection rate of 99.42% was attained using fuzzy ARTMAP neural network on the KDDCup99 dataset.
Among all supervised and unsupervised ML methods, the KNN demonstrated the highest accuracy of 99.89% on the KDD dataset for DoS, Probe, U2R, and R2L attacks, followed by RF on the AWID dataset.
Supervised DL-based IDS: Artificial neural network (ANN), convolutional neural network (CNN), and recurrent neural network (RNN) can be employed in IDS. For instance, ANN has been used to detect DoS, U2R, R2L, and Probe attacks with a high detection rate of more than 90% on the KDD-99 dataset. The limitation of this approach was the need for extensive preprocessing and feature engineering.
Similarly, CNN + Softmax has been utilized to achieve 97.53% accuracy for Smurf, Neptune, satan, ipsweep, and portsweep attacks on KDD99 + NSLKDD dataset, CNN to attain 98.88% for DoS, U2R, R2L, and Probe attacks on KDD Dataset, CNN to realize 99.953% accuracy on KDD 99 dataset, RNN-long short-term memory (LSTM) to realize 96% accuracy for the botnet, bruteforce, DoS, and distributed denial-of-service (DDoS) attacks on CAIDA DDoS 2007 dataset, RNN-bidirectional LSTM (BLSTM) to attain 98.48% accuracy for brute force, DoS, and DDoS attacks on CICIDS2017 dataset, and RNN to realize 83.28% accuracy for DoS, Probe, U2R, and R2L attacks on NSL-KDD dataset.
High accuracy, low false acceptance rates, excellent handling of sequential data, and good ability to work with large datasets are the key advantages of these approaches. However, these supervised DL-based approaches were computationally expensive, limited to binary classification, and required large amounts of training data, which are their major limitations.
Unsupervised DL-based IDS: Autoencoder (AE) and deep belief networks (DBN) are primarily used for IDS. For instance, DBN, AE, and DBN have been employed to achieve 99.3% accuracy, 94.71% accuracy, and 97.5% accuracy on the KDD cup99 dataset, KDD CUP 99 dataset, and NSL-KDD dataset, respectively, for DoS, Probe, U2R, and R2L attacks.
Similarly, deep evolving stream clustering (DESC-IDS) has attained a high accuracy of 96.44% on the KDD CUP 99 dataset. High accuracy in detecting attacks and effective feature reduction and anomaly identification are the major advantages of unsupervised DL-based IDS, while the need for large amounts of training data and computational resources and the inability to work effectively with complex, large, and diverse datasets are the key limitations of these approaches.
Among all supervised and unsupervised DL methods, CNN displayed the highest accuracy of 99.95% on the KDD 99 dataset, followed by DBN on the KDD Cup 99 dataset. CNN also showed the highest accuracy among all supervised and unsupervised ML and DL algorithms, followed by KNN. The accuracy rates of all algorithms varied from 81.2% to 99.95%.
Ensemble-based IDS: Bayesian convolutional neural network (BCNN), RF + AODE, and multilayer perceptron neural network (MPNN) and sequential minimal optimization (SMO) have been used 99.12%, 90.51%, and 95.02% accuracies, respectively. Similarly, the adaptive voting algorithm has attained 85.2% accuracy for DoS, Probe, R2L, and U2R attacks.
High accuracy rate, reduced false positives, low false acceptance rate, and effective feature reduction are the key advantages of these methods. However, ensemble methods are computationally expensive and ineffective with highly imbalanced data, which are the major limitations.
Recent Studies
In a study published in the journal Electronics, researchers proposed a novel IDS based on ensemble methods of ML. They selected features from the CICIDS-2017 dataset to eliminate false positives and improve classification accuracy.
The proposed IDS utilized several ML algorithms, including decision trees, RF, and SVM. An ensemble technique voting classifier was introduced after training the ML models and an accuracy of 96.25% was realized. Additionally, the proposed IDS also incorporated the explainable AI (XAI) algorithm local interpretable model-agnostic explanation (LIME) for better understanding and explainability of the black-box approach to reliable intrusion detection.
LIME is an extensible, modular approach that can describe ML model predictions in a clear and understandable manner. The experimental results confirmed that the XAI LIME was more responsive and explanation-friendly. In recent years, AI-based anomaly detection systems have gained significant attention among different network intrusion detection system (NIDS) technologies, with various AI models being proposed to improve the NIDS performance. However, the issue of data imbalance remains a major challenge, due to which AI models cannot adequately learn malicious behavior.
Thus, the AI models fail to detect all threats in the network accurately. In a study published in the IEEE Internet of Things Journal, researchers proposed a novel AI-based NIDS that can effectively address the data imbalance problem and improve the detection performance of previous AI-based NIDS.
They leveraged a state-of-the-art generative model that can generate plausible synthetic data for minor attack traffic to address the data imbalance problem. Specifically, researchers focused on AE-driven DL models and the reconstruction error and Wasserstein distance-based generative adversarial networks (GANs). Comprehensive evaluations were performed using different datasets to determine the effectiveness of the proposed AI-based NIDS. Results demonstrated that the proposed system significantly outperformed the previous AI-based NIDS.
References and Further Reading
Patil, S., Varadarajan, V., Mazhar, S. M., Sahibzada, A., Ahmed, N., Sinha, O., Kumar, S., Shaw, K., Kotecha, K. (2021). Explainable Artificial Intelligence for Intrusion Detection System. Electronics, 11(19), 3079. https://doi.org/10.3390/electronics11193079
Park, C., Lee, J., Kim, Y., Park, J. –G., Kim, H., Hong, D. (2023). An Enhanced AI-Based Network Intrusion Detection System Using Generative Adversarial Networks. IEEE Internet of Things Journal, 10, 3, 2330-2345. https://doi.org/10.1109/JIOT.2022.3211346.
Sowmya, T., Mary Anita, E. (2023). A comprehensive review of AI-based intrusion detection system. Measurement: Sensors, 28, 100827. https://doi.org/10.1016/j.measen.2023.100827