A High-Performance Multimodal Deep Learning Model for Detecting Minority Class Sample Attacks
Abstract
:1. Introduction
1.1. Research Background
1.2. The Contribution of the Work in this Paper
- A novel high-performance multimodal deep learning method is designed to address the low detection rate of minority attacks. The method consists of a generative adversarial network, a one-dimensional convolutional neural network, and a gated recurrent unit, called GOCG;
- A deep learning method based on multi-perspective learning is designed, and multi-class classification of minority attacks is performed using integrated classifiers;
- Achieved high performance in all metrics on the CICIDS 2017 and the NSL-KDD datasets. The strategy in this paper achieves better experimental results compared to the state-of-the-art.
2. Related Work
3. Materials and Methods
3.1. Data Enhancement
Algorithm 1 Training GAN |
Input: Minority class attack samples |
Output: Generated data |
for G (Generator) |
2: Each iteration: |
} |
} |
} |
6: Back propagation calculates the gradients of D, G |
) |
) |
9: Optimize the loss of D, G with Adam optimizer |
10: When the discriminator determines that the generated sample is true, the generated data are output |
3.2. Data Preprocessing
3.3. Feature Extraction and Classification
Algorithm 2 Training multimodal methods |
Input: dataset X |
1: Data preprocessing: removing redundant data, One-hot coding, and normalization |
3: Circulation |
4: for number of epochs learned do |
7: gradient descent |
8: end for |
9: Until the loss function converges |
11: Circulation |
12: for number of epochs learned do |
13: for mini-batch quantities do |
16: gradient descent |
17: end for |
18: end for |
19: Until the loss function converges |
Algorithm 3 Training the Integrated Classifier |
1: Circulation |
2: for C = 1 in rang(1,6) |
4: backpropagation calculates the gradient and loss values |
5: gradient descent |
6: end for |
7: Until the loss function converges |
4. Experiments
4.1. Dataset Description and Design
4.2. Model Hyperparameters
4.2.1. Hyperparameters of the Data Enhancement Section
4.2.2. Hyperparameters for Feature Extraction Part
4.2.3. Hyperparameters for the Classification Part
4.3. Evaluation Measures
5. Results and Discussion
5.1. Analysis of Experimental Results
5.1.1. Analysis of Accuracy Results
5.1.2. Analysis of Precision Results
5.1.3. Analysis of Recall Results
5.1.4. Analysis of F-Measure Results
5.1.5. Analysis of FPR Results
5.1.6. Analysis of FNR Results
5.2. Comparative Analysis of Different Methods
5.3. Ablation Experiment Analysis
- Classification Layer: Using only gated recurrent units for feature learning and integrated classifier classification, a few attack detections without data augmentation were first evaluated. This method is known as GRU;
- Classification layer following data augmentation: Data augmentation is performed by using a generative adversarial network. After that, the augmented data were fed into the gated recurrent unit for training, and an integrated classifier was used for classification. This method is known as GAN+GRU;
- Multimodal approach following data augmentation: following training dataset augmentation, the multimodal approach learns features from both the 1DCNN and GRU perspectives before classifying the results using an integrated classifier. The notation for this approach is GAN+GRU+1DCNN.
6. Conclusions
- Further explore a multimodal approach for detecting zero-day attacks;
- Conduct experiments using real network traffic data;
- Research on better multimodal techniques in real network detection scenarios;
- Dimensionality reduction using principal component analysis.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Papadimitriou, F. A nexus of Cyber-Geography and Cyber-Psychology: Topos/“Notopia” and identity in hacking. Comput. Hum. Behav. 2009, 25, 1331–1334. [Google Scholar] [CrossRef]
- Mishra, P.; Varadharajan, V.; Tupakula, U.; Pilli, E.S. A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tutor. 2018, 21, 686–728. [Google Scholar] [CrossRef]
- Lansky, J.; Ali, S.; Mohammadi, M.; Majeed, M.K.; Karim, S.H.T.; Rashidi, S.; Hosseinzadeh, M.; Rahmani, A.M. Deep learning-based intrusion detection systems: A systematic review. IEEE Access 2021, 9, 101574–101599. [Google Scholar] [CrossRef]
- Feng, F.; Li, K.-C.; Shen, J.; Zhou, Q.; Yang, X. Using cost-sensitive learning and feature selection algorithms to improve the performance of imbalanced classification. IEEE Access 2020, 8, 69979–69996. [Google Scholar] [CrossRef]
- Zhang, H.; Huang, L.; Wu, C.Q.; Li, Z. An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset. Comput. Netw. 2020, 177, 107315. [Google Scholar] [CrossRef]
- Chuang, P.-J.; Wu, D.-Y. Applying deep learning to balancing network intrusion detection datasets. In Proceedings of the 2019 IEEE 11th International Conference on Advanced Infocomm Technology (ICAIT), Jinan, China, 18–20 October 2019; pp. 213–217. [Google Scholar]
- Louati, F.; Ktata, F.B. A deep learning-based multi-agent system for intrusion detection. SN Appl. Sci. 2020, 2, 675. [Google Scholar] [CrossRef]
- Wang, Z.; Liu, Y.; He, D.; Chan, S. Intrusion detection methods based on integrated deep learning model. Comput. Secur. 2021, 103, 102177. [Google Scholar] [CrossRef]
- Tama, B.A.; Rhee, K.-H. HFSTE: Hybrid feature selections and tree-based classifiers ensemble for intrusion detection system. IEICE Trans. Inf. Syst. 2017, 100, 1729–1737. [Google Scholar] [CrossRef]
- Peng, W.; Kong, X.; Peng, G.; Li, X.; Wang, Z. Network intrusion detection based on deep learning. In Proceedings of the 2019 International Conference on Communications, Information System and Computer Engineering (CISCE), Haikou, China, 5–7 July 2019; pp. 431–435. [Google Scholar]
- Salama, M.A.; Eid, H.F.; Ramadan, R.A.; Darwish, A.; Hassanien, A.E. Hybrid intelligent intrusion detection scheme. In Soft Computing in Industrial Applications; Springer: Berlin/Heidelberg, Germany, 2011; pp. 293–303. [Google Scholar]
- Mehmood, M.; Javed, T.; Nebhen, J.; Abbas, S.; Abid, R.; Bojja, G.R.; Rizwan, M. A hybrid approach for network intrusion detection. CMC-Comput. Mater. Contin 2022, 70, 91–107. [Google Scholar] [CrossRef]
- Savanović, N.; Toskovic, A.; Petrovic, A.; Zivkovic, M.; Damaševičius, R.; Jovanovic, L.; Bacanin, N.; Nikolic, B. Intrusion Detection in Healthcare 4.0 Internet of Things Systems via Metaheuristics Optimized Machine Learning. Sustainability 2023, 15, 12563. [Google Scholar] [CrossRef]
- Malibari, A.A.; Alotaibi, S.S.; Alshahrani, R.; Dhahbi, S.; Alabdan, R.; Al-wesabi, F.N.; Hilal, A.M. A novel metaheuristics with deep learning enabled intrusion detection system for secured smart environment. Sustain. Energy Technol. Assess. 2022, 52, 102312. [Google Scholar] [CrossRef]
- Saif, S.; Das, P.; Biswas, S.; Khari, M.; Shanmuganathan, V. HIIDS: Hybrid intelligent intrusion detection system empowered with machine learning and metaheuristic algorithms for application in IoT based healthcare. Microprocess. Microsyst. 2022, 104622. [Google Scholar] [CrossRef]
- Chalé, M.; Bastian, N.D. Generating realistic cyber data for training and evaluating machine learning classifiers for network intrusion detection systems. Expert Syst. Appl. 2022, 207, 117936. [Google Scholar] [CrossRef]
- Thakkar, A.; Lohiya, R. Fusion of statistical importance for feature selection in Deep Neural Network-based Intrusion Detection System. Inf. Fusion 2023, 90, 353–363. [Google Scholar] [CrossRef]
- Ho, S.; Al Jufout, S.; Dajani, K.; Mozumdar, M. A novel intrusion detection model for detecting known and innovative cyberattacks using convolutional neural network. IEEE Open J. Comput. Soc. 2021, 2, 14–25. [Google Scholar] [CrossRef]
- Andresini, G.; Appice, A.; De Rose, L.; Malerba, D. GAN augmentation to deal with imbalance in imaging-based intrusion detection. Future Gener. Comput. Syst. 2021, 123, 108–127. [Google Scholar] [CrossRef]
- Alqahtani, A.S. FSO-LSTM IDS: Hybrid optimized and ensembled deep-learning network-based intrusion detection system for smart networks. J. Supercomput. 2022, 78, 9438–9455. [Google Scholar] [CrossRef]
- Yin, C.; Zhu, Y.; Fei, J.; He, X. A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access 2017, 5, 21954–21961. [Google Scholar] [CrossRef]
- Moizuddin, M.; Jose, M.V. A bio-inspired hybrid deep learning model for network intrusion detection. Knowl.-Based Syst. 2022, 238, 107894. [Google Scholar] [CrossRef]
- Muhammad, A.R.; Sukarno, P.; Wardana, A.A. Integrated Security Information and Event Management (SIEM) with Intrusion Detection System (IDS) for Live Analysis based on Machine Learning. Procedia Comput. Sci. 2023, 217, 1406–1415. [Google Scholar] [CrossRef]
- Daoud, M.A.; Dahmani, Y.; Bendaoud, M.; Ouared, A.; Ahmed, H. Convolutional neural network-based high-precision and speed detection system on CIDDS-001. Data Knowl. Eng. 2023, 144, 102130. [Google Scholar] [CrossRef]
- Nayyar, S.; Arora, S.; Singh, M. Recurrent neural network based intrusion detection system. In Proceedings of the 2020 International Conference on Communication and Signal Processing (ICCSP), Melmaruvathur, India, 28–30 July 2020; pp. 0136–0140. [Google Scholar]
- Kasongo, S.M. A deep learning technique for intrusion detection system using a Recurrent Neural Networks based framework. Comput. Commun. 2023, 199, 113–125. [Google Scholar] [CrossRef]
- Pingale, S.V.; Sutar, S.R. Remora whale optimization-based hybrid deep learning for network intrusion detection using CNN features. Expert Syst. Appl. 2022, 210, 118476. [Google Scholar] [CrossRef]
- Min, B.; Yoo, J.; Kim, S.; Shin, D.; Shin, D. Network anomaly detection using memory-augmented deep autoencoder. IEEE Access 2021, 9, 104695–104706. [Google Scholar] [CrossRef]
- Maseer, Z.K.; Yusof, R.; Bahaman, N.; Mostafa, S.A.; Foozy, C.F.M. Benchmarking of machine learning for anomaly based intrusion detection systems in the CICIDS2017 dataset. IEEE Access 2021, 9, 22351–22370. [Google Scholar] [CrossRef]
- Marteau, P.-F. Random partitioning forest for point-wise and collective anomaly detection—Application to network intrusion detection. IEEE Trans. Inf. Forensics Secur. 2021, 16, 2157–2172. [Google Scholar] [CrossRef]
- Elnakib, O.; Shaaban, E.; Mahmoud, M.; Emara, K. EIDM: Deep learning model for IoT intrusion detection systems. J. Supercomput. 2023, 79, 13241–13261. [Google Scholar] [CrossRef]
- Abbas, A.; Khan, M.A.; Latif, S.; Ajaz, M.; Shah, A.A.; Ahmad, J. A New Ensemble-Based Intrusion Detection System for Internet of Things. Arab. J. Sci. Eng. 2022, 47, 1805–1819. [Google Scholar] [CrossRef]
- Sohi, S.M.; Seifert, J.-P.; Ganji, F. RNNIDS: Enhancing network intrusion detection systems through deep learning. Comput. Secur. 2021, 102, 102151. [Google Scholar] [CrossRef]
- Liu, J.; Gao, Y.; Hu, F. A fast network intrusion detection system using adaptive synthetic oversampling and LightGBM. Comput. Secur. 2021, 106, 102289. [Google Scholar] [CrossRef]
- Gu, J.; Lu, S. An effective intrusion detection approach using SVM with naïve Bayes feature embedding. Comput. Secur. 2021, 103, 102158. [Google Scholar] [CrossRef]
- Khan, M.A. HCRNNIDS: Hybrid convolutional recurrent neural network-based network intrusion detection system. Processes 2021, 9, 834. [Google Scholar] [CrossRef]
- Gupta, N.; Jindal, V.; Bedi, P. LIO-IDS: Handling class imbalance using LSTM and improved one-vs-one technique in intrusion detection system. Comput. Netw. 2021, 192, 108076. [Google Scholar] [CrossRef]
- Araujo-Filho, P.F.d.; Naili, M.; Kaddoum, G.; Fapi, E.T.; Zhu, Z. Unsupervised GAN-Based Intrusion Detection System Using Temporal Convolutional Networks and Self-Attention. IEEE Trans. Netw. Serv. Manag. 2023, 20, 4951–4963. [Google Scholar] [CrossRef]
- Kumar, V.; Sinha, D. Synthetic attack data generation model applying generative adversarial network for intrusion detection. Comput. Secur. 2023, 125, 103054. [Google Scholar] [CrossRef]
- Yuan, L.; Yu, S.; Yang, Z.; Duan, M.; Li, K. A data balancing approach based on generative adversarial network. Future Gener. Comput. Syst. 2023, 141, 768–776. [Google Scholar] [CrossRef]
- Babu, K.S.; Rao, Y.N. MCGAN: Modified Conditional Generative Adversarial Network (MCGAN) for Class Imbalance Problems in Network Intrusion Detection System. Appl. Sci. 2023, 13, 2576. [Google Scholar] [CrossRef]
- Donkol, A.A.E.-B.; Hafez, A.G.; Hussein, A.I.; Mabrook, M.M. Optimization of Intrusion Detection Using Likely Point PSO and Enhanced LSTM-RNN Hybrid Technique in Communication Networks. IEEE Access 2023, 11, 9469–9482. [Google Scholar] [CrossRef]
- Altunay, H.C.; Albayrak, Z. A hybrid CNN+ LSTMbased intrusion detection system for industrial IoT networks. Eng. Sci. Technol. Int. J. 2023, 38, 101322. [Google Scholar] [CrossRef]
- Han, J.; Pak, W. Hierarchical LSTM-Based Network Intrusion Detection System Using Hybrid Classification. Appl. Sci. 2023, 13, 3089. [Google Scholar] [CrossRef]
- Zavrak, S.; Iskefiyeli, M. Flow-based intrusion detection on software-defined networks: A multivariate time series anomaly detection approach. Neural Comput. Appl. 2023, 35, 12175–12193. [Google Scholar] [CrossRef]
- Rekha, G.; Tyagi, A.K. Necessary information to know to solve class imbalance problem: From a user’s perspective. In Proceedings of the ICRIC 2019: Recent Innovations in Computing, Jammu, India, 8–9 March 2019; pp. 645–658. [Google Scholar]
- Dubey, A.K.; Jain, V. Comparative Study of Convolution Neural Network’s Relu and Leaky-Relu Activation Functions. In Applications of Computing, Automation and Wireless Systems in Electrical Engineering; Springer: Singapore, 2019; pp. 873–880. [Google Scholar]
- Sharafaldin, I.; Lashkari, A.H.; Ghorbani, A.A. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 2018, 1, 108–116. [Google Scholar] [CrossRef]
- Pandey, B.K.; Veeramanickam, M.; Ahmad, S.; Rodriguez, C.; Esenarro, D. ExpSSOA-Deep maxout: Exponential Shuffled shepherd optimization based Deep maxout network for intrusion detection using big data in cloud computing framework. Comput. Secur. 2023, 124, 102975. [Google Scholar] [CrossRef]
Traffic Class | Label | Numbers | Ratio |
---|---|---|---|
Benign | Benign | 2,273,097 | 80.30% |
DDoS | DDoS | 128,027 | 4.52% |
DoS | DoS Hulk | 231,073 | 8.16% |
DoS GoldenEye | 10,293 | 0.36% | |
DoS slowloris | 5796 | 0.20% | |
DoS Slowhttptest | 5499 | 0.19% | |
Port Scan | Port Scan | 158,930 | 5.61% |
Botnet | Bot | 1966 | 0.07% |
Brute Force | FTP-Patator | 7938 | 0.28% |
SSH-Patator | 5897 | 0.20% | |
Web Attack | Web Attack– Brute Force | 1507 | 0.05% |
Web Attack– Sql Injection | 21 | 0.001% | |
Web Attack– XSS | 652 | 0.02% | |
Infiltration | Infiltration | 36 | 0.002% |
Heartbleed | Heartbleed | 11 | 0.001% |
Total | N | 2,830,743 | 100% |
Label | Train (80%) | Test (20%) |
---|---|---|
Benign | 8000 | 2000 |
Heartbleed | 8000 | 2000 |
Infiltration | 8000 | 2000 |
Sql Injection | 8000 | 2000 |
XSS Total | 8000 40,000 | 2000 10,000 |
Class | Label | Total Number of Each Class | Ratio |
---|---|---|---|
dos | Back | 57,727 | 35.99% |
land | |||
neptune | |||
pod | |||
smurf | |||
teardrop | |||
u2r | buffer_overflow | 452 | 0.28% |
loadmodule | |||
perl | |||
rootkit | |||
r2l | ftp_write | 6503 | 4.05% |
guess_passwd | |||
imap | |||
multihop | |||
phf | |||
spy | |||
warezclient | |||
warezmaster | |||
probe | ipsweep | 16,479 | 10.27% |
nmap | |||
portsweep | |||
Satan | |||
normal | normal | 79,206 | 49.39% |
total | n | 160,367 | 100% |
Label | Train (80%) | Test (20%) |
---|---|---|
normal | 8000 | 2000 |
buffer_overflow | 8000 | 2000 |
loadmodule | 8000 | 2000 |
perl | 8000 | 2000 |
rootkit Total | 8000 40,000 | 2000 10,000 |
Attack Classes | Accuracy | Precision | Recall | F-Measure |
---|---|---|---|---|
buffer_overflow | 99.90% | 99.80% | 100% | 99.89% |
rootkit | 99.95% | 99.90% | 100% | 99.94% |
perl | 100% | 100% | 100% | 100% |
loadmodule | 100% | 100% | 100% | 100% |
Attack Classes | Accuracy | Precision | Recall | F-Measure |
---|---|---|---|---|
Sql Injection | 98.30% | 96.60% | 100% | 98.27% |
Heartbleed | 99.95% | 99.90% | 100% | 99.94% |
XSS | 100% | 100% | 100% | 100% |
Infiltration | 100% | 100% | 100% | 100% |
Buffer_Overflow | Rootkit | Perl | Loadmodule | |
---|---|---|---|---|
FPR | 0.20% | 0.10% | 0% | 0% |
FNR | 0% | 0% | 0% | 0% |
Sql Injection | Heartbleed | XSS | Infiltration | |
---|---|---|---|---|
FPR | 3.40% | 0.10% | 0% | 0% |
FNR | 0% | 0% | 0% | 0% |
Research | Avg Accuracy | Avg Precision | Avg Recall | Avg F-Measure |
[37] | 87% 86% | 63% 66.66% | 71.75% 72.33% | 66.5% 66.5% |
[28] | 89.61% 96.05% | 97.33% 93.43% | 91.47% 92.89% | 94.06% 92.86% |
[27] | 93.5% 93% | 91.7% 91.2% | 92.7% 92.2% | 92.2% 91.7% |
[49] | 87.15% 87.75% | 86.25% 86.95% | 85.6% 87.3% | 85.95% 87.1% |
[38] | 97.07% | 97.05% | 97.10% | 97.07% |
Proposed model | 99.56% 99.96% | 99.12% 99.92% | 100% 100% | 99.55% 99.95% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yu, L.; Xu, L.; Jiang, X. A High-Performance Multimodal Deep Learning Model for Detecting Minority Class Sample Attacks. Symmetry 2024, 16, 42. https://doi.org/10.3390/sym16010042
Yu L, Xu L, Jiang X. A High-Performance Multimodal Deep Learning Model for Detecting Minority Class Sample Attacks. Symmetry. 2024; 16(1):42. https://doi.org/10.3390/sym16010042
Chicago/Turabian StyleYu, Li, Liuquan Xu, and Xuefeng Jiang. 2024. "A High-Performance Multimodal Deep Learning Model for Detecting Minority Class Sample Attacks" Symmetry 16, no. 1: 42. https://doi.org/10.3390/sym16010042
APA StyleYu, L., Xu, L., & Jiang, X. (2024). A High-Performance Multimodal Deep Learning Model for Detecting Minority Class Sample Attacks. Symmetry, 16(1), 42. https://doi.org/10.3390/sym16010042