Efficient, Lightweight Cyber Intrusion Detection System for IoT Ecosystems Using MI2G Algorithm

Round 1

Reviewer 1 Report

The purpose of scientific research is not clear enough. But the level of scientific research paper is high and it can be recommended for publication.

Author Response

Reviewer#1, Concern # 1:

The purpose of scientific research is not clear enough. But the level of scientific research paper is high and it can be recommended for publication

Author response #1:  Dear Reviewer thank you for your positive response and support. We highly appreciate your response and honored to have a editor review our research paper.

We have considered the IoT systems which are exposed to attack and send the data to cloud for decision making. Authors feel that decision making can be faster if it is done at the local IoT level. Hence a new feature selection technique, that yields feature which consumes lesser computational power to identify an attack, is needed.

We have added a paragraph to address purpose of the research.

Author action:  We have added following in the introduction. However, most IoT systems are deployed at inaccessible, harsh locations and starve for storage and computational power. This indicates the need for the IDS system for IoT, which can work with less computational power and storage for safeguarding the IoT systems from cyber intrusions [29], and [31]. There are two challenges for an IDS system for IoT to work efficiently. Firstly, a feature selection technique quickly provides the relevant and important features and requires less computational power to find these features. Secondly, the feature selected through the feature selection technique and the classifier should yield higher accuracy in less training time. Hence, targeted research providing industry-ready and low computational cost feature selection is needed IDS for IoT systems

This study introduces the new feature selection technique called MI2G, which is based on Mutual Information, and Information Gain based on the critical concepts of the information theory. The study also introduces the IDS system for IoT using a decision tree (DT) and feature selection algorithm called MI2G. The proposed feature selection algorithm selects the feature which shows the high Mutual Information with the target feature and provides the maximum information about the features in the dataset. The proposed system using MI2G also provide good accuracy with various datasets.

Reviewer 2 Report

Dear Authors,

·        This article's contributions are very limited and most of the content is already available with the published articles.

·        The literature study of this article is poor. There are several works have published in the literature with the same scope.

·        Here, to me the Related work is simply poor meaning incomplete literature overview, especially that the method presented is used in many other works as well, also the same measures

 

·        Introduction is too lengthy with already published content. Most of the content in the introduction is available in many of the article. Important information about the challenge and contributions are not highlighted here.

 

·        Lack of mathematical rigor.
Low quality of language and presentation. English, figures, and typesetting are sub-par, significantly below our journal standards.

 

·        The referees made the following points about this paper.
This paper's novelty and scientific contribution do not appear to be significant enough for further consideration by the journal.

 

·        The novelty of the presented work is not significant.

 

·        The paper also lacks sufficient technical depth.
The overall scientific contribution of the paper is not significant enough.

·        The Conclusion section is underdeveloped as it is about the steps done rather than interpretation

·        Influential citations are missing in this article.

 

 

My opinion is that this manuscript does not provide sufficient contributions to the literature to be published. I suggest that the article is not suitable for publication as it is

Author Response

 

Reviewer#2, Concern # 1: This article's contributions are very limited and most of the content is already available with the published articles.

Author response:  Dear Reviewer We sincerely thank you for comments. There are a huge number of studies on the ML using IDS and few of them use the feature selection techniques. IoT systems usually have less memory and computational power available. Hence the IDS for preventing cyber-attacks are not able to take any decision locally and rely on the network bandwidth for decision from cloud.  Hence a new Feature selection technique which consumes lesser computational power is needed. Also, the use of information theory-based filter techniques to identify features is less explored by the researchers. We have upgraded the paper to cater to this concern.

 

Also, we have rewritten the contribution and added a section in introduction to highlight our contribution.

 

 Author action:  We have added the following section for the concern raised.

This study introduces the new feature selection technique called MI2G, which is based on Mutual Information, and Information Gain based on the critical concepts of the information theory. The study also introduces the IDS system for IoT using a decision tree (DT) and feature selection algorithm called MI2G. The proposed feature selection algorithm selects the feature which shows the high Mutual Information with the target feature and provides the maximum information about the features in the dataset. The proposed system using MI2G also provide good accuracy with various datasets.

1.1 Our Contribution

 

The major contribution of the study is as follows –

 

• A novel, unique hybrid feature selection algorithm MI2G (Mutual Information-Information Gain) by amalgamating the Mutual Information and Information Gain techniques for feature selection.
• Novel lightweight, and less computation cost-intensive IDS system showing good accuracy on CICIDS2018 in reduced training time
• Present detailed experimental findings to understand the suggested strategy as a valuable, universal IoT ecosystem IDS solution technique.
• Verify proposed IDS with reduced dataset obtained by applying the new MI2G feature selection algorithm on CICIDS2018.
• Comparison of the Proposed IDS system with available recent studies with recent datasets CICIDS2018 dataset, and classical UNSWNB15 dataset. The proposed system showed comparable accuracy and other parameters in lesser time.

 

 

Reviewer#2, Concern # 2: The literature study of this article is poor. There are several works have published in the literature with the same scope.Here, to me the Related work is simply poor meaning incomplete literature overview, especially that the method presented is used in many other works as well, also the same measures.

Author response:  Dear Reviewer thank you pointing out, we have added every research after a lot of review. We considered 108 papers, published in various reputed journal from 2018 to 2022, and then finalized 25 papers which used a new technique or showed an accuracy of more than 95%. Also, going by the suggestion we have augmented with the few research published in 2022.

 

IDS problems are classification problem and most of the classification problems use similar parameters to rate the performance. During our study of various studies, we found out that parameters such as accuracy, precision, recall and F1-Score were measured. Authors wanted to compare their study with that of the latest studies so we went ahead and calculated the performance on same parameters.

 

Author action:  The literature review has been augmented as following –

Soleymanzadeh et al.[51] tried the ensemble technique for IDS. They tied the classifiers on the NSLKDD dataset and UNSWNB15 dataset and reported an accuracy of 95%. However, all these ensemble classifiers took a training time of around 9-12 mins. On the other hand,  Carrera et al. [52] used the SHAP values for selecting the critical features and the autoencoders as classifiers. They proposed an IDS system with the Memory-Augmented Deep Autoencoder with Extended Isolation Forest classifier. Proposed IDS showed an accuracy of 95.14% for KDDCU99 and an accuracy of 83.5% for CICIDS2017.

Cao et al. [53] used the CNN and GRU using the hybrid feature selection technique Adaptive Synthetic Sampling (ADASYN) and Repeated Edited nearest neighbours (RENN). Their experiment showed the accuracy of UNSW_NB15 and NSL-KDD datasets, and the experimental results show that the classification accuracy reaches 86.25%, 99.69%,

Kareem et al. [54] proposed a new feature selection technique GTO-BSA, which was based on the Gorilla Troops Optimizer (GTO) based on the algorithm for bird swarms (BSA). They obtained an accuracy of 98.7% on the CICIDS2018 dataset.

 Imrana et al. [55] propose a novel feature-driven intrusion detection system, χ2-BidLSTM, that integrates a χ2 statistical model and bidirectional long short-term memory (BidLSTM). They used the NSL KDD dataset and obtained an accuracy of 95.62%.

Jeyaselvi et al. [56] used the Improved Pearson Correlation Coefficient (IPCC) to identify the features which require less computational power and time in addition to providing higher accuracy. Further, Hussein et al.[57] used the double feature selection technique on IoTID20 and showed accuracy in the range of 78.1% to 92% for features in the range of 20 to 25.    

Following latest studies published in 2022 have been cited –

• Soleymanzadeh, R., Aljasim, M., Qadeer, M. W., & Kashef, R. (2022). Cyberattack, and Fraud Detection Using Ensemble Stacking. AI, 3(1), 22-36.
• Carrera, F., Dentamaro, V., Galantucci, S., Iannacone, A., Impedovo, D., & Pirlo, G. (2022). Combining Unsupervised Approaches for Near Real-Time Network Traffic Anomaly Detection. Applied Sciences, 12(3), 1759.
• Cao, B., Li, C., Song, Y., Qin, Y., & Chen, C. (2022). Network Intrusion Detection Model Based on CNN, and GRU. Applied Sciences, 12(9), 4184.
• Kareem, S. S., Mostafa, R. R., Hashim, F. A., & El-Bakry, H. M. (2022). An effective feature selection model using hybrid metaheuristic algorithms for iot intrusion detection. Sensors, 22(4), 1396.
• Imrana, Y., Xiang, Y., Ali, L., Abdul-Rauf, Z., Hu, Y. C., Kadry, S., & Lim, S. (2022). χ 2-bidlstm: a feature driven intrusion detection system based on χ 2 statistical model, and bidirectional lstm. Sensors, 22(5), 2018.
• Jeyaselvi, M., Dhanaraj, R. K., Sathya, M., Memon, F. H., Krishnasamy, L., Dev, K., Qureshi, N. M. F. (2022). A highly secured intrusion detection system for IoT using EXPSO-STFA feature selection for LAANN to detect attacks. Cluster Computing, 1-16.
• Hussein, A. Y., Falcarin, P., & Sadiq, A. T. (2022). IoT Intrusion Detection Using Modified Random Forest Based on Double Feature Selection Methods. In International Conference on Emerging Technology Trends in Internet of Things and Computing (pp. 61-78). Springer, Cham

 

Reviewer#2, Concern # 2 Important information about the challenge and contributions are not highlighted here.

Author response:  Dear Reviewer thank you feedback. We have added the challenges faced by the industry and the need for a new feature selection technique. Further, We have rewritten the section 1.1 to highlight contribution of the study.

Author action : The introduction section has been updated or rewritten with the following –

However, most IoT systems are deployed at inaccessible, harsh locations and starve for storage and computational power. This indicates the need for the IDS system for IoT, which can work with less computational power and storage for safeguarding the IoT systems from cyber intrusions [29], and [31]. There are two challenges for an IDS system for IoT to work efficiently. Firstly, a feature selection technique quickly provides the relevant and important features and requires less computational power to find these features. Secondly, the feature selected through the feature selection technique and the classifier should yield higher accuracy in less training time. Hence, targeted research providing industry-ready and low computational cost feature selection is needed IDS for IoT systems.

 

This study introduces the new feature selection technique called MI2G, which is based on Mutual Information, and Information Gain based on the critical concepts of the information theory. The study also introduces the IDS system for IoT using a decision tree (DT) and feature selection algorithm called MI2G. The proposed feature selection algorithm selects the feature which shows the high Mutual Information with the target feature and provides the maximum information about the features in the dataset. The proposed system using MI2G also provide good accuracy with various datasets.

 

1.1 Our Contribution

 

The major contribution of the study is as follows –

 

• A novel, unique hybrid feature selection algorithm MI2G (Mutual Information-Information Gain) by amalgamating the Mutual Information and Information Gain techniques for feature selection.
• Novel lightweight, and less computation cost-intensive IDS system showing good accuracy on CICIDS2018 in reduced training time
• Present detailed experimental findings to understand the suggested strategy as a valuable, universal IoT ecosystem IDS solution technique.
• Verify proposed IDS with reduced dataset obtained by applying the new MI2G feature selection algorithm on CICIDS2018.
• Comparison of the Proposed IDS system with available recent studies with recent datasets CICIDS2018 dataset, and classical UNSWNB15 dataset. The proposed system showed comparable accuracy and other parameters in lesser time.

Reviewer#2, Concern # 3 Lack of mathematical rigor..

Author response:  Dear Reviewer thank you feedback. However, we would like to bring the attention to the section 3.1 and section 3.2 of the study. The study uses 14 equations for calculations and involves the concepts of probability theory and information theory as follows

Consider X and Y to be two random variables in the space. Each variable has the probability mass function as p(x) and p(y). Hence Kullback Leibler distance between these two mass functions is given as 

 

             (1)

                                   (2)

 (3)

                    (4)

I (X;Y) = H(X) – H(X|Y)                                                       (5)

By symmetry, it can be written that

                        I (X;Y) = H(Y) – H(Y|X)                               (6)

Since               H(X,Y) = H(X)  = H(Y|X)

                        I(X;Y) = H(X) + H(Y) – H(X,Y)                    (7)

 

The entropy of the variable X is given by H(X) and H(Y). The entropy of X with a known Y value is given as H(X|Y), and the entropy of Y for any given value of X is given as H(Y|X). H(X|Y) can be called the conditional entropy of X over Y and vice versa.

Information Gain (IG) of a feature X defines the reduction in the entropy of dataset D by splitting the values on feature x or it can be explained as the variance caused by feature X. It can be calculated as the difference between the prior entropy of dataset D, and the conditional entropy of the dataset for the feature [19]. Hence for any dataset D such that (x,y) = {x1, x2,………xn}, the IG can be given as

IG (D,x) = H(D) – H(D|x)                                       (8)

 

Reviewer#2, Concern # 4 Low quality of language and presentation. English, figures, and typesetting are sub-par, significantly below our journal standards.

Author response:  Dear Reviewer thank you feedback. We have used the professional editing services for taking care of the concerns raised. Also, we have taken help of our colleagues who are native English speakers.

 

Reviewer#2, Concern # 5: This paper's novelty and scientific contribution do not appear to be significant enough for further consideration by the journal.

Author response:  Dear Reviewer We sincerely thank you for comments. We would like to  to bring your attention to section 6.1 of the study. The section 6.1 compares the proposed system with the latest published studies and finds that proposed system provides comparable accuracy, precision, recall , F1Score for various datasets in less time and consumes less memory. 

Concern about the novelty and contribution have already been  addressed in concern#2.

 

Reviewer#2, Concern # 6: The Conclusion section is underdeveloped as it is about the steps done rather than interpretation.

Author response:  Dear Reviewer We sincerely thank you for comments. Based on your suggestion, we have rewritten the conclusion. The rewritten conclusion is given below.

Author Action : The conclusion has been rewritten to summarize the results and explanation. Also, the conclusion now provides a summary, takes care of the practical application and future directions. Kindly find below the conclusion.

Conclusion

This research proposes a hybrid feature selection technique using the concept of information theory for ML-based IDS to extract the relevant and important features. The new feature selection technique, MI2G, selects the features that show high mutual information with the label and high information gain. The mutual information suggests the statistical dependence of the label on the feature, and information gain indicates the feature which contributes to the randomness in the dataset. The feature selected using the MI2G technique reduced not only the training time (in a range of 30%-60%) but also boosted the accuracy (in a range of 2.7% -5.1%) of the ML algorithms such as LR, LDA, NB, DT, RF, SVM, and GBM.

The DT classifier performed consistently better on all parameters than other ML classifiers. DT was able to classify the attacks with an accuracy of 99.5% and was also able to classify the attacks with similar Precision and recall in a training time of  0.39 seconds. The high accuracy, Precision, and inadequate training or test time confirm that features selected using the MI2G are highly relevant and important. Hence, this study proposes an IDS system using the MI2G feature selection technique. DT should be used for IoT systems with low computational power and more susceptible to cyber-attacks.

The IDS system proposed in this study showed better or comparable accuracy and Precision when compared with the latest studies on various datasets such as CICIDS18 and UNSWNB15.

The authors plan to apply the proposed IDS and feature selection techniques to classify the attacks on cell phone towers located in harsh locations and environments, which often have low computational power available and cater to a huge amount of data. Similarly, the proposed IDS and feature selection technique for the Internet of Medical Things (IoMT) and Internet of vehicles (IoV) prevent cyber-attacks and enable them to take decisions accurately, precisely, fastly, and locally. 

 

 

Reviewer#2, Concern # 7 Influential citations are missing in this article..

 

Author response:  Dear Reviewer We sincerely thank you for comments. This concern is already responded in your concern#2. As per your suggestion, we have added few important citations from SCI journals. These studies were published in 2022 only.

 

Author Response File: Author Response.docx

Reviewer 3 Report

Efficient, Lightweight Cyber Intrusion Detection System for IoT ecosystems using MI2G Algorithm

 

 

The authors propose a MI2G Algorithm to overcome the challenges of computational cost and accuracy. The proposed algorithm is based on the Information theory models to select the feature with very high statistical dependence and entropy reduction in the dataset. According to the authors, this algorithm feature selection algorithm in training time by 27%-63% with different classifiers. In addition, the proposed approach shows an accuracy, precision, recall, and F1-Score of even more than 99% when tested with the CICIDS2018 dataset.

 

 

 

Comments/suggestions

 

-          The research design is not appropriate – must be improved

-          The conclusions are not supported by the results – this is mandatory requirement for the authors

-          The methods are adequately described

-          Editing for English language is required - mandatory

Author Response

Reviewer#3, Concern # 1: The research design is not appropriate – it must be improved

 

Author response:  Dear Reviewer We sincerely thank you for the comments. We have rewritten the experiment set up section of the study. We have also changed the figure 2 and 3. These figures are part of the research design or experimental setup.

 

Author action: The research design or experimental set up has been rewritten as per the suggestion. The updated section is as follow –

 

This section introduces the experiments and steps taken to check the validity of the proposed MI2G algorithm for feature selection. It helps to identify the suitable classifier for the IDS system. In the current research, we applied the data preprocessing steps to remove the unidentified values and applied the transformation and normalisation to ensure that good data leads to accurate results [51-54]. The dataset was split into a training set containing 80% and a validation set containing 20% of the complete dataset. The MI2G feature selection algorithm identified the relevant and important features applied to various machine learning algorithms. Each machine learning algorithm was tuned using GridSearchCV to select the machine learning model with the best accuracy. The parameters for each model with the best accuracy are given in table 3. These models with the best parameters were applied to the validation set to verify the performance [55-59]. In order to avoid the overfitting of the model, models such as LR, LDA, SVM, GBM, and LDA were regularised using the L2 penalty. L2 penalty is susceptible to outliers, but it worked well in the current experiment because the dataset did not contain any outlier data [60-63]. The performance of ML algorithms was evaluated based on the criteria given in table 3. On a high level, the complete experimental setup is depicted in figure 2, and the schematic diagram of the proposed IDS system is given in figure 3.

 

 

Figure 2: Experimental Setup

 

 

 

 

Figure 3: Proposed schematic diagram of proposed IDS system

 

Reviewer#3, Concern # 2: The conclusions are not supported by the results – this is mandatory requirement for the authors

Author response:  Dear Reviewer We sincerely thank you for comments. Based on your suggestion, we have rewritten the conclusion. The rewritten conclusion is given below.

Author Action : The conclusion has been rewritten to summarize the results and explanation. Also, the conclusion now provides a summary, takes care of the practical application and future directions. Kindly find below the conclusion.

Conclusion

This research proposes a hybrid feature selection technique using the concept of information theory for ML-based IDS to extract the relevant and important features. The new feature selection technique, MI2G, selects the features that show high mutual information with the label and high information gain. The mutual information suggests the statistical dependence of the label on the feature, and information gain indicates the feature which contributes to the randomness in the dataset. The feature selected using the MI2G technique reduced not only the training time (in a range of 30%-60%) but also boosted the accuracy (in a range of 2.7% -5.1%) of the ML algorithms such as LR, LDA, NB, DT, RF, SVM, and GBM.

The DT classifier performed consistently better on all parameters than other ML classifiers. DT was able to classify the attacks with an accuracy of 99.5% and was also able to classify the attacks with similar Precision and recall in a training time of  0.39 seconds. The high accuracy, Precision, and inadequate training or test time confirm that features selected using the MI2G are highly relevant and important. Hence, this study proposes an IDS system using the MI2G feature selection technique. DT should be used for IoT systems with low computational power and more susceptible to cyber-attacks.

The IDS system proposed in this study showed better or comparable accuracy and Precision when compared with the latest studies on various datasets such as CICIDS18 and UNSWNB15.

The authors plan to apply the proposed IDS and feature selection techniques to classify the attacks on cell phone towers located in harsh locations and environments, which often have low computational power available and cater to a huge amount of data. Similarly, the proposed IDS and feature selection technique for the Internet of Medical Things (IoMT) and Internet of vehicles (IoV) prevent cyber-attacks and enable them to take decisions accurately, precisely, fastly, and locally. 

 

 

Reviewer#3, Concern # 3: The methods are adequately described.

Author response:  Dear Reviewer thank you for your positive response and support. We highly appreciate your response and honored to have a editor review our research paper.

 

Reviewer#3, Concern # 4: Editing for English language is required – mandatory

Author response:  Dear Reviewer thank you for your feedback. We have taken the professional editing service and have augmented the paper. Also, we have taken the feedback from our colleagues who are native English speakers.

Reviewer 4 Report

The paper describes an interesting topic of the IoT ecosystem. The major contributions of the paper is MI2G Algorithm. The problem in this paper is well defined and the objectives are clear. This paper adequately put the progress it reports in the context of previous work, representative referencing and introductory discussion. It is clearly and concisely written. 

You need to consider the following points in your paper:

* How practical is your proposed solution?

* Some figures are not clear enough, such as figures 5,6,7.

 

 

Author Response

Reviewer#4, Concern # 1: How practical is your proposed solution?.

Author response : Dear Reviewer. Thank you for your feedback. The IDS and FS technique are very practical as it address the industrial problem. Also, one of the authors is applying the proposed technique in a pilot for the protection of cellphone towers and EV charging the stations. We have also metioned this in paper.

Author action :  As responded above, following has been added in introduction and conclusion section.

However, most IoT systems are deployed at inaccessible, harsh locations and starve for storage and computational power. This indicates the need for the IDS system for IoT, which can work with less computational power and storage for safeguarding the IoT systems from cyber intrusions [29], and [31].

 

The authors plan to apply the proposed IDS and feature selection techniques to classify the attacks on cell phone towers located in harsh locations and environments, which often have low computational power available and cater to a huge amount of data. Similarly, the proposed IDS and feature selection technique for the Internet of Medical Things (IoMT) and Internet of vehicles (IoV) prevent cyber-attacks and enable them to take decisions accurately, precisely, fastly, and locally.

 

Reviewer#4, Concern # 2: Some figures are not clear enough, such as figures 5,6,7.

Author response and action : Dear Reviewer. Thank you for your feedback. We went through all the images and we have remade the images which were not clear. Kindly find below the updated figures.

Figure 5: Accuracy comparison of the initial dataset and feature selected through the MI2G dataset

 

Figure 6: Precision comparison of the initial dataset and feature selected through the MI2G dataset

Figure 7: Recall comparison of the initial dataset and feature selected through the MI2G dataset

Figure 8: Comparison of F1-Score of the initial dataset and feature selected through the MI2G dataset

 

Round 2

Reviewer 2 Report

the paper still need more modification in all sections to be accepted for publication 

Reviewer 3 Report

All of the suggestions were considered by the authors.