Performance Evaluation of Deep Learning Models for Classifying Cybersecurity Attacks in IoT Networks

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

While the paper is very well structured, with comprehensive overview of the contemporary literature and relevant research in the respective field, and well organized and sound research methodology and executed statistical analysis (with respectful data set), the result analysis and the conclusions are too narrow and pointing that CNN method is the one to be used for the future reference. However, there is no conclusion and analysis done of other frameworks/architectures (DNN, LSTM) and, if CNN is the one to be used, why other 2 failed to produce better results, i.e. what are the limitations of the DNN and LSTM in this respect. 

The result analysis and the conclusions section has to be enriched with sound elaborations, explanations and conclusions deriving from the statistical results, before paper could be published. 

Author Response

Reviewer's comment

While the paper is very well structured, with comprehensive overview of the contemporary literature and relevant research in the respective field, and well organized and sound research methodology and executed statistical analysis (with respectful data set), the result analysis and the conclusions are too narrow and pointing that CNN method is the one to be used for the future reference. However, there is no conclusion and analysis done of other frameworks/architectures (DNN, LSTM) and, if CNN is the one to be used, why other 2 failed to produce better results, i.e. what are the limitations of the DNN and LSTM in this respect. 

The result analysis and the conclusions section has to be enriched with sound elaborations, explanations and conclusions deriving from the statistical results, before paper could be published. 

Response to the comment.

We sincerely appreciate your detailed and constructive comments on our paper. We value your time and dedication in reviewing our work. We understand your concern about the narrowness of our analysis of results and conclusions, particularly regarding the preference for the CNN method compared to other approaches such as DNN and LSTM. To address this, the text in the document has been considered, as detailed in lines 261-284 and lines 330-362.

We hope that these improvements meet your expectations and contribute to a more comprehensive and balanced presentation of our findings. Once again, we sincerely appreciate your valuable feedback and are committed to improving our work based on your suggestions.

 

Reviewer 2 Report

Comments and Suggestions for Authors

The following comments are made to help improve this paper:

1) Line 8 refers to IoT in industrial fields, but this paper states that IoT is used in many other fields including home usage. Consider whether this reference to industrial usage is appropriate.

2) Line 10 states that the IoT network infrastructure is particularly vulnerable due to its complexity. Other studies conclude that IoT vulnerability is due to the use of open-source software, and manufacturers' reluctance to provide updates for device software. Reconsider whether IoT networks are vulnerable due to complexity.

3) The research results are presented as F1 score and percentage accuracy. Consider adding a paragraph to explain the value of the F1 score and the issues that arise from evaluating research results using percentage accuracy.

4) Consider comparing the related research using the F1 score.

5) Lines 38 - 39, new concerns regarding security, stemming from the vulnerabilities presented by IoT devices, such as their storage capacity, and processing power. Explain how storage capacity and processing power are vulnerabilities.

6) Line 110, Are the "records" referred to here TCP/IP packets?

7) Consider whether identifying the 46 features used in this research would be useful.

8) Line 130, what are the 6 features that only contain zero values?

9) Consider whether you should present your results using percentage accuracy. Would the F1 score be a better statistic?

 

 

 

Author Response

1) Line 8 refers to IoT in industrial fields, but this paper states that IoT is used in many other fields including home usage. Consider whether this reference to industrial usage is appropriate.

We appreciate your observation regarding the reference to IoT in industrial fields in line 8 of our article. We acknowledge that IoT is utilized across a variety of fields, not only in industrial environments but also in domestic usage and many other contexts. In doing so, the following text has been considered, as detailed in lines 8-11 of the document.

2) Line 10 states that the IoT network infrastructure is particularly vulnerable due to its complexity. Other studies conclude that IoT vulnerability is due to the use of open-source software, and manufacturers' reluctance to provide updates for device software. Reconsider whether IoT networks are vulnerable due to complexity.

We appreciate your observation regarding the statement in line 10 of our article regarding the vulnerability of IoT network infrastructure due to its complexity. We acknowledge that there are differing opinions and findings in the literature regarding the causes of vulnerability in IoT networks, including the use of open-source software and manufacturers' reluctance to provide software updates for devices. Taking this into account, the text from lines 8-11 has been considered as detailed in the document.

3) The research results are presented as F1 score and percentage accuracy. Consider adding a paragraph to explain the value of the F1 score and the issues that arise from evaluating research results using percentage accuracy.

We appreciate your suggestion regarding presenting the research results in the form of F1 score and percentage accuracy. We agree that it would be beneficial to add an explanatory paragraph about the value of the F1 score and the issues that may arise when evaluating research results using percentage accuracy. To this end, the following text has been added, as detailed in lines 268-276.

 

4) Consider comparing the related research using the F1 score.

We appreciate your suggestion to compare related research using the F1 score. We will consider this recommendation to enhance the comparison of our research with previous studies, and the following text described in lines 303-310 of the document has been taken into account.

 

5) Lines 38 - 39, new concerns regarding security, stemming from the vulnerabilities presented by IoT devices, such as their storage capacity, and processing power. Explain how storage capacity and processing power are vulnerabilities.

We appreciate your observation regarding lines 38-39 of our article and your request to explain how storage capacity and processing power are vulnerabilities in IoT devices. This statement prompted us to thoroughly review the cited references. Following this analysis, we have decided to remove the mention of "storage capacity and processing power" from the text, as they were used in other contexts. Therefore, we have proceeded to eliminate these mentions from the document.

 

6) Line 110, Are the "records" referred to here TCP/IP packets?

We appreciate your question regarding line 110 of our article. In that context, we do indeed refer to "records" as TCP/IP packets. These records, as detailed in [26], represent the communication data transmitted across the network using the TCP/IP protocol. For further clarity, we have added the related text described in lines 109-114.

 

7) Consider whether identifying the 46 features used in this research would be useful.

We appreciate your suggestion to consider whether it would be useful to identify the 46 features used in this research. In this case, the 46 characteristics have not been considered because upon analyzing the dataset, descriptors were found that have the same value in all records, which would not contribute significance to the proposed models. These characteristics are 'ece_flag_number', 'cwr_flag_number', 'Telnet', 'SMTP', 'IRC', and 'DHCP'.

8) Line 130, what are the 6 features that only contain zero values?

Thank you for your question about line 130 of our article. The 6 attributes that contain only zero values are 'ece_flag_number', 'cwr_flag_number', 'Telnet', 'SMTP', 'IRC', and 'DHCP', as detailed in lines 137-138. Despite their lack of variability in the collected data, these attributes are relevant for our analysis.

9) Consider whether you should present your results using percentage accuracy. Would the F1 score be a better statistic?

We appreciate your suggestion regarding presenting our results using percentage accuracy. We agree that using the F1 score is a more suitable statistic for our research, which is why it has been included in the document as detailed in lines 268-276.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors addressed the main points of concern.