Using Ensemble Learning for Anomaly Detection in Cyber–Physical Systems

Round 1

Reviewer 1 Report (Previous Reviewer 3)

PFA my comments in the pdf. I still find the paper lacks technicality. Please clearly identify strong technical contributions of the proposed approach

Comments for author File: Comments.pdf

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report (Previous Reviewer 4)

This article presents a hybrid anomaly detection approach to identify threats to Cyber-Physical Systems by using signature-based anomaly detection, threshold-based anomaly detection, and behavioral-based anomaly detection using Ensemble Learning to increase accuracy. The paper in this version is nicely written and can be accepted in its current form. 

Author Response

Thank you for your comments. Based on the comprehensive feedback from 5 reviewers, we sincerely believe the paper has been significantly improved by the reviewer’s suggestions and comments.

Reviewer 3 Report (Previous Reviewer 5)

Accept.

Author Response

Thank you for your comments. Based on the comprehensive feedback from 5 reviewers, we sincerely believe the paper has been significantly improved by the reviewer’s suggestions and comments.

Reviewer 4 Report (Previous Reviewer 1)

The content of the paper is acceptable.

Author Response

Thank you for your comments. Based on the comprehensive feedback from 5 reviewers, we sincerely believe the paper has been significantly improved by the reviewer’s suggestions and comments.

Round 2

Reviewer 1 Report (Previous Reviewer 3)

I do not have any new comments. I am ok to accept the paper. 

I am not satisfied with responses to some of my comments. I hope the authors can incorporate some of the unaddressed comments in their future submissions. 

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Recommendation: Major Revision

1.        Enhanced clarity is imperative in explicating how the hybrid model confronts and mitigates the complexities inherent in heterogeneous Cyber-Physical Systems (CPS) environments, delineating its strategic advantages.

2.        An extensive discourse on the inherent limitations of utilizing ensemble learning for anomaly detection would significantly enrich the study, providing a balanced perspective on its applicability and challenges.

3.        A more detailed justification is crucial for the selection of specific base classifiers within the ensemble framework, elucidating the rationale behind their choice and their contribution to the overall model's performance.

4.        The description of the experimental setup should be augmented to include comprehensive details on the data preprocessing methods employed, offering insight into the initial handling and preparation of data for analysis.

5. An in-depth examination of the computational demands and the scalability of the proposed ensemble model is advisable, assessing its feasibility and efficiency in processing large datasets and its adaptability to varying computational environments.

Reviewer 2 Report

- The text above (line 230 - 234) says that threshold-based filters are used on the OT side but here there is also an arrow coming from the IT side. Rework the description explain this or correct the figure 1.

- Abbreviats such as TN, TP in the formulas shall be introduced befoe use.

- The link to github in the conclusions does not exist.

Reviewer 3 Report

My comments are in the attached pdf

Comments for author File: Comments.pdf

The grammar needs improvements in several parts of the paper. 

Reviewer 4 Report

This article presents a hybrid anomaly detection approach to identify threats to Cyber-Physical Systems by using signature-based anomaly detection, threshold-based anomaly detection, and behavioral-based anomaly detection using Ensemble Learning to increase accuracy. the paper is interesting and can be accepted after addressing the following review comments as:

1- The authors have used conventional ML tools which are the SVM, NB, ANN, LR, and KNN and their ensembles to tackle the considered problem. IT is well-known that deep learning approaches might have better accuracy than the original ML tools when considering such applications. It is suggested to improve the current manuscript by adding DL such as 1DCNN or LSTM. Moreover, the hybrid ensemble of ML and DL will certainly boost the accuracy and performance of the models.

2- It is not very clear what type of boosting ensembles have been used. What about boosting ensembles such as  Adaboost, XGboost, etc?

3-  It is recommended to list the main research gaps and contributions as a bullet list at the end of the introduction section. 

4- The abstract section has been written carefully. Please rewrite the abstract section by mimicking (not explicitly) the IMRaD method. 

The authors have used too many long sentences in the manuscript. 

Reviewer 5 Report

1. In the abstract, too much background is introduced, while the work and conclusions obtained in this article are ignored. The abstract should be reorganized.

2. The specific structure and parameter settings of multiple classifiers should be supplemented, including LR, NB, SVM, KNN, and MLP. At present, the technical details are completely unclear. For example, what type of SVM is adopted, RBF kernel or polynomial kernel?

3. Many ensemble methods like voting, stacking, bagging and boosting have been proposed, but no reasonable reason has been given to choose the most suitable one.

4. According to the information in Table 2, this is a class imbalance task. How did the author consider solving this problem? What technologies have been introduced to solve this problem?

5. The training process of these models should be introduced.

6. Deep learning should be discussed in the related work, such as MR-DCAE: Manifold regularization-based deep convolutional autoencoder for unauthorized broadcasting identification, Fine-grained modulation classification using multi-scale radio transformer with dual-channel representation. 

7. The appendix is suggested to be organized on open-source platforms, such as GitHub.

N/A.